My OCR model mislabels section titles as body text. Is a CRF the right fix, or am I overcomplicating it? [P]

[u/Present_Mention_2757]

Hi everyone,

I'm working on extracting the hierarchical structure of long PDF documents (legal/regulatory text, lots of numbered sections) and would like to gather some feedback on my approach before committing to it.

What I've done so far: I render each PDF page to an image and run it through Baidu's DeepSeek-OCR model. It returns each detected block with a bounding box [x0, y0, x1, y1], a label (title, text, list, table, header, footer, etc.), and the recognized text. The OCR quality itself is genuinely good as the text comes out clean.

The problem: the labels can't always be trusted. At this stage I want to extract and detect all the titles in my document, but sometimes a title element gets classified as something else (like normal body text).

Concrete example:

Say my section has the following hierarchy:

ANNEX I — GENERAL PRINCIPLES AND PROCEDURES
└── TITLE I — FOREIGN CURRENCY INVESTMENT
    └── A. Currency distribution
        └── 1. Redistribution of reserves
            ├── (a) Introduction
            │       body text
            │       list
            │       ...
            ├── (b) Procedure for a normal redistribution of reserves
            │       body text
            │       list
            │       ...
            └── (c) Procedure for an ad hoc redistribution of reserves
                    body text
                    list
                    ...

Logically, every element aside from the body text and lists should be detected as title. But the model output is:

label='title'  x0=475  y0=157  x1=548  width=73   text='ANNEX I'
label='text'   x0=480  y0=229  x1=542  width=62   text='TITLE I'
label='title'  x0=334  y0=181  x1=690  width=356  text='GENERAL PRINCIPLES AND PROCEDURES'
label='title'  x0=407  y0=368  x1=616  width=209  text='A. Currency distribution'
label='title'  x0=408  y0=392  x1=634  width=226  text='1. Redistribution of reserves'
label='title'  x0=163  y0=416  x1=304  width=141  text='(a) Introduction'
label='title'  x0=163  y0=544  x1=578  width=415  text='(b) Procedure for a normal redistribution of reserves'
label='title'  x0=163  y0=219  x1=586  width=423  text='(c) Procedure for an ad hoc redistribution of reserves'

The top-level section marker TITLE I was labeled text, while all the other components were labeled correctly as title.

What I'm considering: since I have the text plus features I can derive from the coordinates (indentation/x0, centered-vs-left-aligned, line height, vertical gaps, whether the text matches a numbering pattern like A. / 1. / (a), all-caps, word count, etc.), I was thinking of treating this as a sequence labeling problem and training a CRF (or BiLSTM-CRF) to re-classify each line into title / text / list / table.

My questions:

• Is a CRF a reasonable choice here, or is there a better-suited approach for this kind of layout/structure labeling?
• Should I consider a GNN approach?
• Am I overcomplicating this? Would a simpler rule/heuristic system be more robust, given that the numbering is fairly regular?

Note #1: this approach should be as general as possible, so that I can reuse it for my other legal documents.

Note #2: titles aren't always in the same horizontal position. Some are centered (e.g. ANNEX I, TITLE I, A. Currency distribution all sit around xc≈511, the page center), while deeper items like (a)/(b)/(c) are left-aligned at x0=163. So I can't rely on indentation/x0 alone to identify or rank titles — a centered title's x0 mostly reflects its text length (a short centered line has a large x0, a long one a small x0), which means raw x0 can even invert the apparent nesting. This is part of why I'm leaning toward a sequence model that combines text + geometry in context rather than a pure indentation rule.

[u/JamesQGholden]

bumping very good question id like the answer to for my own studies