Extracting Structured Data from Documents with AI APIs

Document data extraction with AI works as a five-stage pipeline: ingest, OCR and layout parsing, schema-bound model extraction, deterministic validation, then human review for the residue. The model is the least reliable stage. The pipeline built around it is what makes field-level accuracy above 95% achievable on invoices, contracts, and forms at volume.

This guide covers the pipeline contract, why naive “just ask for JSON” collapses on long documents, the schema and validator to ship, and the routing, throughput, and evaluation practices that keep accuracy from drifting.

The extraction pipeline: five stages and one contract

Think of it as a chain where each stage hands the next a typed artifact: pages → IR → candidate fields → accepted fields → corrections. Nothing writes to your database except the validator. The model only proposes.

Stage	Artifact it produces	Failure it prevents
1. Ingest	Per-page: text layer with word bounding boxes, or a 200–300 DPI raster	One scan mode silently degrading the whole document
2. Layout parse	Page-anchored IR: typed blocks, reading order, table grids	Column collapse and shuffled reading order
3. Model extraction	Schema-bound partial objects, each field carrying a verbatim quote and page anchor	Invented values and unparseable JSON
4. Validation	Accepted values + a list of flagged issues	Arithmetic and cross-field errors that look plausible
5. Human review	Corrections, appended to your gold set	Silent errors reaching the ledger

Two decisions at stage 1 matter most. First, decide per page, not per document: real submissions mix a digital cover page with scanned attachments. If page.get_text() returns a meaningful character count, use the embedded text layer — it is cheaper than OCR and preserves exact coordinates, which you need for provenance. Rasterize and OCR only the pages without one. Second, make the stage-2 IR a real contract: page-anchored JSON with typed blocks (text, table, key_value), a reading-order index, and bounding boxes. When you swap OCR engines, only the parser changes — prompts, validators, and the review UI stay untouched.

Capture OCR character confidence at stage 1 while you are there — it is the cheapest predictor of downstream extraction failure you get for free.

Why “just ask for JSON” fails on long and multi-page documents

Four failure modes get lumped together as “the model isn’t good enough.” They have different fixes.

• Silent omission. On a 60-page agreement, fields in the middle third get dropped while the model still returns valid JSON with a plausible subset. There is no error to catch.
• Output truncation. A page with 900 line items exceeds the output budget and the JSON is cut mid-object. A lenient parser makes this worse: it salvages the head of the array and you never learn the tail existed.
• Cross-references broken by chunking. Totals on page 1, line items on pages 2–9, tax rules in the terms section. Page-local extraction returns a total you cannot verify and rows you must sum yourself — and concatenating per-page rows naively double-counts any row that appears in two overlapping chunks.
• Tables linearized into nonsense. Text extraction flattens a table row-major, so a wrapped cell becomes the first token of the next “row,” multi-row headers merge, and column association dies. The model then invents a plausible structure over garbage input.

Free-form “respond in JSON” adds a fifth layer: markdown fences, preambles, invented enum values, thousands separators inside numbers, and locale-specific dates. The fix is structural, not prompt-level. Constrain the decoder with a schema (see structured outputs), and make chunking explicit: extract partial objects per unit, then reduce them deterministically in your own code.

Schema-driven extraction with strict validation

The schema below does two things at once. It constrains decoding so the response is valid JSON by construction, and it makes every field grounded: each value must carry the verbatim source quote that supports it plus the page it came from. Grounding is the cheapest hallucination detector available — you verify it with a substring check, no second model call required.

# schema_constrained_extraction.py
import json
from openai import OpenAI

client = OpenAI(base_url="https://your-gateway/v1", api_key="...")

def field(value_type):
    """A grounded field: the value, the verbatim quote, and the page anchor."""
    return {
        "type": "object",
        "additionalProperties": False,
        "required": ["value", "quote", "page"],
        "properties": {
            "value": value_type,
            "quote": {"type": "string"},
            "page":  {"type": "integer", "minimum": 1},
        },
    }

INVOICE_SCHEMA = {
    "type": "object",
    "additionalProperties": False,
    "required": ["invoice_number", "issue_date", "currency", "subtotal", "total", "line_items"],
    "properties": {
        "invoice_number": field({"type": "string"}),
        "issue_date":     field({"type": "string", "description": "ISO 8601, YYYY-MM-DD"}),
        "currency":       field({"type": "string",
                                 "enum": ["USD", "EUR", "GBP", "JPY", "CNY", "AUD", "CAD"]}),
        "subtotal":       field({"type": "number"}),
        "total":          field({"type": "number"}),
        "line_items": {
            "type": "array",
            "items": {
                "type": "object",
                "additionalProperties": False,
                "required": ["description", "quantity", "unit_price", "amount", "page"],
                "properties": {
                    "description": {"type": "string"},
                    "quantity":    {"type": "number"},
                    "unit_price":  {"type": "number"},
                    "amount":      {"type": "number"},
                    "page":        {"type": "integer", "minimum": 1},
                },
            },
        },
    },
}

resp = client.chat.completions.create(
    model=EXTRACTION_MODEL,
    response_format={"type": "json_schema",
                     "json_schema": {"name": "invoice", "strict": True,
                                     "schema": INVOICE_SCHEMA}},
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},  # stable prefix: cacheable
        {"role": "user",   "content": page_ir_text},   # per-page payload
    ],
)
data = json.loads(resp.choices[0].message.content)

Keep the schema honest about absence. If a field can genuinely be missing, allow null rather than letting the model guess — an explicit null is a signal you can route, an invented value is a signal you cannot see. Put the schema and instructions in a stable system prefix so prompt caching can hit; per-page content goes in the user turn, after it.

Validation is deterministic code, and it must never repair values silently. A mismatch means the extraction is wrong upstream; auto-correcting the total hides the bug and poisons your evaluation set.

from datetime import date
from decimal import Decimal

CENT = Decimal("0.01")

def money(x):
    return Decimal(str(x)).quantize(CENT)

def grounded(f, page_text: str) -> bool:
    """Reject any value whose quote is not literally present in the source."""
    q = " ".join(f["quote"].split())
    p = " ".join(page_text.split())
    return bool(q) and q in p

def validate_invoice(data, pages):
    accepted, flagged = {}, []

    for name in ("invoice_number", "issue_date", "currency", "subtotal", "total"):
        f = data[name]
        if not grounded(f, pages.get(f["page"], "")):
            flagged.append({"field": name, "reason": "ungrounded", "quote": f["quote"][:80]})
        else:
            accepted[name] = f["value"]

    try:
        date.fromisoformat(accepted["issue_date"])
    except (KeyError, ValueError):
        flagged.append({"field": "issue_date", "reason": "bad_format"})

    rows = data["line_items"]
    # Tolerance must absorb per-row rounding, not real mismatches.
    tol = CENT * max(len(rows), 1)
    row_sum = sum(money(r["amount"]) for r in rows)
    if abs(row_sum - money(data["subtotal"]["value"])) > tol:
        flagged.append({"field": "subtotal", "reason": "sum_mismatch",
                        "computed": str(row_sum), "stated": str(data["subtotal"]["value"])})

    for r in rows:
        if abs(money(r["quantity"]) * money(r["unit_price"]) - money(r["amount"])) > CENT * max(r["quantity"], 1):
            flagged.append({"field": "line_items", "reason": "row_math",
                            "page": r["page"], "description": r["description"][:60]})

    return accepted, flagged, rows

The rule that makes this pay off: a value is promoted to “accepted” only by the validator, never by the model. Everything else lands in a review queue with its quote and page anchor attached, which turns review from re-reading a document into confirming a highlighted span.

Tables, multi-page documents, and repeated or sectioned fields

These three cases break page-local extraction in different ways, so handle them with different mechanics.

Tables. Detect the grid before the model sees anything: run a table detector over the page, assign each table a stable table_id, and serialize it to CSV or Markdown with a single header row. If the table has merged cells, multi-row headers, or checkbox columns, crop the region as an image and send that to a multimodal model instead — see multimodal AI APIs. Never hand the model a linearized text dump of a table and hope. Then verify: if the number of extracted rows differs from the number of detected rows, you have a table-extraction failure.

Multi-page documents. Extract one page per call, then reduce deterministically. Document-scoped fields (invoice number, total) resolve by agreement: if two pages report different values, keep both as candidates and flag a conflict rather than letting the last page win. Row-scoped fields (line items) are concatenated and deduplicated on the row’s own identity — page, normalized description, amount — because overlapping chunks re-emit the same row, and silent double counting inflates every downstream total.

def reduce_pages(page_results):
    doc, rows, conflicts = {}, [], []

    for pr in page_results:
        for name, f in pr.get("document_fields", {}).items():
            prev = doc.get(name)
            if prev is None:
                doc[name] = f
            elif prev["value"] != f["value"]:
                conflicts.append({"field": name, "candidates": [prev, f]})

        rows.extend(pr.get("line_items", []))

    seen, deduped = set(), []
    for r in rows:
        key = (r["page"], " ".join(r["description"].split()).casefold(), str(r["amount"]))
        if key not in seen:
            seen.add(key)
            deduped.append(r)

    return doc, deduped, conflicts

Repeated and sectioned fields. Contracts with multiple parties, schedules, and amendments do not fit a flat schema. Model them as an array of labelled groups — {"groups": [{"label": "...", "fields": {...}}]} — and extract per section rather than per document. Give each group a section_id derived from the heading so that “Party B” is unambiguous, and add a deterministic check that the number of groups matches the number of section headings you detected in the layout pass.

Confidence and human-in-the-loop routing

Do not start with the model’s self-reported confidence. It is poorly calibrated, it costs output tokens on every field, and it is the weakest signal you have. Use deterministic signals first, add self-reported confidence only after you have measured its calibration on your own data, and route on expected cost of error rather than on accuracy alone.

Trigger	Signal	Action
Schema violation under strict decoding	Should be impossible; means the route ignored your schema	Hard fail, alert, retry on a different provider route
Grounding failure	Quote not found in the page text	Re-render page at higher DPI and re-extract; review only if it persists
Arithmetic mismatch	Rows do not reconcile to subtotal, or subtotal + tax ≠ total	Review the whole table region, not just the failing total
Cross-page conflict	Two pages report different values for a document-scoped field	Review with both candidates and their anchors side by side
Missing required value	Explicit null where the field must exist	Retry with a narrower crop; then review
Out-of-vocabulary enum	New supplier, currency, or unit type	Review once, then extend the enum and the mapping table
Low OCR character confidence	Page mean below your calibrated threshold	Re-OCR with deskew and higher DPI, re-extract, review only if it survives
Unknown layout fingerprint	First N documents from a new vendor	Full-document review for the first N, then auto-route
High-value document	Amount above a business threshold	Always review, regardless of confidence

Calibrate the thresholds against your gold set: pick the operating point where the marginal review hour costs less than the marginal error it prevents. Keep review decisions append-only — every correction is a labelled example, and that log is how the system improves without a retraining project.

Cost and throughput: batch backfills, page-level parallelism

Three levers dominate extraction economics.

• Batch for anything not interactive. Backfills, nightly ingestion, and archive migrations do not need a synchronous response, and batch AI APIs trade turnaround time for a large discount on the same model. Reserve synchronous calls for the queue where a user is waiting.
• Parallelize per page, not per document. A 200-page document becomes 200 concurrent calls and finishes in roughly the time of its slowest page. The tradeoff is real: each call re-sends the shared instructions, so page-parallel costs more input tokens than one large call. Parallelize when pages are independent (line-item lists, forms) or when latency matters; use one chunked call for short, heavily cross-referential documents.
• Tier by page, and cache the prefix. Classify each page cheaply first — a page with a text layer and zero detected tables is a different job from a skewed scan with merged-cell tables. Send easy pages to a small/fast model and hard ones to a stronger or multimodal model. Keep the system prompt and schema in a stable prefix so prompt caching hits.

Retry at page granularity with an idempotency key of (document_id, page, schema_version), so one bad page never re-runs a 200-page job. Track cost per accepted field, not cost per document: it is the only metric that charges you for retries and for the review time your thresholds create. A cheap model with a 20% review rate is often more expensive than a stronger model with a 5% one. For the broader levers, see reducing AI API costs.

Accuracy and evaluation: a labelled gold set and field-level scoring

Document-level accuracy is the metric that hides the most. “95% accurate” is compatible with a tax ID field that is right 60% of the time. Score per field, with precision and recall.

• Build a stratified gold set of 150–300 documents: multiple vendors, layouts, scan qualities, and languages. Double-annotate a 10% subsample. Where two careful humans disagree, the field is ambiguous — fix the schema definition, not the model.
• Normalize before scoring. Currency to Decimal, dates to ISO 8601, strings to casefolded whitespace-collapsed form, supplier names to canonical entity IDs. Otherwise you score formatting differences as errors and chase phantom regressions.
• Separate the two error classes. A missed line item hurts recall and understates payables; a phantom row hurts precision and inflates them. Weight them per field according to which failure costs more, and report both.
• Measure the silent error rate. Fields that pass schema validation and grounding but are still wrong. This is the number that reaches production, and the gold set is the only thing that can see it.
• Gate changes in CI. Version prompts and schemas like code, re-run the gold set on every change, and block the deploy if any field’s F1 drops beyond a fixed tolerance. Freeze previously-failing documents into a hard set and never remove them.
• Watch confusion pairs. If issue_date and due_date trade errors, add a description or a deterministic ordering check — that is a schema fix, not a model upgrade.

Frequently asked questions

Should I OCR every page, or use the PDF text layer?

Use the text layer whenever it exists and is meaningful. It is cheaper, more accurate than OCR on the same page, and it gives you exact word coordinates for provenance. Rasterize and OCR only pages with no usable text layer, and decide this per page — mixed documents are the norm, not the exception.

How many pages should go into one extraction call?

Start with one page per call for tables and line items, because that keeps output tokens bounded and makes retries cheap. Group up to about five pages only when the fields are document-scoped and the pages are text-light. Measure cross-page field recall at both settings on your gold set before you commit — the answer is document-type specific.

Do I need a multimodal model for extraction?

Not for clean digital PDFs with simple tables. You do need one for skewed scans, merged-cell or multi-row-header tables, checkboxes, stamps, and handwriting, because those structures are destroyed by text extraction and survive in the image. Route those pages specifically rather than paying multimodal cost for every page.

Can I trust the model’s confidence score?

Not out of the box. Self-reported confidence is usually miscalibrated and costs tokens on every field. Rank signals by reliability: schema validation, grounding checks, arithmetic reconciliation, cross-page agreement, OCR character confidence — then self-reported confidence, and only after you have verified its calibration on your own labelled data.

Conclusion

Accurate document extraction is an engineering problem more than a modeling one. Decide the OCR path per page, make the layout IR a real contract, constrain decoding with a schema, and require every value to cite a verbatim quote. Let deterministic validation decide what is accepted, route the residue to humans by expected cost of error, batch the backfills, and score accuracy per field against a frozen gold set. Swapping models then becomes a configuration change instead of a rewrite — and a single OpenAI-compatible endpoint such as qoraapi.com lets you do exactly that across providers without touching the pipeline.

Related reading

• AI Structured Outputs Explained: JSON Mode, Schema Enforcement, Reliable Parsing
• Batch AI APIs: Processing Millions of Requests Affordably
• Multimodal AI APIs: Working with Vision and Audio
• Building a RAG Ingestion Pipeline: Crawling, Parsing, and Syncing
• Detecting and Reducing Hallucinations in Production LLM Apps
• AI Gateway vs API Gateway: Key Differences and When to Use Each
• Top 10 Real-World Use Cases for an AI API in 2026