OCR Invoice Extraction With Human Review: A Design Guide

Invoices and purchase orders arrive as scans, photos and PDFs in every layout imaginable, and what you need is a small set of reliable fields in your accounting or ERP system. Getting from one to the other is no longer the hard part. Doing it so that people can trust the result is. This guide covers how to design a pipeline where automation does the bulk of the work and a person catches what it should not decide alone.

Think in stages

1. Intake. Accept the formats you expect, enforce size limits, detect duplicates by file hash, and reject anything unsafe.
2. Text extraction. Digital PDFs often contain real text that can be read directly. Scans and photos need OCR or a vision model.
3. Field extraction. Turn text and layout into structured fields.
4. Validation. Check the fields against rules that a correct document must satisfy.
5. Routing. Send clean results straight through and uncertain ones to a person.
6. Review and approval. A human corrects and signs off.
7. Export and audit. Deliver the approved data and keep a record of every change.

OCR engines and vision models

There are two broad approaches to reading a document, and many pipelines use both. Dedicated OCR engines run locally, cost nothing per page and keep documents on your own infrastructure, but they return text and positions and leave understanding the layout to you. Vision-language models read the page as an image and can return fields directly, handling unusual layouts better, at a per-page cost and with the document sent to a provider. A common design runs a local engine first and falls back to a vision model when confidence is low. Test candidates on your own documents, because published accuracy numbers rarely match a particular supplier's invoice style.

Extract against a schema

Define the fields before you extract: names, types, which are required, and the formats you expect for dates, currencies and tax identifiers. Ask the model for output that conforms to that schema, and reject output that does not parse. A schema turns a vague request to read this invoice into a precise contract, and it makes later steps simple because the data always has the same shape. Let the schema vary by document type or by customer where needed, instead of forcing one shape on everything.

Validation rules catch what models miss

Rules are the cheapest and most reliable safeguard. Useful ones include:

• Line items add up to the subtotal, and subtotal plus tax equals the total.
• Dates are valid, and the due date is not before the issue date.
• Currency codes and tax identifiers match the expected format for the country.
• Required fields are present.
• The same supplier, number and amount have not been processed before.
• The supplier matches a known record, or is flagged as new.

Record which rule failed, not just that something failed, because that is exactly what the reviewer needs to see.

Confidence and routing

Do not rely only on a number the model reports about itself. Combine signals: rule failures, missing required fields, disagreement between two extraction engines on the same field, and the model's own score as one input among several. Choose a threshold for automatic approval on real data, start conservative, and track two figures together: the share that passes straight through and the error rate among those. Raising the first is only a success if the second stays acceptable. Sample some automatically approved documents for a human audit, so you notice drift.

Design the review queue for speed and trust

The reviewer's screen is where the system earns confidence. Show the original document beside the extracted fields, highlight the fields that failed a rule, let the reviewer correct values quickly and move on with the keyboard, and show why each document was sent for review. Provide clear actions to approve, reject or return a document to the queue. Once approved, lock the record, and keep a history of every edit with who made it and when. Each correction is also data: tracking which fields reviewers fix most tells you where to improve extraction or rules.

Measure it properly

• Field-level accuracy on a labeled set of your own documents, reported per field, because an average hides a field that is often wrong.
• Straight-through rate, the share of documents approved without a person.
• Correction rate per field during review.
• Time per document, before and after.

Privacy and compliance

Invoices contain names, addresses, bank details and tax numbers. Decide how long documents and extracted data are kept, encrypt them at rest, limit who can open them, and understand each provider's data handling terms before sending documents to it. For regulated customers, the location of processing may matter as much as the accuracy.

Roll it out in steps

Start with one document type and one supplier group. Run the system in shadow mode first, producing results next to the manual process so you can compare without risk. Widen the scope as the numbers justify it, and keep the human review in place for anything the rules do not clearly clear.

Starting from a codebase

If you would rather start with the pipeline built, the AI agent and document AI category lists BoilerShelf's Document Workflow Delivery Kit. Whatever you choose, test it on your own documents and measure the figures above before you rely on it.