Where the expensive part of the process lived

Affinda develops intelligent document-processing systems that extract fields from invoices, forms and other unstructured files before sending them into accounting or another business workflow. In an AWS customer story, the company describes its earlier approach: new document types and schemas required fine-tuned open models, labelled data and engineering work.

The problem was not only the cost of one inference. Every new use case carried a setup bill: agree the fields, collect examples, annotate data, configure a model, verify quality and bring an engineer back whenever the form changed.

Affinda moved part of the pipeline to large language models through Amazon Bedrock. The source names Claude Sonnet 3.5 V2, Claude 3.7 Sonnet and Claude Sonnet 4. The new system uses asynchronous inference, retains a hybrid of traditional and generative models, and runs within the company's existing cloud environment using EKS, EC2, SageMaker and CloudFormation. The updated platform began rolling out to customers in late 2024.

The process change matters most. Instead of training a separate extractor for each schema, the team defines the response structure and shows the model a small number of characteristic examples in context. Corrections can be expressed in natural language, allowing some configuration work to move from an ML engineer to an implementation specialist or process owner.

The outcome is a company claim, not a universal benchmark

Affinda reports a 90% reduction in setup time for new extraction use cases and a 90% cost saving for its product-delivery team. The company says that configuration which previously took weeks or months can, in some cases, be completed by users in minutes.

AWS published these numbers as a customer story and attributes them to the participant. The page does not disclose baseline hours, document volume, field-type distribution, test-set composition, accuracy before and after the change, or an independent audit. Another company should not place the 90% figure directly into its budget as an expected saving.

The narrower conclusion is useful: generative extraction can reduce the launch cost of a new template when manual annotation and engineering for a separate model dominate the bill. Whether it works for a particular company's invoices, completion certificates or contracts can only be established on that company's documents.

A reproducible architecture without cloud lock-in

Affinda's principle does not require copying its entire AWS stack. It can be evaluated with a protected API or a local model, provided that the model handles the relevant language, tables and document format well enough.

A practical pipeline separates the following stages:

1. Receive the file with malware scanning, size limits and an operation identifier.
2. Extract text and structure: OCR for scans, block coordinates, page numbers and tables.
3. Classify the document and select a schema version.
4. Use deterministic extraction where a rule is reliable: barcode, tax identifier, a number matched by a regular expression, or a total from a known field.
5. Give an LLM or VLM the document, JSON Schema and a few verified examples of difficult cases.
6. Run syntactic and business validation on the result.
7. Send uncertain records to a human queue; write accepted changes to the target system idempotently.
8. Save operator corrections as new test examples, but do not change production configuration without review.

Ollama supports structured responses constrained by JSON Schema and recommends validation with a typed model. llama.cpp can constrain generation with grammars and convert a supported subset of JSON Schema into GBNF. This reduces malformed JSON; it does not prove that a value is true. A correctly formatted amount can still come from the wrong row.

A schema is a contract, not a hint

Before selecting a model, define every output attribute:

  • exact name and type;
  • whether it is required and whether `null` is allowed;
  • date, currency and decimal-separator formats;
  • allowed categories;
  • normalisation rule;
  • page and evidence-fragment reference;
  • reason for routing the field to review.

For a financial document, `total: 125000` is not sufficient. Store the currency, VAT, original representation, coordinates or evidence quote, and schema version. Ordinary code then checks that line items match the total, the period is valid, the counterparty exists, the number is not a duplicate and values match reference data.

A closed schema that rejects additional properties helps expose unexpected output. Versioning protects integrations: an old form continues under its existing contract while the new schema completes testing.

How to choose the few examples

Few-shot configuration does not mean placing the first three documents in the prompt. Use idiomatic examples that represent genuine ambiguity:

  • one ordinary document;
  • one file with a missing optional field;
  • one difficult layout or table;
  • one correction of a common mistake;
  • one explicit case where the system must abstain from automatic extraction.

Examples become configuration data and need the same owners, versions and review as conventional rules. Do not automatically add every user correction: one operator mistake or unusual document can damage the entire class.

A long demonstration set also consumes context and compute. Information-extraction research indicates that few-shot effects vary by field and task, while added examples increase input volume. Select the number experimentally rather than maximising it.

What should remain deterministic

The hybrid architecture in the case is important. A generative model does not have to replace everything. Stable barcodes, checksums, reference lists, arithmetic relationships and known coordinates are cheaper and more dependable in ordinary code or specialised OCR.

An LLM is useful for variable names, unseen layouts, split context and explanations of why a value belongs to a field. The final decision to write into an ERP system should remain with validators and process rules.

A fine-tuned model can still be appropriate for very high, stable volume; strict latency targets; durable labelled data; or when a base model's quality is insufficient. Few-shot methods mainly optimise the speed of introducing new schemas.

What changes in a local deployment

For Russian companies, Affinda's particular cloud stack is not a ready-made implementation recipe. The same contract can be presented to a local model through Ollama, llama.cpp or another compatible inference server.

A local deployment reduces the need to transfer documents outside a managed environment and simplifies storage control. In exchange, the company owns GPU or CPU capacity, updates, availability, logging, redundancy and licence review for the chosen model.

A minimum data separation is:

  • originals remain in the existing protected repository;
  • the model receives only required pages or images;
  • prompts, examples and schemas are versioned without personal data where possible;
  • the audit log stores the document hash, model and schema versions, validator results and human decision;
  • the test environment cannot write to production systems.

Prompt injection also matters: a document may contain text that attempts to override the instruction. Treat file content as data, not a command. The model should not receive arbitrary tools, and every output must pass the strict schema and business rules.

What to measure in a pilot

For a first test, select one document type and 100–300 sanitised files, including rare and poor scans. This is a modelled pilot range, not a number taken from the Affinda story.

Split the collection into configuration and held-out test sets. Compare three paths:

  • current manual processing;
  • an existing template, OCR pipeline or fine-tuned model;
  • an LLM with a schema and a small number of examples.

Measure field-level quality, the share of documents needing no manual intervention, erroneous automatic-write rate, operator time per exception, p95 latency and cost per accepted document. Test unseen layouts, tables, handwritten corrections, missing fields, multiple currencies and repeat upload of the same file.

The critical metric is straight-through processing under an agreed error ceiling. A system that safely completes 60% of documents and routes the rest to a person may be more valuable than a model with a high average score and rare expensive errors.

An economic model

First record today's cost of launching a new use case:

`annotation + ML/integration hours + testing + corrections + maintenance`.

After the pilot, compare it with schema-and-example configuration:

`field analysis + example preparation + inference + exception review + regression tests`.

Annual benefit depends on the number of new schemas, document volume and reduction in manual work. Subtract infrastructure, licences, observability, quality control and the cost of incorrect writes. If a new document type appears once every few years, a 90% setup reduction may matter less than per-inference cost. With dozens of new forms per quarter, the balance reverses.

The management takeaway

The Affinda case shows where generative AI can change the economics: not necessarily in the price of processing one document, but in the time required to configure the next schema. The part worth reproducing is not the cloud brand but the controlled pipeline—response structure, a few reference examples, deterministic checks and a human exception queue.

The practical next step is to select one expensive-to-configure document type, create a held-out test set, and compare the current method with a local or protected LLM in one week. Decide on scaling from cost per accepted document and erroneous automatic writes, not the 90% headline from someone else's case.