Why valid JSON can still damage the ledger

When a local language model is connected to invoices, requests, or acceptance certificates, the first technical win looks convincing: instead of free text, the model returns tidy JSON. Fields are present, types match, and the application can parse the object. The team concludes that the result is ready to be written directly into the CRM or ERP.

This is where the dangerous part begins. Constrained decoding can prevent an extra comma or an unknown property, but it cannot stop the model from placing the wrong tax identifier in an allowed field, confusing a delivery date with an invoice date, or producing a correct total from incorrect line items. The format becomes more reliable; the meaning does not necessarily follow.

A sound architecture separates four properties:

  • syntactic validity: the response is valid JSON;
  • schema conformance: required fields, types, and allowed values are present;
  • factual correctness: every value is supported by the source document;
  • action eligibility: the record satisfies business rules and may be posted.

Only the last property authorises a change to a production system. The first three are checks on the way to that decision.

What vLLM and llama.cpp provide

Local inference servers can restrict the next-token space. vLLM supports structured outputs based on JSON Schema, choices, regular expressions, and grammars. llama.cpp supports GBNF and can convert part of JSON Schema into a grammar. This is stronger than asking in a prompt to “reply with strict JSON”: tokens that violate the grammar are excluded during generation instead of being repaired afterwards.

Support is not identical across implementations. The llama.cpp documentation explicitly describes a subset of JSON Schema and lists limitations. Certain combinations of `anyOf` or `oneOf`, nested references, string formats, and other features may be unsupported. Some unsupported features may even be skipped silently. A schema that works with one server should not be assumed portable to another without tests.

JSON Schema itself has versions, called dialects. The official documentation recommends declaring `$schema` at the root so that validators and readers know the intended semantics. An implementation may ignore an unknown custom keyword: the instance can pass validation even though the schema author expected an extra constraint.

The project lesson is simple: version the schema together with the model and inference server, and verify actual feature support with positive and negative examples.

Seven stages between a document and the ERP

A reliable workflow is easier to reason about as a pipeline in which the model has no direct production permissions.

1. **Document intake.** The file receives an immutable identifier, checksum, arrival time, and source. The original is stored separately from recognition outputs.
2. **Text extraction.** OCR or a parser returns text with page coordinates. Poor quality, unknown formats, and protected files enter quarantine.
3. **Structured generation.** The local LLM receives a short schema, clear field definitions, and the extracted text. The server constrains the response with the chosen grammar.
4. **Independent validation.** A standard library validates the JSON again against the pinned schema version. The inference server's internal check does not replace this step.
5. **Semantic validation.** Deterministic code recalculates totals and tax, validates dates and reference data, and checks for duplicates and invalid field combinations.
6. **Decision routing.** Safe, fully supported records enter an auto-accept queue; ambiguous records go to a person with the source evidence highlighted.
7. **Idempotent execution.** A separate executor writes to the CRM or ERP using a stable key and records the outcome. A retry cannot create a second operation.

The model remains a transformer rather than the holder of accounting permissions. Even a successful generation cannot initiate a payment, post a document, or alter a customer record on its own.

Designing a schema that can be verified

The first schema should be short. One object covering every document type creates optional branches, expands the prompt, and makes failures harder to locate. A better sequence is to classify the document type with a fixed `enum`, then apply a separate schema for an invoice, certificate, or order.

For every field, define:

  • its type and whether `null` is permitted;
  • the normalised representation;
  • whether it is required for the particular document type;
  • a reference to the supporting page and text span;
  • a reason when the value is absent;
  • the version of the normalisation rule.

Do not ask the model to invent “97% confidence.” Without calibration, that number adds little to the decision. Ask for evidence instead: page number, coordinates, or a short source span. Then have code verify that the span actually exists in the pinned document version.

A refusal path also belongs in the contract. The model should be able to return `needs_review` with a reason code such as blurred scan, multiple candidates, inconsistent totals, or unknown counterparty. If the schema permits only success, constrained decoding will force a syntactically allowed answer even when evidence is insufficient.

Checks that should not be delegated to the model

Business rules belong in ordinary code. For an invoice, these may include:

  • line-item totals plus tax must reconcile with the stated total within a defined tolerance;
  • the currency must come from an approved reference list;
  • dates must fall within an allowed period;
  • the supplier must resolve uniquely through a tax identifier or another stable key;
  • newly observed bank details require a separate approval;
  • document number, supplier, and date must not duplicate an accepted record;
  • totals above a defined threshold always require human review.

These rules are reproducible, testable, and explainable to an auditor. An LLM may propose a candidate, but it should not perform calculations that ten deterministic lines of code can do exactly.

Keep the raw and normalised values separate. Store “1 234,50” beside `1234.50`, the original date beside its ISO representation, and the written supplier name beside the matched reference identifier. A later rule change can then be applied without guessing what the document originally contained.

Why the schema needs a second validator

JSONSchemaBench collected 10,000 real-world schemas and evaluates constrained-decoding engines along three axes: efficiency, coverage of schema features, and output quality. This is a useful warning that “supports JSON Schema” says nothing about completeness, overhead, or the quality of populated values.

An independent validation step after generation solves several problems. It establishes one dialect for all models, catches implementation differences, refuses unknown keywords, and emits precise error codes. When the inference server changes, this layer remains the application's stable contract.

Before upgrading vLLM, llama.cpp, the constrained-decoding backend, or the model, run the same corpus again. It should contain ordinary documents as well as empty fields, long line-item arrays, mixed languages, comma decimals, suppliers with similar names, poor scans, and deliberately inconsistent totals.

Idempotency matters more than a polished demo

A document can be processed twice because of queue retries, crash recovery, or a user uploading it again. The executor should therefore derive its key from stable inputs rather than the model output: company identifier, source-file hash, operation type, and destination version.

Before writing, it checks the operation ledger. If the key already completed, the previous result is returned. If an earlier attempt stopped after calling the ERP but before replying to the application, the executor reconciles by external identifier instead of repeating the call blindly.

Metrics worth showing to management

The valid-JSON rate is a technical metric, not a business outcome. A pilot needs at least four independent groups:

  • **shape:** responses that pass the schema without regeneration;
  • **meaning:** per-field accuracy on a labelled set and the share of documents with every required field correct;
  • **decision:** false auto-acceptance, unnecessary manual review, and missed duplicates;
  • **operation:** cost per accepted document, p95 processing time, retries, and discrepancies with the ERP.

The most dangerous metric is the rate of incorrectly auto-accepted documents. It must not be hidden inside average field accuracy: one incorrect bank account matters more than ten correctly extracted optional comments.

Calculate economics on accepted outcomes. A modelled cost per document equals server depreciation, energy, operations, OCR, inference, and review cost divided by the number of records that pass all checks and require no later correction. Regeneration, quarantine, and incident investigation belong in the numerator.

A two-week pilot

You can begin without write access. Select 200–500 historical documents of one type, remove exact duplicates, and have two employees label critical fields. Pin the schema, model, prompt, inference server, and normalisation rules.

During week one, run in shadow mode and compare outputs with documents already posted. During week two, give operators a confirmation interface while keeping the ERP read-only. Consider automatic posting only after a separate test set demonstrates an acceptable false-auto-acceptance rate and the team verifies recovery from retries.

A successful pilot is not one in which the local model always places braces correctly. Success means incorrect meaning stops before the production system, the operator can see evidence for every value, and a retry cannot create a second record. JSON Schema makes output predictable. The rest of the pipeline creates trust.