Document AI for Archives: What the La Foneria Pilot Proves — and What It Does Not

What La Foneria actually tested

La Foneria is a Spanish microenterprise in the digital humanities sector. It helps museums, archives and documentation centres process and publish digitised collections. The European Digital Innovation Hubs case card lists 1–9 employees and €200,000 in turnover. One representative workflow requires people to find place names across 10,000 images, link them to a controlled vocabulary in Dédalo and prepare the resulting data for search and publication.

For historical-memory archives, the problem is harder than ordinary OCR. A page may contain names, places of origin or imprisonment, sentence types, institutions and events. Recognising a string is not enough: the mention has to be assigned to the right class, linked to a controlled record and stored with provenance. A one-letter OCR error can create a new person, while a correctly recognised name may still be linked to the wrong individual.

La Foneria worked with DIH4CAT and the Computer Vision Center's Document Analysis group on a “test before invest” stage. This distinction matters. The company was not buying a finished automation product; it was testing whether a specific document flow was technically manageable before making a larger investment.

Two routes, not one “smart model”

The published case describes two main routes.

• OCR → NLP/NER. An OCR engine transcribes the page, then a model identifies and classifies people, places, detention centres and sentence types. Dictionaries and domain vocabularies help normalise spelling variants and link mentions to canonical records.
• Word Spotting → selective transcription. The system searches the image for words or recurring fragments and transcribes only the detected regions. This can be useful for repeated formulas and severely degraded pages.

According to the case report, OCR followed by text processing produced more promising results than Word Spotting. Handwriting recognition performed better than the research team expected, while Topic Spotting showed potential for recurring phrases. Yet the case card provides no numerical accuracy. Its authors explicitly say that a larger project with labelled images and objective evaluation is needed.

That is the main business lesson: “the approach appears viable” and “the process is ready for unattended automation” are different statuses. The case card also describes a roadmap from TRL 2–3 to TRL 4 by December 2026. This is a target, not a confirmed production outcome. The projected doubling of revenue after new functionality is launched is likewise a company forecast, not a measured pilot result.

How an SME can transfer the approach to its documents

The same logic applies beyond historical archives. A manufacturing, logistics or service company may hold thousands of scanned acceptance certificates, contracts, equipment passports, requests and legacy customer records. The valuable result is not “chat with a folder”; it is a structured record that can be verified and passed to an ERP, CRM, archive or search index.

A practical pipeline looks like this:

1. Preserve the original. Give every file a stable identifier, checksum, source, ingestion time and access policy.
2. Pre-process the page for rotation, perspective, noise and segmentation. Separate tables, stamps, handwriting and typed regions where useful.
3. Return OCR text together with block coordinates and confidence. Low recognition confidence must influence downstream decisions.
4. Extract entities into an approved schema: counterparty, contract number, date, amount, asset, department or another business master-data field.
5. Link spelling variants to canonical records. “Acme Ltd”, “ACME, Limited” and a former company name must not become three counterparties.
6. Ask a person to approve uncertain values. Thresholds should reflect the cost of error: a wrong document topic is usually less serious than a wrong amount, personal record or payment basis.
7. Send only the accepted version to the target system, together with page coordinates, model version, approval time and correction log.

Local deployment is often justified. These documents may contain trade secrets, personal data and records with long retention periods. OCR, entity extraction and master data can remain inside the company network, while only anonymised metrics leave the perimeter. But local does not mean automatically safe: permissions, backups, logs and evaluation datasets still require governance.

Build the ground truth before choosing the model

Research on historical NER explains why a generic demo says little about a particular archive. Performance changes with language, period, typeface, OCR noise, document genre and entity set. A pilot therefore starts with a representative sample, not the largest available model.

The ground-truth set should include:

• different document types and periods;
• good, average and poor scans;
• typed and handwritten text;
• rare but costly errors;
• real abbreviations, former names and ambiguous people;
• pages where the target entity is absent, so false positives can be measured.

Annotators need one written guide. For each field, define boundaries, permitted formats, uncertainty rules and conflict resolution. Otherwise, the model is compared not with truth but with several employees' incompatible opinions.

A few hundred pages can be enough for an initial pilot if the sample is stratified by document type and quality. Expand it when confidence intervals remain too wide or rare errors are missing. Freeze the test split before tuning thresholds; otherwise the team quietly trains on the exam.

Metrics must mirror the production operation

A single “90% accuracy” number hides almost everything that matters. Measure each stage and the final business operation.

• OCR: character error rate and word error rate, split by document type.
• Extraction: precision, recall and F1 for every field rather than only a global average.
• Linking: share of mentions mapped to the correct master-data record.
• Process: share accepted without edits, average review time, throughput, queue length and reprocessing.
• Risk: false acceptance rate — errors that passed the threshold and reached the system without correction.
• Economics: cost per accepted record, including preparation, infrastructure, annotation and reviewer time.

A published medieval-charter corpus illustrates how wide the range can be even in a research setting: entity-level precision varied from roughly 72.8% to 94.0%, and recall from 58.1% to 81.8%, depending on the entity class. These figures are not forecasts for La Foneria or for Russian business documents. They simply show why an unsegmented average is dangerous.

A modelled business case for 10,000 pages

The following is not a La Foneria result. It is an illustrative management calculation.

Assumptions:

• manual processing takes three minutes per page;
• after the pilot, 60% of pages are accepted after a 45-second review;
• the remaining 40% require two minutes of review and correction;
• schema design, sampling, integration and testing require 120 person-hours;
• the company supplies its own fully loaded labour rate.

The manual process takes 500 hours. Reviewing the automated flow takes about 208 hours: 75 hours for 60% of the pages and 133 hours for the remainder. Gross savings are approximately 292 hours. After the initial 120 hours of preparation, the first batch produces about 172 hours of net gain; a comparable second batch produces about 292 hours if document quality and schema remain stable.

The budget ceiling is straightforward:

`acceptable pilot cost = hours saved × fully loaded hourly rate − infrastructure − error reserve`.

If a costly mistake triggers legal review or renewed customer contact, its expected cost must be subtracted separately. A cheap pipeline with many false acceptances can therefore be worse than a slower system with mandatory approval.

A six-week pilot plan

1. Week 1: select one document flow and no more than 5–8 fields; record a manual baseline for time and errors.
2. Week 2: create a stratified sample, annotation guide and frozen test set.
3. Week 3: compare at least two routes — full OCR+NER and a narrower field or template detector.
4. Week 4: add master data, confidence thresholds and a review screen that highlights the original region.
5. Week 5: run in shadow mode; the system proposes values but does not write them automatically.
6. Week 6: calculate cost per accepted record, false acceptances and correction time; decide whether to scale.

Approve production gates in advance. A reasonable set may include zero false acceptances for critical fields in the test, a required recall level for mandatory attributes, measurable time reduction and the ability to delete or reprocess a record through its stable identifier.

The management takeaway

The La Foneria case is valuable not because of a projected future turnover, but because of the discipline shown at an early stage. The team decomposed the work into OCR, extraction and linking, compared routes and stopped at an honest conclusion: the technology is promising, but automation requires labelled ground truth and numerical validation.

For your own pilot, choose one document type, a few expensive fields and 200–500 representative pages. Measure the manual process first, then run a shadow pipeline with human approval. If you still cannot state the cost per accepted record and the cost of a false acceptance, the pilot is not finished — no matter how convincing the demo looks.