What the authors studied
The EASI-RAG paper describes a staged method for implementing a RAG tool in an industrial SME. The case was a French environmental testing laboratory where an assistant answered operators’ questions using operational procedures. A team with no prior RAG experience brought the first version into use in less than a month and then improved it using user feedback.
The Telegram note about the case reported nine source files in Word and Excel and an initial test of 50 questions, with 14 answers judged incorrect. The authors later reported zero errors in a repeated test, but the answer categories in the paper add up to 51 despite the stated total of 50. After deployment, users found two further incorrect answers. The result should therefore not be read as evidence of perfect accuracy.
The method is more than retrieval
EASI-RAG divides the work into five stages:
1. design the data pipeline, document segmentation, vectorisation and retrieval;
2. define metrics and an initial test in advance;
3. analyse errors and correct bottlenecks;
4. integrate the assistant into the operational process;
5. collect feedback and improve iteratively.
The authors define four roles: end user, data expert, process owner and developer. This division matters in an SME. A developer can configure retrieval, but only the procedure owner can confirm which instruction is current, and operators can show real questions and the consequences of a wrong answer.
What to prepare
For nine documents, start by cataloguing files and owners. Check versions, tables, effective dates, duplicates and access permissions. Then collect ordinary work questions and “trap” questions for which the documentation has no answer. For each question, mark the correct source and acceptable answer.
Test text and table extraction first. Exact lexical search can be useful for technical terms and equipment codes, combined with vector search. Check that chunking does not split instructions into unsafe fragments. Show a link to the source section in results; when there is no evidence, the system should abstain and route the question to a person.
How to evaluate the pilot
Measure extraction coverage, source correctness, factual answer accuracy, abstention rate and how often an operator still has to search manually. Re-run the same questions after every index or prompt change. Add a separate test set for a new process rather than relying on the original 50 examples.
The laboratory’s payback was not measured. The transferable lesson is not a promise of “RAG in three weeks,” but the method: a narrow knowledge scope, named owners, pre-launch tests and a required feedback loop.
