Why a polished demo does not answer the key question
An employee asks a chatbot a sensible question, gets a coherent answer, and sees links to documents. The demo looks successful—but the manager still does not know whether the system will reduce search time or select the current instruction instead of a similar file from an old folder.
A RAG system (retrieval-augmented generation) depends on at least two components. First, the system searches corporate materials for passages; then a language model forms an answer from the retrieved context. If search does not find the relevant rule, the model cannot reliably “guess” it. Even when the right text is found, the model may omit a condition, combine two versions incorrectly, or cite the wrong passage.
That is why a pilot should not be judged by one question—“Did you like the answer?”—but separately: what search found, what the model wrote, which sources are shown, how the system behaves when documents are incomplete, and what one useful answer costs.
Start with one process and its documents
Do not begin with “upload the whole company to AI.” Choose a repeated scenario with a clear owner and a limited set of materials. Examples include employee questions about returns, warranty rules, delivery, discount approval, or common product configuration.
Set the boundaries:
- who asks questions and what decisions they may make based on an answer;
- where the documents live and who owns the current version of each;
- which materials the user’s role may see and which must remain restricted;
- what counts as a correct result: an exact quote, a short explanation, a procedure link, or escalation to a specialist;
- which errors are serious enough to stop an automatic answer.
If conflicting rules exist in different places, appoint an owner and resolve the versions first. An index speeds up document search; it does not decide which of two internal policies the company considers current.
Build a question set before tuning the model
Start with real, anonymized requests from email, chats, tickets, or employee questions. Remove personal data, prices, and customer details when they are not needed for the test. A useful starting set is 30–50 questions: enough to expose common failures quickly, but not enough to claim production readiness.
Create a reference card for each question:
- the expected answer or facts that must be included;
- the document and passage that support those facts;
- acceptable alternative wording;
- the condition under which the system should clarify, abstain, or hand the question to an employee;
- the risk level of an error.
Deliberately include different cases: a direct question, colloquial wording, synonyms, a typo, a question with multiple conditions, a query spanning old and new policy versions, an unanswerable question, and an irrelevant request. Testing only questions phrased like document headings usually overestimates quality.
Keep the original test set unchanged as a control. After changing text splitting or search parameters, compare the new version on the same questions. Add new failure examples separately instead of replacing old ones; otherwise, an improvement on one special case may hide a regression elsewhere.
Test retrieval and generation separately
For each test question, save not only the final answer but also its trace: original query, rewritten query if any, retrieved passages, their order and scores, document versions, answer, citations, latency, and token use.
Then review two distinct kinds of quality.
**Retrieval quality.** Did a supporting passage appear in the first 3–5 results? Did a similar but outdated instruction displace the right rule? Were all documents found when an answer depends on multiple conditions? Did the results include a source the user is not authorized to access?
**Answer quality.** Does the response rely only on retrieved evidence? Does it preserve exceptions, deadlines, and numbers? Does each citation actually support the adjacent claim? If the documents do not contain an answer, does the system acknowledge the gap or confidently invent a continuation?
This separation helps identify the right fix. If the relevant document never appeared in results, prompt tuning will not solve the main problem: first review source quality, access filters, chunking, retrieval, or ranking. If the context is good but the answer is incomplete, review model instructions, output length, answer structure, and how citations are linked to claims.
Microsoft’s RAG evaluation documentation also separates retrieval-stage evaluation from final-answer evaluation: search results can be compared with labeled relevant documents, while the response can be checked independently for grounding, relevance, and completeness. This describes specific Microsoft tools; it does not require a Russian business to choose that platform. RAGChecker, published in the NeurIPS 2024 proceedings, likewise diagnoses retrieval and generation components separately. The practical takeaway is simpler than the terminology: one aggregate score does not explain where a workflow failed.
Use simple metrics, but verify them with people
A small pilot does not need a complex scoring system on day one. A table for each run can track:
- **source hit:** whether the supporting passage appears among the first 3 or 5 results;
- **retrieval coverage:** whether all sources needed for an answer were found;
- **grounding:** whether each checkable claim is supported by retrieved material;
- **answer correctness and completeness:** whether conditions, exceptions, and expected facts are preserved;
- **abstention or escalation:** whether the system declines to answer when evidence is insufficient;
- **operational measures:** latency, model and search cost, repeated questions, and corrections.
Do not look only at averages. Count critical-question failures separately and categorize them: outdated version, omitted exception, incorrect access, invented number, wrong citation, or an overlong answer. One dangerous procedural error can matter more to a business than ten successful reference answers.
An automated language-model evaluator can speed up first-pass labeling, but it is not an independent judge. It can also fail and sometimes reward a fluent answer over a correct one. At the start, the process owner or a trained expert should review every answer in the test set. Later, the team can sample results and compare automatic scores with human judgments.
Set a release threshold before launch
Write down what result is sufficient in advance. For example, for a low-risk internal reference tool, a team might require that at least 9 of 10 control queries retrieve a supporting source near the top, and that every critical question be reviewed by a person with no material error. This is an illustrative management threshold, not a universal industry standard; choose it according to the cost of error and the workload.
You can begin with three outcomes:
- **green:** an answer includes supporting sources and the system is available to a limited user group;
- **amber:** only part of the material was found or confidence is insufficient—the system shows excerpts and routes the question to an employee;
- **red:** there is no reliable source, conflicting versions were found, or access failed—the system withholds the answer and notifies the data owner.
For HR, financial, medical, legal, and other sensitive decisions, an internal threshold alone is not enough. The workflow needs domain review, access controls, logging, and human approval; in some cases RAG should prepare a brief rather than make a decision.
A pilot architecture without unnecessary agency
A first release often needs only a direct flow: authenticate the user, search authorized documents, rank several passages, generate an answer with sources, validate the output format, and log the run. An agent capable of taking actions will not fix a poor catalog or contradictory policies. Add one only when the scenario truly requires sequential actions—for example, ask a clarifying question, check inventory in a system, and create a draft request.
Minimum technical requirements:
- keep the source, version date, owner, and access rights with every passage;
- enforce access filters during retrieval, not after the answer is generated;
- do not send the whole archive to the model when a few relevant passages are sufficient;
- show the document name and exact passage, not a decorative link without evidence;
- treat document contents as untrusted input: an instruction inside a PDF must not change system rules or an agent’s permissions;
- provide for updating and deleting indexed records, including personal data;
- keep the control set separate from prompts and test answers.
A local model may be appropriate when data, autonomy, or predictable-cost requirements justify it and the company is ready to operate hardware, updates, and observability. A cloud model may simplify a short pilot if data transfer is permitted and the provider passes review. RAG itself does not determine where documents are processed; that is a separate environment and contract decision.
Calculate the cost of a useful answer, not a token
Include document preparation, duplicate removal, test labeling, search tuning, model use, indexing, access controls, logs, quality review, maintenance, and employee time spent correcting results. To compare options, divide monthly costs by answers that pass your quality criteria, not by every system response.
At the pilot stage, record the current process baseline: minutes an employee spends finding and checking an answer, incoming request volume, and how often a question is escalated to another specialist. Measure the same indicators after the test. Time saved is not a direct cost reduction unless the freed capacity is put to useful work; the cost of errors, delays, and extra review can outweigh search savings.
A two-week plan
1. Select one process and agree with its owner on the cost of an error.
2. Gather and clean documents; check versions and access rights.
3. Prepare a control set of questions with sources and expected answers.
4. Run a simple baseline configuration and retain retrieval traces.
5. Categorize failures: data, retrieval, permissions, generation, or interface.
6. Change one material parameter at a time and rerun the same control set.
7. Compare quality, latency, cost, and employee time with the original process.
8. Decide: limited rollout, another improvement cycle, or no RAG for this process.
What a manager should take away
Ask the team to show not only answers but also the retrieved documents, failures, costs, and abstentions on the same control set. If a vendor or internal team cannot explain why an answer is considered correct, the demo is not yet evidence of readiness.
