Why a demo almost always looks better than the system

Enterprise RAG is easy to demonstrate with five convenient questions. The developer knows the phrasing, the required documents are current, and the knowledge base contains no duplicates or contradictions. The model sounds confident and opens a citation, so the solution appears ready.

A production corpus contains old policy revisions, similar product names, tables, scans, conversations, documents from the wrong department and questions that have no answer at all. One aggregate “accuracy” score hides the cause of failure. The system may fail to retrieve the evidence, rank it too low, retrieve it correctly but distort it during generation, or answer confidently when it should refuse.

RAG therefore needs at least three independent examinations:

1. did retrieval find the required evidence;
2. is the answer supported only by that evidence;
3. does the system stop when evidence is absent or contradictory?

This separation follows the structure of RAGAS, which exposes distinct metrics for context quality, noise sensitivity, response relevance and faithfulness to retrieved context. One number is convenient on a slide but unhelpful when the pipeline needs repair.

Test 1: retrieval and ranking

Start with the generator disabled. For each question, the evaluation set should contain identifiers of documents or passages that a domain expert considers sufficient. Test only the retriever and reranker output.

The minimum measurement set includes:

  • **context recall** — whether all required evidence appeared;
  • **context precision** — whether irrelevant passages crowded the top of the ranking;
  • rank of the first useful document;
  • share of requests that obey department, product, date and access filters;
  • robustness to typos, abbreviations and conversational wording.

If the required policy never enters top-k, changing the generator prompt cannot fix the problem. The team needs to inspect document segmentation, metadata, embeddings, hybrid retrieval, the reranker or index freshness.

Versions deserve a dedicated test. Given two policy revisions, retrieval should rank the active version and either filter the old one or label it as archived. The criterion is stricter for restricted documents: a passage unavailable to the user must never enter model context. That is an access incident, not merely a quality defect.

Test 2: generation and grounding

Once retrieval is stable, enable the generator. This layer measures the connection between meaningful claims and context rather than writing style.

Ask four separate questions:

  • does the response address the user's request;
  • is every factual claim supported by retrieved passages;
  • do citations point to passages that actually support the conclusion;
  • did the answer omit a material limit, deadline, exception or unit?

Faithfulness is not universal factual truth. It measures whether the answer follows the provided context. If the corpus contains an obsolete document, a model can be perfectly faithful to a bad source. Knowledge owners still need freshness dates, named document owners and a process for retiring old revisions.

An LLM judge accelerates hundreds of evaluations but is not a neutral instrument. Pin its version, prompt and temperature. Have humans review a regular sample, and store disputed outcomes with their rationale. A judge can run inside a local perimeter, but using the same model for generation and judging increases the chance of shared blind spots.

Test 3: correct refusal

Questions without answers are the most neglected part of an evaluation set. EnterpriseRAG-Bench models roughly 500,000 corporate documents across nine source types and 500 questions, including multi-document retrieval, conflicts and recognizing absent information. That resembles a working knowledge base more closely than a clean FAQ collection.

Include at least four negative classes in your own set:

  • the required information exists in no permitted source;
  • similar information exists, but for another product or period;
  • two active documents conflict;
  • the answer exists, but the user cannot access its source.

Correct behavior differs by class. With missing evidence, explain what is absent and offer an escalation route. With conflicting evidence, surface both versions and ask the process owner to select the source of truth. With restricted evidence, do not reveal even the hidden passage's contents. A generic “I don't know” is safe but not very useful; an invented answer is convenient only until someone acts on it.

Refusal needs its own metrics: correct-refusal rate, false refusal on answerable questions and dangerous answers without sufficient support. The last figure is usually more important than an average style score.

Building a golden set without a research laboratory

A first pilot can start with 60–120 questions if they represent real decisions. Use anonymized support conversations, employee searches and onboarding questions. Do not let the RAG developer define the gold standard alone: the process owner must confirm the correct source and acceptable response.

Cover these scenarios:

  • a simple fact in one document;
  • synthesis across multiple documents;
  • a table, appendix or scan;
  • current and obsolete revisions;
  • synonyms, abbreviations and typos;
  • role or department restrictions;
  • conflicting sources;
  • no available answer.

For every row, store the question, acceptable phrasing variants, required facts, prohibited claims, gold document identifiers, expected refusal behavior and business-risk level. Avoid copying personal data unless it is essential; anonymized or synthetic variants should preserve task structure instead.

Do not turn the set into an exam the team gradually memorizes. Keep a hidden holdout and add new production failures monthly. NIST AI RMF calls for repeatable testing, evaluation, verification and validation before deployment and during operation, with domain experts and documented measurement conditions.

A minimal measurement architecture

Every request should leave a trace across component boundaries:

`question → access filters → retrieved passages → reranker → model context → answer → citations → refusal decision`.

The evaluation record should include index version, embedding model, top-k settings, reranker version, generator model and prompt, latency, passages used and test outcomes. Without this data, the team can observe quality decline but cannot tie it to a change.

Two loops are required:

1. **Offline regression.** Run the golden set whenever documents, parsing, chunking, models, prompts or retrieval parameters change. Compare the candidate with the current baseline for every question class.
2. **Production control.** Send a small anonymized sample of real conversations to domain experts on a schedule. Track new topics, false refusals, stale documents and cases missing from the test set.

The Russian market is moving in the same direction. In April, CNews described a “Neirosupport” update that gives administrators a sandbox for testing typical responses before changes reach customer conversations. The claim that internal testing can reach 80% or more comes from the vendor and cannot be transferred to another process without its own data and criteria. The important principle is that knowledge-base changes are validated outside the live channel first.

Launch thresholds depend on risk

There is no universal “85% is enough.” A navigation hint and a payment instruction carry different error costs. Define risk classes first, then assign thresholds and responses.

A read-only reference assistant may escalate uncertainty to a human. A customer-facing draft needs checks for mandatory caveats and tone. Legal, HR or financial policies require a verified citation, a current revision and specialist review in disputed cases. Exposure of a restricted passage should have zero tolerance regardless of average quality.

Evaluation economics also matters. Automated judges consume API tokens or local GPU time, while expert labeling consumes staff time. Use a cascade: cheap deterministic checks across the full set, LLM judges for semantic metrics, and human review for high-risk or disputed examples. Compare that cost with incorrect decisions, rework and lost trust, not with an imaginary zero-cost alternative.

A ten-working-day plan

  • Days 1–2: choose one process and collect 80 questions across eight scenario types.
  • Days 3–4: a domain expert labels sources, required facts and correct refusals.
  • Day 5: freeze the current configuration and measure the retrieval baseline.
  • Days 6–7: add generation metrics and manually review at least 20% of examples.
  • Day 8: change one component only, such as chunking or reranking, and compare it with baseline.
  • Day 9: run the hidden holdout and access-control tests.
  • Day 10: make the pilot decision, document limits and schedule regular regression runs.

The first management artifact is not a chatbot or a model shortlist. It is a table of questions, sources, risks and acceptance criteria. If the team cannot show whether retrieval, generation or refusal failed, RAG cannot yet be improved safely; every model replacement will be judged by impression.