RAG Citations Without Decoration: Linking Answers to Exact Evidence

A Link Does Not Automatically Make an Answer Verifiable

A RAG system can place a neat “[1]” marker after a paragraph while pointing to a document that supports only a neighboring idea. A model may invent a filename, confuse contract versions or attach one citation to several unrelated claims. The answer then looks more reliable than it actually is.

The ALCE study proposed evaluating cited answers not only for fluency but also for citation quality. On one evaluated dataset, even the best tested systems lacked complete citation support roughly half of the time. A later paper, Correctness is not Faithfulness in RAG Attributions, separates two questions: whether a source supports a claim and whether the answer genuinely relied on that source rather than receiving a plausible citation after generation.

The practical business conclusion is that citation markers cannot be delegated entirely to the generative model. A reliable system preserves fragment provenance, allows the model to reference only retrieved identifiers, checks that evidence supports each claim and constructs the final link on the server.

What the User Should Receive

A verifiable citation should answer four questions without another search:

• which exact document was used;
• which document version applied when the answer was created;
• which page or section contains the basis;
• which fragment supports the specific claim.

A filename alone is insufficient when a PDF has hundreds of pages. A URL to the current version is risky when the answer used an older revision. A long quotation without an anchor forces the employee to repeat the retrieval system's work manually.

A practical interface shows a short answer, an evidence marker after every checkable claim and a side card with title, version, page, effective date and a small excerpt. Clicking opens the source at the relevant location. When precise location is unavailable, the system honestly shows its grounding level: document, section, page or character range.

Provenance Starts During Ingestion

Precise citations cannot be added at the end if indexing discarded page and structure information. The ingestion pipeline needs a durable provenance chain.

For every document, store:

• an internal document_id independent of the filename;
• a source-byte hash and version identifier;
• owner, effective date and publication state;
• links to previous and next revisions;
• source and access policy.

For every chunk, keep chunk_id, document_id, version_id, text, heading path, page numbers, character range and, when available, page coordinates. DoclingDocument, for example, represents provenance with page_no, a bounding box and charspan. The application can then open a PDF on the right page and highlight the region without asking the model where it found the text.

A chunk identifier should be reproducible when the same version is processed again. It can be derived from version_id, structural path and normalized text. A random identifier is convenient for insertion but makes index comparison, citation revocation and change investigation harder.

An Evidence-Aware Answer Pipeline

A production query can pass through six controlled stages.

1. The retrieval layer receives the question and user permissions, then returns authorized chunks with stable identifiers and provenance metadata.
2. A reranker narrows the set without dropping the source relationship. Each fragment receives a short temporary evidence_id such as E1 or E2.
3. The model sees only evidence text and temporary identifiers. In a structured response, every factual claim contains a list of evidence_ids. The model never writes a URL, file path or page number itself.
4. A validator checks the schema: every identifier exists, is authorized for this user and belongs to an active version. Unknown markers fail validation.
5. A verification module compares each claim with its fragment. Rules and a separate classifier can cover routine traffic, while disputed high-impact claims go to a person or become an abstention.
6. The server replaces E1 with a safe link from the document registry and adds title, version and page. The application, not the LLM, determines the final destination.

This design separates language generation from data routing. If the model invents E99, the validator cannot find it in the retrieved set. If a document is revoked, the server can hide the citation and require a fresh answer from current evidence.

Exact Quotations and Semantic Support Need Different Checks

For a verbatim claim, use a deterministic test: after whitespace normalization, the quoted string must occur in the stored chunk and within a known source range. This control is inexpensive and easy to explain.

Most business answers paraphrase their sources. There may be no exact match. A verifier must decide whether the claim follows from the fragment and whether it adds new numbers, conditions or causality. An automated classifier is useful at scale, but its judgment is not proof of truth either. Contract values, regulatory deadlines, payment details and legally significant decisions require a deterministic source or human confirmation.

Measure completeness separately: did every material claim receive support? One correct marker after a long paragraph can produce good citation precision while hiding unsupported conclusions. ALCE distinguishes answer quality from citation quality, and the same principle belongs in acceptance testing.

Versions, Revocation and Access Control

A citation must point to the version that participated in the answer. If policy forbids access to an old version, silently redirecting to the new one is misleading because the content may have changed. Mark the answer as stale and offer to recompute it instead.

The source registry tracks version state: active, superseded, revoked or deleted. A document change invalidates related caches. The interface shows when the answer was produced and whether its evidence is still current.

Check permissions twice: before retrieval and when the link is opened. A user may have had access during generation and lost it later. Do not place source text in the URL or create a permanent public link. The application issues a short-lived address after checking the current session again.

A Model Cost View of the Pilot

Assume a knowledge base has 20,000 documents and employees ask 1,000 questions per day. Adding provenance barely changes generation cost: it adds a few fields per chunk and a version registry. Most additional expense comes from claim verification and maintaining a reproducible event log.

Not every internal answer requires a second model call. Define risk tiers:

• navigation and routine reference — schema validation, link existence and exact-match checking for short quotations;
• policies and customer responses — automatic verification of every material claim;
• amounts, deadlines and actions in a system of record — deterministic rules, constrained fields and human approval.

These are model assumptions, not a price quote. Compare options using cost per accepted answer: generation, verification, abstention rate and expert minutes. A cheap system that repeatedly directs employees to the wrong revision creates hidden costs far above token spend.

Metrics That Can Be Audited

A pilot can use 100 to 200 real questions and two subject-matter reviewers. Label separate properties instead of asking whether they liked the answer.

• Citation validity: every marker exists and opens an authorized source.
• Citation correctness: the source supports the associated claim.
• Citation completeness: all material claims have evidence.
• Anchor accuracy: the link opens the correct page or range.
• Version correctness: the current version was used or the older one is explicit.
• Abstention quality: the system refuses when evidence is insufficient.

Ragas defines faithfulness as the share of response claims supported by the retrieved context. It is useful for automated regression testing but does not replace a human sample. The NIST Generative AI Profile also recommends reviewing sources and citations during pre-deployment measurement and ongoing monitoring.

Set thresholds according to process risk. A reference FAQ can tolerate graceful degradation; an unsupported number in a contract-deadline answer should trigger abstention rather than a confident sentence without evidence.

A Two-Week Next Step

During week one, select one policy collection, preserve version and provenance during indexing, and require the model to return JSON with claims and evidence_ids. The server rejects unknown identifiers and constructs links only from its registry.

During week two, collect 100 real questions. Reviewers score support, completeness, version correctness and anchor accuracy. Separately test document replacement, revoked access, deleted versions and a model attempt to reference a nonexistent fragment.

Success does not mean answering every question. It means the system either provides verifiable evidence or clearly abstains; an old version is not disguised as a new one; another user's source does not open; and an invalid marker cannot pass validation.

Practical Takeaway

Verifiable RAG is built around a provenance registry, not attractive answer formatting. Preserve version, page and coordinates during ingestion, give the model only temporary evidence identifiers, validate them on the server and build final links outside the LLM. Measure citation correctness and completeness separately.

The first management test takes an hour: ask the system to answer from a revoked version, provide an unsupported number and cite nonexistent E99. If the interface cannot distinguish those cases from a normal answer, citations are still decorative rather than verifiable.