The problem sits between systems, not inside the model

Consider an ordinary scenario: a manager moves from sales to procurement. Access to a folder containing customers' commercial terms is removed in the file store. Yet an internal assistant has already indexed the documents, and some answers remain in a cache. If the RAG application still searches old chunks through a shared service account, changing the role in the source system alone does not guarantee that the manager stops receiving the information.

This is a test scenario, not a report of an incident at a real company. Running a model locally reduces reliance on sending data outside the organization, but does not replace internal access control. OWASP explicitly identifies weaknesses in vector stores and retrieval as RAG risks and recommends least-privilege access for sensitive information. The useful question is therefore: how long after a right is revoked will the same user stop seeing the document through every access path?

Where copies and traces remain

A typical design contains a chain rather than one “knowledge base.” A connector reads a source document. The system splits it into chunks, creates vectors, and stores text or a text reference with metadata. Search returns relevant chunks, the application assembles context for the model, and the answer may enter chat history, logs, a cache, or an export. Removing permission from the original folder changes only one part of that chain.

Each document should have a stable identifier, version, source, owner, and access status. Each chunk should point back to its document and current visibility scope. If different sections of one file have different audiences, a coarse “available to the department” tag is insufficient: the access boundary must match the retrieval unit. It is better not to index sections whose rights the application cannot represent and verify.

Qdrant, for example, can restrict search using metadata filters. That is a useful mechanism, but a filter is not an authorization service by itself. The application must obtain a verified user identity and current entitlements from a trusted system, then construct the filter on the server. The client must not be allowed to choose its own permitted groups, and a model instruction must not be treated as a permission check.

A minimal revocation architecture

The existing document system or identity directory remains the source of truth for access rights; it need not be duplicated just for RAG. A role change, departure, or project closure should emit an event or be discovered promptly by the connector. The RAG service maps that event to document identifiers and updates chunk metadata; if the source is deleted, it removes the corresponding index points. At the same time it invalidates answer caches and references to previously assembled context.

Every user request needs two barriers:

  • before retrieval, a server-generated filter for organization, project, role, and other relevant attributes derived from current rights;
  • after retrieval, a second check of the returned document identifiers before text goes to the model and before a source link is shown.

The second barrier matters when index synchronization is delayed: a chunk may still exist but must no longer enter an answer. If the rights service is unavailable, the safe behavior for sensitive collections is not to “show the last known result” but to withhold the document until the check recovers. That trades availability for protection and should be agreed before a pilot.

Do not confuse revoking access with physically deleting every trace. Chat history, exports, audit logs, and backups follow different retention rules. Each needs its own retention period, access roles, and deletion or restriction procedure. A message already delivered to a person cannot automatically be taken back; preventing new disclosure after revocation is therefore essential.

Measure a revocation delay instead of promising “instant”

A useful measure is the time from a permission change in the source system to the first guaranteed denial in RAG. Break it down into event delivery, index update, cache invalidation, and request-time checking. For especially sensitive documents, checking the live entitlement when reading provides stronger protection than waiting for background reindexing. It adds a network call and a dependency on the rights directory, however, so measure response latency under your own load.

A test set can start with a few dozen documents belonging to different owners and three roles: owner, another department's employee, and former project member. First confirm that authorized questions find the right sources. Then revoke a right and repeat the same question, a paraphrase, and direct opening of the source link. Test an old chat, a cached answer, and retrieval after a connector failure as well. The required result is absence of protected text, not merely absence of a link.

Track false denials too. An overbroad restriction makes the assistant useless and encourages staff to work around its rules. A pilot therefore needs two kinds of measures: unauthorized disclosures should be zero in the test set, while the authorized-answer rate and retrieval latency stay acceptable. Zero in a test does not prove absolute security, but it finds obvious gaps before wider rollout.

Cost and the next step

For a small business, the difficult work is usually not the GPU but the links among accounts, document versions, and caches. Do not buy another infrastructure layer just to draw a neat permissions diagram if the existing store and directory already provide the necessary identifiers and events. First estimate the connector, request-time authorization, logging, and recurring test costs. Compare them with manual document search and the cost of a mistaken disclosure; there is no universal payback percentage here.

Start with one collection and one process, such as answering questions about supplier contracts. Map the path: source entitlement → document → chunks → response → history and cache. Appoint an access owner, define an acceptable revocation delay, and run a role-change test before enabling a broad employee audience. If current authorization cannot be demonstrated at any step, that chunk should not be used in answers yet.

Image: Cbarille / Wikimedia Commons, CC BY-SA 4.0; square crop of an ACL administration console screenshot. It illustrates a permissions interface and is not an endorsement of a particular product.