Why access control cannot be left until later

In corporate RAG, retrieval finds passages and a language model forms an answer from them. If the system first retrieves documents from a shared index and only then asks the model “not to reveal anything secret,” the data has already been passed to the model. The dependable boundary is to filter the passages the user may access before generation.

This matters especially in a small business, where one shared repository can quickly combine instructions, contracts, HR documents, and correspondence. The convenience of one search box must not turn into access to everything.

Three steps for safer retrieval

1. **Identify the user in the application.** Authenticate them through the organization’s normal mechanism. Obtain immutable user and group identifiers from the token or server session. Do not let the model choose whose identity to search under.
2. **Propagate document permissions to every chunk.** During indexing, retain the source document ID, version, owner, permitted groups, sensitivity level, and source link. All chunks from a document need a consistent set of metadata; otherwise a split document can expose access through a chunk that lacks its ACL.
3. **Apply the filter to every search request.** Pass only server-derived access identifiers to the search index and use them to constrain candidates before any context is sent to the model. If the ACL is missing or stale, or the user’s identity cannot be verified, returning no results is safer than searching without restrictions.

For Azure AI Search, Microsoft documents a security-filter pattern: documents carry a filterable field with user or group identifiers, and the query applies a `search.in` filter. Those identifier strings do not authenticate the user; your application must verify identity and permissions. Setting a field to `retrievable=false` is not a substitute for filtering either: it only keeps that field out of the ordinary search response.

A practical pilot design

Start with one limited corpus, such as support instructions or sales-team procedures. Store these fields for each record:

  • `document_id` and `chunk_id`;
  • `allowed_principals` — user or group IDs from a trusted source;
  • `classification` and, if needed, `tenant_id`;
  • `source_uri`, version, and last synchronization time.

For a query, the application validates the user’s session, obtains their groups, builds the filter, and only then calls search. Only permitted matching passages enter the prompt. The answer includes source links so the employee can verify the document version and origin. Before returning it, the system checks that every cited passage came from the search results authorized for that session.

The model should not receive search-index credentials or construct ACL filters itself. A first pilot needs only read-only search access and one server component that makes access decisions. If agent actions are added later—creating a ticket, sending email, or updating a CRM—authorize each operation separately.

Common failure modes

  • **Permissions are copied once and forgotten.** After an employee leaves or changes roles, the source system may revoke access while the index retains an old group list. Permission-change synchronization and revocation tests are essential.
  • **A filter is not applied on every path.** Chat, similar-question search, fallback routes, or admin diagnostics can bypass shared middleware. Verify the server gateway through which all search requests pass.
  • **The cache ignores the user.** An answer generated for a manager must not be served from a shared cache to an employee. Include authorization context in the cache key or disable caching for sensitive answers.
  • **Chunk metadata drifts.** Updating a document ACL must update or invalidate all its chunks, embeddings, caches, and derived indexes.
  • **A citation points somewhere the user cannot access.** A citation in an answer does not guarantee authorization to open the file; the source system must check the user independently.

OWASP also recommends inheriting access controls on every vector or chunk, isolating context, and testing stale permissions and cache leakage. RAG reduces answers without supporting context, but it does not make content safe automatically: a document can contain errors, sensitive data, or malicious instructions.

Test before launch

Create a small test set of users: a regular employee, a manager, and a user whose access has been revoked. Give them documents at different access levels. Test not only expected answers, but also the absence of forbidden passages from retrieval traces, logs, and caches.

Minimum negative tests:

  • A user without permission asks about salaries or a restricted contract—the restricted passage never enters the context.
  • A user’s group membership is revoked—old access no longer works after synchronization.
  • Two employees ask the same question—their answers and cached results do not cross authorization boundaries.
  • Search fails or ACL data is unavailable—the system fails closed instead of switching to an unrestricted index.
  • An allowed document contains instructions aimed at the model—they are treated as data, not commands.

For a business owner, this is first a project about identity quality and permission synchronization, and only then a vector-database choice. Pilot one corpus and measure both false grants and false denials before connecting contracts, personal data, or financial files.