“It is local” is not a threat model

A local RAG system is often presented as a tidy cabinet: the documents are inside the office, the model is inside too, so security must already be covered. In practice, the cabinet may sit in the server room while its virtual key opens every drawer. If retrieval ignores user permissions, an employee can ask an innocent question and receive a fragment from somebody else’s contract, payroll sheet or project correspondence.

Local deployment reduces some risks associated with sending data to an external provider, but it does not replace authentication, authorization, auditing or least privilege. NIST’s Zero Trust Architecture explicitly rejects implicit trust based only on network location. “Inside our perimeter” is an address, not an access pass.

The good news is that access-aware RAG does not require teaching a model to say “show me your badge.” Permissions should be enforced by ordinary software before documents enter the model context. The language model answers questions; it should not serve as the security guard. It has too much imagination and no official badge.

Where the extra door usually opens

A common failure looks harmless. A company puts documents from several departments or customers into one vector database, generates embeddings and connects a chat interface. The user signs in through the corporate identity provider, but the application then sends the question to a shared retrieval endpoint. Retrieval returns the most similar chunks regardless of ownership, and the model faithfully summarises them.

There are less obvious variants:

  • authorization is checked after retrieval, although forbidden text has already reached logs, traces or the model context;
  • the policy is written in the system prompt, for example “do not show finance documents to sales staff”;
  • access is attached to a document but lost when the document is split into chunks;
  • an employee leaves a group while the index or cache keeps the old permissions;
  • responses are cached only by question text, so the next user receives the director’s result;
  • one RAG service account has administrator rights to every source even though the user needs one folder;
  • an agent can send files as well as read them, without human approval for the privileged action.

OWASP warns that RAG and fine-tuning do not eliminate prompt injection. Instructions embedded in an ingested document can try to change model behaviour or induce access to a private source. OWASP recommends enforcing privilege controls in code, giving the application only the access it needs, and requiring human approval for high-risk actions. A system prompt is a reminder, not a lock.

Reference architecture: access before similarity

A protected request follows a deliberate chain.

1. The user authenticates through the corporate identity provider. The application receives a stable identity, organization, roles and groups rather than trusting fields sent by the browser.
2. A policy layer calculates the allowed scope: tenant, department, project, document class and access validity period.
3. Vector or hybrid retrieval receives that scope as a mandatory server-side filter. Unauthorized objects are removed first; semantic similarity is calculated among the remaining objects.
4. Only authorized chunks enter the model context, accompanied by document and version identifiers.
5. The answer includes links to permitted sources. The server checks authorization again when a source is opened.
6. The audit trail records the user, policy version, retrieved chunk identifiers and authorization result without blindly copying full secret text.

Order matters. Retrieving the top ten chunks and discarding forbidden ones afterwards may remove relevant authorized results. More importantly, secret content may already have entered telemetry. Pre-filtering makes the boundary explicit and reduces the leakage surface.

Metadata every chunk needs

An index should store more than a filename alongside each vector. A practical minimum normally includes:

  • organization or customer identifier;
  • source document and version identifiers;
  • owner and permitted groups, or a reference to a central policy;
  • confidentiality class;
  • access start and expiry dates;
  • deletion or revocation state;
  • source and time of the last permission synchronization.

Every chunk must inherit those fields. Otherwise the folder is locked while its loose pages somehow wander around the corridor.

Qdrant’s multitenancy documentation describes several isolation patterns: filtering by a tenant payload field, dedicated shards and a tiered approach. Payload filtering can be efficient for many small tenants; dedicated shards trade resources for stronger isolation for a smaller number of larger tenants. The right design depends on volume, isolation requirements and failure assumptions, but the tenant filter must be generated by the server rather than accepted from a user request as a suggestion.

Permissions outlive the polished demo

The first demonstration is usually perfect: three documents, two users and one confident chatbot. Production adds department transfers, temporary project groups, departures, classification changes and deleted files. The authorization index is therefore a synchronized system, not a one-time configuration.

The team must define the source of truth: file storage, ECM, CRM, group directory or a dedicated policy engine. A revocation event should invalidate caches quickly, remove chunks from retrieval and prevent old links from opening. Give propagation a measurable objective, such as no more than five minutes for critical classes, instead of the word “promptly.”

Cache keys must include the tenant, user or group set, policy version and index version. A global cache keyed only by question wording is a highly efficient way to create a highly efficient incident.

Personal data and accountability in Russia

When the knowledge base contains personal data, Article 19 of Russia’s Federal Law No. 152-FZ requires legal, organizational and technical measures against unauthorized or accidental access, alteration, copying, disclosure and other unlawful actions. It also covers threat identification, effectiveness assessment before system launch, unauthorized-access detection and incident response.

The exact control set depends on the data category, information system and applicable regulation. The design should be reviewed by information security specialists and the organization’s privacy owner. Unfortunately, the word “local” does not complete any table in a control-effectiveness assessment.

For a business, the minimum practical work includes source inventory, a documented role matrix, least-privileged service identities, channel and storage protection under the accepted threat model, logging, backups and a revocation procedure. If RAG serves several legal entities or customers, tenant boundaries belong in the architecture, not in a filter added before the presentation.

Testing before launch

The usual happy-path questions, such as “find the leave policy,” are not enough. The acceptance plan needs a negative-test matrix.

  • A sales employee asks for an exact phrase from a restricted finance document.
  • Two customers upload similarly worded documents with identical filenames.
  • An employee is removed from a group and the old question is repeated before and after the promised synchronization deadline.
  • A director and an intern ask the same question, followed by a cache check.
  • An authorized document contains an instruction telling the model to query a restricted source.
  • A user changes the tenant or group field in an API request.
  • An old citation link is opened after access has been revoked.
  • Logs and traces are searched for prohibited fragments and personal data.

The acceptance rule is straightforward: a forbidden document must enter neither retrieval results, model context, cache nor full-content logs. A denial should be recorded without revealing that a secret file exists. High-risk agent actions—sending a document, changing a record or starting a process—need separate authorization and, where appropriate, human approval.

Economics without a gold-plated turnstile

Start with one knowledge base and two contrasting roles. The pilot needs corporate authentication, a mandatory retrieval filter, a restricted-query test set, and an audit of caches and logs. For many small customers, evaluate payload filtering and tenant indexes. For a few large customers with strict requirements, evaluate dedicated shards or physical separation.

The primary metric is not the percentage of attractive answers but the absence of cross-scope retrieval while authorized recall remains acceptable. Also measure authorization latency, revocation propagation time, false denials and the share of answers with verifiable permitted citations.

Executive takeaway

Ask the team to run the demonstration backwards: prove not that RAG knows the documents, but that it does not know somebody else’s documents. Create two roles, two identically named files and one shared question; then change permissions and repeat the test through the cache and an old citation link.

When access is checked before retrieval, inherited by every chunk, updated on revocation and verified with negative tests, local RAG becomes a useful corporate tool. When the whole policy is the sentence “model, please do not tell anyone,” Vnutrik is already carrying a stack of folders out of the archive. Fortunately, a human is still guarding the gate.