Why a List of Numbers Can Still Be Personal Data

An embedding is an array of numbers that encodes the meaning of text for semantic search. In a local RAG system, those vectors usually sit next to a chunk identifier, operational metadata and sometimes the original text fragment. Their numerical appearance makes them easy to mistake for anonymized data. That is a dangerous shortcut.

An embedding is derived from source text and deliberately preserves enough information for semantically similar fragments to be close together. The paper Text Embeddings Reveal (Almost) As Much As Text showed that a purpose-built Vec2Text model could reconstruct original short texts from their embeddings under the studied conditions. The authors reported exact recovery for 92% of 32-token inputs in one evaluated setting and demonstrated recovery of full names from clinical notes.

Those figures should not be generalized mechanically to every model, text length or deployment. A later independent study reproduced important Vec2Text results under conditions close to the ideal setup while also showing sensitivity to input length and other assumptions. The business conclusion is narrower but more useful: the absence of readable words does not prove anonymity. Treat a vector as a derivative of the source data and protect it according to the source sensitivity.

Where the Practical Risk Appears in Local RAG

A local deployment reduces disclosure to an external model provider, but it does not remove internal trust boundaries. In most environments, an ordinary configuration mistake is more likely to expose data than a sophisticated inversion attack.

  • The vector database is reachable from unnecessary network segments or listens on a public interface.
  • Authentication is not enabled. Qdrant documentation explicitly warns that a self-hosted open-source instance is insecure by default.
  • One shared API key is used by the application, developers and batch jobs, so compromise gives an attacker excessive reach.
  • A tenant or department filter is supplied by the client and can be omitted or changed.
  • The returned payload contains the original chunk, filename, personal name, contract number or internal path.
  • Collection snapshots, exports and backups are protected less carefully than the live database.
  • User queries, retrieved chunks or reindexing output are copied into logs.
  • An operator with technical access can export a complete collection even though the task requires only a few search results.

A Qdrant snapshot contains collection configuration, points and payloads, making it a copy of the sensitive retrieval layer. OWASP also highlights vector-system risks, including cross-user context leakage.

A Baseline Architecture for a Protected Index

Effective protection starts before a vector is computed. If unnecessary data enters a chunk, a later retrieval filter does not undo the fact that the data was stored.

1. During ingestion, classify and minimize the document. Remove fields that are not needed for retrieval, such as signatures, bank details, personal contacts and internal comments. For especially sensitive fields, keep a reference identifier and fetch the value only after authorization.
2. Give every chunk an owner, tenant, sensitivity class, source-document identifier, version and retention deadline. A server-side ingestion pipeline should assign these fields rather than the user browser.
3. Put the vector database in a private network segment and do not publish it to the internet. Protect service-to-service traffic with TLS, enable authentication and separate credentials by service and operation. The public application calls a retrieval gateway, never the database directly.
4. The gateway derives the user identity from a verified session and injects mandatory tenant and ACL filters. The model does not decide which permissions apply and cannot remove a filter with a textual instruction.
5. Return a limited number of authorized chunks. The reranker and generative model receive only the already filtered set. Enforce request and response size limits, and require a separate role for bulk reading.
6. Encrypt disks, snapshots and backup storage with platform controls. Separate backup access from the live database credential and test restoration in an isolated environment.
7. Audit the subject, tenant, collection, operation type, result count and export volume. Log raw queries and chunk contents only when justified, and give those logs their own retention period.

Qdrant supports read-only and granular keys, TLS and private-interface binding; Milvus provides authentication, TLS and roles. These controls must be enabled and tested with the application policy.

Why a tenant_id Filter Is Not Enough

A tenant_id field is not authorization. A client can alter it, ingestion can write a chunk without a tenant, and one broadly privileged key can bypass the application by calling the API directly.

A practical baseline uses three layers:

  • the network prevents user devices from reaching the database;
  • the service identity is limited to the required operations and collections;
  • the retrieval gateway injects a server-side ACL filter and verifies results before they reach the model.

For highly sensitive data, consider a separate collection or even a separate database project with its own credentials and backup policy. This costs more than a shared index, but it reduces incident radius and simplifies auditing. Base the decision on data classification, not developer convenience.

Deletion Must Cover the Entire Chain

Deleting the source PDF is not enough. Chunks, vectors, payloads, response caches, search logs and snapshots may remain. The system needs one deletion operation based on a stable document identifier.

A reliable workflow looks like this:

  • mark the document as revoked and exclude it from retrieval immediately;
  • let a background job delete related chunks, vectors and payloads;
  • invalidate caches by document version;
  • reconcile the number of removed objects against the source registry;
  • let backups expire and be destroyed according to the retention policy;
  • record completion without copying document contents into the audit log.

If policy requires urgent physical removal from every backup, the backup architecture must support that requirement in advance. Otherwise, a promise to delete on request will fail precisely where the most complete copy is stored.

Do Noise and Quantization Solve the Problem?

Adding noise, reducing vector precision and quantization may make some reconstruction methods harder. The independent Vec2Text evaluation mentions these approaches as possible mitigations, not as universal controls. They can also reduce retrieval quality, while a different attack may exploit other representation properties.

Noise is therefore not a replacement for authentication, isolation and retrieval controls. A team considering this defense should evaluate two outcomes on its own data: the reduction in success of a specified attack and the change in recall, precision and RAG answer quality. Without both measurements, the result is an expensive impression of safety.

A Model Cost View for a Small Deployment

Consider 100,000 documents with an average of six chunks each: roughly 600,000 vectors. Storage for that volume is rarely the largest cost. Data inventory, identity setup, credential separation, server-side ACLs, protected backups and deletion testing usually require more engineering effort.

One collection with mandatory server-side filters may be sufficient for an internal knowledge base where every employee has the same permissions. HR, legal or customer records justify evaluating isolated collections. Isolation creates more configuration and backup jobs, but it reduces the impact of a mistake and the scope of incident review.

This is a model scenario, not a quote. In a pilot, measure the cost of a protected workflow: roles, key rotation, restore testing, isolation audits and provable deletion.

What to Test Before Launch

A two-week pilot can cover one workflow without connecting the entire corporate repository.

  • Use 500 to 2,000 documents with known owners and access classes.
  • Attempt access without a key and with an expired key; the database must not return data.
  • Search as two tenants and deliberately replace the tenant_id value.
  • Insert a synthetic sensitive canary and confirm that it does not appear in logs or another tenant's results.
  • Delete one document and locate every derivative: chunk, vector, payload, cache entry and queued ingestion task.
  • Restore a backup in an isolated environment and inspect who actually gains access.
  • Alert on bulk reads, unusual response volume and requests from an unexpected service identity.

The readiness criterion is not an impressive model answer. It is reproducible isolation under errors and abuse. If the team cannot explain why a user saw those exact chunks and cannot demonstrate how a deleted document disappears, the retrieval layer is not ready for sensitive data.

Practical Takeaway

Local deployment closes an important external boundary, but a vector index remains a database of derivative data. It is not automatically anonymized merely because a person cannot read a vector by eye. Minimize data before indexing, close network access, enable TLS and authentication, separate credentials, enforce ACLs on the server, protect payloads and snapshots, monitor bulk extraction and design complete deletion.

The best first management step is a short negative test: access without a key, request another tenant's data and try to recover a deleted document from backup. Those three attempts reveal the maturity of a RAG deployment faster than another polished demonstration chat.