Why “upload the documents once” is not enough

A RAG system is only as useful as the freshness of its search index. When an employee retires a policy, changes a price, or revokes access to an instruction, old chunks must stop participating in retrieval. Otherwise, the model can confidently cite a document that no longer exists in the operational system.

The failure happens before answer generation, between the source of truth and the index. A nightly import or a manual “reindex” button leaves a window in which the business already follows new rules while the assistant still answers from old ones. A full rebuild narrows that window, but becomes slower and more expensive as the corpus grows.

For changing structured sources, it is usually more practical to send RAG a stream of events rather than another table snapshot: a record was created, updated, or deleted. This pattern is commonly called change data capture, or CDC. PostgreSQL can expose changes through logical decoding; Debezium turns them into events, and a worker updates only the affected documents and chunks.

What the primary sources establish

PostgreSQL describes logical decoding as transforming write-ahead log records into a stream that an external consumer can understand. A replication slot retains the read position and lets processing continue after downtime. The documentation also makes an important warning: after a crash, a client may receive already-seen changes again and must handle the replay safely.

Debezium’s PostgreSQL connector produces row-level events for insert, update, and delete operations. For a deletion, it can also emit a tombstone: a record with the same key and a null value. This is not a meaningless empty event. It is an explicit instruction that the object must no longer exist in a derived view.

Qdrant stores a point as an identifier, a vector, and optional payload. Mutations are first recorded in its own write-ahead log; an upsert with the same identifier updates the point instead of creating an independent copy. Points can be deleted by identifier or filter. These properties support an idempotent worker without rebuilding the whole collection.

The independent DBLog research formalizes the problem of reconciling an initial snapshot with a live change log. The practical RAG lesson is straightforward: the initial load and subsequent stream need a consistent boundary, or some updates will be missed or applied twice.

A minimum architecture for a small business

There is no requirement to begin with a large Kafka cluster. For one database and a moderate document flow, six components are enough:

  • PostgreSQL or another operational system remains the single source of truth;
  • a CDC connector reads the log and emits a key, operation, and position;
  • a small durable queue separates the source from slower file and embedding work;
  • a worker assembles the current document and splits it into chunks;
  • an embedding service runs locally or in an approved external environment;
  • a vector store accepts upsert and delete while a state table records progress.

If Kafka already exists, Debezium fits naturally. If it does not, Debezium Server or a dedicated logical-replication consumer may be sufficient. The mandatory properties remain the same: durable position tracking, replay, a dead-letter or retry path, and the ability to apply an event again.

A stable identifier matters more than a chunk number

A common mistake is to assign sequential chunk numbers: document 42, chunks 1 through 12. Editing a paragraph near the beginning shifts every following number, so the system either leaves old points behind or rewrites nearly the entire document.

A more reliable identifier combines a stable source key, representation version, and fingerprint of normalized content. For example:

  • `source_id` is the record primary key or an immutable file identifier;
  • `representation` names the view, such as product card, instruction, or appendix;
  • `content_hash` is SHA-256 of normalized chunk text;
  • `tenant_id` and the access label belong in payload, without putting secrets in the ID.

For an update, the worker first computes the new target set of identifiers. It upserts the new points, then deletes previous points for that `source_id` that are absent from the new version. This set reconciliation is replay-safe: applying the same event twice converges to the same state.

Deletion needs a first-class path

A soft-delete flag in the source is insufficient if retrieval still returns old points. A tombstone or delete event should follow a short, explicit path:

1. identify the source and tenant from the event key;
2. block new answers for the object before any vector recomputation;
3. delete every point for the matching `source_id` and access scope;
4. clear derived full-text and cached representations;
5. record acknowledgement together with the log position.

For legally sensitive data, application logs should not contain the deleted document body. The identifier, operation type, log position, and technical outcome are enough. Audit data should prove that deletion ran without becoming another uncontrolled copy of the content.

Ordering operations without a distributed transaction

PostgreSQL and a vector database rarely share one transaction. Simulating “exactly once” across separate systems makes the project more complex and still cannot eliminate network failures. A better approach is at-least-once delivery with idempotent effects.

A safe sequence is:

  • receive the event and record its deduplication key;
  • read the current source state unless the event is a delete;
  • build chunks and embeddings;
  • apply upserts and remove points that no longer belong;
  • store a result checksum and only then acknowledge the event.

If the worker fails after upsert but before acknowledgement, the event arrives again. Stable point IDs prevent duplicates. If deletion is temporarily unavailable, the event stays in the retry queue while the object can be immediately excluded with `active=false` or a deny list enforced by the retrieval gateway.

Why reconciliation is still necessary

CDC reduces latency but does not remove the need for periodic checks. A replication slot may lag, a worker may remain in an endless retry loop, or a schema change may break document assembly. A daily or weekly low-cost reconciliation should compare:

  • active object counts in the source and index;
  • sampled hashes of current text and indexed versions;
  • age of the latest successfully applied event;
  • retry-queue size and maximum attempt count;
  • points that remain for deleted `source_id` values;
  • the share of queries retrieving material older than the freshness SLA.

A full rebuild remains an emergency tool and a way to change the embedding model. It should not be the routine delivery mechanism for every edit.

Data, permissions, and infrastructure

The CDC account needs the minimum privileges required to read the log and only the tables that supply the knowledge base. Connection secrets belong outside connector configuration. Network rules should allow the source to feed the connector and the worker to reach the index, without granting the vector database direct access to production PostgreSQL.

Do not copy every row field into an event “for future use.” Send the technical key and minimum routing data; the worker can assemble the full document through a controlled read API. This makes masking, tenant authorization, and excluded-field policies easier to enforce.

An access change must be processed as strictly as a text edit. The new payload must reach the index, old points with the previous visibility scope must disappear, and the retrieval gateway must always add filters derived from the authenticated user.

Limits and economics

CDC is unnecessary for a folder containing five PDFs that change once per quarter. A manifest of content hashes plus a daily sync is simpler and often safer. A change stream becomes justified when the source changes throughout the day, stale answers have material consequences, or full rebuilds already interfere with operations.

Economics should be calculated around the full process, not the connector license. Include engineering time, queueing and observability, embeddings for changed chunks, version storage, and the expected cost of a stale answer. Incremental processing usually reduces GPU work and rebuild time, while adding operational checks.

A pilot can start with a model calculation and explicit assumptions: documents changed per day, average chunks per document, embedding time, acceptable update latency, and human verification cost. When less than one percent of the corpus changes, recomputing only affected objects will often look more rational than a full nightly rebuild, but the decision should be validated with measurements from the company’s own corpus.

A two-week pilot

Start with one changing process rather than the entire repository: a price list, a policy collection, or service records.

  • Define a freshness SLA, for example that a change must disappear from answers within five minutes.
  • Select 30–50 documents and prepare create, update, delete, and permission-change scenarios.
  • Take a consistent initial snapshot and start the stream from the retained position.
  • Apply several events twice and confirm that point count does not grow.
  • Delete a document and test retrieval, cache, citations, and audit records.
  • Stop the worker, accumulate changes, restart it, and verify recovery.
  • Add metrics for lag, retries, and orphaned points.

The readiness criterion is not an impressive model response. It is a reproducible index state: after a replay, crash, or deletion, the index converges to the source of truth. Vnutrik may remember a lot; the business needs him to forget retired instructions on time.