Why changing the model name is not enough

An embedding model maps queries and document chunks into coordinates in one vector space. Replacing it changes more than dimensionality and speed: it changes the geometry of meaning. Even when old and new models return the same number of coordinates, their results cannot be mixed. Similarity is meaningful only inside a space produced by a compatible model and distance function.

Upgrading an embedding model in a production RAG system is therefore a data migration. The team must re-embed the entire corpus, capture documents created or modified during the rebuild, compare quality on real questions, switch search without downtime, and preserve a rollback path.

Qdrant's official documentation describes two zero-downtime approaches: parallel blue-green collections and an additional named vector inside an existing collection. The commands are Qdrant-specific, but the operating pattern applies to OpenSearch, Milvus, pgvector, and other stores: build the new version beside the old one, evaluate both on the same queries, and promote the new version only through a controlled cutover.

What to record before starting

A migration needs the original text. A vector is a derived artifact; it cannot reliably reconstruct the document for a new model. Before budgeting the work, confirm where source chunks live and whether their preprocessing can be reproduced deterministically.

For every chunk, store:

  • a stable `chunk_id` independent of the model;
  • document identifier and version;
  • checksum of normalized text;
  • chunking policy and its version;
  • embedding model, dimensions, and distance function;
  • embedding creation time;
  • access-control and tenant metadata;
  • deletion or current-version status.

Capture a baseline separately: point count, dimensions, distance metric, index parameters, quantization settings, p50/p95 latency, and results for gold queries. Qdrant migration guidance identifies a distance-metric mismatch as a common cause of post-migration quality regression. Without a baseline, the team cannot distinguish a better model from an accidental index-configuration change.

Option 1: blue-green collections

This is the most portable architecture. The old collection continues to serve search while a new collection is created with the dimensions and metric required by the new model.

The sequence is:

1. Create `rag_v2` without changing the production alias `rag_current`, which still points to `rag_v1`.
2. Enable dual writes: encode every new or modified chunk with both models and write it to both collections.
3. Run a background job that reads source documents, re-embeds existing chunks, and idempotently writes them into `rag_v2`.
4. Send a portion of queries to both versions without exposing the new answer to users.
5. Verify completeness, permissions, retrieval quality, latency, and cost.
6. Atomically switch the alias to `rag_v2`.
7. Retain `rag_v1` for an agreed rollback period, then delete it through a separate decision.

The alias matters because applications call a stable logical name. Both Qdrant and OpenSearch document atomic alias changes, so concurrent requests do not observe a transient state in which the name points to the wrong index or no index.

Blue-green migration temporarily duplicates payload and index storage. Simple dual upserts are also insufficient: deletes and partial updates must reach both versions. Qdrant explicitly warns that a migration covering only upserts can reintroduce a deleted document into the new collection during background backfill.

A safer design uses one operation log. Every event receives a monotonic version: `upsert`, `delete`, or permission change. The backfill records a log boundary, processes the main corpus, and then applies all later events. Before cutover, the team reconciles current chunk counts and samples tombstones.

Option 2: two named vectors in one collection

Qdrant 1.18 and later can add a new named vector to an existing collection. Payloads, identifiers, and permissions stay in place; new embeddings enter an additional field, and each query explicitly selects the old or new vector name.

This reduces duplicated document storage and makes deletion simpler: deleting a point removes both vectors. Rollback is also straightforward because search can return to the old name while the old vector remains.

The constraints remain:

  • the collection must support named vectors;
  • dual writes are required for every new or modified chunk;
  • existing points do not acquire new embeddings automatically, so backfill is still necessary;
  • the query service must switch the query model and vector name together;
  • the old vector must remain through the rollback window.

The query-model version and index version form one configuration. If an application encodes a query with the old model and sends it to the new index, or the reverse, the call may succeed technically while retrieval quality becomes unpredictable.

Evaluating quality before cutover

A public benchmark helps shortlist candidates but cannot replace testing on business documents. BEIR demonstrates that retrieval methods behave differently across domains and tasks. A production system needs its own set of at least 100–300 real questions for which experts have identified relevant documents or chunks.

Measure layers separately.

**Retrieval:** Recall@k indicates whether a required chunk entered the candidate set; nDCG@k considers its ranking. Also track queries with no useful result and top-k overlap between versions.

**Answer:** an expert checks source grounding, completeness, unsupported claims, and access compliance. Better Recall does not guarantee a better final answer: a new retriever may return more broadly similar but less precise context.

**Operations:** p95 latency, embedding queue depth, CPU/GPU use, index size, recovery time, and cost per accepted answer. A higher-quality model may increase vector dimensions, slow search, or require a more expensive reranker.

Shadow mode is useful: `v1` supplies the production answer while `v2` runs in parallel for measurement. A small employee cohort can then enter a controlled A/B test. Automatic promotion based on a single metric is unsafe; the process owner and IT jointly approve the cutover.

Modelled re-embedding economics

Consider a hypothetical corpus of 500,000 chunks averaging 600 tokens. Re-embedding requires 300 million input tokens. At an effective throughput of 15,000 tokens per second, pure computation takes about 5.6 hours. Calendar time is longer because of data reads, network latency, retries, indexing, and queue limits.

If the new vector has 1,024 float32 components, raw vectors alone occupy roughly 2.05 GB: `500,000 × 1,024 × 4 bytes`. This excludes graph indexes, payloads, internal metadata, replicas, and disk headroom. A blue-green migration holds old and new versions simultaneously, so capacity cannot be budgeted from final collection size alone.

Add the cost of:

  • re-encoding ongoing changes during dual writes;
  • storing two indexes and backups;
  • preparing gold questions and expert review;
  • observability, reconciliation, and retry handling;
  • retaining rollback capacity after cutover;
  • potentially changing the reranker or generative model.

These figures are modelled and do not describe a particular deployment. For a small business, expert time often costs more than GPU time because experts must identify correct sources, adjudicate disagreements, and confirm that a quality gain changes accepted decisions.

Security and local deployment

When an external embedding API is used, all reprocessed text leaves the local environment. Before migration, review the contract, processing region, request retention, and whether every document class may be transferred. A local embedding service can simplify the data flow for confidential knowledge bases, but it requires internal version control, performance management, and updates.

Dual writing must not bypass authorization filters. Access and tenant metadata travel with the document, and authorization is enforced before vector retrieval in both versions. Shadow queries are logged and follow the same retention limits as production traffic.

Cutover checklist

Switch the alias only when all conditions hold:

  • 100% of current chunks have the new embedding version;
  • deleted documents have not reappeared;
  • checksums and access metadata match;
  • the new retriever passed the gold set without critical regression;
  • p95 latency and cost fit the SLA and budget;
  • monitoring distinguishes model and index versions;
  • rollback was rehearsed in staging;
  • the old index is protected from accidental deletion during the rollback window.

The practical first step is not to start re-embedding but to reconstruct vector lineage. Take 100 documents, repeat chunking, and verify that they produce the same stable `chunk_id` values. Then collect 100 real queries and capture the current result set. If those two operations cannot be reproduced, an embedding migration will expose a data-governance problem before improving quality—and that is useful information in itself.