Not every question needs a graph

A conventional RAG system retrieves relevant document passages and gives them to a language model. For “What delivery date does this contract specify?”, that is often enough. A different task is to determine which suppliers are linked to delayed batches, which contractual exceptions applied at the time, and where revised terms were approved in correspondence. Evidence is spread across contracts, emails, and operational logs. Text similarity does not always reveal the chain of relationships.

GraphRAG is one way to extract entities and relationships from a corpus in advance, organize them into a graph, and prepare summaries of graph communities. At query time, the system can search connected nodes or summaries as well as similar passages. It is not “better RAG by default”: the extra data layer costs compute, maintenance, and quality assurance.

Microsoft GraphRAG documents four query modes. Basic Search resembles conventional passage retrieval; Local Search explores the neighborhood of particular entities; Global Search uses community reports for questions about the corpus as a whole; and DRIFT starts locally while adding broader context. These modes are not interchangeable. Choose the mode for the question type, not for the fashionable label.

Where a business benefit is plausible

The approach is relevant when employees repeatedly connect facts from multiple sources:

  • procurement: the relationship between a supplier, an item, a quality incident, and contract terms;
  • field service: a path from equipment model and component revision to repair history and bulletins;
  • compliance: the connection between a policy, an exception, and supporting records;
  • feedback analysis: recurring themes across many requests rather than one exact match.

These are examples of suitable tasks, not reported deployment outcomes. If most questions ask for one field in one document, first improve hybrid search and links to source passages. A graph can add cost and errors without a meaningful gain.

An independent study of graph-based RAG methods in the Proceedings of the VLDB Endowment reports mixed results across question types: graph methods do not beat ordinary RAG in every test. The authors also compare time and token use, and simpler approaches are often cheaper. Buying a “graph architecture” without a set of your own evaluation questions is therefore a poor way to test its value.

Building the system boundary

The minimum design starts with a governed document corpus, not a graph database. Documents need versions, effective dates, owners, and access rights. The pipeline then splits documents into passages, extracts entities and relationships, preserves links back to the source passages, builds communities and reports, and retrieves evidence for a draft answer. A person remains responsible for decisions where errors are costly.

For a pilot, keep originals, extraction results, the graph, the retrieval index, and quality logs separate. Microsoft GraphRAG writes indexing results to Parquet artifacts and allows a vector-store integration. That does not mean a small firm must deploy another permanent server immediately: storage choices depend on existing infrastructure and data volume.

Local deployment is possible, but it is not a single-switch promise. Project documentation allows other models through LiteLLM and proxies, including Ollama, while warning about malformed JSON. An extraction model must reliably return the required structured output. Test it separately on Russian names, abbreviations, item numbers, and ambiguous organizations. If it confuses entities, the later stages can propagate those mistakes with confidence.

The cost comes before the first answer

With ordinary RAG, initial expenses center on document preparation and embeddings. Standard GraphRAG adds model calls to extract and describe entities and relationships, followed by community reports. Microsoft warns that indexing can be expensive; its documentation estimates graph extraction at roughly 75% of standard indexing cost. Faster FastGraphRAG replaces some work with NLP methods but may produce a noisier graph.

Here is a model calculation, not a quoted price or a customer result. Suppose a pilot has 10,000 text units and one call per unit uses 1,200 input tokens plus 300 output tokens. This stage alone consumes 15 million tokens. Community reports, retries, prompt tuning, embeddings, and reindexing are excluded. With a local model, the API bill becomes GPU time, electricity, queueing, and engineering effort. Measure cost per accepted answer for the relevant question class, not cost per attractive graph.

There is an operational cost too. A changed contract may make dependent relationships and community reports stale. Update, deletion, and reconciliation rules are therefore necessary. A graph does not make the corpus current by itself.

Permissions and verifiable evidence

Extracted relationships may be more sensitive than individual documents because a graph reveals partners, roles, and approval chains. Enforce access rights before graph context reaches the model, and check them again when opening a source document. A system prompt is not an access-control mechanism. Across companies or departments, test negative cases: can a user see a relationship built from a document they cannot read? Does it persist after access is revoked or a document is deleted?

An answer should provide not only a coherent story but references to specific document versions. Evaluate false edges separately: if the extractor merges two people with the same name or similar item numbers, a persuasive answer is more dangerous than “not found.” Keep a human in the loop for legally or financially consequential decisions.

A pilot without a large commitment

Take a limited document set for one process and assemble evaluation questions in advance: simple facts, multi-hop relationships, and themes across the corpus. For each group, record acceptable evidence and when “insufficient information” is the right answer. Compare baseline hybrid RAG with GraphRAG on the same corpus and, where possible, the same model. Measure the share of evidence-backed answers, false relationships, 95th-percentile response time, indexing hours, and expert review effort.

Move forward only if gains on genuinely multi-hop questions justify the extra expense without weakening access control. If not, keep ordinary RAG and improve document and retrieval quality. Microsoft describes its repository as a research project in maintenance mode, not a supported enterprise product; that is a separate operations-planning risk.

The cover shows a diagram from the official Microsoft GraphRAG documentation: © Microsoft Corporation, MIT License; gently center-cropped to a square without adding content.