GraphRAG and friends: when entities and relationships beat similarity

Some questions aren't about what a document says — they're about how things are connected. Vector search can't answer those. A typed knowledge graph can, and the cost is the hard part.

Jul 14, 2026

The problem

Here is a question that breaks every RAG pipeline in this series so far: “which of our suppliers are connected to the legal entities named in this dispute?” The answer isn’t sitting in a chunk anywhere. It’s a path — supplier to parent company to subsidiary to the named party — assembled from facts that live in four different documents. Naive RAG retrieves the four documents independently and has nothing to join them. Advanced RAG reranks those four documents more precisely and still has nothing to join them. The retrieval got better; the question stayed unanswerable, because similarity search returns passages, and the answer is a traversal.

That’s the line that divides this article from the previous two. Vector search hands you the ten business cards that look most like your query. A graph hands you the org chart. When the question is “what does this document say,” the business cards are what you want. When the question is “how are these things connected,” you need the org chart — and building, storing, and querying that org chart is a different region of the knowledge layer, the part of an agent system that decides reliability in production and is an engineering discipline in its own right. This is where structure stops being optional.

How GraphRAG works

The spine: extract entities and relationships from the corpus into a graph, optionally summarize clusters of that graph ahead of time, and at query time either traverse the graph directly or retrieve over those summaries.

Microsoft’s GraphRAG — the paper that named the pattern, Edge and co-authors’ From Local to Global: A Graph RAG Approach to Query-Focused Summarization ([arXiv 2404.16130](https://arxiv.org/abs/2404.16130), April 2024) — does it in two passes. First, an LLM reads the corpus and extracts an entity-and-relationship graph. Then it partitions that graph into communities of closely-related entities using the Leiden algorithm, and pre-writes a summary of each community. A “global” question — “what are the recurring themes across all these incident reports?” — gets answered by map-reducing over the community summaries rather than over raw chunks, which is something flat RAG simply cannot do, because no single chunk contains the global answer.

The dominant production pattern looks slightly different from the research paper. Most teams store a property graph in Neo4j, Memgraph, or an embeddable store like Kùzu; chunks become document nodes linked to the entities they mention; and retrieval combines an LLM writing a Cypher query against the graph with ordinary vector search over the chunk embeddings. Neo4j’s neo4j-graphrag-python SDK and its Knowledge Graph Builder are the de-facto reference here, and LlamaIndex’s PropertyGraphIndex exposes the same idea framework-agnostically.

In practice

The example I’ll use is corporate ownership and supply relationships — the kind of corpus where the relationships are the point, not the prose.

Companies own stakes in other companies, supply each other, and share directors — and those links change as deals close. So the schema is genuinely typed: Company and Person nodes, OWNS edges carrying a stake percentage, SUPPLIES edges, DIRECTOR_OF edges, and a validity stamp on every edge so the graph knows which relationships held on a given date. The base vocabulary isn’t invented — it maps onto established models like the Financial Industry Business Ontology (FIBO) and legal-entity identifiers (LEI), which keeps the schema defensible rather than ad hoc.

Now the question “which suppliers does Beta Retail depend on, and which of those came under new ownership after 2024-01-01?” becomes a short traversal instead of an impossibility:

```cypher
MATCH (b:Company {name: 'Beta Retail'})-[:SUPPLIES*1..3]->(dep:Company)
MATCH (parent:Company)-[o:OWNS]->(dep)
WHERE o.since > date('2024-01-01')
RETURN dep.name, parent.name, o.since ORDER BY o.since
```

Flat RAG can’t express “depend on through the supply chain” — there’s no similarity score for a dependency path. The graph makes it a path query, and crucially it returns provenance: which company, which relationship, valid on which date. For a due-diligence or compliance answer, an uncited or out-of-date link is worse than no answer, and that’s exactly the failure a typed temporal graph is built to prevent.

Where it shines

Two situations make the cost worth it.

The first is relationship and multi-hop questions — anything of the form “how is X connected to Y,” “what depends on Z,” “trace the chain from A to B.” Supply-chain and ownership graphs, compliance and KYC (”is this counterparty linked to a sanctioned entity through any chain of ownership?”), the supplier-dependency case above. These questions are common in finance, law, and regulated industries, and they’re precisely the ones similarity search can’t touch. KAG, Lei Liang and co-authors’ framework from Ant Group ([arXiv 2409.13731](https://arxiv.org/abs/2409.13731), September 2024), leans hard into this — it adds logical-form reasoning over the graph and reported a 19.6% relative F1 gain on the multi-hop benchmark HotpotQA over a strong graph-RAG baseline (HippoRAG), which is the kind of lift you only get when the structure is doing real work.

The second is global sensemaking over a whole corpus — “what are the main themes,” “summarize the recurring risks across a thousand reports.” This is what GraphRAG’s community summaries were built for: the answer isn’t in any one document, it’s a property of the whole set, and the pre-computed summaries are how you reach it without stuffing the entire corpus into a context window.

Where it breaks

Be honest about this, because the hype skips it. A knowledge graph is expensive and brittle in three specific ways.

It’s expensive to build. Entity-and-relation extraction means an LLM call over every chunk of the corpus, and on a large corpus that indexing bill is real — easily orders of magnitude more than the few cents naive RAG cost in Article 2. Microsoft’s own answer to this is LazyGraphRAG (November 2024), which defers the community summarization until query time and cuts indexing cost down to roughly vector-RAG levels while keeping most of the global-query quality — Microsoft reports more than 700× lower cost on global queries than full GraphRAG. LightRAG (Guo and co-authors, EMNLP 2025 Findings, [arXiv 2410.05779](https://arxiv.org/abs/2410.05779)) attacks the same problem from the freshness side, with dual-level retrieval and incremental graph updates so you don’t re-extract the world every time a document changes.

It’s hard to keep fresh. A flat vector store updates by embedding the new document; a graph updates by re-extracting entities, reconciling them against what’s already there, and fixing up edges — which is a much heavier operation, and the one teams underestimate most.

The schema is the hard part, and it can’t be frozen. This is where I spend my research time, so I’ll be opinionated: the genuinely unsolved problem in enterprise knowledge graphs isn’t extraction, it’s schema evolution. You design a typed schema, and then the world hands you an entity type you didn’t anticipate — a new regulatory instrument, a new category of counterparty, a relationship that doesn’t fit your existing edge types. The agent ingesting that new document has to decide, under cost, whether to coerce the new thing into an existing type or propose extending the schema. Get that decision wrong in one direction and the graph rots into a mess of mis-typed nodes; wrong in the other and it fragments into a thousand one-off types nobody can query. Almost nobody has a principled control policy for that decision yet — it’s one of the genuinely unsolved problems in the area.

When not to use it

Most enterprises don’t need a knowledge graph — and that’s worth saying out loud before anyone spends a quarter building one. If your questions are overwhelmingly “what does this document say about X” rather than “how is X connected to Y,” a graph is expensive overkill, and a good advanced-RAG stack will beat a half-maintained graph every time. The graph earns its keep only when relationships are first-class in the questions your users actually ask, and you have the engineering capacity to keep it fresh. A stale graph is worse than no graph, because it answers a connection query with false confidence.

Cost / latency / setup effort:

Combinations and hybrids

In production, GraphRAG is almost never the whole answer — it’s the half that handles relationships, sitting next to a vector store that handles “what does this say.” The standard hybrid lets an agent route: similarity search for descriptive questions, graph traversal for relational ones, often both fused for questions that have a descriptive and a relational part. That routing decision — does this question need the graph at all — is itself a retrieval-as-decision problem, which is the subject of the next article on agentic RAG. The heuristic to remember: vectors for “what does this say”; graphs for “how are these connected.” Most real systems need a little of both, and the engineering is in knowing which question is in front of you.

Production checklist

Version the schema explicitly and treat a schema change as a migration, not a config tweak — log when types are added and why.
Track extraction precision/recall on a labelled sample; a graph built from sloppy extraction is confidently wrong, which is the worst failure mode.
Stamp every node and edge with provenance and validity dates — which document, which version, in force when — so a traversal can return a citation, not just an answer.
Instrument graph freshness: how stale is the graph relative to the corpus, and alert when re-extraction falls behind ingestion.
Reconcile entities deliberately — decide your dedup/merge policy for “is this the same entity” up front; entity resolution drift is silent and corrosive.
Measure how often the graph is actually needed — if most queries route to the vector side, that’s a signal you may not have needed the graph.
Keep a coerce-vs-extend policy for new types, even a crude one, so schema growth is a decision and not an accident.

The take

GraphRAG isn’t a better RAG — it’s a different tool for a different question. When the answer is a passage, use vectors; when the answer is a path, you need a graph, and you should go in clear-eyed about the build cost, the freshness burden, and the fact that the schema is the part nobody has fully solved. Most teams don’t need one. The ones that do — law, finance, compliance, anywhere relationships are the product — get answers from it that no amount of reranking would ever surface.

New articles land here first. Subscribe for the full series. I take on a small number of advisory engagements each year for teams hitting exactly these problems. The fastest way to reach me is on LinkedIn; reach out if that’s you.

The Knowledge Layer

Discussion about this post

Ready for more?