Stop graphing everything: When GraphRAG actually beats vector RAG

If you have built anything with retrieval-augmented generation (RAG) in the last two years, you have lived its central frustration: You chop your documents into chunks, embed them, retrieve the top few that look similar to the question, and hand them to the model. For “What was our Q3 refund policy?” This works beautifully. For “What are the recurring themes across two years of customer complaints?” it falls flat — because no single chunk contains the answer. The fashionable fix is GraphRAG : Instead of feeding the model isolated snippets, you first build a knowledge graph of the entities and relationships in your corpus, then use that structure as context. The pitch is seductive. But seductive pitches deserve scrutiny, so I went through the evidence — the original Microsoft paper plus four independent benchmark studies — to answer a simple question: When you swap text chunks for a context graph, do answers actually get better? The short version: Yes, substantially — but only for the right kind of question, and not for free. Let me show you the receipts. Why text chunks hit a wall Standard vector RAG retrieves the k passages most similar to your query. That design has three structural blind spots: It can’t connect the dots. When an answer requires joining facts that live in different passages through a shared entity, chunks embedded in isolation never reveal the link. It’s blind to global questions. “What are the main themes?” needs the whole corpus, but similarity search only returns the handful of chunks that superficially resemble the question. It severs context at chunk boundaries. The relationships and hierarchy that complex reasoning depends on are exactly what chunking throws away. Microsoft Research framed this crisply when they introduced GraphRAG: Baseline RAG

If you have built anything with retrieval-augmented generation (RAG) in the last two years, you have lived its central frustration: You chop your documents into chunks, embed them, retrieve the top few that look similar to the question, and hand them to the model. For “What was our Q3 refund policy?” This works beautifully. For “What are the recurring themes across two years of customer complaints?” it falls flat — because no single chunk contains the answer. The fashionable fix is GraphRAG : Instead of feeding the model isolated snippets, you first build a knowledge graph of the entities and relationships in your corpus, then use that structure as context. The pitch is seductive. But seductive pitches deserve scrutiny, so I went through the evidence — the original Microsoft paper plus four independent benchmark studies — to answer a simple question: When you swap text chunks for a context graph, do answers actually get better? The short version: Yes, substantially — but only for the right kind of question, and not for free. Let me show you the receipts. Why text chunks hit a wall Standard vector RAG retrieves the k passages most similar to your query. That design has three structural blind spots: It can’t connect the dots. When an answer requires joining facts that live in different passages through a shared entity, chunks embedded in isolation never reveal the link. It’s blind to global questions. “What are the main themes?” needs the whole corpus, but similarity search only returns the handful of chunks that superficially resemble the question. It severs context at chunk boundaries. The relationships and hierarchy that complex reasoning depends on are exactly what chunking throws away. Microsoft Research framed this crisply when they introduced GraphRAG: Baseline RAG