Nine articles of techniques come down to one question a buyer actually has: given my corpus, my update rate, my latency budget, and my team, what should I build? Here’s the decision, made out loud.
The problem
A VP of engineering has read the briefs, sat through the vendor demos, and now has a budget and a deadline. The team is split: one camp wants a knowledge graph because a competitor announced one, another wants an agentic framework because it’s the thing this year, and a third just wants to ship the simple RAG that already half-works. Everyone has a technique. Nobody has a decision, because the decision was never about the techniques. It was about which of them this particular problem actually justifies.
That’s the gap this final article closes. The nine pieces before it mapped the knowledge layer (the part of an agent system that decides reliability in production, and an engineering discipline in its own right) technique by technique. This one runs the other direction: start from the constraints a real buyer has and walk to a recommendation. There is no best knowledge layer, only the right one for a given corpus, update rate, latency budget, reliability bar, and team. The most expensive mistake in this whole space is buying a tier of complexity the problem didn’t ask for. The goal here is to make the choice legible enough that you can defend it in the budget meeting.
How to choose: the five questions
Five constraints decide it. Answer these honestly and the architecture mostly falls out.
Figure 1: The selection tree. Relationships and stakes set the retrieval tier; continuity adds memory; context engineering and evaluation wrap everything regardless of where you land.
1. What shape are the questions: passages or relationships? If users mostly ask “what does the corpus say about X,” you’re in retrieval territory (Articles 2–3). If they ask “how is X connected to Y,” “what depends on Z,” “trace the chain” (and that’s a real share of traffic, not a demo question), you need a graph (Article 4), but only if the documents cross-reference each other densely enough to be worth traversing, which is a property of your corpus you can measure rather than one your domain guarantees. This is the first fork because it’s the one that most changes the build.
2. How expensive is a wrong answer? A wrong answer in an onboarding bot is a follow-up Slack message; a wrong answer in a legal, medical, or financial system is a liability. Low stakes justify naive RAG; high stakes demand the advanced-RAG reliability moves at minimum, and push provenance and temporal correctness from nice-to-haves to requirements.
3. How often does the knowledge change, and do facts supersede each other? A static corpus is easy. A corpus where this year’s policy overrides last year’s needs metadata filtering at least, and a temporal model (Article 4’s typed graph, or Article 6’s temporal memory) if “what was true when” matters. Update frequency is also what kills knowledge graphs: a graph you can’t keep fresh is worse than no graph.
4. What’s the latency budget? Every tier up the stack costs milliseconds-to-seconds. Reranking adds a few hundred ms; an agentic loop adds seconds and 3–5× the model calls. A sub-second product requirement rules out the agentic tier for the hot path no matter how much you’d like the accuracy. Know the budget before you choose, not after.
5. How deep is the team? This is the constraint people lie to themselves about. A knowledge graph or an agentic system is an ongoing operational commitment: extraction pipelines, freshness monitoring, eval harnesses, the lot. A two-person team that builds a graph it can’t maintain has bought a liability. Match the architecture to the team that has to run it at 3am, not the team in the kickoff meeting.
Three scenarios, end to end
Abstract advice is cheap. Here are three concrete builds, each starting from the five questions.
Scenario A: a legal knowledge base
The questions look heavily relational (“which articles does this one depend on, and were any amended after this date?”), wrong answers are expensive and must be cited, and the corpus changes on a relentless amendment schedule where the date a fact was valid is part of the answer. Five questions, five strong signals, and the obvious answer is a typed knowledge graph. I built one, benchmarked it against four other arms, and the obvious answer was wrong.
On a Serbian legal corpus the graph arm came out level with strong flat retrieval (p = 0.69), because Serbian statutes barely cross-reference each other and there was no web to traverse. What paid was the temporal half: on the three validity questions, where the edges genuinely exist, the graph arm was the only fixed-pipeline arm to get all of them. Three questions prove nothing on their own, but they point at where structure still earns its keep, which is validity-aware traversal rather than cross-reference multi-hop. Meanwhile the agentic arm led every quality axis among the deployable options (0.93 correctness, best provenance) at roughly 2.3× advanced’s cost and under twice its latency, which legal research can absorb because it isn’t a sub-second product.
So the recommendation is advanced RAG with temporal validity modelled explicitly (Articles 3 and 6), an agentic layer (Article 5) for the hard tier, and a typed graph (Article 4) only where the corpus is genuinely densely cross-referenced. That last one is an empirical question about your documents, not a property of the domain, and it is exactly the kind of assumption the benchmark discipline in Article 9 exists to kill before you build on it. A naive-RAG assistant that confidently quotes a repealed rule is still the failure mode this entire stack exists to prevent.
Scenario B: an e-commerce shopping assistant
Different shape entirely. The questions are mostly descriptive (“a waterproof jacket under €150 for hiking”), the corpus is a large, fast-moving product catalog, latency is a hard product constraint because a slow assistant loses the sale, and a wrong answer is a bad recommendation, not a compliance event. Relationships barely matter; speed and freshness dominate.
The recommendation is advanced RAG (Article 3): hybrid search so exact terms like a SKU or a brand don’t get lost in semantic fog, aggressive metadata filtering on price, category, and stock, and a reranker tuned for a tight latency budget. No graph (the relationships aren’t the product), no agentic loop on the hot path (latency rules it out), but a memory system (Article 6) is worth it here for returning shoppers: remembering a customer’s size, style, and past purchases is a conversion lever, and it’s the one place this build spends on continuity. This is the scenario where reaching for a knowledge graph because it’s impressive would be the expensive mistake the intro warned about.
Scenario C: an internal-tools agent
The unglamorous, common case. A few hundred employees ask an internal agent to find forms, check policies, and answer “how do I do X” over a wiki and some structured systems. The corpus is small-to-medium and low-stakes, the questions are mixed (some descriptive, some “which system owns this”), and the team maintaining it is small and busy with other work.
The recommendation: start at naive RAG (Article 2) and earn every upgrade. Add advanced-RAG reranking when you can measure that retrieval is the bottleneck, add a light agentic router (Article 5) only if a real fraction of questions genuinely span multiple systems, and lean hard on context engineering (Article 7) to keep it cheap. Crucially, don’t build the graph or the full memory stack this team can’t maintain: the right architecture here is the simplest one that clears the bar, instrumented well enough (Article 9) to know when the bar moves. A different agent in the same company (one tracking a fast-moving, multi-source story over time) would sit at the other end of this spectrum: its whole value is temporal memory of a developing situation, exactly the heavy stack this internal agent shouldn’t touch. Same field, opposite answers, because the constraints are opposite.
Where each tier shines, in one table
The whole series, compressed to a triage card:
Two of these apply regardless of where you land. Context engineering (Article 7) wraps whatever you build, because every architecture eventually has to fit its evidence into a finite window. And evaluation (Article 9) gates every one of these choices, because the difference between “we chose the graph” and “we chose the graph and proved it earned its latency” is the difference between an architecture decision and a guess. The one anti-pattern that spans all of it: knowledge editing (Article 8). When a fact goes stale, the answer is almost never to reach into the model’s weights, it’s to fix the corpus the rest of this stack retrieves from.
Where this goes wrong
Three failure patterns, because the selection itself can be the mistake.
Over-building. The most common and most expensive: a team buys the graph, the agentic loop, and the memory stack for a problem that naive RAG plus a reranker would have served, and now maintains three subsystems that move no metric. Complexity you don’t need is complexity that fails at 3am. The discipline is to start one tier below where you think you need to be and let evaluation pull you up.
Under-building the reliability bar. The opposite mistake, rarer but worse where it happens: shipping naive RAG into a legal, medical, or financial setting because it demoed fine, and discovering that “confidently wrong and uncited” is a category of failure the cheap stack can’t address. High stakes are non-negotiable on provenance and recency; that’s not a tier you skip to save a sprint.
Choosing for the team in the kickoff, not the team in production. An architecture is a maintenance commitment, and the org that designs a graph-and-agent system it lacks the people to keep fresh has chosen a future outage. The right question isn’t “what’s the best architecture,” it’s “what’s the best architecture this team can operate,” and those have different answers more often than anyone likes to admit.
Cost / latency / setup effort (the tiers, side by side):
Concretely, as of October 2026: hybrid search on Elasticsearch or OpenSearch with a Voyage reranker over it (Article 3); Neo4j or LightRAG for the graph tier (Article 4); Mem0, Zep or Letta for managed memory (Article 6). Illustrations, not load-bearing: delete every product name here and the argument above is unchanged.
Production checklist: for the decision itself
Write the five answers down (question shape, stakes, update rate, latency budget, team depth) before anyone names a technique.
Start one tier below your instinct and let evaluation (Article 9) justify each upgrade with a measured delta.
Make “can we keep it fresh” a gate on the graph and memory tiers specifically: freshness debt is the silent killer.
Separate the hot path from the slow path: an agentic loop can serve the hard 10% of questions while advanced RAG serves the fast 90%.
Set the reliability bar by the cost of a wrong answer, and make provenance and recency hard requirements wherever that cost is high.
Budget for the layer you’ll always need: context engineering and evaluation aren’t optional add-ons; price them in from day one.
Revisit the choice on a schedule: the field moves in months, and the right tier today can be the wrong one in two model cohorts.
The take
There is no best knowledge layer: there’s the one your corpus, your stakes, your update rate, your latency budget, and your team actually justify, and the whole skill is reading those five honestly and buying exactly the tier they ask for. Start simple, make every upgrade earn its place against a number, keep the changing knowledge outside the model where you can govern it, and wrap all of it in context engineering and evaluation no matter where you land. Nine articles of technique reduce to that one disciplined decision, and getting it right is worth more than any single technique in the stack, because it’s the difference between a knowledge layer that’s reliable in production and an expensive one that merely looks the part.
That closes the core ten. If you want to see what happens when the corpus changes underneath you, the finale already ran: three agent memories, six months of contradicting news, and what it actually cost to keep each one correct (<finale-url>).
If your team is staring at exactly this decision and wants a second pair of eyes on it, that’s the kind of work I take on. DM me.
What to read next
Internal:
Article 1 — Your agent’s problem isn’t the model, it’s the knowledge layer
Article 2 — Naive RAG: the baseline you’ll always benchmark against
Article 3 — Advanced RAG: what you actually run in production
Article 4 — GraphRAG and friends: when entities and relationships beat similarity
Article 5 — Agentic RAG: when retrieval becomes a decision, not a pipeline
Article 6 — Vector vs graph vs episodic: a tour of agent memory systems
Article 7 — Context engineering: the discipline that replaces prompt engineering
Article 8 — Knowledge editing and why it isn’t the answer for enterprise updates
Article 9 — Evaluation: how do you know your agent setup is working?
Article 10 — Choosing your knowledge layer: a practical enterprise selection guide (coming soon)
External:
Lewis et al. (NeurIPS 2020), Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. https://arxiv.org/abs/2005.11401. Where the whole stack started.
Edge et al. (April 2024), From Local to Global: A Graph RAG Approach to Query-Focused Summarization. https://arxiv.org/abs/2404.16130. The graph tier, for the relationship-heavy case.
Anthropic (September 2025), Effective context engineering for AI agents. https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents. The wrap-everything discipline, current.
Es et al. (EACL 2024), RAGAS: Automated Evaluation of Retrieval Augmented Generation. https://arxiv.org/abs/2309.15217. How you prove the choice was right.
Singh, Ehtesham, Kumar, Talaei Khoei & Vasilakos, Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG (arXiv preprint, v4 revised 1 April 2026). https://arxiv.org/abs/2501.09136. A taxonomy of agentic RAG by agent cardinality, control structure, autonomy and knowledge representation, and the clearest current map of where the agentic tier is and isn’t mature.