RAG Architecture Benchmark

Summary

Seven RAG architectures run against one 4,200-question evaluation set: naive vector retrieval, hybrid, reranked, query rewriting, multi-hop, GraphRAG, and a control that simply fills a long context window. The set, the scripts and every raw output are public.

The finding is plain. Hybrid retrieval with reranking reaches 0.87 recall, 19 points above naive vector search, for 240ms of added end-to-end latency. Multi-hop and GraphRAG do not beat it on this set and cost three to seven times as much.

The distribution of failure modes matters more than the headline. 61% of naive retrieval errors are misses; after reranking that falls to 22%, and the residual errors become "retrieved but unused" — a class far more sensitive to prompt structure than to the retriever. Section four separates the two with ablations…

Contents