Why retrieval quality beats model size
Teams obsess over which model to use. The real lever in document Q&A is what you feed it — chunking, deduplication, and metadata.
Priya Raman
Founder · Lumen
Every week a new language model tops the benchmarks, and every week the same thing happens in production: a team swaps the model, accuracy barely moves, and the roadmap slips. When your product answers questions over documents, the model is rarely the bottleneck. Retrieval is.
Garbage in, confident garbage out
A state-of-the-art model given the wrong passage will answer fluently and incorrectly. That's worse than failing loudly, because users can't tell the difference. The fix isn't a bigger model — it's a better index: clean text extraction, sensible chunk boundaries, and metadata that survives the trip from Drive to vector store.
- Chunk on semantic boundaries (headings, sections), not fixed token counts.
- Deduplicate near-identical versions of documents — old drafts poison answers.
- Keep page numbers and section titles attached to every chunk, forever.
- Re-index on change; a stale index quietly erodes trust.
How we measure it
We evaluate Lumen with a simple discipline: every answer must either cite a source or decline. When retrieval quality went up 18% after reworking our chunking, citation accuracy followed — and 'I don't know' rates fell by half. Users don't need the biggest model. They need the right paragraph.
“The best RAG system is a search engine that happens to talk.”