AI Reality Checks
Your RAG system is probably a search problem wearing an AI costume
What I tell consulting clients before they spend months blaming the model for retrieval, permissions, and content-quality failures.
5 min read · 2026-06-08
Most broken RAG projects are not broken because the model is dumb. They are broken because the source content is messy, the retrieval strategy is vague, permissions are duct-taped on later, and nobody defined what a correct answer looks like before the demo got applause.
A competent architecture review starts with boring questions: who is allowed to see which documents, how freshness is enforced, what gets logged, how answers are evaluated, and where a human can override the machine. If those answers are fuzzy, the system is not production-ready. It is an expensive autocomplete box.
I no longer take on implementation or building projects. I only offer paid consultations and strategic advisory. In a paid AI Reality Check, I will tell you whether your RAG idea is worth pursuing, where it will break, and which assumptions need to die before they eat your budget.
If this sounds uncomfortably familiar, get a paid review.
I no longer take on implementation or building projects. I only offer paid consultations and strategic advisory, which means the guidance is not a setup for selling you a build.
Book a paid consultation