case study · August 2026
Doccrew
Grounded question answering over documents, built around one hard rule: no sentence ships without evidence. Live at doccrew.vercel.app, where you can watch the pipeline think in real time.
The problem
Most RAG systems ask the model to cite its sources and hope. Nothing checks. The failure mode is an answer that reads confidently, cites plausibly, and is wrong, which is worse than no answer, because the reader cannot tell. Doccrew's premise: grounding must be a gate, not a request.
The design: a verifier that cannot hallucinate
The pipeline is planner, retriever, writer, verifier. The first three use an LLM; the last one deliberately does not. The verifier is pure Python: it splits the drafted answer into sentences and checks each against the retrieved chunks using token-overlap scoring. Unsupported sentences are struck, visibly. If nothing survives, Doccrew refuses to answer instead of guessing. Because the verifier is deterministic code, the grounding gate itself cannot hallucinate, which is the property everything else stands on.
Making the pipeline legible
The UI streams the run live over SSE: the subqueries the planner produced, the chunks retrieval returned with scores, the writer's cited draft, and the verifier's kept-or-struck verdict on every sentence. This started as a debugging tool and became the product's best feature: a stranger can watch it work and understand exactly why an answer says what it says, and what got cut.
What went wrong
Converting the pipeline to the Anthropic SDK surfaced a lesson about provider capabilities: server-side model fallbacks are accepted only by certain model tiers, and the smaller model the public demo runs rejects the parameter with a 400. The fix gates the feature by model family, discovered the honest way, in a stack trace. Deployment taught its own lesson: the FastAPI backend and Next.js frontend ship as two Vercel projects, and getting per-IP rate limiting right on the public demo mattered as much as the pipeline itself.
Limits, stated plainly
Token-overlap verification catches fabricated facts but not all paraphrase, and not negation: a sentence sharing most of its words with a chunk while inverting the meaning can survive. The blog post analyzing this tradeoff is linked below; the direction for closing it is an entailment-based check layered behind the fast lexical one. Stating that limit is part of the design: a verifier you cannot distrust correctly is not a verifier.
Try it live at doccrew.vercel.app, read the code at github.com/tsaipraveen99/doccrew, or read the design argument: A verifier that can't hallucinate.