Hook 40 · Version A · Problem-Example-Solution (scoped client story)
We helped a UK startup team improve its reported RAG accuracy from 65% to 96% after a two-day workshop.
2026-11-03 · 11:30 Europe/Chisinau
Score 90/100Scheduled
Show postHide post
We helped a UK startup team improve its reported RAG accuracy from 65% to 96% after a two-day workshop.
The startup had spent a year developing an email-answering system for an oilfield production company. This is one owner-supplied engagement result, not a typical outcome or a guarantee. The available case notes do not define the metric or establish which technical change caused the improvement.
The transferable lesson is a check your team can run before its next RAG change:
Input: A representative question and the source material needed to answer it.
Failure: The answer is wrong or unsupported.
Diagnosis: Did retrieval find the right source? Did the answer use it faithfully? Did it cover the user's full request?
Verification: Compare a focused change against the same labeled cases and inspect regressions.
Keep retrieval coverage separate from answer quality. When an LLM judge is involved, test its decisions against expert-reviewed failures before using its score as evidence.
The aim of coaching is a method your engineers can repeat across projects, not a promise to reproduce one client's number.
For production RAG, evals are the most important transferable capability. Coaching engineers to define cases, inspect retrieval and test the judge gives them a method for the next product. One reported client improvement is useful context, not a guarantee that another team will reach it.
Comment “RAG” if you want the short guide to separating retrieval from answer quality.
P.S. We'll start sending it once at least 10 people ask for it here.
P.P.S. Connect with me so I can send it by DM after that.
Copy post includes bold formatting when the destination supports it. Copy with bold letters uses special Unicode characters; they may be less accessible to screen readers.