Keeping uncertainty
out of the output
In a domain where a confident wrong answer is worse than admitting doubt, raw LLM output is not something to bet on. This engine makes every key answer checkable and traceable — and says so when it is unsure.
- Input
- Multi-source candidates (model + rules/data)
- Cross-check
- Rule validation
- Confidence score
- Pass → output | Unsure → human review
Every key output passes the validation layer first; low confidence routes to a human instead of forcing a confident-looking wrong answer.
Problem
In this domain, a confident wrong answer does more damage than “I’m not sure”. Wiring raw LLM output straight downstream just pushes uncontrolled errors further along.
Constraint
No single model or single data source can be trusted alone; errors must be caught before they reach downstream; and the system must be able to flag “not sure” rather than always answering.
Decision
Validation-first, not model-first. Domain rules live in a testable validation layer, not buried in a prompt and hoped for; when confidence is low, it returns “needs review” rather than an answer.
Result
Key outputs became verifiable and traceable; uncertain cases are routed to humans instead of failing silently. Errors moved from “found later” to “caught at the moment”.
What was hard
Not wiring the model — defining how much confidence is “enough”, giving conflicting sources a clear adjudication rule, and stopping the validation layer from becoming another unmaintained black box.