Let it say I don't know
A system that always gives you an answer is easy to build and hard to trust.
That sentence is most of what I believe about applied AI, and it took me a while to arrive at, because refusing to answer feels like failure when you are the one who built the thing. Every instinct you have as a builder says that coverage is the product.
The trouble is that a language model has no natural way to run out of confidence. Ask it something the source documents do not cover and it will still produce fluent, well-formatted, entirely plausible prose. The output looks identical whether the evidence was overwhelming or absent. In a consumer chat product that is a nuisance. In legal work, where somebody has to defend the answer to a partner or a court, it is the whole problem.
Abstention is a feature you have to build
So it has to be designed in. In practice that means the retrieval layer has to report not just what it found but how well it covered the question, and the decision layer has to be allowed to return something other than an answer. Three outcomes instead of one: answer, flag, escalate. The uncertain path ends at a person, on purpose.
This is unglamorous work. Nobody demos the case where the software declines. But the value of a cited answer comes entirely from the fact that the system could have declined and didn't — otherwise the citation is decoration.
The cost of being wrong is not symmetric
The reason to accept a lower answer rate is that the errors are not the same size. A missed answer costs a few minutes of somebody's time. A confident wrong answer, carried into a filing with a citation that looks real, costs considerably more than that, and it costs it later, when it is expensive to unwind.
Optimizing for coverage buys you a demo. Optimizing for the cost of being wrong buys you something a professional will put their name on.