Deciding Not to Use AI for a Feature

The most valuable decision in this cluster is the one that produces no feature. It is also the hardest to make, because the arguments against building are structural and boring and the argument for building is a demo somebody already showed the room.

Why the no is hard to say

Three asymmetries make it hard, and naming them helps, because each one has a counter.

The evidence is asymmetric. A prototype can be built in an afternoon and shown; the failure modes take weeks to find and cannot be shown at all. Whoever demos first has evidence and whoever objects has predictions. The counter is to make the objection concrete and cheap to test — not “this will not be reliable enough” but “here are twenty real inputs; the proposal needs to handle eighteen”.

The cost is asymmetric in time. Building is a one-time visible cost; operating is a permanent invisible one. Prompt maintenance, model migrations, evaluation upkeep, incident response and the review queue are all real recurring load that does not appear in the estimate. The counter is to quote the ongoing cost, not the build cost, and to note that the ongoing cost does not decline.

The incentive is asymmetric. Shipping an AI feature is legible to leadership and to the market in a way that not shipping one is not. This one has no clean counter, only honesty: name it out loud, because an unnamed incentive wins every argument it is present for.

Two questions that end it

Before any discussion of models, prompts or cost, two questions disqualify a proposal outright. Both are about the shape of the problem rather than the quality of the technology, which is why the answer does not change when a better model ships.

Is there a correct answer that the system must produce?

Not usually, not on average — must. A tax calculation, a permission check, a price, a legal deadline, an eligibility determination against written rules. A model produces a plausible answer, and plausible is categorically the wrong property when a rule exists that defines the right one. If the rule is written down anywhere, implement the rule.

The useful refinement: a model may still help by reading unstructured input and handing structured facts to the rule engine. That is extract-then-reason, and it is the answer to most “we need AI for this calculation” proposals. The model reads; the code decides. What is being rejected is the model doing the deciding, not the model being involved.

Can anyone tell when it is wrong?

If a wrong output is indistinguishable from a right one to whoever receives it, and no downstream check exists, then the error rate is not merely unmeasured — it is unmeasurable, and it will stay that way. You cannot evaluate it, you cannot detect a regression, and you cannot tell whether the feature is helping or quietly degrading the thing it touches.

This is the question that kills features that sound compelling: automatic prioritisation of a queue, sentiment scoring that nobody verifies, a recommendation with no measurable outcome, a summary of something nobody has time to read in full. If the answer is no, you are not building a feature, you are adding an unfalsifiable claim to your product. The fix is to build the verification first — a sample reviewed by a person, a downstream outcome you can measure — and only then the feature, which frequently reveals that the verification was the valuable half.

Where a model is genuinely the right tool

A framework that only says no is not used. The positive criteria are just as specific: reach for a model when the problem has at least two of these properties, and preferably three.

• The input is unstructured and the variety is real. Free text, documents, images, speech — where writing rules means writing an unbounded number of them. This is the core competence and everything else is secondary to it.
• Approximately right is genuinely useful. A draft, a first pass, a suggestion, a summary, a ranking. Cases where the consumer of the output expects to exercise judgement over it.
• The alternative is nothing. Not a worse feature — nothing. Documents that go unread, tickets that go untagged, transcripts nobody makes. The bar for an imperfect system is much lower when the status quo is absence, and this is where AI features most reliably earn their keep.
• The output is verifiable or reversible. A schema can check it, code can test it, a person reviews it, or it can be undone. This does not motivate building, but it determines whether the build is safe.

The alternatives people forget

Most rejected proposals have a cheaper mechanism sitting behind them, and the reason it is not considered is that it is unfashionable rather than that it is worse.

The pattern across the table is the same: where the structure exists, use it. A model is the right tool for the part of the problem where structure is absent, and using it for the parts where structure is present is paying per token for something a query already answers.

Saying it, and revisiting it

A no that is a refusal gets overturned. A no that comes with a condition is a decision the room can accept, and it is also more honest, because these decisions genuinely are conditional.

• Name what would change your mind. “If we had a way to verify the output, this becomes viable” or “if this were a suggestion rather than an action, the risk is acceptable”. Frequently the proposer will take the condition, and the resulting feature is the one that should have been proposed.
• Propose the smaller version. Level one of gradual autonomy — compute it, log it, act on none of it — costs almost nothing and answers the accuracy question with real data within weeks. It is the correct response to “we do not know if this would work”, and it is available immediately.
• Write down the decision and its reason. Two paragraphs: what was proposed, why not, what would change it. The same proposal returns every six months, and without a record the discussion restarts from zero each time.
• Put a date on revisiting. Some of these decisions genuinely expire — a capability improves, a price falls, a verification path appears. A no with a review date is a decision; a no without one is a position, and positions age badly in this field.