The AI mistakes that cost you don't look like mistakes

When people talk about AI errors they picture invention: a book that does not exist, a number nobody published. That error is annoying, but it is the cheapest one — it looks wrong. You read it, you frown, you check.

The expensive ones are different. This week I built a small tool. It reads what a few communities are discussing and tells me what is worth writing about. Two hundred lines or so, written alongside an AI assistant — which is how I work now: I say what I want, it writes, I check. It worked on the first try, produced a clean ranking, and was wrong in three different ways. None of them looked like an error.

I am not saying that as a disclaimer. I am saying it because it explains the rest: in all three cases, what failed was not the code being written — it was what I said I wanted.

First: it measured what was easy to measure

The ranking sorted topics by "signal" — how often a topic showed up, weighted by the reach of where it showed up. The top topic had twelve signals. Looked solid.

Twelve signals were six conversations, and half the weight came from one: an old thread with eighty thousand views, from which the tool had pulled three separate excerpts. Each excerpt counted as eighty thousand. The ranking was not measuring a repeated topic. It was measuring one big conversation wearing six hats.

The part that matters: I had written, in the document that defines the criterion, that what counts is repetition across conversations. The tool did the opposite of what I had specified — and the result did not give it away, because a plausible number came out.

Second: it erased the trail

Fixing the count should have taken two minutes: change how the numbers are added up. It took eleven, because the tool was adding as it went, without keeping any of the model's answers. To change the arithmetic, everything had to be asked again.

It is a structural decision that feels like a detail while you are writing it, and turns into a toll every time you change your mind. And with AI you change your mind constantly — that is the working mode, not a planning failure. Storing the raw answer and computing later costs three lines at the right moment.

Third: it classified by word, not by concept

One category was "the workflow AI shortens". In came keyboard shortcuts, text expanders, email automations — all fine things, none of them involving AI at all. The category said "workflow", and the model matched the word.

That was not the model being dumb. That was my definition being loose: I described the category by what it looked like, not by what it required. The fix was one sentence — these topics require AI to be explicitly present in the item — and the leak stopped.

What the three have in common

None is a hallucination. All three are specification errors that produce a plausible result.

Which is why reviewing the output does not catch them. The text reads well, the number has the right shape, the code runs. The error sits between what you meant and what you said — and the model faithfully executed the second one.

What I do now, and it is not much

Check the first output against the written criterion, not against your impression. I had the criterion in writing. Reading the two side by side surfaced the bug in thirty seconds. I did not read it, because the result looked good — which is exactly where the problem lives.

Separate asking from concluding. Whoever calls the model writes the raw answer down and stops there. Whoever computes reads the file. With both in one place, every change of criterion pays the full bill again.

Define a category by what it requires, not by what it resembles. If the definition does not say what it demands, the model will match the nearest word. And it will do it again tomorrow.

Where this does not work

None of it catches a judgement error. If the criterion is wrong on paper, checking the output against it confirms the mistake with more confidence — and that is the most expensive failure of the three, because now there is process behind it.

There is a real cost too: keeping everything raw and checking the first output makes the work slower up front. Worth it when the result decides something. For a script you throw away tomorrow, it is ceremony.

What did not change

None of these three made me build less with AI. The tool was done in an afternoon; without it, days. What changed is where I look: it is not the output that fools you, it is the spec you did not reread.