You wrote a prompt. The LLM reformatted your JSON. It charged you 800 tokens. It hallucinated a field name. You added a retry loop. It worked on the third try. You shipped it. You felt productive.
You weren’t productive. You were burning money to make a language model do what a for-loop does for free.
I’ve watched this scene play out in dozens of codebases. Smart engineers, people who can write a regex in their sleep, suddenly forget that deterministic code exists the moment an API key lands on their desk. They load raw JSON into a context window, ask a model to reformat data that requires zero reasoning, and then act surprised when the output has a stray comma that breaks everything downstream.
The bottleneck was never the model. It was your default instinct to throw tokens at every problem like it’s a hammer and everything looks like a nail.
Here’s the uncomfortable truth nobody on Twitter will tell you: most of what you’re routing through an LLM right now doesn’t need reasoning. It needs transformation. Those are fundamentally different jobs. A language model is a probabilistic engine — it guesses the next token based on patterns. That’s magical when you need interpretation, summarization, or creative synthesis. It’s actively harmful when you need a field renamed from user_id to userId.
Think about what happens when you ask an LLM to reformat structured data. You’re paying for computation, latency, and the privilege of uncertainty. You’re introducing a probabilistic failure mode into a pipeline that could have been deterministic. You’re shipping a brittle system that works 97% of the time and fails on the 3% of edge cases you didn’t prompt for — at 3 AM, in production, on a Friday.
Every unnecessary LLM call is a tax you pay for choosing convenience over engineering judgment.
The trap catches almost everyone right now, and I understand why. The interface is seductive. You type English, you get output. No schema design, no parsing logic, no error handling for malformed inputs. The LLM just… handles it. Until it doesn’t. Until it decides null should be an empty string. Until it wraps your JSON in markdown because it felt like being helpful. Until it invents a value that doesn’t exist in the source data because the pattern felt right.
A 15-line Python script doesn’t have opinions. It doesn’t get creative. It doesn’t hallucinate. It does exactly what you told it to do, every single time, at near-zero cost, in milliseconds instead of seconds.
I’m not saying stop using LLMs. I’m saying stop using them for everything. The real skill in AI engineering isn’t knowing how to prompt — it’s knowing when not to prompt at all. The best AI systems being built today aren’t the ones that route everything through a single model. They’re the ones that use deterministic code for structure-preserving tasks and reserve probabilistic reasoning for the jobs that actually require it.
If your LLM call doesn’t need to reason, it doesn’t need to be an LLM call. Full stop.
So the next time you reach for the API key, ask yourself one question: am I asking the model to think, or am I asking it to transform? If it’s the latter, close the tab. Open your editor. Write the script. Your infrastructure bill, your latency budget, and your on-call engineer will all thank you.
The future of AI isn’t more tokens. It’s knowing exactly when tokens are the wrong tool for the job.
FAQ
Q: But isn't it faster to just prompt an LLM than write parsing code?
A: Faster to write, slower to run, and fragile in production. You're trading 10 minutes of engineering for a permanent tax on every API call — plus hallucination risk on structured data that should never be probabilistic.
Q: How do I know which tasks to keep on the LLM?
A: Ask: does this require interpretation, judgment, or generation of novel content? If yes, use the LLM. If it's pure data transformation — renaming fields, filtering, reformatting — write deterministic code. The line is reasoning, not complexity.
Q: Isn't this just gatekeeping against people who can't code?
A: No — it's the opposite. If you CAN code, stop pretending you can't. The people truly hurt by over-tokenization are the ones shipping brittle systems that fail silently. Engineering judgment isn't gatekeeping; it's the job.