You’ve probably been there. You build a slick new CLI tool, you wire it up to your favorite LLM, and you feed it perfectly structured JSON. You wait for the AI to work its magic. Instead, it burns through your token budget, hallucinates half the data, and gets completely lost in the syntax.
You did everything the tutorials told you to do. You used clean, structured data. So why is the model acting like it’s reading a foreign language?
Because it is. We spent a decade making data readable for humans and parseable for machines, only to realize LLMs are neither.
Recently, a developer on Hacker News highlighted this exact frustration. They chose JSON as the output format for their CLI tool, assuming that structure equaled clarity. Instead, they noticed the LLMs easily got lost and consumed a massive amount of tokens just trying to parse the output.
Here is the hard truth: JSON and Markdown are failing us. JSON is bloated with curly braces, quotes, and commas. Markdown is verbose and relies on visual spacing that means nothing to a token parser. Every curly brace and comma in your JSON is a tax on your model’s attention span. You are paying for syntactic overhead, not semantic meaning.
Think about how LLMs are actually trained. Yes, they read Wikipedia and Reddit. But they also ingest GitHub. They read Python, C++, and Rust. They understand the dense, compressed logic of programming languages far better than they understand the rigid, bloated syntax of web data formats.
The optimal format for an LLM isn’t a beautifully formatted Markdown table, and it certainly isn’t a nested JSON nightmare. The LLM doesn’t want a beautifully formatted Markdown table; it wants the dense, semantic shorthand of code.
What we need is a compressed symbolic DSL (Domain Specific Language). Instead of spelling out a table in JSON, we should be encoding it in a compact, code-like expression that leverages the LLM’s deep understanding of programming syntax. We need formats that strip away the syntactic fat and deliver pure semantic muscle.
If you are building pipelines that feed data into AI agents, this isn’t just a theoretical debate. It directly impacts your cost, your latency, and your accuracy. Stop translating your data for human eyes or legacy machines. Speak the LLM’s native tongue, and watch your token bills plummet while your model’s comprehension finally clicks into place.
FAQ
Q: But LLMs are trained on JSON and Markdown. Won't they understand it fine?
A: They understand it, but they pay a massive token tax to do so. Just because a model can parse bloated syntax doesn't mean it's the most efficient or accurate way to feed it data. Token overhead directly degrades the model's attention span.
Q: How does this actually save me money?
A: By stripping out syntactic bloat (like quotes, brackets, and verbose keys) and using a compressed code-like DSL, you drastically reduce the token count per request. Less tokens means lower API costs and faster latency.
Q: Are you saying we should just feed raw minified code to the model?
A: No. The goal is a new format that balances minimal token count with maximum semantic meaning. It should look like a highly compressed programming expression, leveraging the LLM's training on code syntax rather than prose or web data formats.