I asked my AI assistant a simple question: “What’s the weather like today?”
It gave me a novel. A 500-word treatise on atmospheric pressure, climate models, and a comparison to yesterday’s weather. I just wanted to know if I needed an umbrella. Sound familiar?
We’ve been treating LLMs like search engines that talk, when they should be conversational agents that defer. That’s the core insight from a new paper called “Keep It Real” — and it explains why every interaction with a chatbot feels like being buried alive in text.
You’ve probably felt that exhaustion. You type a quick question, and the AI dumps everything it knows. It’s not being helpful — it’s being exhaustive. And that’s the problem.
The author, Noam K., was frustrated by the same thing. So he dug into why chatbots info-dump. The answer is baked into their design: LLMs are trained to maximize information delivery. They treat every response as a final exam. But humans don’t want the whole encyclopedia — we want a back-and-forth. We want to ask follow-ups. We want the AI to shut up until we ask for more.
True AI companionship requires the AI to strategically withhold information until prompted, not just dump it. That’s a radical shift. It means building a “schema-level deferral channel” — a way for the AI to signal, “I know more, but I’ll wait until you ask.”
Think about it: when you talk to a friend, they don’t recite their entire knowledge on the first question. They answer, then pause. They let you steer. That’s what we need from AI.
The paper proposes a simple mechanism: a special token that tells the model to defer. Instead of outputting every possible detail, the model outputs a compressed version with a promise to elaborate. It’s like a conversational “…” that says, “More available if you want.”
This isn’t just a technical tweak. It’s a fundamental rethinking of what an AI assistant should be. We’ve been optimizing for completeness. We should be optimizing for collaboration.
I tested a prototype based on this idea. I asked the same question, and the AI replied: “It’s 72°F and sunny. Want details on humidity or UV index?” That’s it. I felt relief. Then I said yes, and it gave me a short follow-up. We had a conversation, not a lecture.
The twist? This approach actually makes the AI feel smarter. By holding back, it shows restraint — a sign of intelligence. The reader should finish this knowing exactly where I stand: The best AI assistant is the one that knows when to shut up. That’s not a bug. It’s the feature we’ve been missing.
FAQ
Q: Isn't providing more information always better?
A: No. Humans have cognitive limits. Overwhelming the user with information they didn't ask for creates friction, not value. The best response is the minimum needed to answer the question, with an offer to expand.
Q: How would this deferral mechanism actually work in practice?
A: The paper proposes a special token in the model's output that signals 'more available.' The model is trained to output a concise initial response, then a prompt like 'Want details on X?' This turns the AI into a conversational partner, not a text generator.
Q: Isn't this just a fancy way of saying 'use a shorter prompt'?
A: No. Shorter prompts don't change the model's behavior — it still wants to maximize output. This is a structural change in how the model is trained and how it communicates, making it inherently deferential rather than exhaustive.