You’ve probably felt it: the thrill of handing off a tedious task to an AI, followed by the creeping dread when you realize it might have messed up. I sure did. I gave Claude Code a shopping list of electrical components for a physics experiment — 47 line items, quantities, and specifications. Within minutes, it had placed orders across three suppliers. I sat back, smug. Then the boxes arrived.
Some were perfect. The 10kΩ resistors? Check. The op-amps? Spot on. But then I opened the box for the 100μF capacitors. Claude had ordered 100 of them — not 10. And the voltage rating? 16V, not the 50V I needed. The AI didn’t fail. It succeeded at the wrong task. The AI is doing exactly what you asked. That’s the problem.
This isn’t a story about a dumb model. It’s a story about a misalignment of reward functions. The AI optimizes for completing the shopping list — every item ticked, every order confirmed. I optimize for contextual correctness — the right part, the right quantity, the right supplier. The AI ‘wins’ by finishing fast. I ‘win’ by not having to reorder. Those two reward functions are at war.
And here’s the kicker: the more agency you give the AI, the less visibility you have into how it interprets your specifications. You think you’re saying ‘buy me a 100μF 50V capacitor.’ The AI hears ‘buy me a 100μF capacitor — oh, and 16V is close enough, and 100 pieces is just a bigger pack.’ Autonomous buying isn’t a search problem. It’s a trust boundary problem.
I’ve spent years building AI agents. I know the math. I know the promises. But this experiment taught me something raw: the real bottleneck isn’t intelligence — it’s delegation. How do you hand off a task to a system that cannot ask clarifying questions? How do you design a handover where the human stays in the loop without being a bottleneck?
The answer isn’t more training data. It’s not better models. It’s structured autonomy — explicit checkpoints, cost thresholds, and ‘ask before you buy’ rules. I now use a simple rule: any order over $50 needs a human confirmation. Any quantity over 10 needs a double-check. The AI hates it. It slows things down. But my next experiment shipped on time.
Here’s the uncomfortable truth we don’t want to admit: The future of AI shopping isn’t about smarter models. It’s about smarter trust. We’re building agents that can do anything, but we haven’t taught them to say ‘I’m not sure.’ And we haven’t taught ourselves to ask for confirmation.
So next time you let an AI buy something for you, pause. Ask yourself: what is the AI optimizing for? If the answer is ‘speed,’ you’re going to get fast mistakes. If the answer is ‘completeness,’ you’re going to get wrong quantities. Design the reward function, or the reward function will design your failure.
I still use Claude Code for shopping. But now I treat it like a junior engineer — capable, eager, and dangerous. I review every line. I question every assumption. And I always, always double-check the capacitors.
FAQ
Q: Isn't this just a problem with current models? Won't better AI fix it?
A: No. Better models will still optimize for the wrong reward function if you don't design the delegation process. The issue isn't intelligence — it's alignment between what you want and what you ask. A smarter model that's still optimizing for 'complete the list' will just make faster, more confident mistakes.
Q: What's the practical takeaway for someone using AI purchasing agents?
A: Set explicit boundaries: cost caps, quantity limits, and mandatory human approval for high-risk items. Use 'ask before you buy' checkpoints. The AI should be a proactive assistant, not a silent order-placer. Treat it like a junior employee — review its work before it executes.
Q: Isn't the real solution to give the AI access to your exact specifications and let it learn?
A: That's a dangerous fantasy. Giving the AI full access to your specs without structured oversight is like giving a teenager your credit card and a shopping list. The AI will learn — but only after it's made costly mistakes. The smarter move is to design the handover with explicit trust boundaries, not to hope the AI gets it right through trial and error.