You’ve felt the sting. You drop a small fortune on a maxed-out Mac Studio M3 Ultra, expecting the silicon to crush local AI workloads. You download the brand new Qwen 3.8 27B model, hit generate, and… you wait. You watch a blinking cursor stutter out text at 14 tokens per second. Meanwhile, a free web chat on your phone feels instantaneous.
Spending four grand on a Mac Studio doesn’t buy you speed; it buys you the privilege of watching a progress bar.
The real gut-punch? The previous version of that exact same model—Qwen 3.6, at the exact same parameter size and quantization—ran twice as fast on your machine. An “upgrade” just halved your performance. You didn’t change your hardware. The silicon didn’t degrade. So what happened?
The dirty secret of the local LLM community is out: your hardware isn’t the bottleneck. The software is.
In the local LLM world, an upgrade isn’t an upgrade—it’s a regression disguised by a version number.
We are currently at the mercy of fragmented toolchains. If you’re running models locally, you’re likely juggling LM Studio, Unsloth, or terminal scripts, tweaking Q4_K_M quantizations and praying for dflash2 support. You’re acting as a systems engineer just to get a chatbot to output a paragraph without choking. The fact that Qwen 3.8 runs at half the speed of 3.6 on identical hardware proves that raw compute is no longer the issue. Optimization maturity is.
Some users are fleeing back to old AMD datacenter cards like the Instinct MI50, trading the sleek silence of Apple silicon for the noisy heat of 1TB/s VRAM bandwidth just to get acceptable throughput. But swapping hardware is missing the point entirely. The local LLM scene doesn’t need more RAM or faster GPUs. It needs a “Draw Things” moment.
We don’t need another terminal command; we need a polished, opinionated app that hides the configuration mess and standardizes optimizations.
Draw Things revolutionized local image generation on Mac by absorbing the complexity. You didn’t need to know how the sausage was made; you just clicked a button and got a picture. Local text models are currently stuck in the pre-Draw Things era, trapped in a labyrinth of developer-first tools that punish the end user.
If you’re deciding whether to buy local LLM hardware right now, stop. Don’t assume “same size = same speed” or “newer = faster.” The software stack dictates your real-world tokens per second, and right now, that stack is a house of cards.
Until the software catches up to the silicon, your expensive hardware investment is just a very expensive paperweight waiting for an optimization patch.
FAQ
Q: Isn't 14 tokens per second still readable for most use cases?
A: Technically yes, but anything under 100 tok/s feels broken compared to modern cloud models. When a free Gemini instance feels instant and your $4,000 machine stutters, the local experience is objectively inferior.
Q: Should I hold off on buying a Mac Studio for AI development?
A: Don't buy it expecting out-of-the-box parity with cloud models. If you need local control and privacy, expect to spend hours fiddling with fragmented toolchains. The hardware is ready; the software ecosystem is not.
Q: Is local AI just a hobbyist toy right now?
A: For the average consumer, yes. Until we get a unified, aggressively optimized app that standardizes model optimizations, local LLMs are a sysadmin's hobby, not a seamless consumer product.