It’s 3 AM. Your PagerDuty is screaming. Your application is timing out. You check the metrics and realize your database has ballooned to 40TB in just two years. You thought you’d never hit those limits. You were wrong.
The panic sets in. You start frantically researching btrfs, tablespaces, and how to stitch together multiple 10TB volumes on Hetzner or AWS. You’re trying to rebuild an airplane engine mid-flight. But while you’re obsessing over raw capacity, you’re completely missing the actual executioner standing behind you.
When scaling a database, capacity is a vanity metric; throughput is the real killer.
We’ve all been conditioned to think our storage problems are about gigabytes and terabytes. The cloud providers sell you on volume sizes—10TB here, 64TB on the biggest EBS volume there. But when you hit that wall and start carving up your data into separate logical units just to fit within arbitrary hardware limits, you’re treating the symptom, not the disease.
The real bottleneck isn’t how much data you can store. It’s how fast you can write to it. And if you’re running a massive Postgres instance, you are likely bleeding performance right now without even knowing it.
I saw this firsthand in a recent infrastructure deep-dive. A team running a ~35TB Postgres database was hitting mysterious performance ceilings. They had massive NVMe drives, top-tier hardware, and yet their throughput was choking. The culprit? Write amplification in the Write-Ahead Log (WAL).
When your database is choking, don’t just give it bigger lungs; give it a separate airway.
Here is the twist nobody tells you about scaling: the WAL was quietly eating 20-30% of their total NVMe throughput. The system was competing with itself. Every transaction had to fight the background noise of its own safety mechanisms. They didn’t need more storage. They needed a better layout.
The solution wasn’t to buy more 64TB volumes or hack together a btrfs pool. They simply separated pg_wal onto its own dedicated volume. Instantly, the write amplification stopped cannibalizing their main I/O paths. Throughput recovered. The crisis ended.
This is why blindly following the ‘just add more storage’ mantra is dangerous. Neutrality and best practices will get you killed in the trenches of high-scale engineering. You have to take a side. You have to look at your volume layout and aggressively isolate your I/O profiles. If your transaction logs are sharing physical pathways with your heavy read/write tables, you are actively sabotaging your own architecture.
The next time you hit a storage wall, stop reaching for your credit card. Stop trying to duct-tape multiple volumes together. Look at your I/O. Find the write amplification. Isolate the noise. Because in the end, the size of your database doesn’t matter if your disks can’t breathe.
FAQ
Q: Doesn't just buying a bigger single volume solve the I/O problem anyway?
A: No. A bigger volume gives you more capacity, but if your WAL and your core data are on the same disk, they will still compete for the same I/O queue. Write amplification will still choke your throughput regardless of how many terabytes you add.
Q: What's the practical implication for my current infrastructure?
A: Stop treating your database as a single monolithic block. You need to map your I/O profiles. Separate your transaction logs (like pg_wal) and potentially your indexes onto dedicated, high-performance volumes to prevent internal resource contention.
Q: Is the cloud provider's 64TB volume limit actually a feature, not a bug?
A: Exactly. Those hard caps force you to stop being lazy with your architecture. They push you to break your monolithic data into logical, isolated units, which is the exact pattern you need to optimize throughput and survive at scale.