Stop Using NFS Hard Mounts. Here’s What Actually Protects Your Data.

You’ve been there. It’s 2 AM. Your monitoring dashboard is screaming red. A process is hung — completely stuck — because an NFS server somewhere went down and your hard mount decided that waiting forever was better than letting go. You can’t kill the process. You can’t unmount. You’re staring at a frozen system and wondering why you ever trusted a network to behave like a local disk.

Hard mounts don’t protect your data. They protect your ego — the illusion that if you just wait long enough, everything will be fine.

Here’s the dirty secret of NFS hard mounts: they were designed for a world that doesn’t exist anymore. A world where network interruptions were rare, transient blips. A world where a server going down meant someone tripped over a cable, not that your cloud provider just had a regional outage. In that world, waiting indefinitely made sense. In our world, it’s operational malpractice.

Let’s talk about what actually happens when you use hard mounts in production. Your application blocks. Your I/O queue fills up. Your users start seeing timeouts. Your load balancer marks nodes as unhealthy. And you — the person responsible for keeping things running — are locked out of your own system because every command that touches that filesystem is now wedged behind an unreachable server.

Now let’s talk about soft mounts. Most sysadmins treat them like a dirty word. “Soft mounts cause data corruption!” they say, repeating the same folklore they heard from a senior engineer years ago. But here’s what they miss: soft mounts with carefully tuned retries, retransmits, and timeouts give you something hard mounts never can — control.

The choice was never between safety and danger. It was between being stuck and being free.

Think about it. A soft mount with retries set to a high value — say, enough to cover a 72-hour window — gives you a deliberate maintenance window. Server goes down? Your applications get errors they can actually handle. They can retry, fail over, or degrade gracefully. And you? You get to fix the problem instead of staring at a hung terminal wondering if you should just reboot the whole box.

I saw this firsthand at a company that ran hard mounts across a fleet of 200 storage clients. When their NFS server crashed during a firmware update, 200 machines froze. Not one or two — all of them. Processes couldn’t be killed. Filesystems couldn’t be unmounted. The team spent four hours rebooting servers one by one, hoping nothing corrupted during the forced shutdowns. The irony? The hard mounts they chose for “reliability” caused more downtime in those four hours than a year of soft mount failures ever would have.

Reliability isn’t about never failing. It’s about failing in ways you can recover from.

Now, I’m not saying soft mounts are perfect. They’re not. If your application doesn’t handle I/O errors gracefully — and most don’t — you can lose data. A write that returns success but never actually hit the disk is a nightmare scenario. But that’s not a mount option problem. That’s an application design problem. And if your data integrity depends on a kernel flag rather than your application’s error handling, you’ve already lost.

The real answer is this: stop treating hard vs. soft as a religious war. Start treating it as a design decision. What happens when the server disappears? Hard mounts say, “freeze and pray.” Soft mounts say, “fail and adapt.” In a world where infrastructure fails constantly — where cloud providers have multi-hour outages, where network partitions are routine, where your storage server might be rebooted for patches every other week — which response do you actually want?

If your system can’t survive a failure, then your system was never reliable — it was just lucky.

So here’s what I’d tell anyone setting up NFS today: use soft mounts. Tune your retries to give yourself a real maintenance window. Make sure your applications handle I/O errors instead of pretending they’ll never happen. And for the love of everything operational, stop trusting hard mounts to save you. They won’t. They’ll just make sure that when things go wrong, you can’t do anything about it.

The best filesystem configuration isn’t the one that never fails. It’s the one that fails on your terms.

FAQ

Q: But won't soft mounts corrupt my data when writes silently fail?

A: If your data integrity depends on a kernel mount flag rather than your application's error handling, you already have a data integrity problem. Fix the application. Handle I/O errors explicitly. Don't outsource data safety to NFS.

Q: What retry and timeout values should I actually use?

A: Set retries high enough to cover your longest plausible maintenance window — 72 hours is a common recommendation. The point isn't to never fail; it's to fail on a timeline you control, not one the kernel decides for you.

Q: Are you seriously saying hard mounts are always wrong?

A: No. Hard mounts make sense in narrow cases where data integrity absolutely must block availability — like write-once archival storage. But for 90% of production workloads, the ability to fail and recover beats freezing indefinitely. Hard mounts are the default, not the exception, and that's the problem.

📎 Source: View Source