You’ve seen the GitHub repos. You’ve read the benchmarks. A new Rust key-value store drops, promising “insane speed” and “durability.” The stars climb. The hype builds. We all love a beautifully engineered Rust project that pushes hardware to its limits.
TurboKV is exactly that. It boasts a persisted Bloom-filter format using hardware AES and built-in LZ4 compression. It is a masterclass in optimizing the fast path.
But then you read the comments section. That’s where the real engineering happens.
Someone spots DbOptions::durable(). They look at the implementation. It appends to the Write-Ahead Log (WAL) without a per-write sync.
In the storage engine arms race, speed is the marketing budget, but durability is the invoice.
What does “durable” actually mean here? It means the data survives a process restart. If your application crashes, the OS page cache still holds your data, and TurboKV can recover it. But if the power cord is yanked? The OS never flushed that page cache to the physical disk. Your data vanishes.
This is the critical distinction between a cache and a database. A cache survives process restarts. A database survives power loss.
A hardware-accelerated Bloom filter is completely irrelevant when a power outage turns your WAL into vapor.
The commenters gleefully tear into this contradiction. One simply writes: “Oops you built a database!” It’s the classic developer hubris. You start by building a fast cache. You add a WAL for safety. You add compression for efficiency. Suddenly, you’re maintaining a database—except you don’t have the decades of battle-testing that Postgres or RocksDB have.
If you are evaluating embedded KV stores, you must check the durability semantics under power loss, not just process restart. Marketing language like durable() can be dangerously misleading.
And don’t let the language fool you. Rust’s memory safety guarantees won’t save you from a semantic design flaw. The compiler ensures you don’t segfault; it does not ensure your data hits the physical NAND.
Rust makes memory safety a breeze, but it makes data durability exactly as hard as it’s always been.
The hardest problem in databases isn’t speed. Speed is just SIMD instructions and compression algorithms. The hardest problem is knowing what is actually durable. And when you sacrifice fsync for the sake of a benchmark, you haven’t built an insanely fast database. You’ve just built a cache that lies.
FAQ
Q: Doesn't appending to the WAL still write to the OS page cache?
A: Yes, and that's exactly the problem. The OS page cache is in RAM. If you lose power, RAM is wiped. Without an explicit fsync call to force the OS to write to the physical disk, your 'durable' data dies with the power supply.
Q: Should I ever use TurboKV?
A: Use it as a cache or for ephemeral data where a power-loss wipe is acceptable. Do not use it as a primary database if the data actually matters.
Q: Is fsync just too slow for modern systems?
A: No, fsync is exactly as slow as the physical hardware requires. Blaming fsync for bad benchmarks is like blaming gravity for a plane crash. It's the fundamental reality of persistent storage.