You’ve probably stared at your database query, waiting for it to spin up, wondering why the very system designed to make code run faster is taking its sweet time to get off the ground. We’ve all been there, watching the latency metrics spike because the JIT compiler is busy “optimizing.”
We’ve been sold a lie about Just-In-Time compilation: we’re told it’s “just in time,” but in reality, it’s usually just too late.
The traditional approach—like the LLVM-based JIT in PostgreSQL—feels like building a custom racecar engine while the driver is waiting at the starting line. It’s bloated, it’s slow, and as one frustrated developer recently pointed out, it’s exactly why people lament Postgres’s JIT performance. The paradox is maddening: the tool meant to eliminate startup latency is the very thing causing it.
But what if we didn’t need to build the engine at all?
Enter copy-and-patch compilation. It sounds like a hack, and it is—the best kind of hack. Instead of running expensive optimization passes at runtime, you pre-compile a massive library of machine code templates. When the code is needed, you don’t compile from scratch. You just copy the right template and patch in the specific variables.
True speed isn’t building the engine while driving; it’s having the engine pre-assembled and just slotting it into the chassis.
The result? JIT compilation in 5 microseconds. Not 5 milliseconds. 5 microseconds. That number should make you do a double-take. It’s a figure that feels almost impossible at the bleeding edge of systems performance, triggering a curiosity about what else we’ve been doing the hard way.
For anyone working on databases, runtime systems, or latency-sensitive applications, this isn’t just a neat trick. It eliminates the traditional trade-off between startup time and execution speed. You get the raw power of machine code without the startup penalty. New architectures that were previously impractical are suddenly on the table.
Skeptics will argue that JIT compilation is inherently insecure, or that we should just rely on older paradigms like Common Lisp where the programmer manually dictates what gets compiled. But that misses the forest for the trees. We don’t need more manual control; we need invisible, instantaneous execution.
The future of runtime performance isn’t about doing more work faster; it’s about doing less work, period.
Copy-and-patch redefines the design space for runtime code generation. It proves that we don’t need full optimization passes to get efficient code. We just need to be smarter about what we pre-compute.
The bottleneck is gone. The only question left is: what will you build when your compiler takes less time than a CPU cache miss?
FAQ
Q: Isn't JIT compilation inherently insecure?
A: Security in JIT comes from sandboxing and verification, not from the compilation method itself. Copy-and-patch doesn't magically introduce vulnerabilities; it just changes how fast the code is assembled. The security model remains independent of the compilation speed.
Q: What's the practical implication of 5-microsecond JIT?
A: It means databases and runtime systems can finally generate optimized machine code on the fly without the dreaded startup penalty. You can dynamically optimize hot paths in real-time, even for short-lived queries or transient workloads where traditional JIT overhead would outweigh the benefits.
Q: Why is this better than traditional LLVM-based JIT?
A: LLVM is an incredible compiler infrastructure, but it's heavy. Running full optimization passes at runtime is like calling a construction crew to build a shed. Copy-and-patch is like snapping together prefabricated parts. It skips the expensive analysis and gets you running in microseconds instead of milliseconds.