Your RTX 4090 Is Being Held Back on Purpose
When you run LLMs on consumer RTX GPUs, vLLM and SGLang silently fall back to FlashAttention-2 — a kernel from 2022. Not because your hardware can’t handle FA-3/4, but because nobody bothered to port them. A first-principles rebuild of attention kernels proves the core techniques are architecture-agnostic, meaning you’re leaving real performance on the table every single inference call.