Diffusion Models

We’ve been throwing massive compute at discrete token-based models, hoping to brute-force our way to AGI. But what if the very concept of the ‘token’ is the bottleneck? Continuous-latent diffusion language modeling proposes a radical shift: replacing rigid discrete tokens with continuous latent variables, potentially breaking the scaling plateau and blurring the lines between language and thought.