Are LLMs a dead end on the path to AGI? Mimicking human speech and ideas is impressive, but they’re unable to perform complex reasoning due to the underlying architecture.
An LLM is trained to predict the next token in a stream of text. Its entire reference pool is a symbol soup cooked by real humans. They’re great at predicting how words follow words, but there is no abstract understanding or reasoning beneath the language.
Conversely, the Joint Embedding Predictive Architecture (JEPA) builds an underlying model of reasoning and interpretation based on its data. Rather than predicting raw output — the next word, or every pixel of the next visual frame — JEPA predicts in representation space. It encodes a scene into an abstract latent / concept, then predicts what comes next.
That distinction sounds arbitrary until you see the difference in output. A system that predicts abstract future states can reason about consequences, cause and effect. It can ask what state results if I take this action, and that capacity for internal simulation is precisely what an LLM cannot do.
The LeWorldModel paper from Yann LeCun’s group is the first JEPA to train “stably end-to-end from raw pixels”, using roughly 15 million parameters on a single GPU with impressive speed. They raised over $1 billion for AMI Labs at a $3.5 billion valuation to prove this vision, reported as Europe’s largest seed round ever.
This is the strongest the case for JEPA has ever looked. It is also where the honesty has to begin.
The architecture is not a solved problem, and the most candid voice on its flaws is LeCun himself. He has described JEPA’s pull toward a trivial solution by admitting the system “happily wants to collapse”. Left unchecked, a latent predictor finds the laziest answer available: it maps every input to the same constant. The prediction loss drops to zero while the model learns nothing of value. The entire framework leans on regularisers that stop this from happening, and they are notoriously difficult to tune.
A paired benchmark found current world models collapse under minor visual shifts, and the identifiability result has so far been validated only in simulated environments.
So which path actually reaches AGI? The honest answer is that neither is there yet. The timeline is likely several years to a decade rather than months.
However — still in its infancy compared to LLMs — I believe JEPA’s ability to reason and understand abstraction will eventually have it overtake LLMs, and go beyond them to find the true path towards AGI.