World Models: Predicting Consequences, Not Just Text

From a Debate Reference to a Real Technical Lineage

"The AGI Debate" cited Yann LeCun's world-model proposal as one side of the field's central disagreement, but treated it mostly as a position to be steelmanned rather than a technical subject to explain. This article does the explaining: world models have a real, decade-plus research history, distinct concrete implementations with published results, and a genuine (if contested) claim to solving problems the current recipe (Article 6's RL, Article 1's pretraining) doesn't obviously solve.

As with every open-bet article in this series, the honest framing matters: world models are a serious, active research direction with real results (Part 3, Part 4 below), not a fully proven replacement for the current dominant approach — this article documents what exists and how it works, without pre-declaring it the field's future.

Predicting Consequences, Not Predicting Text

Article 1's pretraining paradigm trains a model to predict the next token in text — a powerful, general-purpose signal, but one entirely about language statistics. A world model is trained on a different kind of prediction: given the current state of an environment and an action taken in it, predict what state comes next. This is a direct, learned analogue of Article 6's MDP framework (state, action, next state) — except instead of hand-designing the environment's rules, the model learns them from observed experience.

The practical payoff is planning: with a good enough learned model of "if I do X, Y will probably happen," an agent can mentally simulate many possible action sequences and pick a good one, before acting in the real world — rather than learning purely through costly trial and error (Article 6's Part 2 value functions, learned from real, executed actions).

Ha & Schmidhuber's "World Models" and the Model-Based RL Lineage

David Ha and Jürgen Schmidhuber's "World Models" (2018) is, appropriately, the paper that gave this whole research direction its name. Their agent has three components: a compressed visual encoding of what it sees, a learned predictive model of how that encoding evolves over time and in response to actions, and a small controller that picks actions based on the model's predictions. Remarkably, they showed a controller could be trained almost entirely inside the learned model's own "imagined" rollouts, then transferred successfully to the real environment — a striking early demonstration that a good enough learned simulator can substitute for costly real-world trial and error.

This built on an older idea in classical RL: Sutton's 1991 "Dyna" architecture, which alternated between learning from real experience and learning from a model's simulated experience, decades before deep learning made a rich, learned visual model of this kind practical.

Mastering Games Without Ever Being Told the Rules

DeepMind's MuZero (Schrittwieser et al., Nature, 2020) is the clearest, most rigorously validated demonstration of a world model's practical power. Unlike its predecessor AlphaGo (Article 6, Scenario B), which was given the actual rules of Go to plan with, MuZero is given no rules at all — it learns its own internal model of "what happens next" purely from experience, then uses that learned model to plan ahead via tree search (the same search-plus-value-estimate combination from Article 6), and matched or exceeded AlphaGo-level performance not just in Go but in Chess, Shogi, and Atari games with completely different rules, using the same general algorithm.

MuZero's result directly answers a natural skeptical question about world models: does a learned, imperfect model of an environment actually help, compared to just learning values or policies directly from real experience (Article 6, Parts 2 and 4)? The answer, at least in these game domains, was a clear yes.

"Dreaming" as a Training Strategy

Hafner et al.'s DreamerV3 (2023) pushes Ha & Schmidhuber's original idea to a much more general, robust extreme: an agent that learns a world model from a modest amount of real experience, then trains its policy almost entirely on imagined rollouts generated by that model — literally practicing inside its own head rather than the real environment — and does this successfully across a very wide range of different task domains without per-task tuning, including, notably, being the first algorithm to obtain diamonds in Minecraft from scratch without human-crafted guidance, a task requiring long, complex sequences of sub-goals.

This directly demonstrates why world models matter for sample efficiency (a theme from "The AGI Debate"): real-world interaction — a physical robot, a costly simulation, a human providing feedback (Article 6) — is expensive, while imagined rollouts inside a learned model are comparatively cheap once the model itself is trained.

Why LeCun's Proposal Avoids Predicting Pixels

LeCun's "A Path Towards Autonomous Machine Intelligence" (2022, previously cited in "The AGI Debate") proposes the Joint Embedding Predictive Architecture (JEPA) as a specific answer to a real weakness in naive world modeling: predicting raw future pixels or tokens forces a model to waste capacity on unpredictable, irrelevant detail (the exact texture of leaves rustling in the wind) instead of the actually meaningful, predictable structure (an object's trajectory). JEPA instead predicts in an abstract, learned representation space — predict what the compressed, meaningful representation of the future will be, not its literal pixel values.

Assran et al.'s I-JEPA (2023) is a concrete image-based implementation of this idea, and Meta AI's follow-up video-based V-JEPA work extends it to predicting future video representations — both real, published systems, though — consistent with this series' honest-uncertainty discipline — whether this specific architectural family will scale to match or surpass the current Transformer-plus-scaling recipe (Article 1, "After Transformers") on genuinely general tasks remains an open, actively contested question, not a settled result.

A System That Can Predict Consequences Raises the Stakes on Article 8

Key Papers to Read First

Courses to Complete

Real Scenario Walkthroughs

Self-Assessment Checklist

⚠️ What's Genuinely Uncertain

Where This Series Goes Next

Article 10 moves from world models to systems — the distributed-training and infrastructure engineering covered practically in Article 1's compute section, revisited here with more technical depth on the actual systems-design tradeoffs (data/tensor/pipeline parallelism, covered in Article 2's self-assessment, now explained mechanically). From there, a capstone article ties every foundational concept from all eleven articles back into one unified map.

🎥 Recommended Videos

🧭 Closing — A Real Research Direction, Not Yet a Settled Answer