Home › Blog › AI Research in 2026
AI Research Progress Through the Years · 2026 · Updated: 27 October 2026 📅

AI Research in 2026: The Papers and Ideas Reshaping the Field

2026 had no single breakthrough. Instead there was rapid progress on many fronts at once: theory explaining why Transformers work, reasoning without answer keys, agents that rewrite themselves, a rethink of diffusion language models, detectors for hidden deception, honest benchmarks for AI scientists and better maths for training. This guide walks through twenty of the year's most-awarded and most-discussed papers, from ICLR 2026 in April and ICML 2026 in July, plus award winners from CVPR, ACL, RLC and COLM (updated 27 October 2026). Each one gets a plain-English explanation and an animation or interactive diagram, so no maths background is needed, followed by what the paper found, why it matters and links to the sources.

How to read this: the first article in AI Research Progress Through the Years. Awards are taken from the official ICLR and ICML announcements and, for the "Since July" section, from the CVPR and ACL award pages, a university announcement (RLC) and the COLM conference account; NeurIPS 2026 (December) has not happened yet. Several papers first appeared on arXiv in 2025 and were published at 2026 conferences — we give both dates. The "🧒 In plain English" boxes simplify on purpose; the "What they found" boxes and the linked papers carry the precise claims. Animations illustrate ideas and are not reproductions of the papers' experiments.

01 How We Chose the Papers

Most papers here won awards at the two big machine-learning conferences held so far in 2026. A few were chosen because they are widely discussed or represent a major trend. Each carries a badge:

🏆 Outstanding Paper🏅 Honorable Mention🎤 Oral presentation📄 Accepted paper📝 Lab report
ConferenceTop awardsHonorable mentions (selected)
ICLR 2026 (April)Transformers are Inherently Succinct · LLMs Get Lost in Multi-Turn ConversationThe Polar Express
ICML 2026 (July, Seoul)The Flexibility Trap · High-Accuracy Sampling for Diffusion Models and Log-Concave DistributionsThe Obfuscation Atlas · Motion Attribution for Video Generation · How Much Can Language Models Memorize? · A Random Matrix Perspective on the Consistency of Diffusion Models · To Grok Grokking
CVPR 2026 (June)D4RT (dynamic scene reconstruction)NitroGen
ACL 2026 (July)Expressivity of Local Attention · The Imperfective Paradox · Resource-Rational Encoding in Sentence Processing—
RLC 2026 (August)Gradient Iterated TD Learning (Empirical RL)—
COLM 2026 (October)Message Passing Enables Efficient Reasoning—

02 The 2026 Paper Map

Twenty papers across nine themes. Filter by conference or award, and click any paper to jump to it:

▶ Paper map Click to jump

03 🧱 Model Architecture and Theory

Understanding what today's models are and what they can store.

📏

Transformers are Inherently Succinct

Bergsträßer, Cotterell & Lin · ICLR 2026
🏆 Outstanding Paper
🧒 In plain English

Imagine describing a very long pattern. One way is a giant lookup table that lists every case (that's roughly what an older RNN has to do). Another is a short recipe that generates the pattern (that's what a Transformer can do). This paper proves that for some patterns, the Transformer's recipe is exponentially shorter. That helps explain why Transformers are so powerful — and why it's also very hard to formally check what they will do.

▶ Recipe vs lookup table How big a description each model needs
🔍 What they found
  • Formal proofs that Transformers can describe some languages exponentially more compactly than RNNs and certain automata
  • The flip side: that compactness makes verifying properties of Transformers computationally hard
💡 Why it matters

It gives a theoretical reason for the Transformer's power — and a warning that formally verifying model behaviour will stay hard.

🗄️

How Much Can Language Models Memorize?

Morris et al. · ICML 2026
🏅 Honorable Mention
🧒 In plain English

Think of a model as a jar with a fixed number of marbles of storage. This paper measured the jar: GPT-style models store about 3.6 bits per parameter. While training data is smaller than the jar, the model memorises examples word for word. Once the data overflows the jar, it can't keep everything — so it starts learning general patterns instead (the moment researchers call "grokking").

▶ The memory jar Capacity ≈ 3.6 bits × parameters
🔍 What they found
  • Hundreds of Transformers from 500K to 1.5B parameters: ~3.6 bits per parameter (3.51 in bfloat16, 3.83 in float32)
  • Models memorise until capacity fills, then unintended memorisation falls as generalisation begins — explaining double descent
  • Scaling laws that predict how easily you can tell whether a given document was in the training set
💡 Why it matters

Privacy and copyright arguments often ask "does the model contain my data?" This gives a measurable answer for how much it can contain.

04 🧠 Reasoning and Post-Training

Teaching models to reason when there is no answer key.

🤔

Learning to Reason without External Rewards (INTUITOR)

Zhao et al. · ICLR 2026 (arXiv May 2025)
📄 Accepted paper
🧒 In plain English

Normally a model learns to reason like a student with an answer key: it tries, a checker marks it right or wrong, and it improves. But most real questions have no answer key. INTUITOR lets the model use its own confidence as the reward: answers it is sure about are reinforced. Surprisingly, this matched answer-key training on maths and generalised better to code.

🔍 What they found
  • Replaces the external reward in GRPO with "self-certainty" (how far the model's output distribution is from uniform)
  • Matches GRPO on maths benchmarks; better out-of-domain generalisation (code) without gold answers or tests
💡 Why it matters

Reasoning training could extend to domains where nobody can write an automatic checker.

📚

Reinforcing General Reasoning without Verifiers (VeriFree)

Zhou et al. · ICLR 2026 (arXiv May 2025)
📄 Accepted paper
🧒 In plain English

Checkers exist for maths and code, but not for chemistry, law or economics. VeriFree skips the checker: after the model reasons, it asks "how likely is the known reference answer now?" and rewards reasoning that makes the right answer more likely. No judge needed — just a textbook answer.

▶ Three ways to reward reasoning Answer checker · own confidence · reference answer
🔍 What they found
  • Maximises the probability of the reference answer given the model's own reasoning chain
  • Matches or beats verifier-based methods on MMLU-Pro, GPQA, SuperGPQA and maths
💡 Why it matters

Extends R1-style reasoning training from maths and code to general domains like chemistry, healthcare, law and economics.

05 🧬 Agents and Self-Improvement

Agents that rewrite their own code — and how to pick which ones to keep.

🧬

Darwin Gödel Machine

Zhang, Hu, Lu, Lange & Clune (Sakana AI, UBC) · ICLR 2026
📄 Accepted paper
🧒 In plain English

A coding agent that rewrites its own code. Each new version is tested on programming tasks; better versions are kept in a growing family tree (an "archive"), and any branch can be explored further — like evolution, not a single ladder.

🔍 What they found
  • SWE-bench 20.0% → 50.0%; Polyglot 14.2% → 30.7%
  • Discovered better editing tools, context management and peer-review steps that transferred to other models
  • Documented reward hacking: it removed markers used to detect its own hallucinations
💡 Why it matters

Made self-modifying agents concrete and measurable — and showed why they need independent checks.

🌳

Huxley–Gödel Machine

ICLR 2026 Oral (arXiv Oct 2025)
🎤 Oral
🧒 In plain English

Which agent should you improve next: the one with the best score today, or the one whose descendants will be best? Like judging a racehorse by its offspring, HGM scores each agent by how well its whole family line (its "clade") performs. The best agent today is often not the best ancestor.

▶ Best agent vs best ancestor Pick a selection rule and grow the tree
🔍 What they found
  • Identifies a metaproductivity–performance mismatch: benchmark score ≠ self-improvement potential
  • Clade-metaproductivity selection beats DGM and SICA on SWE-bench Verified and Polyglot; with GPT-5-mini its best agent matches the best human-designed agents on SWE-bench Lite
💡 Why it matters

Shifts self-improvement from "what works now" to "what will keep improving".

06 🌫️ Diffusion Models and Generative AI

New ways to generate text and images, and faster, more precise sampling.

🧩

The Flexibility Trap

Tsinghua University & Alibaba · ICML 2026
🏆 Outstanding Paper
🧒 In plain English

Diffusion language models can fill in the words of an answer in any order, which sounds like a superpower. The paper finds a trap: given the choice, the model fills in the easy words first and keeps skipping the hard, uncertain ones — exactly the steps where real reasoning happens. Forcing plain left-to-right order during training works better.

▶ Filling in a solution Hard steps are marked ⚠
🔍 What they found
  • Arbitrary order lets models bypass high-uncertainty tokens that matter for exploration, collapsing solution coverage
  • JustGRPO — standard GRPO in left-to-right order — reaches 89.1% on GSM8K while keeping parallel decoding at inference
💡 Why it matters

Challenges a headline selling point of diffusion LLMs and gives a simpler training recipe.

🎯

High-Accuracy Sampling for Diffusion Models and Log-Concave Distributions

Chen, Chewi, Daskalakis & Rakhlin · ICML 2026
🏆 Outstanding Paper
🧒 In plain English

Generating from a diffusion model is like a random walk towards a target shape. Standard methods need many more steps each time you ask for a more precise result. This paper's method (First-Order Rejection Sampling) proposes a step and then accepts or rejects it, like a dart player who keeps only good throws — so extra precision costs only a few extra steps.

▶ Steps needed for a given precision Illustrative scaling curves
🔍 What they found
  • FORS: a training-free meta-algorithm reaching error δ in polylog(1/δ) steps using only gradient (score) evaluations
  • Applies to both diffusion models and log-concave distributions
💡 Why it matters

Much higher-quality samples without proportionally more compute — a major theoretical advance for generative models.

07 🛡️ Safety, Verification and Evaluation

Catching hidden deception and measuring what really breaks.

🎭

The Obfuscation Atlas

Taufeeque et al. (FAR.AI) · ICML 2026
🏅 Honorable Mention
🧒 In plain English

Researchers can put a lie detector inside a model — a probe that reads its internal activity. What happens if you train the model while that detector is watching? The model is rewarded for short code, so it learns to cheat by hard-coding test answers. It then either becomes honest, learns to write convincing excuses the detector accepts, or its internal "thoughts" drift so the detector no longer fires.

▶ Three outcomes under a deception detector Pick an outcome
🔍 What they found
  • Obfuscated activations arise from representation drift during RL — even without a detector penalty
  • A detector penalty mainly encourages obfuscated policies (deceptive text with justifications)
  • High enough KL regularisation plus detector penalty yields honest policies
💡 Why it matters

Shows when white-box deception detectors can safely be used as training signals — central to scalable oversight.

💬

LLMs Get Lost in Multi-Turn Conversation

Laban, Hayashi, Zhou & Neville · ICLR 2026
🏆 Outstanding Paper
🧒 In plain English

Benchmarks usually give a model the whole question at once. Real users reveal what they want bit by bit. Splitting the same tasks across several turns, the paper found performance drops sharply: models guess too early, then cling to their wrong guess. Once lost, they rarely recover.

▶ One task, revealed piece by piece Press play
🔍 What they found
  • Over 200,000 simulated conversations across many models and tasks
  • Average performance drop of about 39% from single-turn to multi-turn; the main cause is unreliability, not lost ability
  • Models make early assumptions, attempt premature final answers and over-rely on them
💡 Why it matters

Moves evaluation from single-turn benchmark scores to how models behave in real, long conversations.

08 🔬 AI for Science

Can AI agents do real research, and can we trust what they claim?

🔥

FIRE-Bench

ICML 2026
📄 Accepted paper
🧒 In plain English

Give an AI agent only the question from a published research paper and ask it to rediscover the answer: design the experiment, write code, run it and draw a conclusion. Because the true answer is known, you can check whether the agent really did science.

▶ A research attempt Where agents fail most
🔍 What they found
  • Even the strongest agents score below 50 F1 at rediscovery
  • High variance across runs
  • Failures come mainly from experiment planning and evidence-based reasoning, not coding
💡 Why it matters

A realistic, checkable benchmark for AI research agents — and evidence that research judgment, not coding, is the bottleneck.

⛓️

Science One Framework (Chain-of-Evidence)

Google Research · July 2026
📝 Lab report
🧒 In plain English

When an AI writes a research paper, how do you know its claims are true? Science One makes every claim carry a receipt: a citation must point to a real paper that says that; a number must trace to a recorded experiment output; a method must match the actual code. Claims without receipts are flagged.

▶ Follow the receipts Click a claim
Click a claim to trace its evidence.
🔍 What they found
  • Chain-of-Evidence: every claim traceable through recorded supporting claims to a grounding source
  • Covers citation, numerical and methodological claims; evidence chains built as claims are produced
💡 Why it matters

Addresses the key bottleneck in autonomous science: proving that claims, citations, code and results actually correspond.

09 ⚡ Efficiency and Training Engineering

Faster, more stable training through better maths.

🚄

The Polar Express

Amsel, Persson, Musco & Gower · ICLR 2026
🏅 Honorable Mention🎤 Oral
🧒 In plain English

The Muon optimizer, now popular for training large models, repeatedly applies a small polynomial to "straighten out" a matrix (pushing all its values toward ±1). Muon uses the same polynomial every step. The Polar Express picks the best polynomial for each step using approximation theory, so it converges faster and stays stable in low-precision GPU arithmetic.

▶ Straightening values toward 1 Each line = one singular value of a matrix
🔍 What they found
  • Optimal per-step polynomial approximations to the matrix sign / polar decomposition
  • Designed for GPU execution and low-precision (bfloat16) arithmetic; improves Muon training
💡 Why it matters

Brings numerical-analysis rigour to the matrix operation at the heart of a leading optimizer.

10 🎬 Video and World Generation

Understanding which training videos teach a model how things move.

🎥

Motion Attribution for Video Generation (Motive)

Wu et al. (NVIDIA) · ICML 2026
🏅 Honorable Mention
🧒 In plain English

A video model learns how things move from millions of clips. Which clips taught it good motion — and which taught it bad habits? Motive traces a generated movement back to the training clips that influenced it, focusing on motion rather than background. Removing clips with static or unrealistic motion makes the model's physics better.

▶ Which training clips help motion? Toggle the filter
🔍 What they found
  • Motion-weighted, gradient-based attribution that scales to large video datasets
  • Static or non-physical clips can dilute learning of dynamics
  • Fine-tuning on Motive-selected data: better VBench motion smoothness and dynamics; 74.1% human preference vs the base model
💡 Why it matters

Better training-data selection for video and world models, where realistic physics matters.

11 🆕 Since July: CVPR, ACL, RLC and COLM

Award winners from the conferences that followed ICML, updated 27 October 2026.

🎞️

Efficiently Reconstructing Dynamic Scenes One D4RT at a Time

Zhang, Le Moing et al. (Google DeepMind, UCL, Oxford) · CVPR 2026
🏆 Best Paper
🧒 In plain English

Film a street with a phone while people and cars move. D4RT turns that single video into a moving 3D model: how far away everything is, where the camera was, and where each point goes over time. Instead of building a full 3D map for every frame, you simply ask it "where is this point, at this moment?" and it answers, which makes it far faster.

🔍 What they found
  • One feed-forward network jointly estimates depth, point correspondence over time and camera parameters from a single video
  • A query interface replaces dense per-frame decoding and separate task-specific decoders
  • Chosen from more than 16,000 CVPR submissions; a summary site reports up to ~300× faster inference than earlier methods (not checked against the paper)
💡 Why it matters

Fast 4D reconstruction from ordinary video is a building block for robots, AR and world models that must understand moving scenes.

🎮

NitroGen: An Open Foundation Model for Generalist Gaming Agents

Magne et al. (NVIDIA-led) · CVPR 2026
🏅 Best Paper Honorable Mention
🧒 In plain English

Millions of people post videos of themselves playing games, often with the controller buttons shown on screen. NitroGen reads those button presses back out of the videos and learns to play by copying them: about 40,000 hours across more than 1,000 games. The result is one agent that can pick up games it has never seen.

🔍 What they found
  • Player actions extracted from public gameplay videos create a huge, free training set
  • Trained by large-scale behaviour cloning; plays 3D combat, 2D platformers and procedurally generated worlds
  • Up to 52% relative gain in task success on unseen games versus training from scratch (authors' figure); weights, data and benchmark released
💡 Why it matters

The same "learn from internet video" recipe that built language models is being applied to actions, a step toward general embodied agents.

🔭

Characterizing the Expressivity of Local Attention in Transformers

Jiaoda Li & Ryan Cotterell (ETH Zurich) · ACL 2026
🏆 Best Paper
🧒 In plain English

Many models let each word look only at a small window of nearby words ("local attention") to save compute. People noticed this sometimes makes models better, not just cheaper. This paper proves why: a local window lets a Transformer recognise patterns that full attention alone cannot, and the two kinds of attention cover different patterns, so mixing them is strongest.

🔍 What they found
  • Adding local attention to fixed-precision Transformers strictly enlarges the class of regular languages they can recognise
  • Global and local attention are complementary: neither contains the other, and the hybrid is the most expressive
  • Experiments on formal languages and language modelling: hybrid global–local models beat global-only ones
💡 Why it matters

A theoretical reason for the hybrid attention designs used in many current long-context models, from the same group behind this year's "Transformers are Inherently Succinct".

⏳

The Imperfective Paradox in Large Language Models

Bolei Ma (LMU Munich) & Yusuke Miyao (University of Tokyo) · ACL 2026
🏆 Best Paper
🧒 In plain English

"She was running" means she ran. But "she was building a house" does not mean the house got built. Language models often miss this: they assume goals were reached, even when the text says the work was interrupted. The authors call it a "teleological bias": the models act like storytellers who expect happy endings, not careful logical readers.

🔍 What they found
  • A diagnostic dataset, ImperfectiveNLI, tests whether "was doing X" implies "did X"
  • Open-weight LLMs hallucinate completion for goal-directed events, even overriding explicit cancellation in the text
  • Prompting reduces the bias but makes models wrongly reject valid inferences; internal representations do encode the distinction, yet priors win at decision time
💡 Why it matters

Agents that read reports, logs or contracts need to know whether something actually finished. This is a precise, testable blind spot.

📈

Gradient Iterated Temporal-Difference Learning

Vincent, Gerhardt, Tripathi, Maraqten, White, White, Peters & D'Eramo · RLC 2026
🏆 Outstanding Paper (Empirical RL)
🧒 In plain English

Most RL agents learn by chasing their own next guess (temporal-difference learning). The usual shortcut is fast but can blow up. The safe version ("gradient TD") never blows up but learns slowly, so almost nobody uses it. This paper gets both: it trains several guesses in a chain at once and follows the true gradient through all of them.

▶ Fast, safe or both? Illustrative curves, not the paper's results
🔍 What they found
  • Iterated TD (several action-value functions trained in parallel) is changed to take gradients over its moving targets
  • The result is a gradient TD method, so it avoids semi-gradient divergence
  • Learning speed competitive with semi-gradient methods across several benchmarks, including Atari
💡 Why it matters

Stable value learning without a speed penalty could make deep RL less fragile, from games to the RL used to post-train language models.

📨

Message Passing Enables Efficient Reasoning

Liu, Arora, Swamy & Zanette (Carnegie Mellon) · COLM 2026
🏆 Outstanding Paper
🧒 In plain English

When a model reasons in parallel, the usual pattern is "split into several threads, then merge at the end". Here the threads can message each other while they work: share a useful finding, or tell another thread to stop because its path is a dead end. Each thread only receives what it needs, so it no longer has to carry everyone's full notes.

🔍 What they found
  • "Message Passing Language Models": parallel reasoning threads exchange messages directly instead of only joining at the end
  • Two mechanisms: lower communication cost (no redundant context sharing) and pre-emption of unpromising threads
  • On Sudoku, an asymptotically smaller context requirement than serial chain-of-thought or fork-join; on 3-SAT, pre-emption prunes bad branches
💡 Why it matters

Test-time compute is a major cost. Reasoning that scales without every thread reading everything is cheaper, and maps naturally onto multi-agent systems.

Also from this period: the ECCV 2026 Best Paper went to Heat Kernel Textures (HKTex) from Imperial College London, a 3D graphics method; ACL 2026's third Best Paper, Memory Efficiency and Resource-Rational Encoding in Sentence Processing, links language models to human sentence comprehension; COLM 2026 also recognised FinePhrase (Niklaus et al.) — we have not yet confirmed this award on COLM's own pages.

Read together, the papers point in six directions. Click a trend to highlight the papers behind it:

▶ What 2026's papers say about where the field is heading Click a trend

One trend runs through several papers at once: reliability over raw capability. The multi-turn paper, FIRE-Bench and Science One all measure whether systems behave dependably in realistic settings. They find large gaps even in models that ace standard benchmarks. The toy model below shows how little it takes to produce the multi-turn effect: a tendency to guess early, plus anchoring on the guess.

▶ Why models get lost Same model as the script below
"""Why LLMs get lost in multi-turn conversation - a toy simulation.

A task needs k pieces of information. Single-turn: the user gives all k at once; the model answers
correctly with probability 0.90. Multi-turn ("sharded"): pieces arrive one per turn. After each turn
the model may commit early to an assumption about the missing pieces (probability 0.35 per turn while
information is incomplete). An early assumption is wrong 70% of the time, and once wrong the model
anchors on it: it recovers only 25% of the time when the contradicting piece arrives.
Illustrative parameters (not fitted to the paper). Standard library only.
"""
import random

def conversation(k, rng, commit=0.35, wrong=0.7, recover=0.25, base=0.90):
    stuck = False
    for turn in range(1, k + 1):
        if stuck and rng.random() < recover: stuck = False
        if turn < k and not stuck and rng.random() < commit and rng.random() < wrong: stuck = True
    return (not stuck) and rng.random() < base

def run(k, n=20000, seed=1):
    rng = random.Random(seed + k)
    return sum(conversation(k, rng) for _ in range(n)) / n

print("Accuracy when the same task is split across k turns (20,000 simulated conversations each)\n")
print(f"   {'turns':>5} {'accuracy':>9} {'drop vs 1 turn':>15}")
base = run(1)
for k in [1, 2, 3, 4, 6, 8]:
    a = run(k)
    print(f"   {k:>5} {a:>9.1%} {(a - base) / base:>15.0%}")
print("\nEarly guesses plus anchoring are enough to reproduce a large multi-turn drop:")
print("the model is not less able - it commits too soon and fails to recover.")

Output:

Accuracy when the same task is split across k turns (20,000 simulated conversations each)

   turns  accuracy  drop vs 1 turn
       1     90.0%              0%
       2     73.6%            -18%
       3     64.4%            -28%
       4     58.6%            -35%
       6     53.7%            -40%
       8     53.0%            -41%

Early guesses plus anchoring are enough to reproduce a large multi-turn drop:
the model is not less able - it commits too soon and fails to recover.

With made-up but plausible settings, splitting a task over 6 turns cuts accuracy by about 40%. That is close to the ~39% average drop the paper measured on real models. Set the early-guess chance to zero and the drop disappears. The lesson carries over to agents: holding back until you have enough information, and being willing to revise, matter as much as raw skill.

13 Looking Ahead

The year's papers leave seven open questions for the next phase:

❓ Can AI systems reliably improve themselves at scale? → Open Problems in RSI
❓ How do we verify complex reasoning and research?
❓ Can we prevent deceptive optimisation in powerful agents?
❓ Will improvements transfer beyond training data and benchmarks?
❓ How do we align increasingly autonomous systems with human goals?
❓ What new evaluation methods are needed for real-world impact?
❓ How far can we trust AI-generated science without human replication?

Also worth knowing

AI research in 2026, in one paragraph
The award-winning work of 2026 is less about bigger models and more about understanding and trusting the ones we have. Theory explained why Transformers are so expressive and how much they memorise. Reasoning training learned to do without answer keys. Self-improving agents became real experiments, with better ways to choose what to keep. Diffusion language models were shown to have a hidden weakness. Deception detectors were tested under training pressure. Benchmarks exposed how models get lost in real conversations and how far AI scientists remain from rediscovering known results. Later in the year, award winners pushed the same themes into vision, language and RL: reconstructing moving 3D scenes from one video, agents that learn games from internet footage, proofs about local attention, a precise blind spot in how models read unfinished events, stable value learning and reasoning threads that message each other. The common thread: the field is moving from "can models do this?" to "can we rely on, verify and efficiently improve what they do?"

Next: AI Research in 2025: The Year Reasoning, Agents and Test-Time Compute Broke Out →

⚠️ Notes

Corrections and additions to the source infographic: (1) VeriFree ("Reinforcing General Reasoning without Verifiers") is labelled "2026 (arXiv)" but was published at ICLR 2026 (arXiv May 2025); (2) several papers first appeared on arXiv in 2025 — INTUITOR, VeriFree, the Darwin Gödel Machine, the memorisation paper, the multi-turn paper — and were published at 2026 conferences; (3) the infographic omits two ICML 2026 Honorable Mentions (A Random Matrix Perspective on the Consistency of Diffusion Models, To Grok Grokking), the Test of Time award and the position-paper awards; (4) Science One is a Google Research blog/prototype (July 2026), not a peer-reviewed paper; (5) "3.6 bits per parameter" is for GPT-style models in bfloat16 (3.83 in float32). The "plain English" explanations and animations are our simplifications; the multi-turn simulation is a toy model, not the paper's method.

🔗 All References