AI Research in 2022: The Year Reasoning, Alignment and Generative AI Converged
2022 laid the technical foundation for ChatGPT, reasoning models, agents and multimodal AI. Scaling became a science: Chinchilla showed models should grow with their data. Instruction tuning and RLHF turned raw models into assistants. Chain-of-thought made reasoning visible, and ReAct joined reasoning to action. Diffusion models set off the generative-image explosion. Flamingo and FlashAttention showed that multimodality and systems engineering mattered as much as size. This guide covers seventeen papers in eight research themes, each explained in plain English with an animation, followed by what it found, why it matters and its sources.
01 Venues and Awards
| Venue | Covered here |
|---|---|
| NeurIPS 2022 (New Orleans) | Outstanding Papers: Chinchilla ("An empirical analysis of compute-optimal LLM training") · Imagen. Also: InstructGPT · Chain-of-Thought · Minerva · Flamingo · FlashAttention |
| ICLR 2023 | ReAct (Oral) · Self-Consistency (both posted in 2022) |
| Other venues | Latent Diffusion (CVPR 2022) · SayCan (CoRL 2022) · RT-1 (RSS 2023) · AlphaCode (Science) · Emergent Abilities (TMLR) · PaLM and Flan (JMLR) · DALL·E 2 (preprint) |
02 Paper Map
Seventeen papers in eight themes. Filter and click to jump:
03 📈 Scaling Gets Smarter
More parameters alone → compute-optimal parameters + data → unexpected capabilities at scale.
PaLM: Scaling Language Modeling with Pathways
Chowdhery et al. (Google) · arXiv Apr 2022 · JMLR 2023PaLM was a 540-billion-parameter model trained across thousands of Google's TPU chips at once. At that size it could explain jokes, do multi-step reasoning with worked examples and write code — and on some tasks performance jumped sharply as the model grew.
- 540B dense decoder-only Transformer trained on 6,144 TPU v4 chips with the Pathways system
- Breakthrough few-shot results on reasoning, code and language tasks; some tasks showed discontinuous improvement from 62B to 540B
Showed what extreme scale plus good prompting could do — the base for Flan-PaLM and Minerva.
Chinchilla: Training Compute-Optimal Large Language Models
Hoffmann et al. (DeepMind) · NeurIPS 2022If you have a fixed amount of computing power, should you build a bigger model or train it on more text? DeepMind trained over 400 models to find out: grow both together. Their 70B Chinchilla used the same compute as the 280B Gopher but four times more data — and beat it.
- Over 400 models (70M–16B params, 5–500B tokens): parameters and tokens should scale in equal proportion
- Chinchilla (70B, 1.4T tokens) outperformed Gopher (280B), GPT-3 (175B) and Megatron-Turing NLG (530B) at the same compute as Gopher; 67.5% on MMLU
- Presented at NeurIPS 2022 as "An empirical analysis of compute-optimal large language model training" (Outstanding Paper)
Changed how every lab sized its models; the 20-tokens-per-parameter rule of thumb comes from here.
Emergent Abilities of Large Language Models
Wei et al. (Google, Stanford, DeepMind) · TMLR 2022Some skills seemed to be absent in small models and then switch on in large ones, like water suddenly boiling. The paper catalogued dozens of such "emergent" abilities — sparking huge interest in scaling (and, a year later, a debate over whether some were an illusion of how they were measured).
- Defines an ability as emergent if not present in smaller models but present in larger ones
- Catalogues emergent few-shot tasks and prompting strategies (e.g. chain-of-thought helps only above ~100B parameters)
Framed scaling research for years — counterpoint in 2023's Mirage paper.
04 🧑🏫 Instruction Following and RLHF Become the Product Layer
Pretrained model → SFT → human preferences → RLHF → useful assistant.
InstructGPT: Training Language Models to Follow Instructions with Human Feedback
Ouyang et al. (OpenAI) · NeurIPS 2022GPT-3 could write anything — but it didn't reliably do what you asked. OpenAI had people write good answers, rank model answers, trained a model of those rankings, and used it to steer GPT-3 with reinforcement learning. People preferred a 1.3B InstructGPT over the 175B GPT-3.
- Three stages: supervised fine-tuning on demonstrations, a reward model from human rankings, PPO against the reward
- Labelers preferred 1.3B InstructGPT outputs to 175B GPT-3 outputs on the API prompt distribution
- More truthful and less toxic, with small performance regressions on some benchmarks
The recipe behind ChatGPT, released later in 2022.
Scaling Instruction-Finetuned Language Models (Flan-PaLM)
Chung et al. (Google) · arXiv Oct 2022 · JMLR 2024Instead of teaching a model one task, teach it 1,800 different tasks, each phrased as an instruction ("Translate…", "Summarise…", "Explain step by step…"). It learns to follow instructions in general — including for tasks it never saw.
- Instruction tuning scaled to ~1.8K tasks, with and without chain-of-thought data
- Flan-PaLM 540B reached 75.2% five-shot on MMLU; Flan-T5 checkpoints released publicly
- Gains grew with both model size and number of tasks
Showed instruction tuning is cheap, general and complementary to RLHF.
05 💡 Reasoning Becomes Visible
Answer immediately → think step by step → explore several reasoning paths → quantitative reasoning.
Chain-of-Thought Prompting Elicits Reasoning in LLMs
Wei et al. (Google) · NeurIPS 2022Show the model a few examples where the answer comes with the working — "60 miles per hour for 3 hours is 60 × 3 = 180" — and it starts showing its own working too. For big models, that simple change makes maths word problems far more likely to be right.
- Eight worked examples in the prompt; no fine-tuning
- PaLM 540B with chain-of-thought reached state of the art on GSM8K, beating fine-tuned GPT-3 with a verifier
- Helps only at large scale (~100B parameters)
Made reasoning visible — the ancestor of every reasoning model.
Self-Consistency Improves Chain of Thought Reasoning
Wang et al. (Google) · arXiv Mar 2022 · ICLR 2023Ask the model to reason through the same problem many times, then take a vote on the final answers. Wrong paths tend to disagree with each other; right ones tend to agree.
- Sample diverse chain-of-thought paths, then marginalise to the most consistent answer
- Large gains: GSM8K +17.9%, SVAMP +11.0%, AQuA +12.2% absolute (with PaLM 540B / GPT-3)
The first test-time-compute trick: spend more samples, get more accuracy.
Minerva: Solving Quantitative Reasoning Problems with Language Models
Lewkowycz et al. (Google) · NeurIPS 2022Take PaLM and let it read a lot of maths and science — arXiv papers and web pages with equations kept intact. Without any calculator, it began solving competition-level maths problems step by step.
- Continued training of PaLM on 118GB of scientific papers and maths web pages
- 50.3% on the MATH benchmark with majority voting (previous best ~6.9%); strong results on MMLU-STEM and university-level problems
Showed domain data plus chain-of-thought could unlock quantitative reasoning.
06 🦾 Reasoning Meets Action
Chain-of-thought + external actions + observations = the first agent loop.
ReAct: Synergizing Reasoning and Acting in Language Models
Yao et al. (Princeton, Google) · arXiv Oct 2022 · ICLR 2023Instead of only thinking, or only acting, the model alternates: think about what to do, take an action (like searching Wikipedia), read what comes back, think again. Every modern agent loop descends from this.
- Interleaves reasoning traces with actions and observations
- Reduces hallucination on HotpotQA and FEVER by grounding in Wikipedia lookups
- ALFWorld and WebShop: +34% and +10% absolute success over imitation/RL baselines with only one or two examples
The template for agent harnesses from LangChain to today's coding agents.
SayCan: Do As I Can, Not As I Say
Ahn et al. (Google, Everyday Robots) · CoRL 2022Ask a robot "I spilled my drink, can you help?" A language model knows what sounds useful (find a sponge); the robot knows what it can actually do from where it stands. SayCan multiplies the two scores to pick each next step.
- Combines LLM task-grounding scores with learned value functions (affordances)
- 84% planning success and 74% execution success on 101 real kitchen instructions
Grounded language models in the physical world.
RT-1: Robotics Transformer for Real-World Control at Scale
Brohan et al. (Google) · arXiv Dec 2022 · RSS 2023Apply the foundation-model recipe to robots: collect a huge dataset of real robot demonstrations and train one Transformer on all of it. The result follows hundreds of instructions and copes better with new objects and settings.
- 130k episodes covering 700+ tasks, collected with 13 robots over 17 months
- 97% success on seen instructions; better generalisation to new tasks, distractors and backgrounds than baselines
Set up RT-2's vision-language-action models in 2023.
07 💻 Code Generation Becomes Serious Problem Solving
Generate → sample massively → execute and filter → select.
AlphaCode: Competition-Level Code Generation
Li et al. (DeepMind) · Science 2022Competitive programming needs real problem-solving, not autocomplete. AlphaCode writes up to a million candidate programs, throws out those that fail the example tests, groups the rest by behaviour and submits a handful — reaching about the median human competitor.
- Massive sampling, filtering on example tests, and clustering by program behaviour
- Average ranking in the top 54.3% in simulated Codeforces contests
An early version of generate-and-verify — before interactive coding harnesses.
08 🎨 The Diffusion Generative-AI Explosion
Pixel diffusion → latent diffusion → language-conditioned diffusion → consumer generative AI.
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach et al. (LMU Munich, Runway) · CVPR 2022Diffusion models create images by removing noise step by step — slow and expensive on millions of pixels. Latent diffusion first compresses the image into a much smaller code, does the denoising there, then expands it back. Much cheaper. This became Stable Diffusion.
- Autoencoder compresses images; diffusion runs in its latent space
- Cross-attention conditioning on text and other inputs
- Large compute savings at high resolution; basis of Stable Diffusion
Put high-quality image generation on consumer GPUs.
DALL·E 2: Hierarchical Text-Conditional Image Generation with CLIP Latents
Ramesh et al. (OpenAI) · arXiv Apr 2022DALL·E 2 works in two steps: first turn your text into an idea of an image (a CLIP image embedding), then paint that idea with a diffusion decoder. It produced strikingly realistic and creative pictures and brought text-to-image to the public.
- A prior maps CLIP text embeddings to CLIP image embeddings; a diffusion decoder generates the image
- Improved diversity with comparable photorealism; enabled variations and edits
Brought generative AI into mainstream awareness in 2022.
Imagen: Photorealistic Text-to-Image Diffusion with Deep Language Understanding
Saharia et al. (Google) · NeurIPS 2022Imagen's surprise: to make better pictures from words, improve the part that reads the words. A big frozen text model (T5-XXL) mattered more than a bigger image model.
- Frozen T5-XXL text encoder + cascaded diffusion models
- Zero-shot FID of 7.27 on COCO; human raters preferred it on DrawBench
- Scaling the text encoder improved fidelity and alignment more than scaling the diffusion model
Linked language understanding to image quality — a NeurIPS 2022 Outstanding Paper.
09 🦩 Multimodal Foundation Models
Vision encoder + language model + interleaved image/text → few-shot multimodal intelligence.
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac et al. (DeepMind) · NeurIPS 2022Show Flamingo a couple of examples mixing pictures and text — "[cat photo] this is a cat; [dog photo] this is a dog" — and it handles new images the same way. No retraining, just a few examples, like GPT-3 but with pictures and video.
- Frozen vision encoder and frozen language model bridged by a Perceiver Resampler and gated cross-attention layers
- 80B model set few-shot state of the art on many image and video benchmarks, sometimes beating fine-tuned models
The template for multimodal assistants like GPT-4V and Gemini.
10 ⚡ Systems Engineering Becomes a Research Frontier
Algorithms + architecture + GPU-aware engineering, not just bigger parameter counts.
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Dao, Fu, Ermon, Rudra & Ré (Stanford) · NeurIPS 2022The slow part of attention isn't the maths — it's moving data between the GPU's big slow memory and its small fast memory. FlashAttention works on small tiles that fit in fast memory and never writes the huge attention matrix out. Same exact result, much faster, far less memory.
- IO-aware exact attention with tiling and recomputation; memory linear in sequence length
- 15% faster BERT-large training vs the MLPerf 1.1 record; 3× faster GPT-2; enabled longer contexts
Now standard in virtually every LLM training and inference stack.
11 Chinchilla's Law, Computed
Chinchilla's fitted loss formula is simple enough to explore directly. Here it is applied to Gopher's compute budget: the same FLOPs spent on a smaller model and more data give a lower predicted loss, and the optimal split keeps shifting toward more tokens as budgets grow:
"""Chinchilla in a few lines: bigger model, or more data?
Hoffmann et al. (2022) fitted final pretraining loss as
L(N, D) = E + A / N^alpha + B / D^beta
(N = parameters, D = training tokens) with E = 1.69, A = 406.4, B = 410.7, alpha = 0.34, beta = 0.28
("Approach 3"; this fit favours smaller models than the paper's other two
methods, and a 2024 replication by Besiroglu et al. refitted it). Training compute is about C = 6 N D FLOPs. For a fixed budget we try splits of C
between model size and data and find the loss-minimising one. Standard library only.
"""
E, A, B, ALPHA, BETA = 1.69, 406.4, 410.7, 0.34, 0.28
def loss(n, d):
return E + A / n ** ALPHA + B / d ** BETA
def best_split(c):
best = None
for i in range(400):
n = 10 ** (8 + i * 0.01) # 1e8 .. 1e12 parameters
d = c / (6 * n)
l = loss(n, d)
if best is None or l < best[2]: best = (n, d, l)
return best
gopher = (280e9, 300e9); chinchilla = (70e9, 1.4e12)
c = 6 * gopher[0] * gopher[1]
print(f"Same compute budget: {c:.2e} FLOPs")
print(f" Gopher 280B params, 300B tokens -> predicted loss {loss(*gopher):.3f}")
print(f" Chinchilla 70B params, 1.4T tokens -> predicted loss {loss(*chinchilla):.3f}")
n, d, l = best_split(c)
print(f" Loss-optimal split under this fit: {n / 1e9:.0f}B params, {d / 1e12:.1f}T tokens, loss {l:.3f}")
print(f"\n{'compute (FLOPs)':>16} {'optimal params':>15} {'optimal tokens':>15} {'tokens/param':>13}")
for c in [1e21, 1e22, 1e23, 1e24, 1e25]:
n, d, _ = best_split(c)
print(f"{c:>16.0e} {n / 1e9:>14.1f}B {d / 1e9:>14.0f}B {d / n:>13.0f}")
print("\nParameters and tokens should grow together: Gopher was over-sized and under-trained.")Output:
Same compute budget: 5.04e+23 FLOPs
Gopher 280B params, 300B tokens -> predicted loss 1.993
Chinchilla 70B params, 1.4T tokens -> predicted loss 1.937
Loss-optimal split under this fit: 30B params, 2.8T tokens, loss 1.936
compute (FLOPs) optimal params optimal tokens tokens/param
1e+21 1.8B 92B 50
1e+22 5.1B 325B 63
1e+23 14.5B 1153B 80
1e+24 41.7B 3998B 96
1e+25 117.5B 14186B 121
Parameters and tokens should grow together: Gopher was over-sized and under-trained.The exact optimum depends on which of the paper's three fitting methods you use; this one ("Approach 3") favours smaller models, and later work refitted it. The direction is robust, though: Gopher was over-sized and under-trained, and later open models such as LLaMA (2023) went further, training small models on far more tokens because a smaller model is cheaper to serve.
12 The 2021 → 2026 Roadmap
From scaling to self-improving systems. Press play:
Next: AI Research in 2021: The Year the Foundation-Model Stack Emerged →