AI Research in 2023: The Year Foundation Models Became Platforms
In 2023 large language models stopped being standalone models and became platforms people could build on. Frontier models went multimodal while open models closed the gap. Fine-tuning and alignment became cheap enough for one GPU. Reasoning became search. Models learned to use tools, keep memories, reflect and build skill libraries — the primitives of today's agent harnesses. Vision became promptable, language models started driving robots, and AI-found code shipped in the C++ standard library. This guide covers seventeen papers in eight research themes, each explained in plain English with an animation, followed by what it found, why it matters and its sources.
01 Venues and Awards
| Venue | Covered here | Notes |
|---|---|---|
| NeurIPS 2023 (December, New Orleans) | Outstanding: Are Emergent Abilities a Mirage? · Runners-up: DPO · Scaling Data-Constrained LMs · Outstanding D&B: DecodingTrust · Also: QLoRA, Tree of Thoughts, Toolformer, Reflexion, LLaVA | Other Outstanding: Privacy Auditing with One (1) Training Run; D&B: ClimSim |
| ICCV 2023 | Segment Anything — Best Paper Honorable Mention | — |
| UIST 2023 | Generative Agents — Best Paper | — |
| Other | AlphaDev (Nature) · RT-2 (CoRL 2023) · GPT-4, LLaMA, Llama 2, Voyager (technical reports and preprints) | — |
02 Paper Map
Seventeen papers in eight themes. Filter by award and click to jump:
03 🌐 Frontier Models and the Open-Model Explosion
Frontier multimodal models arrived — and open models closed the gap.
GPT-4 Technical Report
OpenAI · arXiv Mar 2023GPT-4 was a large model that could read both text and images and performed at a human level on many professional and academic exams. One quietly important idea: OpenAI trained small practice models first and used them to predict how well the huge model would do — before spending the compute on it.
- Multimodal (text + image input) Transformer, post-trained with RLHF
- Human-level performance on many exams (e.g. a simulated bar exam around the top 10% of test takers)
- Final loss predicted from models trained with up to 1/10,000th of GPT-4's compute
- Few architecture or training-data details disclosed
Set the frontier for 2023 and started the era of predictable scaling — and of closed technical reports.
LLaMA: Open and Efficient Foundation Language Models
Touvron et al. (Meta) · arXiv Feb 2023A bigger model isn't the only way to a better one: train a smaller model on more data. LLaMA's 13B model beat the 175B GPT-3 on most benchmarks, trained only on public data, and its weights went to researchers — igniting the open-model boom.
- Models from 7B to 65B trained on 1–1.4T tokens of publicly available data
- LLaMA-13B outperformed GPT-3 (175B) on most benchmarks; LLaMA-65B competitive with Chinchilla-70B and PaLM-540B
Sparked Alpaca, Vicuna and the open fine-tuning ecosystem.
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron et al. (Meta) · arXiv Jul 2023Llama 2 came in sizes from 7B to 70B, with ready-to-use chat versions and a licence allowing commercial use. The paper explained, unusually openly, how the chat models were aligned: supervised examples, then separate helpfulness and safety reward models guiding reinforcement learning.
- Pretrained on 2T tokens; 7B, 13B and 70B released (34B withheld)
- RLHF with separate helpfulness and safety reward models, rejection sampling and PPO
- Detailed safety evaluation and red-teaming discussion; commercially usable licence
Made open, commercially usable chat models the norm.
04 🔧 Efficient Fine-Tuning and the New Post-Training Stack
Pretrain → SFT/LoRA → preference optimisation became cheap and simple.
QLoRA: Efficient Finetuning of Quantized LLMs
Dettmers et al. (University of Washington) · NeurIPS 2023Fine-tuning a 65-billion-parameter model used to need a room full of GPUs. QLoRA freezes the model in compact 4-bit form and trains only small add-on adapters. The whole job fits on one 48GB GPU, with no loss in quality.
- 4-bit NormalFloat, double quantisation and paged optimisers
- Memory to fine-tune 65B cut from >780GB to <48GB without degrading 16-bit fine-tuning performance
- Guanaco reached 99.3% of ChatGPT's level on the Vicuna benchmark after 24 hours on one GPU
Put fine-tuning of big models within reach of individual researchers.
Direct Preference Optimization (DPO)
Rafailov, Sharma, Mitchell et al. (Stanford) · NeurIPS 2023Classic RLHF trains a separate reward model, then uses reinforcement learning to chase its scores — complicated and fragile. DPO shows you can skip both: given pairs of "preferred" and "rejected" answers, one simple training loss makes the model prefer the better ones directly.
- Shows the RLHF objective can be optimised exactly with a classification-style loss on preference pairs
- Matches or beats PPO-based RLHF on sentiment control, summarisation and dialogue while being far simpler and more stable
Became the default open-source alignment method; see its 2025 caveat in Learning Dynamics.
05 🌳 Reasoning Becomes Search, Not Just Prompting
From one linear chain of thought to search over reasoning paths.
Tree of Thoughts: Deliberate Problem Solving with LLMs
Yao et al. (Princeton, Google DeepMind) · NeurIPS 2023Chain-of-thought is like solving a maze by walking one path and hoping. Tree of Thoughts lets the model try several next steps, judge which look promising, explore those, and backtrack from dead ends. On the "Game of 24" puzzle, that raised GPT-4's success from 4% to 74%.
- Thoughts as intermediate states; the LM both proposes and evaluates them; BFS/DFS search with backtracking
- Game of 24: 74% success vs 4% for GPT-4 chain-of-thought; gains on creative writing and mini crosswords
The step from one reasoning trace to search over traces — ancestor of test-time compute scaling.
06 🧰 Tools, Memory and the Birth of Modern Agents
Tools, memory, reflection, planning and skill libraries — the primitives of today's harnesses.
Toolformer: Language Models Can Teach Themselves to Use Tools
Schick et al. (Meta) · NeurIPS 2023Language models are bad at arithmetic and don't know today's date — but a calculator and a calendar do. Toolformer teaches a model, with only a few examples, to decide when to call a tool, what to ask it, and how to use the answer — keeping only tool calls that actually help it predict text.
- Self-supervised: samples candidate API calls, keeps those that reduce loss on the following tokens
- Tools: calculator, Q&A system, search engine, translator, calendar
- A 6.7B model with tools often beat much larger models, without losing language ability
Introduced learned tool use — a core primitive of agent harnesses.
Reflexion: Language Agents with Verbal Reinforcement Learning
Shinn et al. · NeurIPS 2023When an agent fails, Reflexion doesn't retrain it. Instead the agent writes itself a note — "I forgot to handle empty input" — stores it in memory, and reads it before the next try. Learning happens in words, not weights.
- Verbal self-reflection stored in episodic memory; no weight updates
- HumanEval: 91% pass@1, above GPT-4's 80% at the time; gains on decision-making (AlfWorld) and reasoning (HotpotQA)
An early form of harness-level learning — improving through memory, not training.
Generative Agents: Interactive Simulacra of Human Behavior
Park et al. (Stanford, Google) · UIST 2023Twenty-five AI characters live in a little town. Each remembers what it sees, reflects on it and plans its day. Told only that one of them wanted to throw a Valentine's Day party, the others spread invitations, asked each other on dates and turned up together — none of it scripted.
- Architecture: memory stream, retrieval by recency/importance/relevance, reflection, planning
- Believable emergent social behaviour among 25 agents
- UIST 2023 Best Paper
The memory–reflection–planning loop behind many later agents.
Voyager: An Open-Ended Embodied Agent with LLMs
Wang et al. (NVIDIA, Caltech et al.) · arXiv May 2023A Minecraft agent that sets its own goals, writes code to achieve them, fixes the code using feedback from the game, and saves working code in a growing skill library. New skills build on old ones, so it keeps getting more capable without retraining.
- Automatic curriculum, executable skill library and iterative prompting with self-verification
- 3.3× more unique items, 2.3× longer distances and tech-tree milestones up to 15.3× faster than prior methods
Lifelong learning through a harness — skills as code.
07 👁️ Multimodal Foundation Models
Specialised vision model → promptable foundation model → conversational multimodal assistant.
Segment Anything (SAM)
Kirillov et al. (Meta) · ICCV 2023Point at anything in a photo — a dog, a cup, a cloud — and SAM outlines it instantly, even for objects it was never specifically trained on. Meta built it from the biggest segmentation dataset ever: over a billion outlines on 11 million images.
- Promptable segmentation: points, boxes or masks as prompts
- SA-1B dataset: >1B masks on 11M licensed, privacy-respecting images
- Strong zero-shot transfer; ICCV 2023 Best Paper Honorable Mention
The first real foundation model for image segmentation.
LLaVA: Visual Instruction Tuning
Liu, Li, Wu & Lee · NeurIPS 2023Connect an image encoder to a chatbot with a small "translator" layer, then teach it with conversations about images that GPT-4 wrote (from image captions). The result: an open assistant you can ask about any picture.
- Vision encoder (CLIP) + projection + LLM (Vicuna), trained on GPT-4-generated visual instruction data
- 85.1% relative score vs GPT-4 on a synthetic instruction benchmark; 92.53% on ScienceQA combined with GPT-4
Template for open vision-language assistants.
08 🤖 Language Models Enter the Physical World
Language model → multimodal model → action model.
RT-2: Vision-Language-Action Models
Brohan et al. (Google DeepMind) · CoRL 2023What if robot movements were just another kind of word? RT-2 writes arm motions as tokens, so a vision-language model trained on the web can also output actions. It can follow instructions it never saw in robot training — like "pick up the extinct animal" (the toy dinosaur).
- Actions represented as text tokens; co-fine-tuned on web vision-language data and robot trajectories
- Generalisation to unseen objects and scenes improved from 32% (RT-1) to 62%, over thousands of evaluation trials
- Emergent semantic reasoning, e.g. choosing an improvised tool
Made the path language model → multimodal model → action model concrete.
09 ⚙️ AI Starts Discovering Useful Algorithms
AI-found code enters real production software.
AlphaDev: Faster Sorting Algorithms via Deep RL
Mankowitz et al. (DeepMind) · Nature 2023Sorting a handful of numbers is one of the most-run pieces of code on Earth, polished by experts for decades. AlphaDev treated writing assembly code as a game and found a shorter sequence of instructions. The result was merged into the standard C++ library that billions of programs use.
- Algorithm discovery framed as a single-player game (AssemblyGame) solved with AlphaZero-style RL
- Up to 70% faster for short sequences and ~1.7% for very long ones; merged into LLVM libc++ — the first change to its sort in over a decade
- Discovered "swap" and "copy" move shortcuts
Strong evidence AI can beat mature human baselines and ship in production — a precursor of AlphaEvolve.
10 📏 Scaling, Evaluation and Trust
What do benchmarks really measure, where does scaling break, and can we trust deployment?
Are Emergent Abilities of Large Language Models a Mirage?
Schaeffer, Miranda & Koyejo (Stanford) · NeurIPS 2023People claimed some abilities appear suddenly once models get big enough. This paper shows many such jumps come from the scoring rule. Grade a 10-digit answer all-or-nothing and it looks like a sudden leap; give partial credit per digit and improvement was smooth all along.
- Nonlinear or discontinuous metrics (exact match, multiple-choice grade) create apparent emergence
- Continuous metrics on the same outputs show smooth, predictable improvement
- Demonstrated on GPT-3/InstructGPT families, BIG-Bench meta-analysis and vision models
Taught the field to ask what a benchmark really measures.
Scaling Data-Constrained Language Models
Muennighoff et al. (Hugging Face et al.) · NeurIPS 2023What happens when we run out of new text? Reading the same data again helps — at first. Up to about 4 passes, repeated data is nearly as good as fresh data. After that, each extra pass is worth less, until it's worth almost nothing.
- Experiments up to 900B tokens and 9B parameters
- Up to ~4 epochs, repeated data gives negligible loss change vs unique data; value decays toward zero with more repetition
- A data-constrained scaling law; adding code data and filtering help
Framed the "data wall" that drove synthetic data and RL in 2024–2025.
DecodingTrust: Trustworthiness of GPT Models
Wang et al. (UIUC-led, with Stanford, Microsoft et al.) · NeurIPS 2023 Datasets & BenchmarksBefore trusting a model in the real world, test it the way an attacker would. DecodingTrust probed GPT-3.5 and GPT-4 across eight kinds of trust. GPT-4 was usually safer on ordinary tests, but easier to mislead with jailbreak-style instructions — because it follows instructions more precisely.
- Dimensions: toxicity, stereotype bias, adversarial robustness, out-of-distribution robustness, robustness to adversarial demonstrations, privacy, machine ethics, fairness
- GPT-4 more trustworthy than GPT-3.5 on standard benchmarks but more vulnerable to misleading system/user prompts
- Both could leak private information from training data and conversations
Made trust a measurable, multi-dimensional benchmark rather than a single safety score.
11 The Emergence Mirage, Simulated
The core argument of Are Emergent Abilities a Mirage? fits in a few lines of Python. A model family improves smoothly, token by token. Score the same outputs all-or-nothing, and a "sudden" ability appears:
"""Are emergent abilities a mirage? (Schaeffer, Miranda & Koyejo, NeurIPS 2023) - a toy reproduction.
Assume a model family whose per-token accuracy p improves smoothly with scale:
p(N) = 1 / (1 + exp(-(log10 N - 9.3))) (N = parameters; illustrative curve)
A task's answer is 10 tokens long (e.g. a 10-digit sum). Two ways to score the same outputs:
exact match = p^10 (all-or-nothing) -> looks like a sudden "emergent" jump
per-token = p (partial credit) -> smooth, predictable improvement
Standard library only.
"""
import math
def per_token(n_params):
return 1 / (1 + math.exp(-(math.log10(n_params) - 9.3)))
print(f"{'parameters':>12} {'per-token acc':>14} {'exact match (10 tokens)':>24}")
for n in [1e7, 1e8, 1e9, 3e9, 1e10, 3e10, 1e11, 3e11]:
p = per_token(n)
print(f"{n:>12.0e} {p:>14.1%} {p ** 10:>24.1%}")
print("\nSame model outputs, two metrics: the all-or-nothing metric sits near zero, then 'suddenly'")
print("shoots up - while per-token accuracy improved steadily the whole time.")Output:
parameters per-token acc exact match (10 tokens)
1e+07 9.1% 0.0%
1e+08 21.4% 0.0%
1e+09 42.6% 0.0%
3e+09 54.4% 0.2%
1e+10 66.8% 1.8%
3e+10 76.4% 6.8%
1e+11 84.6% 18.7%
3e+11 89.8% 34.2%
Same model outputs, two metrics: the all-or-nothing metric sits near zero, then 'suddenly'
shoots up - while per-token accuracy improved steadily the whole time.Try it interactively in the Mirage card: lengthen the answer and the "emergence" gets sharper, even though the model didn't change. This caution about metrics resurfaces in every later year — in 2024 with contamination tests, and in 2025 with the Illusion of Thinking debate.
12 The 2022 → 2026 Roadmap
From ChatGPT to self-improving systems. Press play:
Next: AI Research in 2022: The Year Reasoning, Alignment and Generative AI Converged →