Home › Blog › AI Research in 2023
AI Research Progress Through the Years · 2023 🏗️

AI Research in 2023: The Year Foundation Models Became Platforms

In 2023 large language models stopped being standalone models and became platforms people could build on. Frontier models went multimodal while open models closed the gap. Fine-tuning and alignment became cheap enough for one GPU. Reasoning became search. Models learned to use tools, keep memories, reflect and build skill libraries — the primitives of today's agent harnesses. Vision became promptable, language models started driving robots, and AI-found code shipped in the C++ standard library. This guide covers seventeen papers in eight research themes, each explained in plain English with an animation, followed by what it found, why it matters and its sources.

How to read this: the fourth article in AI Research Progress Through the Years, and the prequel to 2024, 2025 and 2026. Awards come from official announcements (NeurIPS 2023, ICCV 2023, UIST 2023). Papers are grouped by theme, not ranked. The "🧒 In plain English" boxes simplify on purpose, and the animations illustrate ideas; exact claims are in "What they found" and the linked papers.

01 Venues and Awards

🏆 Best / Outstanding Paper🥈 Outstanding runner-up🏅 Honorable Mention📄 Accepted paper📝 Preprint / report📰 Nature
VenueCovered hereNotes
NeurIPS 2023 (December, New Orleans)Outstanding: Are Emergent Abilities a Mirage? · Runners-up: DPO · Scaling Data-Constrained LMs · Outstanding D&B: DecodingTrust · Also: QLoRA, Tree of Thoughts, Toolformer, Reflexion, LLaVAOther Outstanding: Privacy Auditing with One (1) Training Run; D&B: ClimSim
ICCV 2023Segment Anything — Best Paper Honorable Mention—
UIST 2023Generative Agents — Best Paper—
OtherAlphaDev (Nature) · RT-2 (CoRL 2023) · GPT-4, LLaMA, Llama 2, Voyager (technical reports and preprints)—

02 Paper Map

Seventeen papers in eight themes. Filter by award and click to jump:

▶ 2023 paper map Click to jump

03 🌐 Frontier Models and the Open-Model Explosion

Frontier multimodal models arrived — and open models closed the gap.

🧠

GPT-4 Technical Report

OpenAI · arXiv Mar 2023
📝 Preprint / report
🧒 In plain English

GPT-4 was a large model that could read both text and images and performed at a human level on many professional and academic exams. One quietly important idea: OpenAI trained small practice models first and used them to predict how well the huge model would do — before spending the compute on it.

▶ Predicting a big model from small ones Press predict
🔍 What they found
  • Multimodal (text + image input) Transformer, post-trained with RLHF
  • Human-level performance on many exams (e.g. a simulated bar exam around the top 10% of test takers)
  • Final loss predicted from models trained with up to 1/10,000th of GPT-4's compute
  • Few architecture or training-data details disclosed
💡 Why it matters

Set the frontier for 2023 and started the era of predictable scaling — and of closed technical reports.

🦙

LLaMA: Open and Efficient Foundation Language Models

Touvron et al. (Meta) · arXiv Feb 2023
📝 Preprint / report
🧒 In plain English

A bigger model isn't the only way to a better one: train a smaller model on more data. LLaMA's 13B model beat the 175B GPT-3 on most benchmarks, trained only on public data, and its weights went to researchers — igniting the open-model boom.

▶ Smaller model, more data Size vs quality
🔍 What they found
  • Models from 7B to 65B trained on 1–1.4T tokens of publicly available data
  • LLaMA-13B outperformed GPT-3 (175B) on most benchmarks; LLaMA-65B competitive with Chinchilla-70B and PaLM-540B
💡 Why it matters

Sparked Alpaca, Vicuna and the open fine-tuning ecosystem.

🔗 References
🦙

Llama 2: Open Foundation and Fine-Tuned Chat Models

Touvron et al. (Meta) · arXiv Jul 2023
📝 Preprint / report
🧒 In plain English

Llama 2 came in sizes from 7B to 70B, with ready-to-use chat versions and a licence allowing commercial use. The paper explained, unusually openly, how the chat models were aligned: supervised examples, then separate helpfulness and safety reward models guiding reinforcement learning.

▶ How Llama 2-Chat was aligned Press play
🔍 What they found
  • Pretrained on 2T tokens; 7B, 13B and 70B released (34B withheld)
  • RLHF with separate helpfulness and safety reward models, rejection sampling and PPO
  • Detailed safety evaluation and red-teaming discussion; commercially usable licence
💡 Why it matters

Made open, commercially usable chat models the norm.

04 🔧 Efficient Fine-Tuning and the New Post-Training Stack

Pretrain → SFT/LoRA → preference optimisation became cheap and simple.

💾

QLoRA: Efficient Finetuning of Quantized LLMs

Dettmers et al. (University of Washington) · NeurIPS 2023
📄 Accepted paper
🧒 In plain English

Fine-tuning a 65-billion-parameter model used to need a room full of GPUs. QLoRA freezes the model in compact 4-bit form and trains only small add-on adapters. The whole job fits on one 48GB GPU, with no loss in quality.

▶ Memory needed to fine-tune a 65B model Toggle
🔍 What they found
  • 4-bit NormalFloat, double quantisation and paged optimisers
  • Memory to fine-tune 65B cut from >780GB to <48GB without degrading 16-bit fine-tuning performance
  • Guanaco reached 99.3% of ChatGPT's level on the Vicuna benchmark after 24 hours on one GPU
💡 Why it matters

Put fine-tuning of big models within reach of individual researchers.

🎯

Direct Preference Optimization (DPO)

Rafailov, Sharma, Mitchell et al. (Stanford) · NeurIPS 2023
🥈 Outstanding runner-up
🧒 In plain English

Classic RLHF trains a separate reward model, then uses reinforcement learning to chase its scores — complicated and fragile. DPO shows you can skip both: given pairs of "preferred" and "rejected" answers, one simple training loss makes the model prefer the better ones directly.

▶ RLHF vs DPO Toggle the pipeline
🔍 What they found
  • Shows the RLHF objective can be optimised exactly with a classification-style loss on preference pairs
  • Matches or beats PPO-based RLHF on sentiment control, summarisation and dialogue while being far simpler and more stable
💡 Why it matters

Became the default open-source alignment method; see its 2025 caveat in Learning Dynamics.

05 🌳 Reasoning Becomes Search, Not Just Prompting

From one linear chain of thought to search over reasoning paths.

🌳

Tree of Thoughts: Deliberate Problem Solving with LLMs

Yao et al. (Princeton, Google DeepMind) · NeurIPS 2023
📄 Accepted paper
🧒 In plain English

Chain-of-thought is like solving a maze by walking one path and hoping. Tree of Thoughts lets the model try several next steps, judge which look promising, explore those, and backtrack from dead ends. On the "Game of 24" puzzle, that raised GPT-4's success from 4% to 74%.

▶ Game of 24: make 24 from 4, 9, 10, 13 Press search
🔍 What they found
  • Thoughts as intermediate states; the LM both proposes and evaluates them; BFS/DFS search with backtracking
  • Game of 24: 74% success vs 4% for GPT-4 chain-of-thought; gains on creative writing and mini crosswords
💡 Why it matters

The step from one reasoning trace to search over traces — ancestor of test-time compute scaling.

06 🧰 Tools, Memory and the Birth of Modern Agents

Tools, memory, reflection, planning and skill libraries — the primitives of today's harnesses.

🧮

Toolformer: Language Models Can Teach Themselves to Use Tools

Schick et al. (Meta) · NeurIPS 2023
📄 Accepted paper
🧒 In plain English

Language models are bad at arithmetic and don't know today's date — but a calculator and a calendar do. Toolformer teaches a model, with only a few examples, to decide when to call a tool, what to ask it, and how to use the answer — keeping only tool calls that actually help it predict text.

▶ A model deciding to call tools Press write
🔍 What they found
  • Self-supervised: samples candidate API calls, keeps those that reduce loss on the following tokens
  • Tools: calculator, Q&A system, search engine, translator, calendar
  • A 6.7B model with tools often beat much larger models, without losing language ability
💡 Why it matters

Introduced learned tool use — a core primitive of agent harnesses.

🪞

Reflexion: Language Agents with Verbal Reinforcement Learning

Shinn et al. · NeurIPS 2023
📄 Accepted paper
🧒 In plain English

When an agent fails, Reflexion doesn't retrain it. Instead the agent writes itself a note — "I forgot to handle empty input" — stores it in memory, and reads it before the next try. Learning happens in words, not weights.

▶ Three tries with a memory Press play
🔍 What they found
  • Verbal self-reflection stored in episodic memory; no weight updates
  • HumanEval: 91% pass@1, above GPT-4's 80% at the time; gains on decision-making (AlfWorld) and reasoning (HotpotQA)
💡 Why it matters

An early form of harness-level learning — improving through memory, not training.

🏘️

Generative Agents: Interactive Simulacra of Human Behavior

Park et al. (Stanford, Google) · UIST 2023
🏆 Best / Outstanding Paper
🧒 In plain English

Twenty-five AI characters live in a little town. Each remembers what it sees, reflects on it and plans its day. Told only that one of them wanted to throw a Valentine's Day party, the others spread invitations, asked each other on dates and turned up together — none of it scripted.

▶ Smallville: a party spreads Press play
🔍 What they found
  • Architecture: memory stream, retrieval by recency/importance/relevance, reflection, planning
  • Believable emergent social behaviour among 25 agents
  • UIST 2023 Best Paper
💡 Why it matters

The memory–reflection–planning loop behind many later agents.

⛏️

Voyager: An Open-Ended Embodied Agent with LLMs

Wang et al. (NVIDIA, Caltech et al.) · arXiv May 2023
📝 Preprint / report
🧒 In plain English

A Minecraft agent that sets its own goals, writes code to achieve them, fixes the code using feedback from the game, and saves working code in a growing skill library. New skills build on old ones, so it keeps getting more capable without retraining.

▶ A growing skill library Press explore
🔍 What they found
  • Automatic curriculum, executable skill library and iterative prompting with self-verification
  • 3.3× more unique items, 2.3× longer distances and tech-tree milestones up to 15.3× faster than prior methods
💡 Why it matters

Lifelong learning through a harness — skills as code.

07 👁️ Multimodal Foundation Models

Specialised vision model → promptable foundation model → conversational multimodal assistant.

✂️

Segment Anything (SAM)

Kirillov et al. (Meta) · ICCV 2023
🏅 Honorable Mention
🧒 In plain English

Point at anything in a photo — a dog, a cup, a cloud — and SAM outlines it instantly, even for objects it was never specifically trained on. Meta built it from the biggest segmentation dataset ever: over a billion outlines on 11 million images.

▶ Click to segment Click a point
🔍 What they found
  • Promptable segmentation: points, boxes or masks as prompts
  • SA-1B dataset: >1B masks on 11M licensed, privacy-respecting images
  • Strong zero-shot transfer; ICCV 2023 Best Paper Honorable Mention
💡 Why it matters

The first real foundation model for image segmentation.

🖼️

LLaVA: Visual Instruction Tuning

Liu, Li, Wu & Lee · NeurIPS 2023
📄 Accepted paper
🧒 In plain English

Connect an image encoder to a chatbot with a small "translator" layer, then teach it with conversations about images that GPT-4 wrote (from image captions). The result: an open assistant you can ask about any picture.

▶ Ask about an image Press ask
🔍 What they found
  • Vision encoder (CLIP) + projection + LLM (Vicuna), trained on GPT-4-generated visual instruction data
  • 85.1% relative score vs GPT-4 on a synthetic instruction benchmark; 92.53% on ScienceQA combined with GPT-4
💡 Why it matters

Template for open vision-language assistants.

08 🤖 Language Models Enter the Physical World

Language model → multimodal model → action model.

🦾

RT-2: Vision-Language-Action Models

Brohan et al. (Google DeepMind) · CoRL 2023
📄 Accepted paper
🧒 In plain English

What if robot movements were just another kind of word? RT-2 writes arm motions as tokens, so a vision-language model trained on the web can also output actions. It can follow instructions it never saw in robot training — like "pick up the extinct animal" (the toy dinosaur).

▶ Words in, actions out Pick an instruction
🔍 What they found
  • Actions represented as text tokens; co-fine-tuned on web vision-language data and robot trajectories
  • Generalisation to unseen objects and scenes improved from 32% (RT-1) to 62%, over thousands of evaluation trials
  • Emergent semantic reasoning, e.g. choosing an improvised tool
💡 Why it matters

Made the path language model → multimodal model → action model concrete.

09 ⚙️ AI Starts Discovering Useful Algorithms

AI-found code enters real production software.

10 📏 Scaling, Evaluation and Trust

What do benchmarks really measure, where does scaling break, and can we trust deployment?

🌫️

Are Emergent Abilities of Large Language Models a Mirage?

Schaeffer, Miranda & Koyejo (Stanford) · NeurIPS 2023
🏆 Best / Outstanding Paper
🧒 In plain English

People claimed some abilities appear suddenly once models get big enough. This paper shows many such jumps come from the scoring rule. Grade a 10-digit answer all-or-nothing and it looks like a sudden leap; give partial credit per digit and improvement was smooth all along.

▶ Same outputs, two metrics Drag model size
🔍 What they found
  • Nonlinear or discontinuous metrics (exact match, multiple-choice grade) create apparent emergence
  • Continuous metrics on the same outputs show smooth, predictable improvement
  • Demonstrated on GPT-3/InstructGPT families, BIG-Bench meta-analysis and vision models
💡 Why it matters

Taught the field to ask what a benchmark really measures.

🔁

Scaling Data-Constrained Language Models

Muennighoff et al. (Hugging Face et al.) · NeurIPS 2023
🥈 Outstanding runner-up
🧒 In plain English

What happens when we run out of new text? Reading the same data again helps — at first. Up to about 4 passes, repeated data is nearly as good as fresh data. After that, each extra pass is worth less, until it's worth almost nothing.

▶ Value of re-reading the same data Drag epochs
🔍 What they found
  • Experiments up to 900B tokens and 9B parameters
  • Up to ~4 epochs, repeated data gives negligible loss change vs unique data; value decays toward zero with more repetition
  • A data-constrained scaling law; adding code data and filtering help
💡 Why it matters

Framed the "data wall" that drove synthetic data and RL in 2024–2025.

🔐

DecodingTrust: Trustworthiness of GPT Models

Wang et al. (UIUC-led, with Stanford, Microsoft et al.) · NeurIPS 2023 Datasets & Benchmarks
🏆 Outstanding D&B Paper
🧒 In plain English

Before trusting a model in the real world, test it the way an attacker would. DecodingTrust probed GPT-3.5 and GPT-4 across eight kinds of trust. GPT-4 was usually safer on ordinary tests, but easier to mislead with jailbreak-style instructions — because it follows instructions more precisely.

▶ Eight dimensions of trust Toggle the prompt type
🔍 What they found
  • Dimensions: toxicity, stereotype bias, adversarial robustness, out-of-distribution robustness, robustness to adversarial demonstrations, privacy, machine ethics, fairness
  • GPT-4 more trustworthy than GPT-3.5 on standard benchmarks but more vulnerable to misleading system/user prompts
  • Both could leak private information from training data and conversations
💡 Why it matters

Made trust a measurable, multi-dimensional benchmark rather than a single safety score.

11 The Emergence Mirage, Simulated

The core argument of Are Emergent Abilities a Mirage? fits in a few lines of Python. A model family improves smoothly, token by token. Score the same outputs all-or-nothing, and a "sudden" ability appears:

"""Are emergent abilities a mirage? (Schaeffer, Miranda & Koyejo, NeurIPS 2023) - a toy reproduction.

Assume a model family whose per-token accuracy p improves smoothly with scale:
    p(N) = 1 / (1 + exp(-(log10 N - 9.3)))        (N = parameters; illustrative curve)
A task's answer is 10 tokens long (e.g. a 10-digit sum). Two ways to score the same outputs:
  exact match  = p^10        (all-or-nothing)       -> looks like a sudden "emergent" jump
  per-token    = p           (partial credit)       -> smooth, predictable improvement
Standard library only.
"""
import math

def per_token(n_params):
    return 1 / (1 + math.exp(-(math.log10(n_params) - 9.3)))

print(f"{'parameters':>12} {'per-token acc':>14} {'exact match (10 tokens)':>24}")
for n in [1e7, 1e8, 1e9, 3e9, 1e10, 3e10, 1e11, 3e11]:
    p = per_token(n)
    print(f"{n:>12.0e} {p:>14.1%} {p ** 10:>24.1%}")
print("\nSame model outputs, two metrics: the all-or-nothing metric sits near zero, then 'suddenly'")
print("shoots up - while per-token accuracy improved steadily the whole time.")

Output:

  parameters  per-token acc  exact match (10 tokens)
       1e+07           9.1%                     0.0%
       1e+08          21.4%                     0.0%
       1e+09          42.6%                     0.0%
       3e+09          54.4%                     0.2%
       1e+10          66.8%                     1.8%
       3e+10          76.4%                     6.8%
       1e+11          84.6%                    18.7%
       3e+11          89.8%                    34.2%

Same model outputs, two metrics: the all-or-nothing metric sits near zero, then 'suddenly'
shoots up - while per-token accuracy improved steadily the whole time.

Try it interactively in the Mirage card: lengthen the answer and the "emergence" gets sharper, even though the model didn't change. This caution about metrics resurfaces in every later year — in 2024 with contamination tests, and in 2025 with the Illusion of Thinking debate.

12 The 2022 → 2026 Roadmap

From ChatGPT to self-improving systems. Press play:

▶ Eight stages in four years Click a stage or press play
2023Turn foundation models into programmable platforms
→
2024Build richer systems and environments around them
→
2025Give those systems stronger reasoning and agency
→
2026Begin making them persistently improve themselves
AI research in 2023, in one paragraph
2023 turned large models into programmable platforms. GPT-4 set the frontier and made scaling predictable, while LLaMA and Llama 2 opened the weights. QLoRA and DPO made adaptation and alignment cheap and simple. Tree of Thoughts reframed reasoning as search. Toolformer, Reflexion, Generative Agents and Voyager assembled the building blocks of modern agents: tools, memory, reflection, planning and skill libraries. SAM and LLaVA made vision promptable and conversational, and RT-2 turned a vision-language model into a robot controller. AlphaDev put AI-discovered code into the C++ standard library. The Mirage, data-constrained scaling and DecodingTrust papers made measurement, data limits and trust first-class research problems. Almost every idea in the 2024–2026 articles traces back to one of these.

Next: AI Research in 2022: The Year Reasoning, Alignment and Generative AI Converged →

⚠️ Notes

Corrections to the source infographic: (1) DPO is labelled "NeurIPS 2023 Outstanding Paper"; it was an Outstanding Main Track Runner-Up; (2) Scaling Data-Constrained Language Models is credited to "Allen Institute"; it is by Muennighoff et al., led from Hugging Face, and was also a NeurIPS 2023 Outstanding runner-up; (3) DecodingTrust carries a Microsoft logo; it was a UIUC-led collaboration including Microsoft and others, and won the NeurIPS 2023 Outstanding Datasets & Benchmarks award; (4) awards and venues missing from the infographic are added: Segment Anything (ICCV 2023 Best Paper Honorable Mention), Generative Agents (UIST 2023 Best Paper), RT-2 (CoRL 2023), and NeurIPS 2023 for QLoRA, Tree of Thoughts, Toolformer, Reflexion and LLaVA. The brief's details all checked out (QLoRA on one 48GB GPU, ToT on Game of 24, SA-1B's billion masks, RT-2 generalisation, AlphaDev in LLVM). The mirage simulation uses an illustrative accuracy curve.

🔗 All References