Models Teaching Models: How Synthetic Data Became the Fastest-Growing Source of Training Tokens
When human text runs short, labs manufacture more. Models rephrase web pages into cleaner prose, write textbooks to order, answer millions of generated questions, solve maths problems that a checker can verify, and play out agent tasks in simulated environments. Synthetic data now dominates post-training and is growing in pre-training. It can also go badly wrong. This part explains the main recipes, how generated data is filtered and verified, what "model collapse" really shows, and the legal fights over training on other models' outputs.
1. Case file
2. Timeline
Synthetic data, 2022 โ 2025
Tap a milestone. Colour: instruction data, pre-training data, post-training at scale, risks and disputes.
3. Why synthetic
4. Six families of synthetic data
5. Generate, verify, keep
Good synthetic data is mostly thrown away. The standard loop samples many candidates per prompt, scores or checks each one, keeps the best, deduplicates and decontaminates (Part 3), and only then adds them to the mix. Meta's Llama 3 paper describes sampling 10 to 30 responses per prompt and keeping the one its reward model scored highest ("rejection sampling"), and generating unit tests to discard code that failed them. DeepSeek collected about 800,000 samples by rejection sampling after its RL run and used them to train smaller "distilled" models.
Rejection sampling, step by step
A generator proposes 8 answers to one maths problem; a verifier checks each. Illustration.
6. Yield and cost
How many tokens to make a dataset?
Our own model of a verifier-filtered dataset. Prices are inputs, not quotes.
7. Diversity: the hidden failure
Ask a model the same kind of question a million times and you get a million variations of the same answer: the same phrases, structures and blind spots. Synthetic datasets fight this by seeding generation from varied real material. Cosmopedia prompted Mixtral with topics drawn from web datasets and with different audiences and formats (textbooks, blog posts, stories, WikiHow articles); persona-based methods prompt the generator to write as one of a very large number of synthetic personas โ ๏ธ. Deduplication then removes the near-copies that still slip through.
8. Model collapse
In 2024 Nature published Shumailov et al.'s warning: models trained recursively on data generated by earlier models degenerate, first losing the rare "tails" of the distribution and eventually collapsing toward a narrow average. The catch is the setup: each generation's data replaced the last. Gerstgrasser et al. showed that when synthetic data accumulates alongside the original real data, collapse is avoided, and a later study ("Collapse or Thrive?") found adding a little synthetic data helps when real data is scarce but hurts when real data is plentiful.
Collapse simulator
A real (tiny) experiment run in your browser: fit a bell curve to 40 samples, draw 40 new samples from the fit, repeat. In replace mode each generation trains only on the last generation's samples; in accumulate mode it trains on everything so far, including the original real data.
9. How much synthetic, and where
Synthetic share by stage
Disclosed examples (solid) and an illustrative 2026 range (hatched, โ ๏ธ) for frontier-style training. Most labs do not publish their shares.
10. Licences and disputes
Most commercial model providers' terms forbid using their outputs to build competing models. In January 2025 OpenAI told the Financial Times it had seen some evidence of "distillation" it suspected came from DeepSeek, without sharing details, and said it works with Microsoft to detect and block such attempts. Distillation from your own models, or from open-weight models whose licences permit it, is routine: DeepSeek released R1 distills built on Qwen and Llama, and NVIDIA licensed Nemotron-4 340B specifically for synthetic data generation. Synthetic data also raises a quieter question: if a generator was trained on copyrighted text, does rephrasing that knowledge "launder" it? Courts have not settled this (Part 8).
11. Who uses what
| Organisation | Disclosed synthetic data use |
|---|---|
| Microsoft (Phi) | Synthetic textbooks and exercises at the core of the Phi series, starting with 1B tokens for Phi-1 |
| NVIDIA | 98%+ synthetic alignment data for Nemotron-4 340B; 1.9T rephrased tokens in Nemotron-CC; open generation pipeline |
| Meta | Llama 3: rejection-sampled SFT data, code execution feedback, translation and back-translation |
| DeepSeek | ~800K rejection-sampled samples; distilled models; R1-generated reasoning data |
| Hugging Face | Cosmopedia (25B tokens), SmolLM training data |
| Apple | WRAP rephrasing research |
| OpenAI, Anthropic, Google | Use acknowledged in general terms; shares not disclosed |
12. How much synthetic?
A rough guide
An editorial guide.
Inside the Data Layer
13. Reading the numbers
- Primary
- Phi-1: 1.3B params, 6B filtered web + 1B GPT-3.5-generated tokens, 50.6% HumanEval, 55.5% MBPP (paper). Cosmopedia: 25B tokens, 30M+ files, Mixtral-8x7B-Instruct (repository). WRAP: ~3ร faster pre-training on C4 (paper). Nemotron-4 340B: 98%+ synthetic alignment data; 9T pre-training tokens (report). Llama 3: 10โ30 samples per prompt with reward-model selection (paper). DeepSeek-R1: ~800K samples (paper, via coverage). Model collapse: Shumailov et al. (Nature 2024, replace); Gerstgrasser et al. 2024 (accumulate); "Collapse or Thrive?" 2024.
- Secondary โ ๏ธ
- R1's 600K reasoning / 200K other split; Cosmopedia's ~10K H100-hours; persona-based generation details; the OpenAIโDeepSeek claims (unproven, reported by the FT).
- Our illustrations
- Family examples, rejection-sampling demo, yield and cost calculator, stage shares (hatched) and the chooser. The collapse simulator is a real but toy experiment (Gaussian fitting), not a language model.
- Papers and releases
- Textbooks Are All You Need (Phi-1)
- Cosmopedia
- Rephrasing the Web (WRAP)
- Nemotron-4 340B technical report
- The Llama 3 herd of models ยท DeepSeek-R1
- Shumailov et al.: AI models collapse when trained on recursively generated data
- Is model collapse inevitable? ยท Collapse or thrive?
- Reporting
- Fortune: OpenAI, DeepSeek and distillation
- Part 3: filtering and deduplication