Home โ€บ Blog โ€บ Synthetic data
Inside the Data Layer ยท Part 4 ยท Synthetic

Models Teaching Models: How Synthetic Data Became the Fastest-Growing Source of Training Tokens

When human text runs short, labs manufacture more. Models rephrase web pages into cleaner prose, write textbooks to order, answer millions of generated questions, solve maths problems that a checker can verify, and play out agent tasks in simulated environments. Synthetic data now dominates post-training and is growing in pre-training. It can also go badly wrong. This part explains the main recipes, how generated data is filtered and verified, what "model collapse" really shows, and the legal fights over training on other models' outputs.

FL
FrontierAGI Team
โš ๏ธ How this was researched (as of 27 Oct 2026) Sources are the Phi-1, WRAP, Nemotron-4 340B, Llama 3 and DeepSeek-R1 papers, Hugging Face's Cosmopedia release, the model-collapse papers (Shumailov et al. in Nature; Gerstgrasser et al.; "Collapse or Thrive?") and reporting on the OpenAIโ€“DeepSeek distillation dispute. Figures from secondary sources are marked โš ๏ธ. The collapse simulator runs a real (tiny) experiment in your browser; the cost calculators and stage shares are our own illustrations. Disclosure: Claude, made by Anthropic, helps write this site; most labs, including Anthropic, use synthetic data and do not disclose how much.

1. Case file

โœจ
Synthetic data, October 2026Stage 6 ("enrich") of the data supply chain
Phi-1 (Microsoft, 2023)1B synthetic + 6B filtered tokens โ†’ 50.6% HumanEval
Cosmopedia (Hugging Face, 2024)25B tokens, 30M+ files, by Mixtral-8x7B
WRAP (Apple, 2024)Rephrasing C4 โ†’ ~3ร— faster pre-training
Nemotron-4 340B (NVIDIA, 2024)98%+ of alignment data synthetic
Nemotron-CC (Dec 2024)1.9T synthetic of 6.3T tokens
DeepSeek-R1 (Jan 2025)~800K generated samples to train distilled models

2. Timeline

Synthetic data, 2022 โ†’ 2025

Tap a milestone. Colour: instruction data, pre-training data, post-training at scale, risks and disputes.

Tap a milestone.

3. Why synthetic

ScarcityPublic human text is finite (Part 1's data wall) and increasingly blocked (Part 2).
GapsSome skills are rare on the web: step-by-step maths, tool calls, long reasoning, low-resource languages.
ControlGenerated data can be made at a chosen difficulty, format and style, with known answers.
VerificationFor maths and code, answers can be checked automatically, so wrong examples are thrown away.
Cost and rightsModel-written text is cheaper than expert humans and, when generated from your own model, avoids some licensing questions.

4. Six families of synthetic data

5. Generate, verify, keep

Good synthetic data is mostly thrown away. The standard loop samples many candidates per prompt, scores or checks each one, keeps the best, deduplicates and decontaminates (Part 3), and only then adds them to the mix. Meta's Llama 3 paper describes sampling 10 to 30 responses per prompt and keeping the one its reward model scored highest ("rejection sampling"), and generating unit tests to discard code that failed them. DeepSeek collected about 800,000 samples by rejection sampling after its RL run and used them to train smaller "distilled" models.

Rejection sampling, step by step

A generator proposes 8 answers to one maths problem; a verifier checks each. Illustration.

Problem: A train travels 180 km in 2.5 hours. What is its average speed in km/h? (answer: 72)

6. Yield and cost

How many tokens to make a dataset?

Our own model of a verifier-filtered dataset. Prices are inputs, not quotes.

7. Diversity: the hidden failure

Ask a model the same kind of question a million times and you get a million variations of the same answer: the same phrases, structures and blind spots. Synthetic datasets fight this by seeding generation from varied real material. Cosmopedia prompted Mixtral with topics drawn from web datasets and with different audiences and formats (textbooks, blog posts, stories, WikiHow articles); persona-based methods prompt the generator to write as one of a very large number of synthetic personas โš ๏ธ. Deduplication then removes the near-copies that still slip through.

8. Model collapse

In 2024 Nature published Shumailov et al.'s warning: models trained recursively on data generated by earlier models degenerate, first losing the rare "tails" of the distribution and eventually collapsing toward a narrow average. The catch is the setup: each generation's data replaced the last. Gerstgrasser et al. showed that when synthetic data accumulates alongside the original real data, collapse is avoided, and a later study ("Collapse or Thrive?") found adding a little synthetic data helps when real data is scarce but hurts when real data is plentiful.

Collapse simulator

A real (tiny) experiment run in your browser: fit a bell curve to 40 samples, draw 40 new samples from the fit, repeat. In replace mode each generation trains only on the last generation's samples; in accumulate mode it trains on everything so far, including the original real data.

9. How much synthetic, and where

Synthetic share by stage

Disclosed examples (solid) and an illustrative 2026 range (hatched, โš ๏ธ) for frontier-style training. Most labs do not publish their shares.

Most commercial model providers' terms forbid using their outputs to build competing models. In January 2025 OpenAI told the Financial Times it had seen some evidence of "distillation" it suspected came from DeepSeek, without sharing details, and said it works with Microsoft to detect and block such attempts. Distillation from your own models, or from open-weight models whose licences permit it, is routine: DeepSeek released R1 distills built on Qwen and Llama, and NVIDIA licensed Nemotron-4 340B specifically for synthetic data generation. Synthetic data also raises a quieter question: if a generator was trained on copyrighted text, does rephrasing that knowledge "launder" it? Courts have not settled this (Part 8).

11. Who uses what

OrganisationDisclosed synthetic data use
Microsoft (Phi)Synthetic textbooks and exercises at the core of the Phi series, starting with 1B tokens for Phi-1
NVIDIA98%+ synthetic alignment data for Nemotron-4 340B; 1.9T rephrased tokens in Nemotron-CC; open generation pipeline
MetaLlama 3: rejection-sampled SFT data, code execution feedback, translation and back-translation
DeepSeek~800K rejection-sampled samples; distilled models; R1-generated reasoning data
Hugging FaceCosmopedia (25B tokens), SmolLM training data
AppleWRAP rephrasing research
OpenAI, Anthropic, GoogleUse acknowledged in general terms; shares not disclosed

12. How much synthetic?

A rough guide

An editorial guide.

VerdictSynthetic data is no longer a stopgap: it is how post-training data is made, and a growing slice of pre-training. It works best where answers can be checked and where it is anchored to real data, and worst when models feed on their own unchecked output. The scarce input has shifted from text to things that make text trustworthy: verifiers, strong teacher models, varied seeds and fresh human data (Part 5) to keep the loop honest.

Inside the Data Layer

13. Reading the numbers

Primary
Phi-1: 1.3B params, 6B filtered web + 1B GPT-3.5-generated tokens, 50.6% HumanEval, 55.5% MBPP (paper). Cosmopedia: 25B tokens, 30M+ files, Mixtral-8x7B-Instruct (repository). WRAP: ~3ร— faster pre-training on C4 (paper). Nemotron-4 340B: 98%+ synthetic alignment data; 9T pre-training tokens (report). Llama 3: 10โ€“30 samples per prompt with reward-model selection (paper). DeepSeek-R1: ~800K samples (paper, via coverage). Model collapse: Shumailov et al. (Nature 2024, replace); Gerstgrasser et al. 2024 (accumulate); "Collapse or Thrive?" 2024.
Secondary โš ๏ธ
R1's 600K reasoning / 200K other split; Cosmopedia's ~10K H100-hours; persona-based generation details; the OpenAIโ€“DeepSeek claims (unproven, reported by the FT).
Our illustrations
Family examples, rejection-sampling demo, yield and cost calculator, stage shares (hatched) and the chooser. The collapse simulator is a real but toy experiment (Gaussian fitting), not a language model.