Home › Blog › AI Research in 2017
AI Research Progress Through the Years · 2017 🌱

AI Research in 2017: The Year the Modern AI Architecture Was Born

2017 supplied several of the deepest building blocks of modern AI at once. The Transformer became the architecture. AlphaGo Zero showed self-play as a way to generate experience. PPO gave reinforcement learning a scalable loop. Human preferences became a reward signal. Architecture search and population-based training let AI start designing AI, and sparse experts opened a route to enormous capacity. Generative vision became controllable, and capsules explored a different path. This guide covers eleven papers in eight research themes, each explained in plain English with an animation, followed by what it found, why it matters and its sources.

How to read this: the tenth article in AI Research Progress Through the Years. It follows 2016, and the story continues in 2018, 2019, 2020, 2021, 2022, 2023, 2024, 2025 and 2026. Papers are grouped by theme, not ranked. The "🧒 In plain English" boxes simplify on purpose, and the animations illustrate ideas; exact claims are in "What they found" and the linked papers.
▶ 2017 already contained four ingredients of self-improving AI Press build · click to jump

01 Venues

📄 Accepted paper📰 Nature📝 Preprint
VenueCovered here
NIPS 2017 (Long Beach)Attention Is All You Need · Deep RL from Human Preferences · Dynamic Routing Between Capsules
ICLR 2017Neural Architecture Search · Sparsely-Gated Mixture-of-Experts
ICML, CVPR, ICCV 2017Wasserstein GAN (ICML) · pix2pix (CVPR) · CycleGAN (ICCV)
NatureAlphaGo Zero
PreprintsPPO · Population Based Training

None of these took a NIPS 2017 Best Paper award; those went to other work, including the subgame-solving paper behind the poker AI Libratus. Most of the papers here were recognised as landmarks only in hindsight.

02 Paper Map

Eleven papers in eight themes. Filter and click to jump:

▶ 2017 paper map Click to jump

03 ⚡ Attention Is All You Need: The Transformer

Sequential recurrence → every position attends to every other, in parallel.

⚡

Attention Is All You Need

Vaswani et al. (Google Brain, Google Research, University of Toronto) · NIPS 2017
📄 Accepted paper
🧒 In plain English

Older language models read a sentence one word at a time, like passing a note down a line. The Transformer lets every word look at every other word at once and decide which ones matter to it. That made training far faster on GPUs — and it turned out to work for almost everything.

▶ RNN vs self-attention Toggle, then run
🔍 What they found
  • Encoder–decoder built only from multi-head self-attention, feed-forward layers, residual connections, layer normalisation and positional encodings
  • 28.4 BLEU on WMT 2014 English→German (over 2 BLEU above previous best, including ensembles) and 41.8 BLEU on English→French after 3.5 days on eight GPUs
  • Far more parallelisable than recurrent models, with short paths between distant positions
💡 Why it matters

The architecture behind GPT, BERT, T5, ViT and every frontier model since.

04 ♟️ Self-Play Creates Its Own Training Data

A system that generates the experience that trains its next version.

⚫

AlphaGo Zero: Mastering the Game of Go without Human Knowledge

Silver et al. (DeepMind) · Nature 2017
📰 Nature
🧒 In plain English

The first AlphaGo learned from human expert games. AlphaGo Zero started from random play and only played against itself. Each round of games trained a better network, which played better games, which trained an even better network.

▶ The self-play loop Press play
🔍 What they found
  • A single residual network outputs both move probabilities and a value; Monte Carlo tree search guided by the network generates self-play games that train it
  • No human games or hand-crafted features — only the rules
  • Beat the version that defeated Lee Sedol 100–0 after 3 days; beat AlphaGo Master 89–11 after 40 days
💡 Why it matters

The clearest early example of a system generating the data that trains its next version.

05 🧑‍⚖️ Human Preferences Become a Reward Function

Behaviour A vs behaviour B → human choice → reward model → RL.

🧑‍⚖️

Deep Reinforcement Learning from Human Preferences

Christiano, Leike, Brown, Martic, Legg & Amodei (OpenAI, DeepMind) · NIPS 2017
📄 Accepted paper
🧒 In plain English

Some goals are hard to write as a score ("do a nice backflip"). Instead, show a person two short clips and ask which is better. A reward model learns from those choices, and the agent learns from the reward model.

▶ Which clip is better? Press run
🔍 What they found
  • Learned a reward model from human comparisons of short trajectory segments, and trained the policy with RL against it — asynchronously, as feedback arrived
  • Solved Atari games and simulated robot tasks with human feedback on less than 1% of the agent's interactions
  • Learned novel behaviours such as a simulated backflip from about an hour of human feedback
💡 Why it matters

The conceptual core of RLHF — later used in 2020's summarisation work, InstructGPT and ChatGPT.

06 📎 PPO Makes Reinforcement Learning Easier to Scale

Collect experience → estimate advantage → clipped update → repeat.

📎

Proximal Policy Optimization Algorithms (PPO)

Schulman, Wolski, Dhariwal, Radford & Klimov (OpenAI) · arXiv Jul 2017
📝 Preprint / report
🧒 In plain English

Reinforcement learning can be unstable: one over-eager update can wreck a good policy. PPO limits how far each update can move the policy — if an action already became much more likely, it stops pushing. Simple to implement, and it just works.

▶ The clipped objective Drag the probability ratio
🔍 What they found
  • Clipped surrogate objective: the probability ratio between new and old policy is clipped to [1−ε, 1+ε] (ε = 0.2 in the paper)
  • Several epochs of minibatch updates per batch of experience
  • Better balance of sample efficiency, simplicity and wall-clock time than other online policy-gradient methods on MuJoCo control and Atari
💡 Why it matters

Became the default RL algorithm — used for OpenAI Five, Dactyl and the PPO stage of RLHF.

07 🧬 AI Begins Designing AI

Generate candidate models → train and evaluate → reward, select, copy and mutate.

🏗️

Neural Architecture Search with Reinforcement Learning

Zoph & Le (Google Brain) · ICLR 2017
📄 Accepted paper
🧒 In plain English

Let one network design another. A controller writes out a network's blueprint, the network is trained and tested, and its accuracy is the controller's reward. After thousands of tries, the controller proposes architectures as good as expert designs.

▶ Controller → child network → reward Press search
🔍 What they found
  • An RNN controller samples architecture descriptions; child networks are trained; validation accuracy is the reward for REINFORCE
  • CIFAR-10: 3.65% test error, competitive with the best human-designed networks; Penn Treebank: a new recurrent cell with 62.4 perplexity
  • Used 800 GPUs concurrently for the search
💡 Why it matters

An early case of AI searching over the design of AI — ancestor of AutoML and of today's automated research.

🧫

Population Based Training of Neural Networks

Jaderberg et al. (DeepMind) · arXiv Nov 2017
📝 Preprint / report
🧒 In plain English

Train a whole population of models at once. Every so often, weak ones copy a strong one's weights (exploit) and tweak its settings, such as the learning rate (explore). Settings end up changing over time, like an evolving schedule.

▶ Exploit and explore Press run
🔍 What they found
  • Asynchronous population training that jointly optimises weights and adaptive hyperparameter schedules
  • Faster training and better final performance across deep RL (DeepMind Lab, Atari, StarCraft II), machine translation and GANs, with the same compute as a normal hyperparameter search
💡 Why it matters

Selection, inheritance and mutation — the pattern behind evolutionary agents, AlphaEvolve and harness populations.

08 🔀 Sparse Mixture-of-Experts

Store many parameters, use only a few per input.

🔀

Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer

Shazeer et al. (Google Brain) · ICLR 2017
📄 Accepted paper
🧒 In plain English

Build a network with thousands of specialist sub-networks, and a small gate that picks just a few of them for each input. The model can hold enormous knowledge while each input costs only a little computation.

▶ Dense vs sparse Toggle, then route
🔍 What they found
  • A trainable noisy top-k gate selects a few experts out of up to thousands per example, with load-balancing losses
  • Models up to 137 billion parameters, placed between LSTM layers, with only minor losses in computational efficiency
  • Better than the state of the art on language modelling and machine translation at lower compute
💡 Why it matters

The start of sparse scaling: GShard (2020), Switch Transformer (2021) and many of today's frontier models.

09 🎨 Generative Vision Becomes Controllable

GANs that translate between image domains, with or without paired examples, and train more stably.

✏️

pix2pix: Image-to-Image Translation with Conditional Adversarial Networks

Isola, Zhu, Zhou & Efros (UC Berkeley) · CVPR 2017
📄 Accepted paper
🧒 In plain English

Give a network many pairs of pictures — a sketch and the photo, a map and the aerial view — and it learns to turn one into the other. The same method works for all of them; only the training pairs change.

▶ One framework, many translations Pick a task
🔍 What they found
  • Conditional GAN with a U-Net generator and a PatchGAN discriminator that judges local image patches
  • One general framework learns both the mapping and the loss, across many paired translation tasks
💡 Why it matters

Made conditional generation practical; its community demos (e.g. #edges2cats) spread widely.

🦓

CycleGAN: Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks

Zhu, Park, Isola & Efros (UC Berkeley) · ICCV 2017
📄 Accepted paper
🧒 In plain English

There are no photos of the same horse as a zebra. CycleGAN learns horse→zebra and zebra→horse at the same time, with one rule: turn a horse into a zebra and back, and you should get the original horse.

▶ Cycle consistency Press translate
🔍 What they found
  • Two generators (G: X→Y, F: Y→X) with adversarial losses plus a cycle-consistency loss F(G(x)) ≈ x
  • Unpaired translation: horses↔zebras, summer↔winter, photos↔paintings in the style of Monet, Van Gogh and others
💡 Why it matters

Removed the need for paired data; cycle consistency reappears in many later methods.

📏

Wasserstein GAN

Arjovsky, Chintala & Bottou (NYU, Facebook AI) · ICML 2017
📄 Accepted paper
🧒 In plain English

GANs were notoriously hard to train, and their loss number told you almost nothing. WGAN measures the gap between real and fake images as the effort needed to move one pile of sand into the shape of another. That measure gives a smooth signal, so training is steadier and the loss actually tracks image quality.

▶ Why the distance matters Drag the fake distribution
🔍 What they found
  • Replaces the Jensen–Shannon-based GAN objective with an approximation of the Wasserstein (earth mover's) distance, using a weight-clipped critic
  • More stable training, less mode collapse, and a loss that correlates with sample quality
  • Weight clipping was later replaced by a gradient penalty (WGAN-GP, NIPS 2017)
💡 Why it matters

Put GAN training on firmer theoretical ground, feeding into BigGAN and StyleGAN.

10 🧿 Alternative Architectures and Representation Learning

RNNs? CNNs? Capsules? Attention? In 2017 the winner wasn't yet clear.

🧿

Dynamic Routing Between Capsules

Sabour, Frosst & Hinton (Google Brain) · NIPS 2017
📄 Accepted paper
🧒 In plain English

A normal image network detects "eye", "nose" and "mouth" but not always how they fit together. Capsules output vectors that also encode pose, and lower-level parts "vote" for the whole object; parts whose votes agree get routed together.

▶ Routing by agreement Toggle
🔍 What they found
  • Capsules: groups of neurons whose activity vector represents an entity's presence (length) and pose (orientation)
  • Routing-by-agreement sends each part's output to the parent capsules its prediction agrees with
  • 0.25% test error on MNIST, and much better recognition of heavily overlapping digits (MultiMNIST) than a comparable CNN
💡 Why it matters

Didn't become the dominant architecture, but influenced work on equivariance, part–whole structure and object-centric learning.

11 Self-Attention, From Scratch

The Transformer's core operation takes a dozen lines. Each token builds a query, a key and a value; attention weights come from comparing queries with keys, and each token's new vector is a weighted mix of all the values:

"""Self-attention from scratch: the core operation of the Transformer.

Attention(Q, K, V) = softmax(Q K^T / sqrt(d)) V   (Vaswani et al., 2017)

Each token asks a question (query), advertises what it contains (key), and
carries information (value). Every token looks at every other token in ONE
step, so all positions can be computed in parallel - unlike an RNN, which
must process position 1 before position 2. Toy 4-dimensional vectors,
hand-picked so the result is easy to read. Standard library only.
"""
import math

tokens = ["The", "animal", "was", "tired"]
# toy embeddings: dims = [is-noun, is-verb, animacy, state]
X = [[0.1, 0.0, 0.0, 0.0], [1.0, 0.0, 1.0, 0.0], [0.0, 1.0, 0.0, 0.2], [0.0, 0.3, 0.2, 1.0]]
# projections chosen so "state" words query for animate nouns
WQ = [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 1, 0], [0, 0, 3, 0]]
WK = [[1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [0, 0, 0, 1]]
WV = WK

mat = lambda A, B: [[sum(a * b for a, b in zip(r, c)) for c in zip(*B)] for r in A]
Q, K, V = mat(X, WQ), mat(X, WK), mat(X, WV)
d = len(Q[0])

def softmax(xs):
    m = max(xs); e = [math.exp(x - m) for x in xs]; s = sum(e)
    return [v / s for v in e]

weights = [softmax([sum(q * k for q, k in zip(Q[i], K[j])) / math.sqrt(d) for j in range(len(tokens))]) for i in range(len(tokens))]
out = mat(weights, V)

print("Attention weights (row = token asking, column = token attended to)")
print(f"{'':>8}" + "".join(f"{t:>8}" for t in tokens))
for t, row in zip(tokens, weights):
    print(f"{t:>8}" + "".join(f"{w:>8.2f}" for w in row))
i = tokens.index("tired")
print(f"\n'tired' attends most to '{tokens[max(range(4), key=lambda j: weights[i][j])]}'"
      f" (weight {max(weights[i]):.2f}): its new vector mixes in the noun it describes.")
print("   new 'tired' vector:", [round(v, 2) for v in out[i]])

print("\nSequential steps needed to connect the first and last token")
print(f"{'length':>8} {'RNN':>6} {'self-attention':>15}")
for n in [4, 64, 1024]:
    print(f"{n:>8} {n - 1:>6} {1:>15}")
print("\nAll pairs interact in one layer: more parallel, and shorter paths for long-range dependencies.")

Output:

Attention weights (row = token asking, column = token attended to)
             The  animal     was   tired
     The    0.25    0.25    0.25    0.25
  animal    0.21    0.35    0.21    0.23
     was    0.23    0.31    0.23    0.24
   tired    0.12    0.59    0.12    0.17

'tired' attends most to 'animal' (weight 0.59): its new vector mixes in the noun it describes.
   new 'tired' vector: [0.61, 0.17, 0.63, 0.19]

Sequential steps needed to connect the first and last token
  length    RNN  self-attention
       4      3               1
      64     63               1
    1024   1023               1

All pairs interact in one layer: more parallel, and shorter paths for long-range dependencies.

In a real Transformer the projection matrices are learned, there are many attention heads per layer and dozens of layers, and vectors have hundreds or thousands of dimensions. The principle is the same: every token looks at every other in a single step. That is why Transformers train so efficiently on GPUs, and why distant words can influence each other directly.

12 The 2017 → 2026 Roadmap

The whole series in one line. Press play:

▶ Ten years, ten stages Click a stage or press play
2017Invent the architecture and learning loops
→
2018Reusable pretraining
→
2019Pretraining dominant
→
2020Predictable scaling
→
2021Foundation-model stack
→
2022Reasoning, alignment, action
→
2023Programmable platform
→
2024Systems around the model
→
2025Reasoning and agency
→
2026Persistent self-improvement
AI research in 2017, in one paragraph
2017 supplied several of modern AI's deepest primitives at once. The Transformer became the architecture. AlphaGo Zero made self-play a way to generate experience, and PPO a scalable learning loop. Human preferences became a reward signal. NAS and PBT used automated search and selection to design and tune future models, and sparse experts opened a route to enormous capacity. pix2pix, CycleGAN and WGAN made image generation controllable and more stable, and capsules explored a road not taken. Experience generation, candidate generation, selection and learned evaluation are the ingredients that recursive self-improvement research is still combining in 2026.

Next: AI Research in 2016: The Year Deep Learning Became a General-Purpose Engine →

⚠️ Notes

About the source infographic and brief: (1) AlphaGo Zero's "100 : 0" was against the version that beat Lee Sedol, after three days of training; against the stronger AlphaGo Master it won 89–11 after 40 days; (2) the WGAN panel shows photorealistic dogs, birds and castles; images of that quality came from later models such as BigGAN (2018), while the WGAN paper showed small, low-resolution samples such as 64×64 LSUN bedrooms; (3) the "example discovered architecture" with separable-convolution and max-pool branches resembles the NASNet cell from the follow-up paper (Zoph et al., posted July 2017), not the original NAS paper, which searched over layer settings and a recurrent cell; (4) the MoE "2 of 64 experts" is a simplified example — the paper used noisy top-k gating over up to thousands of experts. Other figures checked out against the papers (under 1% human feedback, 137B parameters, MNIST results for capsules, PPO's clipped objective). No paper here won a NIPS 2017 Best Paper award; recognition came later. Widget numbers not taken from a paper are marked illustrative.

🔗 All References