🗺️ Frontier AI Models
A Decade of Breakthroughs
Every landmark model from 2017 to June 2026 — key innovations, differentiating factors, competitive moats, and how each shapes the future of AI.
● OpenAI
● Anthropic
● Google
● Mistral
● DeepSeek
● xAI
● Others
2017
Google Brain
Jun 2017
Transformer
"Attention Is All You Need"
InnovationSelf-attention replaces RNNs entirely. Fully parallelizable training unlocked massive scale.
DifferentiatorNo sequential bottleneck — every token attends to every other token simultaneously.
MoatFoundational architecture — every major LLM (GPT, BERT, Claude, Gemini, LLaMA) is a Transformer variant.
Future PositionArchitecture still dominant in 2026; attention variants (MLA, GQA, FlashAttention) refine but don't replace it.
2018
OpenAI
Jun 2018
GPT-1
117M parameters · Generative pre-training
InnovationFirst demonstration that unsupervised pre-training + supervised fine-tuning generalizes across NLP tasks.
DifferentiatorLeft-to-right language modelling at a scale that generalized without task-specific architectures.
MoatProof-of-concept that sparked OpenAI's GPT scaling roadmap.
Google
Oct 2018
BERT
340M parameters · Bidirectional Encoder
InnovationMasked language modelling — attends to left AND right context simultaneously. Shattered 11 NLP benchmarks.
DifferentiatorBidirectional understanding vs GPT's left-to-right; superior for classification, NER, Q&A.
MoatSearch engine integration (Google Search 2019) — biggest real-world NLP deployment ever.
2019
OpenAI
Feb 2019
GPT-2
1.5B parameters · "Too dangerous to release"
InnovationCoherent long-form text generation at an alarming quality level. First model to raise mainstream AI safety concerns.
DifferentiatorZero-shot task performance without fine-tuning — hints at emergent capabilities from scale.
MoatMedia moment that established OpenAI's brand as frontier-first AI lab.
Google
Oct 2019
T5
11B parameters · Text-to-Text Transfer Transformer
InnovationUnified framework: every NLP task reformulated as text-in → text-out. Enabled massive multi-task training.
DifferentiatorEncoder-decoder architecture; better than BERT on generation tasks while retaining understanding.
MoatFoundation for PaLM, Flan-T5, and instruction-tuning research lineage.
2020
OpenAI
May 2020
GPT-3 👑 Watershed
175B parameters · In-context learning era begins
InnovationFew-shot prompting without gradient updates. Emergent capabilities (arithmetic, translation, code) never trained for explicitly.
DifferentiatorAPI-first monetization model. First LLM sold as a product, not a paper. Spawned an entire startup ecosystem.
Moat$12M in compute, exclusive Azure partnership, 175B weights. Competitors took 2+ years to match.
Future PositionTemplate for the LLM-as-API business model that defines the industry today.
2021
OpenAI
Aug 2021
Codex
GPT-3 fine-tuned on 54M GitHub repos
InnovationFirst model to write production-quality code from natural language. Powers GitHub Copilot.
DifferentiatorCode as a modality. Proved domain-specific fine-tuning could unlock developer productivity at scale.
MoatGitHub partnership (Microsoft) gave Copilot immediate distribution to 80M+ developers.
2022
OpenAI
Jan 2022
InstructGPT
RLHF · "Align to human intent, not next token"
InnovationReinforcement Learning from Human Feedback (RLHF) at scale. Models that follow instructions rather than just predict text.
DifferentiatorSmaller 1.3B InstructGPT outperformed 175B GPT-3 on human preference — quality over scale.
MoatRLHF methodology became the standard alignment technique. Direct precursor to ChatGPT.
Google / DeepMind
Mar 2022
Chinchilla
70B parameters · Scaling laws rewritten
InnovationChinchilla scaling laws: optimal training requires 20× more tokens per parameter than previously believed. Smaller models on more data beat larger models on less.
Differentiator70B Chinchilla matched 280B Gopher. Shifted the entire industry toward compute-optimal training.
MoatPublished paper; open knowledge. Entire industry (LLaMA, Mistral, Phi) uses these laws as baseline.
Google
Apr 2022
PaLM
540B parameters · Chain-of-thought emerges
InnovationFew-shot chain-of-thought prompting discovered — step-by-step reasoning unlocks math and logic at large scale.
DifferentiatorFirst to show reasoning can be elicited by prompting alone, without fine-tuning.
MoatGoogle's TPU infrastructure; Pathways training system across 6144 chips simultaneously.
OpenAI
Nov 2022
ChatGPT 👑 Industry-defining
GPT-3.5 + RLHF · 1M users in 5 days
InnovationConsumer-facing chat interface for a capable LLM. Made AI accessible to non-technical users overnight.
DifferentiatorFastest product to 100M users in history (2 months). Created the "AI assistant" category.
MoatBrand recognition, first-mover consumer trust, Microsoft $10B investment, Azure integration.
Future PositionEstablished OpenAI as the default public face of AI. Competition forced every major tech company to accelerate.
2023
Feb 2023
LLaMA 1
7B–65B · Open weights · Research license
InnovationFirst open-weights competitive LLM. Democratized research — anyone with a consumer GPU could run frontier-quality models.
Differentiator65B LLaMA matched GPT-3 quality. Weights leaked; community explosion (Alpaca, Vicuna, WizardLM).
MoatOpen-source ecosystem lock-in. Meta's distribution through open weights became a strategic moat against closed labs.
OpenAI
Mar 2023
GPT-4 👑 New SOTA
Multimodal · Near-human on professional exams
InnovationFirst frontier multimodal model. Passed bar exam (top 10%), USMLE, GRE. Reliable instruction-following at scale.
DifferentiatorReliability jump over GPT-3.5. Much less hallucination. First model enterprises trusted for production use.
MoatClosed weights, post-training recipe, safety evals. RLHF/RLAIF at massive scale. Enterprise contracts.
Future PositionBenchmark that all 2023-2024 models raced to beat. Still competitive 18 months after release.
Anthropic
Mar 2023
Claude 1
Constitutional AI · Safety-first alignment
InnovationConstitutional AI (CAI): model critiques and revises its own outputs against a set of principles. Reduces need for human labeling in RLHF.
DifferentiatorPositioned on safety and honesty over raw capability. Long context (100K tokens) vs GPT-4's 8K.
MoatEx-OpenAI safety team, unique alignment research, Amazon $4B investment, Bedrock distribution.
Google
May 2023
PaLM 2 / Bard
Multilingual · Medical (Med-PaLM 2) · Code
InnovationStrong multilingual performance (100+ languages). Med-PaLM 2 reached expert physician-level on medical benchmarks.
DifferentiatorChinchilla-optimal training. Powers Google Workspace (Duet AI), Search, Android.
MoatGoogle's distribution across 3B+ Android devices and $280B Search revenue stream.
Jul 2023
LLaMA 2
7B–70B · Commercial license · Microsoft partnership
InnovationFirst open-weights model with commercial use allowed. Ghost Attention for multi-turn consistency. Fine-tuned on 1M human annotations.
DifferentiatorFree commercial use — thousands of companies built products on LLaMA 2 without API costs.
MoatEcosystem: Hugging Face, Azure Marketplace, AWS, embedded in countless startup stacks.
Mistral AI
Sep 2023
Mistral 7B
7B · Grouped-query attention · Sliding window attention
Innovation7B model outperformed 13B LLaMA 2 across all benchmarks. Sliding window attention for efficient long-context. Apache 2.0 license.
DifferentiatorPerformance per parameter record-holder. Proved data quality and architecture choices beat brute-force scale.
MoatEuropean AI sovereignty narrative, $113M Series A at $260M valuation on model quality alone. Andreessen Horowitz backed.
Mistral AI
Dec 2023
Mixtral 8×7B
46.7B total · 12.9B active · Mixture of Experts
InnovationSparse Mixture-of-Experts (MoE) at 8 experts — only 2 activate per token. Same cost as 12.9B dense model with 46.7B capacity.
DifferentiatorMatched GPT-3.5 on most benchmarks at a fraction of inference cost. MoE democratized efficient large models.
MoatMoE architecture blueprint — GPT-4, Gemini 1.5, and most 2024-2025 frontier models use MoE.
Google DeepMind
Dec 2023
Gemini 1.0
Natively multimodal · Ultra / Pro / Nano tiers
InnovationFirst model trained natively on text, code, audio, image, and video simultaneously — not retrofitted. Three-tier deployment (Ultra/Pro/Nano) for device-to-cloud.
DifferentiatorAlphaCode 2 integration (competitive programming). Nano runs on-device (Pixel 8 Pro).
MoatGoogle's unique TPU v4/v5 infrastructure, Search integration at $280B revenue scale, Android device footprint.
Microsoft Research
Dec 2023
Phi-2
2.7B · "Textbooks are all you need"
InnovationProved data quality > data quantity. Synthetic "textbook-quality" data let 2.7B Phi-2 outperform many 13B models.
DifferentiatorRuns on phone/laptop. Microsoft's counter-narrative to the scale arms race.
MoatEdge AI positioning — where cloud APIs are too expensive, private, or latent.
2024
Google DeepMind
Feb 2024
Gemini 1.5 Pro
1M token context · Multi-needle-in-haystack
Innovation1 million token context window — equivalent to 1 hour of video, 11 hours of audio, or the entire codebase of a large app in a single prompt.
DifferentiatorMoE architecture. "Multi-needle" retrieval: found specific facts buried in 750K tokens with near-perfect accuracy.
MoatLong context is infrastructure-expensive — Google's TPU advantage enables what costs competitors 10× more.
Anthropic
Mar 2024
Claude 3 (Haiku / Sonnet / Opus)
First model family to beat GPT-4 head-to-head
InnovationTiered model family at three price-performance points. Claude 3 Opus became #1 on Chatbot Arena — first non-OpenAI model to hold the top spot.
Differentiator200K context, near-perfect recall, vision across all tiers. Haiku fastest small model in class at launch.
MoatEnterprise safety positioning, AWS Bedrock exclusivity, Amazon $4B investment, Constitutional AI differentiation.
Apr 2024
LLaMA 3
8B & 70B · First open-source model competitive with GPT-4
InnovationGQA across all sizes, 15T token training corpus (2× LLaMA 2), 128K vocab tokenizer. LLaMA 3 70B reached MMLU 82%.
DifferentiatorFirst open model truly competitive with closed frontier models — democratized "GPT-4-class" capabilities.
MoatMeta's 600M+ daily users across WhatsApp, Instagram, Facebook as distribution channel for Meta AI.
OpenAI
May 2024
GPT-4o
Omni · Real-time audio-visual · "Her" moment
InnovationTruly multimodal — text/audio/image in a single unified model (not stitched). Real-time voice conversation with natural emotional tone, <320ms latency.
DifferentiatorEnd-to-end audio (not ASR+LLM+TTS pipeline). GPT-4 quality at GPT-3.5 price. Free tier for all users.
MoatReal-time voice moat — custom audio tokenizer trained end-to-end; competitors needed 12+ months to match.
Future PositionAdvanced Voice Mode template — now powering phone calls, customer service, companion apps at millions of users.
Anthropic
Jun 2024
Claude 3.5 Sonnet
Best-in-class coding · Artifacts · Computer Use preview
InnovationDethroned GPT-4o on coding benchmarks (HumanEval 92%). Artifacts feature — generate and preview runnable code in-chat. First computer use demo.
DifferentiatorSonnet-tier pricing (2× cheaper than Opus) with Opus-beating capability — best price-performance in history at launch.
MoatDeveloper adoption flywheel — most used Anthropic model; most cited in "Claude is my preferred coding model" surveys.
OpenAI
Jul 2024
GPT-4o mini
Replaces GPT-3.5 · 15× cheaper than GPT-4
InnovationGPT-4-class quality at near-GPT-3.5 price ($0.15/M input tokens). Obsoleted GPT-3.5 overnight.
DifferentiatorMultimodal at tiny model price. Enabled AI features at scale in apps with thin margins.
MoatOpenAI's infrastructure efficiency advantage — same quality at lower compute through distillation.
Jul 2024
LLaMA 3.1 405B
First open-source frontier model · GPT-4 competitive
Innovation405B open-weights model matching GPT-4o on many benchmarks. First time "frontier" capability was freely downloadable.
Differentiator128K context, multilingual (8 languages). Mark Zuckerberg declared open-source AI a strategic imperative for Meta.
MoatMeta becomes "open-source AI infrastructure" — LLaMA 3.1 runs on 16,000+ H100 clusters, enabling on-premise deployments no API can match.
OpenAI
Sep 2024
o1 👑 Paradigm Shift
Think before you answer · Chain-of-thought at inference
InnovationReinforcement learning on reasoning traces — model learns to think through problems step-by-step before answering. PhD-level on GPQA (78%). IMO gold-medal level math.
DifferentiatorScaling inference compute (not just training) unlocks new capability. Fundamentally different from all prior models.
MoatProcess reward models, custom RL training infrastructure. Inference-time compute scaling opened a new scaling axis competitors scrambled to replicate.
Future PositionEstablished inference-time scaling as the dominant research direction for 2025-2026.
Sep 2024
LLaMA 3.2
Vision (11B/90B) · Edge models (1B/3B)
InnovationFirst LLaMA with vision. 1B/3B models specifically designed for on-device use (Snapdragon, Apple Silicon).
DifferentiatorTrue multimodal open-weights. Edge models enable offline AI on phones — no API required.
MoatiPhone/Android on-device distribution — privacy-preserving AI that needs no internet.
Google DeepMind
Dec 2024
Gemini 2.0 Flash
Agentic · Multimodal streaming · Real-time
InnovationFirst "agentic" model built for multi-step tasks — native tool use, code execution, search grounding. Real-time audio/video streaming.
DifferentiatorSimultaneous input AND output of text, audio, image, video. Thinking mode (Flash Thinking) — Gemini's answer to o1.
MoatGoogle's real-time infrastructure, Search grounding with live web data — no hallucinations on current events.
DeepSeek
Dec 2024
DeepSeek V3 🌏 China Frontier
671B MoE · GPT-4o quality · $6M training cost
Innovation671B MoE (37B active) matching GPT-4o/Claude 3.5 on benchmarks for a reported $6M training cost — 10-50× cheaper than US labs. Multi-head Latent Attention (MLA) dramatically reduces KV cache.
DifferentiatorOpen-weights frontier model from China. $0.27/M input tokens vs GPT-4o's $5/M — 18× cheaper.
MoatEfficiency research moat — MLA, FP8 training, pipeline parallelism innovations publishable and widely adopted.
Future PositionTriggered global cost recompression. Forced all API providers to cut prices 30-60% in Q1 2025.
OpenAI
Dec 2024
o3 (Preview)
ARC-AGI: 87.5% · PhD-level across all disciplines
InnovationScored 87.5% on ARC-AGI (abstract reasoning benchmark humans score 85%). First model to cross human-level on this task. 25.2% on FrontierMath.
DifferentiatorInference compute scaling pushed further — longer thinking budgets yield measurably better answers.
MoatExtended thinking architecture, custom process reward models, safety infrastructure for high-capability models.
2025
DeepSeek
Jan 2025
DeepSeek R1 👑 Market Shock
Open-source o1 competitor · Dropped NASDAQ 3%
InnovationOpen-source reasoning model matching o1 on math, coding, science. Trained using Group Relative Policy Optimization (GRPO) — novel RL without value networks.
DifferentiatorFully open weights AND reasoning traces. API at $3/M tokens vs o1's $60/M — 20× cheaper. Wiped $600B from Nvidia market cap in one day.
MoatEfficiency research discipline + open publication strategy — made DeepSeek a household name globally in 48 hours.
Future PositionSparked "efficiency race" — labs pivoted from "scale more" to "train smarter." GRPO adopted widely across the industry.
Anthropic
Feb 2025
Claude 3.7 Sonnet
Hybrid reasoning · Extended thinking · SWE-bench #1
InnovationFirst model with switchable extended thinking — users control reasoning depth and cost. Top SWE-bench score (62.3%) — best software engineering AI at launch.
DifferentiatorHybrid: standard responses for simple tasks, deep reasoning for complex ones. Cost-adaptive thinking budget.
MoatAgentic coding workflows — Claude Code launched same month; dominant in developer tools (Cursor, Windsurf, VS Code).
xAI
Feb 2025
Grok 3
2T token training · DeepSearch · Think mode
InnovationTrained on 2 trillion tokens of X/Twitter data + web. DeepSearch: real-time web research with citation. Think mode: extended reasoning chain.
DifferentiatorReal-time social data training — knows current events better than any model. 100,000 H100 cluster (Colossus) — largest training run ever at time.
MoatX/Twitter data exclusivity: unique social signals no other lab can replicate. Elon's built-in distribution (X Premium subscribers).
OpenAI
Feb 2025
GPT-4.5
Largest dense model · Emotional intelligence · $75/M
InnovationOpenAI's largest dense (non-MoE) model. Strongest "EQ" — nuanced, emotionally calibrated responses. Near-perfect factual recall.
DifferentiatorFor creative writing, long-form content, and tasks where style matters more than reasoning depth.
MoatDistillation target — GPT-4.5 knowledge used to train smaller models in o3/GPT-5 family.
Google DeepMind
Mar 2025
Gemini 2.5 Pro 👑 #1 All Benchmarks
Reasoning + Long context · Topped all major leaderboards
InnovationFirst model to simultaneously top coding (SWE-bench), reasoning (GPQA), math (AIME), and long-context (MRCR) leaderboards. Thinking budget control. 1M context + 2M in preview.
Differentiator"Think" vs "don't think" toggle. Deep Code Execution — runs actual code during reasoning, verifies answers.
MoatTPU v5e/v5p cluster advantage. Google's 10-year TPU investment paid off — unmatched long-context efficiency.
Future PositionPositioned as the research/enterprise workhorse — Deep Research feature processes hundreds of sources per query.
OpenAI
Apr 2025
o3 & o4-mini
Multimodal reasoning · Tool use in chain-of-thought
InnovationFirst reasoning models with native vision — can reason over images during chain-of-thought. Tools (code execution, web search) invoked mid-reasoning, not just at output.
Differentiatoro4-mini: tiny model with o3-level reasoning at 10× lower cost. Best "intelligence per dollar" at launch.
MoatAgentic infrastructure — models that use tools while thinking are qualitatively different from models that think then use tools.
Apr 2025
LLaMA 4 (Scout / Maverick / Behemoth)
10M context · Natively multimodal MoE · 2T params (Behemoth)
InnovationScout: 17B (active) / 109B total MoE with 10M token context — longest context of any open model. Maverick: 17B active beats GPT-4o. Behemoth: 2T parameter training model, STEM benchmark record.
DifferentiatorisoFLOP multimodal training from scratch — not retrofitted vision. Natively understands interleaved images in 10M-context conversations.
Moat10M context is a new compute moat. Meta's 350K+ H100 cluster only lab besides Google that can sustain such context economics.
Alibaba (Qwen)
Apr 2025
Qwen3
Hybrid thinking · 0.6B to 235B · Top open-source
InnovationFull model family from 0.6B edge models to 235B MoE flagship. Hybrid thinking mode (on/off). Best open-source model at time of release on most benchmarks.
Differentiator119 languages, strongest multilingual reasoning. Qwen3-235B matched Claude 3.7 and GPT-4.1 on key benchmarks.
MoatAlibaba Cloud distribution, Taobao/AliExpress integration — massive Asian market. China's answer to LLaMA dominance.
Anthropic
May 2025
Claude 4 (Sonnet 4 & Opus 4)
Agentic memory · Parallel tool use · Computer Use GA
InnovationFirst Claude models with persistent memory across sessions. Parallel tool calls — multiple browser/code tasks simultaneously. Computer Use generally available: controls entire desktop autonomously.
DifferentiatorSonnet 4 best for agentic workflows. Opus 4 highest-capability research model. Both designed for multi-hour autonomous tasks.
MoatEnterprise agentic positioning — Claude Agents for Salesforce, AWS, Slack. SDK + Claude Code CLI deep developer integration.
Google DeepMind
May–Jun 2025
Gemini 2.5 Flash
Thinking budget · Fastest reasoning model · Free tier
InnovationControllable thinking budget (0 to 24,576 tokens). User pays only for reasoning compute actually used. Fastest reasoning model at comparable quality.
DifferentiatorCost-efficient reasoning at scale — 10× cheaper than 2.5 Pro while retaining 80% of capability for most tasks.
MoatFree Gemini API tier makes it the default for startups. Generative Media integration (Veo, Imagen, Lyria) in same API.
OpenAI
Jun 2025
GPT-5 & o3 Pro
Unified model · "Best OpenAI model ever" · Agentic scale
InnovationGPT-5 collapses reasoning and standard response into one model — user doesn't choose thinking mode, model decides. o3 Pro: unlimited compute budget for hardest problems.
DifferentiatorOperator-level API for enterprise agentic deployments. Multi-agent orchestration built-in — spawns subagents for subtasks.
Moat$300B+ enterprise pipeline, ChatGPT's 300M+ weekly users, Azure OpenAI Service for regulated industries.
Future PositionOpenAI's first genuinely "general" model — positioned for autonomous knowledge work without specialist prompting.
2026
Anthropic
Early 2026
Claude 4.5 / Sonnet 4.6
Enhanced agentic · Web browsing · Deeper memory
InnovationContinuous improvement on agentic capabilities — multi-hour autonomous sessions, proactive planning, improved computer use reliability.
DifferentiatorIterative release cadence — Claude models improving on 2-3 month cycles. Best-in-class code generation sustained.
MoatClaude Code CLI, deep IDE integrations (Cursor, VS Code), enterprise workflow embedding.
DeepSeek
Early 2026
DeepSeek R2 / V4
Next-gen efficiency · Continued cost disruption
InnovationBuilding on R1/V3 efficiency architecture — expected further MLA and training optimizations. Multimodal reasoning capabilities.
DifferentiatorContinued open-weights strategy — deepening US-China competitive dynamic in AI.
MoatEfficiency research + open publication loop — each DeepSeek release resets cost expectations globally.
2026
LLaMA 5
Open AGI-adjacent · Meta's strategic bet on openness
InnovationExpected: full multimodal + reasoning + agentic capabilities in open-weights form. Training on Meta's proprietary social graph and AR/VR data.
DifferentiatorFirst open model integrating social + AR/VR + web data at scale. Ray-Ban Meta glasses as primary deployment vector.
Moat600M+ daily users, proprietary multimodal social data, wearable hardware distribution — unique advantage no competitor can replicate.
Google DeepMind
2026
Gemini 3.0 / Project Astra
Always-on ambient AI · Real-world grounding · Google I/O 2026
InnovationProject Astra: AI that sees through your phone/glasses continuously, remembers context across days, acts as a persistent personal assistant. Gemini 3.0 as its foundation.
DifferentiatorAmbient/persistent AI vs session-based chat. Real-world grounding through Google Maps, Calendar, Gmail ecosystem access.
MoatAndroid + Search + Maps + Calendar data gives Google the richest personal context of any AI assistant. Unmatched distribution through 3B+ devices.
📡 Where the Frontier is Heading — H2 2026 and Beyond
Agentic by Default
Models are no longer chat assistants — they are autonomous workers completing multi-hour tasks. The shift from "answer questions" to "complete projects" is the defining transition of 2026.
Inference-Time Scaling
The next capability gains come from thinking longer, not training larger. Models with "thinking budgets" of hours will surpass models with training budgets of millions of dollars.
Multimodal is Table Stakes
Any model without vision, voice, and code execution will be uncompetitive by end of 2026. The new competition is on reliability, memory, and real-world grounding.
Open vs Closed Narrows
LLaMA 4, DeepSeek R2, and Qwen3 prove open-source reaches GPT-4 level within 6 months of release. The "open/closed capability gap" may close for all but the absolute frontier by 2027.
Price Compression Continues
GPT-3 API cost in 2020: $60/M tokens. GPT-4o equivalent in 2025: $2.50/M. DeepSeek R2 equivalent: <$1/M. By 2027, frontier inference approaches pennies per million tokens.
Compute as Moat Weakens
DeepSeek proved frontier models can train on 2K GPUs instead of 16K. Hardware moats are softening; algorithm and data quality are the new durable advantages.