The ML Layer, Mapped: From PyTorch to Production in One Diagram
Between a rack of GPUs and an agent answering a question sit about eight layers of software: kernels and compilers, a framework, parallelism libraries, post-training and RL tools, evaluation harnesses, an inference engine and an agent runtime. Most of it is open source, and much of it now lives under one foundation. This first part draws the whole map, shows what the big labs use at each layer and traces one request through it; the rest of the series zooms in layer by layer.
1. Why a map
The Who Runs the AI Factory series followed the people who own each layer of the AI factory: kernel engineers, ML systems engineers, inference engineers. This series follows the software itself. A model that answers your question has passed through code written by dozens of projects and thousands of contributors, and choosing that code is one of the biggest decisions a lab or startup makes: it decides which chips you can use, how fast you train, how cheaply you serve and how easily you hire.
2. The eight layers
The ML layer of the AI factory
Top: closest to users. Bottom: closest to the chips. Tap a layer for what it does, the main open-source options, who owns it and which part covers it.
Two things cut across the layers. The communication libraries (NVIDIA's NCCL and its equivalents, DeepSeek's DeepEP for mixture-of-experts traffic) move data between GPUs for both training and inference. And the hardware abstraction at the bottom decides which chips the rest can use: CUDA for NVIDIA, ROCm for AMD, Neuron for Amazon's Trainium and XLA for Google's TPUs.
3. Trace a request
Two journeys through the stack
Pick a journey and step through it. Each step lights up the layer involved. Simplified.
4. Who governs the open stack
Much of the open ML stack now sits under one roof. Meta handed PyTorch to the newly formed PyTorch Foundation, part of the Linux Foundation, in 2022. In May 2025 the foundation became an umbrella for other projects, accepting the vLLM inference engine and Microsoft's DeepSpeed training library; Ray, the distributed computing framework, joined in October 2025, followed by Helion and Safetensors. PyTorch itself keeps moving fast: version 2.13 shipped with 3,328 commits from 526 contributors.
The PyTorch Foundation's projects
Which layer each hosted project sits in.
Google's stack (JAX, XLA, Pathways and MaxText) sits outside the foundation, as do NVIDIA's projects (Megatron-Core, TensorRT-LLM, NCCL), OpenAI's Triton compiler and SGLang, an inference engine that grew out of university research. The foundation matters because it gives competitors a neutral place to collaborate on shared plumbing.
5. PyTorch vs JAX
The framework layer has converged on two choices. PyTorch dominates research and most industry training; estimates of its share of research papers range from about 55% to 85% in 2026 comparisons โ ๏ธ. JAX, built on Google's XLA compiler, is the natural choice on TPUs and is used across Google DeepMind. TensorFlow, once the leader, is now mostly found in older production systems.
Two philosophies
Share estimates from blog comparisons that disagree with each other โ ๏ธ; we found no authoritative 2026 count. Shown as ranges.
6. What the labs use
Lab ร layer
Publicly disclosed choices only. Grey cells mean not disclosed. Tap a cell for the source.
The best-documented stacks belong to the labs that publish. Meta trains Llama on PyTorch with its own parallelism and fault-tolerance code, followed end to end in Producing Intelligence. Google DeepMind uses JAX, XLA and Pathways on TPUs (Case Study 6). DeepSeek released much of its in-house infrastructure during an "Open Source Week" from 24 to 28 February 2025: FlashMLA (attention decoding kernels for Hopper GPUs), DeepEP (expert-parallel communication for mixture-of-experts models), DeepGEMM (FP8 matrix multiplication), DualPipe (a bidirectional pipeline-parallel schedule) and the 3FS file system with the smallpond data framework; sources list the five daily releases slightly differently. xAI said when it announced Grok-1 in 2023 that it used a custom training and inference stack built on JAX, Rust and Kubernetes. OpenAI created the Triton GPU programming language but says little about the rest of its stack, and Anthropic, which trains on Trainium, TPUs and NVIDIA GPUs, has not described its framework publicly.
7. Build or borrow
Who builds which layers in-house
A rough editorial reading. Bars show how much of the stack each type of organisation tends to build itself.
Frontier labs rebuild layers when off-the-shelf code becomes their bottleneck at scale, when they use unusual hardware or architectures (DeepSeek's kernels exist because of its mixture-of-experts and FP8 design), or to keep an edge. Startups usually assemble the open stack: PyTorch, a parallelism library, TRL or a similar RL toolkit, an eval harness and vLLM or SGLang for serving, which is the stack assumed in the frontier lab playbook.
8. Timeline
The open ML stack, 2015 โ 2026
Solid dots are confirmed by the project, hollow dots come from secondary sources. Click a dot.
9. The series
10. Reading the numbers
- Governance
- PyTorch Foundation formed 2022 under the Linux Foundation; vLLM and DeepSpeed accepted 7 May 2025; Ray October 2025; Helion and Safetensors later; PyTorch 2.13: 3,328 commits, 526 contributors (PyTorch newsletter).
- Labs
- DeepSeek Open Source Week 24โ28 Feb 2025 (repo index; lineup varies by source); xAI JAX/Rust/Kubernetes (xAI, 2023); Meta PyTorch (Llama 3 paper); Google JAX/XLA/Pathways. OpenAI and Anthropic stacks largely undisclosed.
- Shares โ ๏ธ
- PyTorch research share estimates of ~55โ85% and JAX ~10% come from conflicting blog comparisons.
- Editorial
- The "build vs borrow" bars and layer descriptions are simplified.
- Projects and governance
- PyTorch Foundation expands, welcomes vLLM and DeepSpeed
- Linux Foundation: PyTorch Foundation welcomes Ray
- PyTorch newsletter, July 2026
- Wikipedia: PyTorch
- Wikipedia: vLLM
- Baseten: how open-source LLM training works
- Labs
- DeepSeek open-infra index
- MLOps newsletter: DeepSeek Open Source Week
- xAI: announcing Grok (2023)
- Meta: The Llama 3 Herd of Models
- Frameworks
- JAX vs PyTorch 2026 benchmarks
- PyTorch vs TensorFlow 2026 (research share)