Home โ€บ Blog โ€บ The ML layer, mapped
Inside the ML Layer ยท Part 1 ยท The map

The ML Layer, Mapped: From PyTorch to Production in One Diagram

Between a rack of GPUs and an agent answering a question sit about eight layers of software: kernels and compilers, a framework, parallelism libraries, post-training and RL tools, evaluation harnesses, an inference engine and an agent runtime. Most of it is open source, and much of it now lives under one foundation. This first part draws the whole map, shows what the big labs use at each layer and traces one request through it; the rest of the series zooms in layer by layer.

FL
FrontierAGI Team
โš ๏ธ How this was researched (as of 26 Oct 2026) Sources are the PyTorch Foundation's announcements and newsletter, project documentation, DeepSeek's open-infra repository index and coverage of its Open Source Week, Meta's Llama 3 paper, Google's JAX and Pathways material, xAI's Grok-1 announcement and framework comparisons. Labs disclose their in-house stacks unevenly: where we found nothing, the map says "not disclosed" rather than guessing. Framework market-share figures come from blogs that disagree, and are marked โš ๏ธ. See Reading the numbers.

1. Why a map

The Who Runs the AI Factory series followed the people who own each layer of the AI factory: kernel engineers, ML systems engineers, inference engineers. This series follows the software itself. A model that answers your question has passed through code written by dozens of projects and thousands of contributors, and choosing that code is one of the biggest decisions a lab or startup makes: it decides which chips you can use, how fast you train, how cheaply you serve and how easily you hire.

๐Ÿ’ก A stack, not a toolNo single project trains and serves a frontier model. Even the most vertically integrated labs combine an open-source framework with in-house parallelism, kernels and serving. The interesting question is where each lab draws the line between borrowing and building.

2. The eight layers

The ML layer of the AI factory

Top: closest to users. Bottom: closest to the chips. Tap a layer for what it does, the main open-source options, who owns it and which part covers it.

Two things cut across the layers. The communication libraries (NVIDIA's NCCL and its equivalents, DeepSeek's DeepEP for mixture-of-experts traffic) move data between GPUs for both training and inference. And the hardware abstraction at the bottom decides which chips the rest can use: CUDA for NVIDIA, ROCm for AMD, Neuron for Amazon's Trainium and XLA for Google's TPUs.

3. Trace a request

Two journeys through the stack

Pick a journey and step through it. Each step lights up the layer involved. Simplified.

4. Who governs the open stack

Much of the open ML stack now sits under one roof. Meta handed PyTorch to the newly formed PyTorch Foundation, part of the Linux Foundation, in 2022. In May 2025 the foundation became an umbrella for other projects, accepting the vLLM inference engine and Microsoft's DeepSpeed training library; Ray, the distributed computing framework, joined in October 2025, followed by Helion and Safetensors. PyTorch itself keeps moving fast: version 2.13 shipped with 3,328 commits from 526 contributors.

The PyTorch Foundation's projects

Which layer each hosted project sits in.

Google's stack (JAX, XLA, Pathways and MaxText) sits outside the foundation, as do NVIDIA's projects (Megatron-Core, TensorRT-LLM, NCCL), OpenAI's Triton compiler and SGLang, an inference engine that grew out of university research. The foundation matters because it gives competitors a neutral place to collaborate on shared plumbing.

5. PyTorch vs JAX

The framework layer has converged on two choices. PyTorch dominates research and most industry training; estimates of its share of research papers range from about 55% to 85% in 2026 comparisons โš ๏ธ. JAX, built on Google's XLA compiler, is the natural choice on TPUs and is used across Google DeepMind. TensorFlow, once the leader, is now mostly found in older production systems.

Two philosophies

PyTorchEager by default, Python-first, compiled when needed (torch.compile). Huge ecosystem: Megatron-Core, DeepSpeed, FSDP2, TorchTitan, vLLM. Runs on NVIDIA, AMD and others. Governed by the PyTorch Foundation.
JAXFunctional, compiled by XLA from the start; parallelism expressed by sharding arrays across a device mesh. Strongest on TPUs (MaxText, Pathways). Led by Google.

Share estimates from blog comparisons that disagree with each other โš ๏ธ; we found no authoritative 2026 count. Shown as ranges.

6. What the labs use

Lab ร— layer

Publicly disclosed choices only. Grey cells mean not disclosed. Tap a cell for the source.

Tap a cell.

The best-documented stacks belong to the labs that publish. Meta trains Llama on PyTorch with its own parallelism and fault-tolerance code, followed end to end in Producing Intelligence. Google DeepMind uses JAX, XLA and Pathways on TPUs (Case Study 6). DeepSeek released much of its in-house infrastructure during an "Open Source Week" from 24 to 28 February 2025: FlashMLA (attention decoding kernels for Hopper GPUs), DeepEP (expert-parallel communication for mixture-of-experts models), DeepGEMM (FP8 matrix multiplication), DualPipe (a bidirectional pipeline-parallel schedule) and the 3FS file system with the smallpond data framework; sources list the five daily releases slightly differently. xAI said when it announced Grok-1 in 2023 that it used a custom training and inference stack built on JAX, Rust and Kubernetes. OpenAI created the Triton GPU programming language but says little about the rest of its stack, and Anthropic, which trains on Trainium, TPUs and NVIDIA GPUs, has not described its framework publicly.

7. Build or borrow

Who builds which layers in-house

A rough editorial reading. Bars show how much of the stack each type of organisation tends to build itself.

Frontier labs rebuild layers when off-the-shelf code becomes their bottleneck at scale, when they use unusual hardware or architectures (DeepSeek's kernels exist because of its mixture-of-experts and FP8 design), or to keep an edge. Startups usually assemble the open stack: PyTorch, a parallelism library, TRL or a similar RL toolkit, an eval harness and vLLM or SGLang for serving, which is the stack assumed in the frontier lab playbook.

8. Timeline

The open ML stack, 2015 โ†’ 2026

Solid dots are confirmed by the project, hollow dots come from secondary sources. Click a dot.

Click a milestone.

9. The series

The takeawayThe ML layer is a stack of about eight layers, most of them open source and increasingly governed together. The frameworks have converged on PyTorch and JAX; the differences between labs show up lower down, in kernels and parallelism, and higher up, in serving and agents. The rest of this series takes each layer in turn, starting with the frameworks.

10. Reading the numbers

Governance
PyTorch Foundation formed 2022 under the Linux Foundation; vLLM and DeepSpeed accepted 7 May 2025; Ray October 2025; Helion and Safetensors later; PyTorch 2.13: 3,328 commits, 526 contributors (PyTorch newsletter).
Labs
DeepSeek Open Source Week 24โ€“28 Feb 2025 (repo index; lineup varies by source); xAI JAX/Rust/Kubernetes (xAI, 2023); Meta PyTorch (Llama 3 paper); Google JAX/XLA/Pathways. OpenAI and Anthropic stacks largely undisclosed.
Shares โš ๏ธ
PyTorch research share estimates of ~55โ€“85% and JAX ~10% come from conflicting blog comparisons.
Editorial
The "build vs borrow" bars and layer descriptions are simplified.