Home › Blog › Security and Access
Who Runs the AI Factory · Part 12 · Security

Guarding the Crown Jewels: The Engineers Who Protect Model Weights, Data and the AI Supply Chain

Everything in this series ends in a set of files. Months of compute, trillions of tokens and the work of thousands of people become a few terabytes of model weights, and whoever copies them gets the model without paying for it. Add training data, user data and the open-source software every lab depends on, and AI factories are among the most attractive targets in computing. This final part follows the security engineers who protect them, from insider-threat detection to egress controls and supply-chain integrity.

FL
FrontierAGI Team
⚠️ How this was researched Sources are RAND's 2024 report "Securing AI Model Weights", Anthropic's May 2025 announcement of ASL-3 protections and coverage of it, reporting on the 2023 LLaMA leak and OpenAI forum breach, the PyTorch maintainers' advisory on the torchtriton attack, Hugging Face and security-firm research on malicious model files, a benchmark of H100 confidential computing, and real OpenAI job postings. Labs say little publicly about security, so detail is limited, and we found no Meta security engineering publication. The ladder's example defences, exfiltration simulator and other demos are illustrations. See Reading the numbers.

1. Where we are in the stack

The AI factory stack, by role

2. The roles: infrastructure security and insider-threat detection

OpenAI's postings show how specialised the work has become. Its Security Engineer, Infrastructure Security covers GPU clusters, Kubernetes, firmware and storage, including the access pathways for highly sensitive model weights and user data, and building controls across layers against sophisticated adversaries and insider threats. Its Security Engineer, Insider Threat Detection & Response builds detection and investigation workflows for threats ranging from access abuse to IP theft, including in AI infrastructure, and asks for 5+ years in detection, response or insider risk. Other detection-and-response postings describe watching for unauthorised access to training datasets and model weights across distributed infrastructure.

Two roles, two skill profiles

Our reading of postings (1–5). The day timelines are composite illustrations.

3. What's worth stealing

A trained frontier model can cost hundreds of millions of dollars in compute, yet its weights fit on a handful of drives. That asymmetry is why weight security has become a policy issue, not just a company concern. Click an asset.

The assets

4. The threat model

RAND's 2024 report "Securing AI Model Weights" gives the field a shared vocabulary. It identifies 38 meaningfully distinct attack vectors, grouped into categories such as running unauthorised code, compromised credentials, supply-chain attacks and exploiting human weaknesses, and matches them to attackers of increasing capability, from opportunistic, financially motivated criminals to the most capable nation-state operations. It defines five security levels, SL1 to SL5, each aimed at stopping a more capable attacker, and sketches benchmark security systems for each. The example defences below are our illustration of the progression, not RAND's specification.

The SL1–SL5 ladder

Click a level. Attacker descriptions follow RAND's framing; example defences are illustrative.

5. What lives at this layer

The security stack

6. Case study: Anthropic's ASL-3 security standard

Some labs tie security to model capability. Under its Responsible Scaling Policy, Anthropic activated its AI Safety Level 3 (ASL-3) protections on 22 May 2025 alongside the launch of Claude Opus 4, as a precaution: it said it had not determined that the model definitely required them. The security side aims to make model weights much harder to steal; coverage reported more than 100 security controls, including two-person authorisation, change-management protocols and egress bandwidth controls that restrict and monitor outbound data so that a multi-terabyte export would be slow and conspicuous. (The deployment side, separate from this article, uses classifiers to block help with chemical, biological, radiological and nuclear weapons.)

Why egress limits work

Illustrative: how long it takes to copy model weights out through a restricted outbound link, and how soon monitoring notices.

7. Incidents and lessons

Public AI security incidents are rare, but the ones that are known each teach something. Click one and toggle the control that would most likely have stopped it.

Four incidents

8. Confidential computing

A newer defence protects data while it is being used. NVIDIA's H100 has a confidential computing mode: a hardware-enforced trusted execution environment on the GPU that keeps code, data and model parameters hidden from the rest of the system, including the host operating system. It needs a matching trusted environment on the CPU side, such as an Intel TDX or AMD SEV-SNP virtual machine. A third-party benchmark found the overhead below 5% for most typical LLM queries. Vendors advertise similar setups on Blackwell GPUs, which we could not confirm from NVIDIA's own documentation. For AI factories, this matters most when running models on infrastructure the model owner does not fully control, such as a cloud or a customer's site.

9. Handoffs and careers

Who security engineers work with

🧭 Getting into the role
Typical backgrounds
Cloud and infrastructure security engineers, detection and response analysts, insider-risk specialists, security researchers, government security staff.
Core skills
Identity and access management, Kubernetes and cloud security, network segmentation, detection engineering, forensics, supply-chain security, threat modelling, and enough ML infrastructure knowledge to know where the weights live.
Growth path
Security engineer → owner of weight protection or insider-risk programme → head of security for a lab, increasingly a board-level role.
Further reading
AI Safety, Alignment & Interpretability · Sandboxing AI Agents · Is It Ready?

🗂️ Case file

Framework
RAND: 38 attack vectors, SL1–SL5.
Anthropic
ASL-3 security (May 2025): 100+ controls, two-person authorisation, egress bandwidth controls.
Meta
LLaMA weights leaked via 4chan (March 2023); PyTorch's torchtriton supply-chain attack (December 2022).
OpenAI
Internal forum breached in early 2023 (design discussions, not core systems); disclosed publicly in July 2024 by the NYT.
Verdict Security is the layer that decides whether everything else in the factory stays yours. The threats span the whole stack, from firmware and drivers to open-source packages and insiders with legitimate access, and the defences are increasingly tied to how capable a model is. Frameworks like RAND's security levels and capability-linked standards like ASL-3 suggest where the industry is heading: treating frontier model weights with the seriousness once reserved for national secrets.

🏁 The series in one view

Twelve layers, twelve kinds of engineer, one factory.

Next: a hub page bringing the series together, with a role finder and career paths.

🔢 Reading the numbers

📚 References