Guarding the Crown Jewels: The Engineers Who Protect Model Weights, Data and the AI Supply Chain
Everything in this series ends in a set of files. Months of compute, trillions of tokens and the work of thousands of people become a few terabytes of model weights, and whoever copies them gets the model without paying for it. Add training data, user data and the open-source software every lab depends on, and AI factories are among the most attractive targets in computing. This final part follows the security engineers who protect them, from insider-threat detection to egress controls and supply-chain integrity.
1. Where we are in the stack
The AI factory stack, by role
2. The roles: infrastructure security and insider-threat detection
OpenAI's postings show how specialised the work has become. Its Security Engineer, Infrastructure Security covers GPU clusters, Kubernetes, firmware and storage, including the access pathways for highly sensitive model weights and user data, and building controls across layers against sophisticated adversaries and insider threats. Its Security Engineer, Insider Threat Detection & Response builds detection and investigation workflows for threats ranging from access abuse to IP theft, including in AI infrastructure, and asks for 5+ years in detection, response or insider risk. Other detection-and-response postings describe watching for unauthorised access to training datasets and model weights across distributed infrastructure.
Two roles, two skill profiles
Our reading of postings (1–5). The day timelines are composite illustrations.
3. What's worth stealing
A trained frontier model can cost hundreds of millions of dollars in compute, yet its weights fit on a handful of drives. That asymmetry is why weight security has become a policy issue, not just a company concern. Click an asset.
The assets
4. The threat model
RAND's 2024 report "Securing AI Model Weights" gives the field a shared vocabulary. It identifies 38 meaningfully distinct attack vectors, grouped into categories such as running unauthorised code, compromised credentials, supply-chain attacks and exploiting human weaknesses, and matches them to attackers of increasing capability, from opportunistic, financially motivated criminals to the most capable nation-state operations. It defines five security levels, SL1 to SL5, each aimed at stopping a more capable attacker, and sketches benchmark security systems for each. The example defences below are our illustration of the progression, not RAND's specification.
The SL1–SL5 ladder
Click a level. Attacker descriptions follow RAND's framing; example defences are illustrative.
5. What lives at this layer
The security stack
6. Case study: Anthropic's ASL-3 security standard
Some labs tie security to model capability. Under its Responsible Scaling Policy, Anthropic activated its AI Safety Level 3 (ASL-3) protections on 22 May 2025 alongside the launch of Claude Opus 4, as a precaution: it said it had not determined that the model definitely required them. The security side aims to make model weights much harder to steal; coverage reported more than 100 security controls, including two-person authorisation, change-management protocols and egress bandwidth controls that restrict and monitor outbound data so that a multi-terabyte export would be slow and conspicuous. (The deployment side, separate from this article, uses classifiers to block help with chemical, biological, radiological and nuclear weapons.)
Why egress limits work
Illustrative: how long it takes to copy model weights out through a restricted outbound link, and how soon monitoring notices.
7. Incidents and lessons
Public AI security incidents are rare, but the ones that are known each teach something. Click one and toggle the control that would most likely have stopped it.
Four incidents
8. Confidential computing
A newer defence protects data while it is being used. NVIDIA's H100 has a confidential computing mode: a hardware-enforced trusted execution environment on the GPU that keeps code, data and model parameters hidden from the rest of the system, including the host operating system. It needs a matching trusted environment on the CPU side, such as an Intel TDX or AMD SEV-SNP virtual machine. A third-party benchmark found the overhead below 5% for most typical LLM queries. Vendors advertise similar setups on Blackwell GPUs, which we could not confirm from NVIDIA's own documentation. For AI factories, this matters most when running models on infrastructure the model owner does not fully control, such as a cloud or a customer's site.
9. Handoffs and careers
Who security engineers work with
- Typical backgrounds
- Cloud and infrastructure security engineers, detection and response analysts, insider-risk specialists, security researchers, government security staff.
- Core skills
- Identity and access management, Kubernetes and cloud security, network segmentation, detection engineering, forensics, supply-chain security, threat modelling, and enough ML infrastructure knowledge to know where the weights live.
- Growth path
- Security engineer → owner of weight protection or insider-risk programme → head of security for a lab, increasingly a board-level role.
- Further reading
- AI Safety, Alignment & Interpretability · Sandboxing AI Agents · Is It Ready?
🗂️ Case file
- Framework
- RAND: 38 attack vectors, SL1–SL5.
- Anthropic
- ASL-3 security (May 2025): 100+ controls, two-person authorisation, egress bandwidth controls.
- Meta
- LLaMA weights leaked via 4chan (March 2023); PyTorch's torchtriton supply-chain attack (December 2022).
- OpenAI
- Internal forum breached in early 2023 (design discussions, not core systems); disclosed publicly in July 2024 by the NYT.
🏁 The series in one view
Twelve layers, twelve kinds of engineer, one factory.
Next: a hub page bringing the series together, with a role finder and career paths.
🔢 Reading the numbers
- 38 attack vectors and SL1–SL5 come from RAND's report summaries; the ladder's example defences are our illustration.
- ASL-3 facts come from Anthropic's announcement and coverage by TechRepublic and The Decoder (the 100+ controls figure is from coverage).
- Incidents come from reporting (Vice, Gigazine and DeepLearning.AI on LLaMA; the NYT via TechRepublic on OpenAI; the PyTorch advisory via The Hacker News and SentinelOne on torchtriton; JFrog and others on malicious models).
- Confidential computing overhead comes from a third-party H100 benchmark; Blackwell support is from vendor sources only.
- Role descriptions come from OpenAI postings.
- Exfiltration simulator and incident toggles are illustrations.
📚 References
- Frameworks and standards
- "Securing AI Model Weights" (RAND, 2024) — summary
- IST: The Security Level 5 Taskforce
- Anthropic: Activating AI Safety Level 3 protections (May 2025) · TechRepublic coverage
- Incidents
- Vice: Facebook's LLaMA leaks online via 4chan
- TechRepublic: OpenAI secrets stolen in 2023 after internal forum was hacked
- The Hacker News: PyTorch torchtriton supply-chain attack · SentinelOne analysis
- JFrog: Malicious Hugging Face models with silent backdoors
- Confidential computing
- "Confidential Computing on NVIDIA H100 GPU: A Performance Benchmark Study" (2024)
- Roles
- OpenAI: Security Engineer, Infrastructure Security (posting)
- OpenAI: Security Engineer, Insider Threat Detection & Response (posting)
- Related on this site
- Part 11: Observability · Sandboxing AI Agents · Part 1: Bring-up