Home › Blog › AGI Researcher Foundations — The Self-Assessment Deep Dive
AGI Researcher Foundations · Article 2 of 11 ✅

The Self-Assessment Deep Dive: Why Each Skill Matters, and How to Actually Build It

Article 1 left you with an eight-item checklist. This article opens each item up — why frontier and research labs actually treat it as a real signal, a concrete how-to path with sourced tutorials and repos, and videos to work through — so "can you check this box" turns into a plan you can execute.

FL
FrontierAGI Team
September 12, 2026 · 65 min read
Opening 1. Training Loop 2. Reproduction 3. Paper → Code 4. Remote GPU/SSH 5. Parallelism 6. Experiment Tracking 7. Explain Chinchilla 8. Publish Publicly What's Missing Where This Series Goes All Videos References Verdict

Why This Checklist Deserves Its Own Article

The Technical Stack article ended with eight yes/no questions — the kind of list that's satisfying to skim and easy to lie to yourself about. "Have you reproduced a well-known result?" is a fair question, but a bare checkbox tells you nothing about why a lab would care, or what the actual first step looks like if the honest answer is no. This article treats each item as its own mini-investigation: the real signal it represents (sourced, not assumed), and a concrete, linkable path from zero to a checked box — the tutorials, repos, papers, and videos that get you there.

One framing worth keeping in mind throughout: none of these eight things are IQ tests. They are all things a disciplined self-taught researcher can complete in a few weeks to a few months, in roughly the order presented, since each builds on the last (a training loop before a reproduction, a reproduction before reading distributed-training papers).

The checklist isn't testing whether you're smart enough for research. It's testing whether you've actually built things, versus read about people who build things.
1Implement a training loop in PyTorch without copying from a tutorial
Why This Is a Real Signal
Modern frameworks hide the training loop behind a Trainer class or a one-line .fit() call, and it's entirely possible to ship a working fine-tuning script without understanding what happens between the forward pass and the optimizer step. That's fine for production use, but it collapses the moment something breaks — a loss that's NaN, a gradient that's not flowing, a shape mismatch three layers deep. The four steps (forward pass, loss, backward pass, optimizer step) are the atomic unit of everything else in this article; being unable to write them from memory means every other item on this list is memorized rather than understood.
How to Actually Build It
Start with PyTorch's own "Learn the Basics" series, then deliberately avoid nn.Module shortcuts on your first pass by working through Andrej Karpathy's micrograd — a ~150-line autograd engine he built specifically to demystify backpropagation by making you implement it, not import it. Once that clicks, write a plain PyTorch MNIST or CIFAR-10 classifier by hand — no Lightning, no Trainer — and only then move to reading production training scripts and recognizing which lines correspond to which of the four steps.
2Reproduce at least one well-known result from scratch
Why This Is a Real Signal
Reproduction is where research literacy gets tested in the real world — it's the exact task Article 1's Scenario A walked through, and it's a recurring pattern in how frontier labs evaluate research engineering candidates: not "do you understand the idea," but "can you go from a paper and a sparse README to a trained artifact that actually matches reported numbers." That process forces you through the unglamorous parts research abstracts skip — tokenization edge cases, learning-rate schedules, checkpointing, and debugging a model that trains but doesn't learn.
How to Actually Build It
Karpathy's nanoGPT is the standard on-ramp: the repository's own README walks through reproducing GPT-2 (124M) on OpenWebText, and its smaller shakespeare_char example trains a working character-level model on a single GPU in minutes, which is the right starting scale before attempting anything bigger. Once that's done, Tanishq Kumar's beyond-nanogpt (referenced in Article 1) gives a structured next set of reproductions — implementations of ideas from more recent papers (speculative decoding, MoE, and others) built the same from-scratch way.
3Translate a paper's architecture diagram into working code
Why This Is a Real Signal
Most people's first contact with a new architecture is someone else's implementation on GitHub — which is a fine way to use an idea, but a poor way to prove you understand it. The actual skill labs care about is going the other direction: reading Figure 1 and the surrounding equations in a paper and producing your own working module, catching the details papers gloss over (exact normalization placement, masking conventions, initialization schemes) that only show up when you try to make the numbers match.
How to Actually Build It
The standard exercise is the Transformer from "Attention Is All You Need" (Vaswani et al., 2017), because Harvard NLP's The Annotated Transformer gives you a paragraph-by-paragraph, line-by-line pairing of the original paper's text with a working PyTorch implementation — read a section, then look away and write your own version of that section before checking theirs. Do this once, and diagram-to-code stops being abstract.
4Run a remote GPU training job that survives a dropped SSH connection
Why This Is a Real Signal
Almost no real training happens on a laptop — it happens on a rented or shared remote machine, and a training run that dies the moment your laptop sleeps or your Wi-Fi hiccups is a recurring, entirely avoidable source of lost compute and lost time. This is exactly the kind of unglamorous infrastructure literacy the Day in the Life article's "operating manual" flagged as a daily reality, not a one-time setup step.
How to Actually Build It
Rent a cheap GPU hour on Lambda Labs, RunPod, or Vast.ai (all covered in Article 1's compute section) and practice the pattern that solves the dropped-connection problem: start a long-running job inside a tmux or GNU Screen session so it keeps running on the remote machine independent of your local connection, then reattach later to check progress. Combine this with checkpointing (saving model state periodically) so that even a genuine crash loses minutes, not hours.
A specific beginner-friendly tmux video wasn't independently re-verified for this article's current-year availability — search "tmux tutorial" on YouTube for an up-to-date walkthrough rather than relying on any single linked video going stale.
5Know the difference between data, tensor, and pipeline parallelism
Why This Is a Real Signal
The moment a model or dataset stops fitting on one GPU, training becomes a systems problem, not just a modeling one — and the OPT-175B logbook referenced in Article 1's Scenario B (56 days, 992 GPUs, dozens of manual restarts) is what that problem looks like at scale. You don't need to have run a 175B-parameter job to be credible here, but you do need to know which parallelism strategy solves which bottleneck (data parallelism for more throughput, tensor parallelism when a single layer doesn't fit on one GPU, pipeline parallelism when the whole model doesn't fit) — because interviewers and collaborators will assume this vocabulary as a baseline the moment a conversation touches infrastructure.
How to Actually Build It
Hugging Face's parallelism documentation is the clearest single overview of the different strategies and when each applies. For the underlying research, read the Megatron-LM paper (Shoeybi et al., 2019) for tensor parallelism and the ZeRO paper (Rajbhandari et al., 2019, underlying DeepSpeed) for how memory is partitioned across data-parallel workers. Lilian Weng's widely-cited blog post on training large models ties these strategies together with clear diagrams.
6Set up experiment tracking instead of relying on manually saved logs
Why This Is a Real Signal
Research is only as good as your ability to compare run #47 against run #12 six weeks later, and "I remember it was better" is not reproducible science. This is one of the cheapest signals on this whole list to fix — it costs nothing and takes under an hour to set up — which is exactly why its absence stands out: it suggests someone hasn't yet run enough real experiments to feel the pain of losing track of them.
How to Actually Build It
Add Weights & Biases to any of the reproduction projects from item 2 above — the free personal tier is enough, and the quickstart guide takes a few lines of code (wandb.init(), then logging your loss each step). The goal isn't mastering every feature; it's building the habit of every run being logged, comparable, and shareable by default.
7Explain "compute-optimal training" (Chinchilla) to a non-technical person
Why This Is a Real Signal
Being able to recite that "Chinchilla showed you should scale data and parameters together" is memorization. Being able to explain why — that earlier models like GPT-3 were trained on too little data for their size, wasting compute on parameters the data couldn't fully use — to someone with zero ML background, using an analogy rather than jargon, is understanding. Frontier labs use exactly this kind of "explain it simply" question in interviews precisely because it's hard to fake.
How to Actually Build It
Read the original Chinchilla paper (Hoffmann et al., 2022, already cited in Article 1) closely enough to reconstruct its central chart from memory — compute budget on one axis, and the finding that smaller-but-more-data-trained models beat larger-but-undertrained ones at the same compute cost. Then practice the explanation out loud on someone non-technical; if you have to use the word "parameters" more than once, the explanation isn't finished yet.
A specific accessible video walkthrough of the Chinchilla paper wasn't independently re-verified as currently live for this article — several well-known paper-review channels have covered it in the past, but search rather than rely on one link.
8Publish or share your own technical work publicly
Why This Is a Real Signal
This is the single item this series keeps returning to — Article 1's Scenario D traced how Neel Nanda's public interpretability write-ups and Callum McDougall's public ARENA/MATS work became the actual path to a job at a frontier lab, not a private portfolio nobody saw. Public work is verifiable in a way "I've done a lot of projects" on a resume is not, and it compounds: each piece is discoverable by the next person deciding whether to take a chance on you.
How to Actually Build It
Pick the lowest-friction format and ship one thing: a GitHub repo with an honest README describing what you built and what didn't work, a short write-up on a personal blog or Substack, or — for interpretability-adjacent work specifically — a post on the AI Alignment Forum, which is where a large share of public interpretability research is actually discussed and where Nanda-style work first surfaces. The bar is not "impressive"; the bar is "exists, and is linkable."
Notice what's absent from all eight: a requirement for a specific degree, a specific employer, or access to a specific cluster. Every path above starts from a laptop, a free tier, and time.

⚠️ What's Missing or Unverified in This Article

Read before treating any single link as the final word: a few references above could not be independently re-verified as currently live or exactly matching the description given, and are flagged inline rather than presented as fully confirmed — specifically the tmux tutorial video (item 4) and the Chinchilla paper-review video (item 7). The papers, official documentation, and repositories linked throughout (arXiv IDs, GitHub repos, official docs) were checked and are real.

Where This Series Goes Next

Article 3 moves from tooling and self-assessment down to the mathematical foundations underneath all of it — linear algebra, probability, optimization, and information theory — with the same distinction this series has kept throughout: what's used daily in research versus what's textbook-only. From there, the series continues through core deep learning concepts, generalization and learning theory, RL foundations, interpretability, alignment, world models, systems, and research methodology, before a capstone article ties everything back into one map.

🎥 All Recommended Videos

The Spelled-Out Intro to Neural Networks and Backpropagation
Andrej Karpathy
Let's Build GPT: From Scratch, in Code, Spelled Out
Andrej Karpathy
Attention in Transformers, Visually Explained
3Blue1Brown
Neural Networks: Zero to Hero
Andrej Karpathy — full series
Weights & Biases — Official Channel
Experiment-tracking walkthroughs

🧭 Closing — Eight Checkboxes, One Underlying Habit

🎯 The Bottom Line
Look back across all eight items and one pattern repeats: each one is really asking "have you done the thing yourself, from a real primary source, and can you show it." Not read a summary of nanoGPT — run it. Not heard of Chinchilla — explain it. Not built something privately — put it somewhere linkable. None of the eight requires a lab affiliation, a specific degree, or paid access to anything beyond a few dollars of GPU time — every path above starts from a public paper, a public repo, or a public tutorial. The honest self-assessment isn't "am I smart enough" — it's "have I actually done the reps," and every unchecked box above now has a concrete next step attached to it.