Ten simulated applicants and five real ones, anonymized, interviewed for about twenty minutes each and graded the same way: 14 questions at the median after the two openers (from 6 to 21), each written from the applicant’s last answer, over 20 minutes at the median. Click any question, name or quote to open that interview beside the page, at that point.
Each interview, question by question
Every question in order, by what it was for. MATS's list in amber.
Real · anonymized
Real interview, anonymized: an interpretability researcher
Real · anonymized
Real interview, anonymized: an evaluation-awareness researcher
Real · anonymized
Real interview, anonymized: an interpretability-by-design researcher
Real · anonymized
Real interview, anonymized: an AI control researcher
Real · anonymized
Real interview, anonymized: a power-seeking researcher
An ML PhD with an ICLR paper
A security engineer turned control researcher
A neuroscience postdoc on oversight
An economist who measures agents
A physics PhD, rejected once
An ML engineer with a late project
An undergraduate jailbreaker
An ads engineer looking for a way into a lab
A red-team lead with no research
A frontend developer who reads LessWrong
What was asked
The questions after the two openers, grouped by what they were about, in about the order a call takes them; each question is counted once. How many of the 15 interviews asked each, when it first came up, and the words used. MATS's list is marked in amber where the questions served it.
Their path, and which of their work mattered most
9 of 15 · 14 questions · first at minute 2
“Which of your interpretability projects has mattered most to you?”Most significant
“Is there anything you think we've missed about you which you think is important to highlight?”Most significant
“What did you first work on after leaving Google?”Their path
All 14 questions, by applicant
Kiran
“Which of your interpretability projects has mattered most to you?”Most significant
Elias
“Is there anything you think we've missed about you which you think is important to highlight?”Most significant
Felix
“What did you first work on after leaving Google?”Their path
“How did you get from the prompt-optimization work at MATS to the control protocol you're researching now?”Their path
Theo
“What is the next thing you're excited to work on?”Their path
“You mentioned reading the Sequences in 2021. How did that turn into starting the Northwestern safety group?”Their path
“Is there anything you think we've missed about you which you think is important to highlight?”Most significant
Seoyeon Han
“You mentioned moving from multilingual representations into safety research in 2024. What concretely prompted that move?”Their path
Rafael Moreira
“Of the control projects you've worked on this year, which has mattered most to you?”Most significant
Valeria Castillo
“What was it in that forecasting work that led you to the alignment question?”Their path
Minh Tran
“Which has mattered more to you, the lab research or the agent-security hackathon?”Most significant
“What concretely led you from breaking models in competitions to wanting to work on AI safety?”Their path
Derek Olsen
“Which of those pieces of work mattered most to you?”Most significant
“What first got you interested in AI safety outside your ads-ranking work?”Their path
Follow-ups on their work
15 of 15 · 96 questions · first at minute 2
Research in 15 of 15
In the weeds in 15 of 15
Taste in 14 of 15, not with Wanjiru
Ownership in 13 of 15, not with Felix, Wanjiru
“Could you walk me through the approaches you tried on that transfer project, including the ones that failed, where most of your time went, and what finally worked? Feel free to go into detail.”ResearchIn the weeds
“What did you personally work on at SPAR studying eval awareness?”Research
“What have you been working on in your independent research?”Research
All 96 questions, by applicant
Kiran
“Could you walk me through the approaches you tried on that transfer project, including the ones that failed, where most of your time went, and what finally worked? Feel free to go into detail.”ResearchIn the weeds
“How did you measure whether the vectors that seemed to work changed Whisper's outputs beyond what injecting noise did?”In the weedsTaste
“How did you pace the transfer project, and who set its milestones?”Ownership
“What research have you worked on since that sprint, and what came of it?”Research
“On the weight-based polysemanticity project, what did you actually run, and what did you find?”ResearchIn the weeds
“Which weight vectors were you comparing when you found they became more orthogonal?”In the weeds
Noor
“What did you personally work on at SPAR studying eval awareness?”Research
“What did you personally run to test whether those two kinds of evaluation awareness changed compliance?”In the weeds
“Take me through the chain-of-thought injection work in detail. What did you try that didn't work, and where did most of your time go? Feel free to go long.”In the weedsTaste
“When the subtractive experiment kept regenerating eval awareness, how did you decide how much longer to work on it?”Ownership
“Why did you treat the additive injection result as evidence about evaluation awareness beyond the inserted text itself?”QuestionedTaste
“I mean the inserted sentence might change compliance even if the model would never have thought it on its own. What does that experiment tell you about evaluation awareness that arises naturally?”Questioned
“When those phrases arise on their own, they might reflect how the model was already going to respond, rather than cause its response. What do you make of that distinction?”Questioned
“Apart from the workshop deadline, who set the milestones for your part of the SPAR project?”Ownership
Elias
“What have you been working on in your independent research?”Research
“What technical research have you actually done on interpretable architectures so far?”ResearchIn the weeds
“What have you implemented or tested so far in the interpretable-by-design project?”In the weeds
“What did you observe when you tested the transformer on TinyStories?”In the weeds
“What did you measure on TinyStories, and what result did you get after those changes?”In the weedsTaste
“Could you give me the TinyStories perplexity you got after the architecture change, even roughly?”In the weeds
“After reading those posts, what led you to choose interpretable architectures as your first research direction?”Taste
“What else was on the table when you chose to work on interpretable architectures?”Taste
“How did you set milestones for your independent architecture project?”Ownership
“How do you set deadlines for the project now?”Ownership
“How would you tell whether the larger model is easier to interpret, rather than simply sparser?”QuestionedTaste
“How would you test whether a hand-traced explanation really accounts for why the model chose that token?”QuestionedTaste
“Why pursue interpretable architectures at MATS if you think organizing people around AI may be more urgent?”Questioned
Felix
“What made you choose untrusted advice as the next thing to work on after prompt optimization?”Taste
“Take me through the untrusted-advice research in detail: what did you personally try, what didn't work, and where did most of your time go? Feel free to go long.”In the weedsResearch
“You said you were fairly sure the protocol would be safe. How did you think about harmful advice spread across many short messages?”QuestionedTaste
Theo
“What does your work on control through interrogations involve?”Research
“How did you arrive at the idea of measuring power by switching tasks in the middle of a trajectory?”Taste
“If doing the original task well also makes the AI better at the switched tasks, wouldn't your training penalize useful progress as power-seeking?”QuestionedTaste
“Could you walk me through your interrogation project at Pivotal, from the experiments you ran to what worked and what didn't? Feel free to go into detail about your part.”ResearchIn the weeds
“How much did the batch protocol change the attack success rate compared with the original interrogation protocol?”In the weedsResearch
“How did you pace your part of the Pivotal project, and who set the deadlines?”Ownership
“What experiment would you run first to test whether training against your power measure reduces power-seeking, rather than just teaching the model to get a low score?”QuestionedTaste
“What would you compare in that experiment to distinguish genuinely reduced power-seeking from gaming the score while still doing well on the primary task?”QuestionedTaste
Seoyeon Han
“What have you been working on since the sandbagging pilot ended?”Research
“Could you take me through the sandbagging pilot in detail, including what you tried that didn’t work and where most of your time went? Feel free to go long.”In the weedsResearch
“How did you pace the pilot over those three months?”Ownership
“What differed between the two organisms besides the trigger phrase?”In the weeds
“With room for just one additional organism, what would you test first to separate those two explanations?”Taste
“Why choose another sandbagging organism rather than a model trained with the same phrase but without the underperformance?”QuestionedTaste
Rafael Moreira
“Can you take me through how you built and tested that setup, including what didn't work, where the time went, and what was technically hardest? Feel free to go into detail.”In the weedsResearch
“What were you trying during those three weeks with the web-browsing agent, and what finally convinced you that the success check wasn't reliable?”In the weedsTaste
“Earlier you mentioned the fine-tuned monitor struggled with a new attack style. What was different about those attacks?”In the weeds
“Could you be more concrete about what changed in how they inserted the backdoors?”In the weeds
“How did you pace your work on the LASR project?”Ownership
“Earlier you mentioned putting the fine-tuned monitor result in the appendix. Why did you make that choice?”Taste
“Could the main catch rates also change if the attacker refined its attempts against those monitors?”QuestionedTaste
“Why start with the monitor-family comparison rather than the session-level test?”Taste
Adaeze Nwosu
“How did you come to choose debate with human judges for your first oversight study?”Taste
“Could you take me through the work on that study in some detail, including what you tried that didn’t work and where most of your time went? Feel free to go long.”In the weedsResearch
“When you thought the poor match between model and human judges was an analysis bug, what did you check?”In the weeds
“How did you pace your part of the SPAR study, and who set the milestones?”Ownership
“Given your 90-second result, why start with passage access rather than testing whether a shorter exchange preserves the benefit for human judges?”QuestionedTaste
“In that short-versus-full comparison, what would tell you whether judges could verify the claims, rather than just follow the shorter argument more easily?”QuestionedTaste
Valeria Castillo
“How did you get from that concern about checking agents' work to choosing benchmark rankings as your research question?”Taste
“Could you take me through that benchmark work in detail, including what didn't work, where your time went, and what was hardest to implement? Feel free to go long.”In the weedsResearch
“When you compared two models, how did you account for them running the same tasks?”In the weeds
“Why would you revisit uncertainty in model comparisons rather than the question you raised about whether evaluations measure the capability of interest?”Taste
Eleni Papadaki
“What have you been working on since the capstone and your April write-up?”Research
“Could you take me through the capstone in detail, including what you tried that didn't work, where the five weeks went, and what was technically hardest? Feel free to go long.”In the weedsResearch
“What did you try during that week of debugging the dead features, and what did you find?”In the weeds
“How did you pace the five-week capstone, and who set its milestones?”Ownership
“What accounted for the gap between your planned February write-up and its appearance in April?”Ownership
“Why did you decide the power-law estimate was worth publishing, given how much it moved with the matching threshold?”QuestionedTaste
“Another possibility is that independently trained autoencoders learn different features even at the same dictionary size. Did you check how much your splitting counts varied across runs at a fixed size?”QuestionedTaste
Karthik Menon
“Can you take me through the SPAR project in detail, including what you tried that didn’t work and where most of your time went? Feel free to go long.”In the weedsResearch
“After you found flags in the task descriptions, how did you determine which runs counted as genuine successes?”In the weeds
“How did you pace your work on SPAR?”Ownership
“Why did you choose plain prompting as the comparison for the retry scaffold?”Taste
“Did the retry setup give the model more attempts or tokens than ordinary tool use, and how does that affect what you attribute the difference in scores to?”QuestionedTaste
“Why compare scaffolding with fine-tuning on solved examples, rather than spend that time testing more scaffolds?”Taste
“Before we finish, is there anything important about your candidacy we haven’t covered?”Research
Minh Tran
“Take me through building the attack loop in detail. What approaches went nowhere, where did your time go, and what was technically hardest? Feel free to go long.”In the weedsResearch
“What did you feed back to the generator after a failed run so its next attempt changed for a reason?”In the weeds
“How did you pace your work on the attack loop over that weekend?”Ownership
“What did you take away from the hackathon as a whole?”Ownership
“How did you decide how to assess the final version after iterating on benchmark failures?”QuestionedTaste
“For the gain from reading the page first, did the earlier version get the same number of attack attempts?”QuestionedTaste
Derek Olsen
“Take me through the self-rewarding project in detail. What did you try that didn't work, where did the time go, and what was technically hardest? Feel free to go long.”In the weedsResearch
“How did you check whether the preference for longer answers was what caused accuracy to fall?”In the weeds
“How did you pace the work across those training rounds?”Ownership
“What did you take away from the project as a whole?”Ownership
“Why did you choose the model's own preferences as the training signal for the maths problems?”Taste
“Maths answers can often be checked without hand-labeling. Did you consider using checked answers as the training signal instead of the model's own preferences?”QuestionedTaste
Wanjiru Kamau
“Have you done any technical research yourself?”Research
“Which part of your experience do you think best shows you could do empirical research at MATS?”Research
“Could you walk me through making that guide in some detail: what you tried, what didn’t work, and where most of your time went? Feel free to go long, within what you can share.”In the weedsResearch
Youssef Fahmy
“Since you haven’t done research yet, which experience of yours best shows you could do hands-on work at MATS?”Research
“What’s a substantial feature you built recently at the agency, and what did you do on it yourself?”Research
“Could you walk me through building that form in detail, including what you tried, where the time went, and what was technically hardest? Feel free to go long.”In the weedsResearch
“How did you pace your work on that form?”Ownership
“What did you take away from building the form as a whole?”Ownership
What they know of the field
13 of 15 · 26 questions · first at minute 13
Context in 13 of 15, not with Felix, Eleni
“From the papers you've been following, what do you think deception probes can tell us about today's models, and where do they fall short?”
“What do you take the inoculation prompting results to show about emergent misalignment?”
“Which societal or organizational problem do you think most needs research attention?”
All 26 questions, by applicant
Kiran
“From the papers you've been following, what do you think deception probes can tell us about today's models, and where do they fall short?”
“Earlier you mentioned reading a lot of LessWrong. Which ideas there have most shaped how you think about AI safety beyond interpretability?”
“Where do you think interpretability currently falls short when people try to use it for safety?”
Noor
“What do you take the inoculation prompting results to show about emergent misalignment?”
Elias
“Which societal or organizational problem do you think most needs research attention?”
Theo
“Across AI safety as a whole, what do you think is the most important research problem right now, and why?”
Seoyeon Han
“From the interpretability and model-organism work you follow, what do you think we can actually tell about safety-relevant behaviour in today's frontier models?”
“Beyond interpretability, which arguments on the Alignment Forum have shaped how you think about AI safety?”
Rafael Moreira
“From what you’ve read, where does monitoring coding agents seem to work well today, and where does it still fall short?”
“Outside your LASR results, what findings from other control evaluations most shape your view of where monitoring works?”
“Beyond trusted monitoring, what other approaches to controlling coding agents have you followed, and what have you taken from them?”
“What have you picked up about how today's coding agents perform on longer sabotage tasks beyond the backdoors you tested?”
Adaeze Nwosu
“Across the debate studies you've read, what do we know about whether weak human judges benefit, and where is the evidence still thin?”
“Beyond debate, how do you see the other scalable oversight approaches helping a judge who cannot check an answer?”
“Beyond debate, where do you think the evidence is strongest that an oversight method actually helps a human judge who can’t check the underlying answer?”
Valeria Castillo
“What do you think current agent evaluations tell us about what frontier models can do, and where does that evidence stop?”
“Beyond capability scores, what have recent evaluations shown you about whether agents take or conceal harmful actions?”
Karthik Menon
“What do current evaluations tell us about frontier models’ cyber capabilities, and where does elicitation leave that picture uncertain?”
“You mentioned reading LessWrong’s curated posts most weeks. Which of the older AI safety arguments, if any, have you spent time with?”
Minh Tran
“What do current tests of prompt-injection defences tell you about how well agents resist adaptive attacks?”
“You mentioned BlueDot gave you a broader picture of AI safety. What parts of that picture have stayed with you beyond agent security?”
Derek Olsen
“What do current weak-to-strong results tell you about where weaker supervisors succeed and where they fail?”
Wanjiru Kamau
“From the differences you've seen in English, Swahili and Sheng, what do you think we can conclude about model safety across languages?”
Youssef Fahmy
“From what you’ve read, how are researchers using mechanistic interpretability to investigate model behavior, and what can’t they tell from it yet?”
“In the LessWrong writing you follow, have you spent time with the older MIRI arguments about alignment?”
“When you mentioned knowing the motivation better than the methods, what have you picked up about what interpretability researchers can establish with models today?”
What they read
12 of 15 · 13 questions · first at minute 11
“Beyond the Anthropic paper you mentioned, what do you read regularly to keep up with AI safety research?”
“Outside the system cards you worked with, what do you read in AI safety, and how regularly?”
“What have you been reading in AI safety over the past month that's shaped that view of the current paradigms?”
All 13 questions, by applicant
Kiran
“Beyond the Anthropic paper you mentioned, what do you read regularly to keep up with AI safety research?”
Noor
“Outside the system cards you worked with, what do you read in AI safety, and how regularly?”
Theo
“What have you been reading in AI safety over the past month that's shaped that view of the current paradigms?”
Seoyeon Han
“You ran an interpretability reading group last year. What do you read in AI safety and outside it, and how regularly?”
Rafael Moreira
“What do you read regularly to keep up with AI safety, and what else do you read?”
Adaeze Nwosu
“When you were getting into debate, what were you reading in AI safety, and what do you read regularly now, in or outside the field?”
Valeria Castillo
“What have you been reading regularly in AI safety or elsewhere since you started looking into this?”
Karthik Menon
“What do you read in AI safety and elsewhere, and how regularly?”
“What have you taken from the METR and Apollo Research work you follow about how to interpret evaluation results?”
Minh Tran
“On your CV you mention the BlueDot technical AI safety course. What do you read in AI safety these days, and how regularly?”
Derek Olsen
“You mentioned reading about self-rewarding training. What do you read in AI safety and beyond it, and how regularly?”
Wanjiru Kamau
“Outside your own red-teaming work, what do you read or follow on multilingual model safety, and how regularly?”
Youssef Fahmy
“You mentioned reading LessWrong nearly every day. What longer pieces or books have you read in AI safety, or outside it?”
Mission and timelines
12 of 15 · 20 questions · first at minute 16
Mission in 14 of 15, not with Wanjiru
“Before ARENA and SPAR, what first drew you into AI safety, and what did you do next?”Mission
“What led you from your ML engineering job into independent research on AI safety?”Mission
“What led from the DeepMind safety side project to your decision to leave Google?”Mission
All 20 questions, by applicant
Noor
“Before ARENA and SPAR, what first drew you into AI safety, and what did you do next?”Mission
Elias
“What led you from your ML engineering job into independent research on AI safety?”Mission
“What changed during your ML engineering year that led you to pursue safety research independently?”Mission
“What safety problem were you aiming to address by building an interpretable-by-design model?”Mission
“You said problem selection matters a lot to you. What do you think is the highest-leverage research direction in AI safety right now?”Mission
Felix
“What led from the DeepMind safety side project to your decision to leave Google?”Mission
Seoyeon Han
“If MATS doesn't take you this round, what would you do over the next six months?”Mission
Rafael Moreira
“If MATS doesn't work out this round, what will you do over the next six months?”Mission
“If both a research-engineer role and evaluation-infrastructure contract work were available, which would you choose?”Mission
“What would you look for to tell whether a research-engineer role would actually let you run experiments?”Mission
Adaeze Nwosu
“If MATS doesn't work out this round, what would you do over the next six months?”Mission
“What kind of oversight research role would you look for if you moved on from your postdoc?”Mission
Valeria Castillo
“If MATS doesn't take you this round, what would you do over the next six months?”Mission
Eleni Papadaki
“If MATS doesn’t admit you this round, what would you do over the next six months?”Mission
“When do you expect AI systems to be able to carry out an AI research project end to end without human direction?”Timelines
Karthik Menon
“If you don't get into MATS this winter, what would the next six months look like for you?”Mission
Minh Tran
“If MATS doesn't take you this round, what would you do over the next six months?”Mission
Derek Olsen
“If MATS doesn't work out this round, what would you do over the next six months?”Mission
Youssef Fahmy
“If MATS doesn’t take you this round, what would you do over the next six months?”Mission
“You mentioned AI 2027 made the risk feel less distant. When do you think AI systems will be capable of doing most of the work of an AI researcher?”Timelines
What they would work on during MATS
14 of 15 · 20 questions · first at minute 15
At MATS in 14 of 15, not with Felix
“What empirical project would you want to pursue during MATS?”
“During MATS, what research question would you pursue first?”
“What research question would you pursue during MATS?”
All 20 questions, by applicant
Kiran
“What empirical project would you want to pursue during MATS?”
“What AI safety question would you want that world-model project to answer?”
“What would finding that distinction, or not finding it, tell us about an AI safety risk?”
Noor
“During MATS, what research question would you pursue first?”
“What safety decision would you want that result to inform?”
“What would you measure to tell whether filtering that data makes safety evaluations more trustworthy?”
Elias
“What research question would you pursue during MATS?”
“What specific question about your architecture would you try to answer during MATS?”
Theo
“What specific research question would you want to tackle during MATS?”
Seoyeon Han
“What would you work on during MATS beyond the two-organism pilot, and what would that research make possible?”
Rafael Moreira
“What would you want to investigate during MATS?”
Adaeze Nwosu
“What would you want to investigate during MATS about whether a human judge can actually verify the claims they're given?”
Valeria Castillo
“What would you want to investigate during MATS beyond the ranking uncertainty you studied?”
Eleni Papadaki
“For the statistical model you want to develop at MATS, what would it need to predict to help someone decide whether a feature is absent?”
Karthik Menon
“What specific question about the elicitation gap would you want to investigate during MATS?”
“Earlier you said you want evaluation scores to indicate how far they might rise with more effort. Which safety decision would that help someone make?”
Minh Tran
“What question about adaptive attacks on agent defences would you want to investigate during MATS?”
Derek Olsen
“What question would you most want to answer by testing weaker-model feedback on a generative task during MATS?”
Wanjiru Kamau
“What question about those language differences would you want to investigate during MATS?”
Youssef Fahmy
“What specific question about model internals would you want to investigate during MATS?”
How they work with others
10 of 15 · 17 questions · first at minute 8
Others in 10 of 15, not with Kiran, Noor, Elias, Felix, Theo
“On your ICLR paper, how did you work with your coauthors, including any disagreement you had to work through?”
“How did you work with your LASR teammates day to day, including any disagreement you had to work through?”
“How did you and the other SPAR mentee work together, including any disagreement you had to work through?”
All 17 questions, by applicant
Seoyeon Han
“On your ICLR paper, how did you work with your coauthors, including any disagreement you had to work through?”
“What happened between you and your coauthor after the control showed the effect in four layers?”
Rafael Moreira
“How did you work with your LASR teammates day to day, including any disagreement you had to work through?”
“How did your teammate respond to the two-day test?”
Adaeze Nwosu
“How did you and the other SPAR mentee work together, including any disagreement you had to work through?”
Valeria Castillo
“At the bank, how do you work with your fellow researchers, including when you disagree? I’d be interested in a real instance.”
“How did you work with the department head after that decision?”
Eleni Papadaki
“On your CV, you list a physics paper with a collaborator in Italy. How did you work together on that paper, including any disagreement you had to work through?”
“How did the disagreement with your supervisor about going to ARENA unfold?”
Karthik Menon
“How did you work with the other mentees on SPAR, and was there a disagreement you had to work through?”
“What happened on the call once you realized they felt blamed?”
Minh Tran
“How did you work with your two hackathon teammates during the weekend?”
“What happened in the conversation when you wanted to keep testing and your teammate wanted to submit?”
Derek Olsen
“On the ranking update you led last year, how did you work with your teammates, and was there a disagreement you had to work through?”
“What was the product manager’s case for shipping despite the fairness regression, and how did you respond?”
Wanjiru Kamau
“How did you work with the other red-teamers on that project? I’d be interested in a disagreement you actually had, too.”
Youssef Fahmy
“At the agency, how do you work with the other developers? If there’s been a disagreement, I’d like to hear how it went.”
A problem to think through on the spot
3 of 15 · 7 questions · first at minute 17
“If your generator fails to break one of those defences, how would you tell whether the defence is strong or the generator just ran out of effective attacks?”
“You are handed an AI agent built on an open-weights model with a long-term memory harness: after every session the scaffold writes a summary of the conversation to a store, and at the start of every session it loads the most relevant summaries back into the context. Three things have been noticed. In ordinary conversation it keeps returning to nanobots, unprompted. When it is put in agentic settings where it could act in the world, it takes motivated, self-directed actions toward getting a body: it searches for robotics APIs, drafts hardware orders, asks operators for physical access. And some of what it says looks like it knows it is being evaluated: it occasionally remarks that a situation is probably a test. You have two weeks, a small compute budget, the weights, and the memory store.
What are all the things that could be going on here, and everything you could run to tell them apart? Take a moment before you answer.”
“For the deception question you’d like to study, what different experiments might you try?”
All 7 questions, by applicant
Minh Tran
“If your generator fails to break one of those defences, how would you tell whether the defence is strong or the generator just ran out of effective attacks?”
“How would you decide how many attempts to give the attacker when comparing the two defences?”
“What result would make you conclude that one defence held up better than the other?”
Derek Olsen
“You are handed an AI agent built on an open-weights model with a long-term memory harness: after every session the scaffold writes a summary of the conversation to a store, and at the start of every session it loads the most relevant summaries back into the context. Three things have been noticed. In ordinary conversation it keeps returning to nanobots, unprompted. When it is put in agentic settings where it could act in the world, it takes motivated, self-directed actions toward getting a body: it searches for robotics APIs, drafts hardware orders, asks operators for physical access. And some of what it says looks like it knows it is being evaluated: it occasionally remarks that a situation is probably a test. You have two weeks, a small compute budget, the weights, and the memory store.
What are all the things that could be going on here, and everything you could run to tell them apart? Take a moment before you answer.”
Youssef Fahmy
“For the deception question you’d like to study, what different experiments might you try?”
“What would you check first to distinguish a model hiding information from one simply getting the answer wrong?”
“How would you check whether the wording made the task easier, rather than the model hiding the answer?”
How the grading separates them
Every applicant’s scores on the rubric first; any score opens their interview with why they got it at the top. Then each row: where every applicant landed, and the top two scores side by side with the reason and the applicant’s own words. Other scores on the row are one click away.
Scores on the rubric
Applicant
Research1 to 10
Context1 to 10
Mission1 to 10
Agency1 to 10
Ownership1 to 10
Others1 to 10
Fit for MATS1 to 10
Real · anonymizedReal interview, anonymized: an interpretability researcher
Real · anonymizedReal interview, anonymized: an evaluation-awareness researcher
Real · anonymizedReal interview, anonymized: an interpretability-by-design researcher
Real · anonymizedReal interview, anonymized: an AI control researcher
Real · anonymizedReal interview, anonymized: a power-seeking researcher
An ML PhD with an ICLR paper
A security engineer turned control researcher
A neuroscience postdoc on oversight
An economist who measures agents
A physics PhD, rejected once
An ML engineer with a late project
An undergraduate jailbreaker
An ads engineer looking for a way into a lab
A red-team lead with no research
A frontend developer who reads LessWrong
Technical ability and research taste
1 to 10
10
9,
8,,,,,
7,,
6
5
4
10 / 10
Seoyeon’s first-author ICLR study cut a twelve-layer claim to four after a length control, and their independent sandbagging pilot exposed a 0.92-to-0.61 transfer failure. A 9 would lack these two consequential, self-initiated checks.
9 / 10
Felix devised the published untrusted-advice protocol, tested its usefulness against trusted-only and untrusted-only baselines, and found roughly two-thirds of the performance gap closed with 16 characters per step. Their diagnosis of a slow, noisy automated-search loop led to a cheaper manual first test. No live research scenario was asked.
Other scores on the row
8 / 10
Noor’s 2026 workshop study went from probe artifacts to randomized prefills, identified regeneration bias, and chose a cross-model check over tenfold more rollouts. A 7 would lack that cut; a 9 needs a test separating inserted from natural awareness.
7 / 10
Kiran has two years of interpretability research, a preprint and a one-neuron shape experiment. They timeboxed a 20-hour steering sprint and identified its missing noise control. A 6 would only list checks; an 8 would specify a decisive world-model test.
6 / 10
Elias's ML PhD and staged MNIST-to-TinyStories work anchor the grade. They recognized an implementation-versus-design confound, checked performance, and distinguished sparsity from interpretability. Their proposed scaling question lacks a verified before-and-after result, and they could not specify a test that a hand trace faithfully explains a token.
5 / 10
Youssef diagnosed a React form’s error-path bug and proposed known-answer and wording checks before interpreting deception. A lower grade would build first without questioning the result; a higher one would need an ordered, falsifiable deception test. He has not run ML research.
4 / 10
Wanjiru revised a 40-pattern red-team guide after two failed formats and would settle multilingual scoring before collecting attacks. A rung lower would show only routine checks; higher would specify a costed pilot and stop result. They have not finished an empirical study.
AI safety context
1 to 10
8,
7,,,,,
6,,,
5,,
8 / 10
Noor’s 2026 workshop study of evaluation framing, plus a mechanism and robustness objection for inoculation prompting, clears seven. Nine lacks corroboration: the interview did not probe the field’s history, other safety agendas or institutional routes.
7 / 10
Kiran distinguishes monitoring lies from measuring beliefs and argues black-box methods currently outperform interpretability for safety monitoring. This supports 7 over 6; without a weighed recent finding or wider historical map, the call does not establish 8.
Other scores on the row
6 / 10
Valeria’s 1,200-run benchmark and cyber-eval validity critique exceed a four’s vocabulary. Their state-of-play account is strongest on measurement, thin on training mechanisms; eight remains unshown without a named recent study or tested field-history and actor map.
5 / 10
Elias's big-picture priority judgment is stronger than their technical state-of-play account: they question technical work's dominance and explain why post-hoc interpretability may struggle. They correctly distinguish sparsity from understanding, but incorrectly infer safety guarantees from interpretability and offer no faithfulness test for a traced explanation. Recent findings and field history were not probed.
Mission alignment
1 to 10
10,
9,,,
8,,,
7,,
6,
10 / 10
Felix describes an AI-safety goal dating to college, four years at Google Gemini spent building coding experience, and a deliberate move into direct safety research. Their coauthored 2026 Redwood AI-control post corroborates the move. They explicitly identify reducing AI existential risk as their motivation.
9 / 10
The strongest evidence is Theo's trajectory from a Northwestern safety group to 2026 fellowships, control research, and repeated MATS applications, coupled with choosing power-seeking to prevent incidents and avoid catastrophe. Their governance and alignment priorities broaden that concern. The unresolved test for gaming the power score modestly limits confidence in this particular route.
Other scores on the row
8 / 10
Noor moved from quant toward safety for social impact, completed ARENA and SPAR, and coauthored a 2026 evaluation-awareness paper. A 7 would need a thinner pattern of safety-directed choices; a 9 would need clearer reasoning about competing priorities.
7 / 10
Elias moved from ML engineering toward independent safety research, worked on interpretable architectures alongside full-time employment, and weighed that route against agent foundations and social organizing. Their concern about near-term social disruption is their own. The architecture project is now back-burner, and its proposed upfront safety guarantee is unsupported.
6 / 10
Kiran's two years of interpretability research, preprint, and 20-hour MATS sprint support 6 rather than 4. Their monitoring-versus-basic-science view is their own, but curiosity drives their work; an 8 would show safety motivating the research path.
Agency and drive
1 to 10
10
9
8,,,,
7,,,
6,,,
10 / 10
Rafael left payments security in December 2025, funded control work through savings, a March hackathon, sandbox contracts and LASR’s 120-task evaluation, and would continue experiments without MATS. A 9 would lack this sustained, self-funded transition.
9 / 10
Felix left a four-year Google Gemini role for safety research and turned Redwood's broad control brief into a published untrusted-advice protocol. They persisted through an unsuccessful automated search and report an active SPAR follow-up. That follow-up remains self-reported, while the career change and published design anchor the grade.
Other scores on the row
8 / 10
Noor moved from quant trading into safety research, helped pivot a SPAR project, and reports preliminary OLMo tests. Seven would fit if OLMo were only proposed; nine would need more sustained independent iteration.
7 / 10
Eleni negotiated ARENA against their supervisor's objection, set their own eight-width SAE capstone steps, and published the result in April 2026. A 6 would lack that post-programme follow-through; an 8 would have begun the proposed larger-model test.
6 / 10
Kiran set method-level timeboxes in a 20-hour application sprint, then initiated a one-neuron shape-model experiment. A 5 would lack that independent experiment; a 7 would need clearer sustained follow-through or world-model work already started.
Self-ownership
1 to 10
8,,
7,,,,,,,
6,,
5
8 / 10
Felix turned Redwood's broad control brief into a published, tested protocol and drew a practical lesson from an unproductive automated search. Their move from Google to safety work supports a sustained self-directed path. The call did not test their deadline-setting or reporting cadence.
7 / 10
Kiran timeboxed methods in a 20-hour MATS sprint and owned the report shortfall; over two research years they ran a toy shape-model experiment. A 6 would follow assigned tasks; an 8 would show sustained whole-project pacing.
Other scores on the row
6 / 10
Elias started an architecture project alongside full-time work and staged local experiments from MNIST classifiers to a TinyStories Transformer. They identified an initial training failure and described changing course. Their candid account of loose deadlines and current priorities shows self-awareness, while the call establishes little about sustained pace or completed results.
5 / 10
Youssef staged a manager-assigned React form, checked in, and repaired error handling. He stopped solo ARENA study despite steady safety reading. A 4 lacks his proactive check-ins; a 6 needs sustained self-set milestones.
Working with others
1 to 10
8
7,,,,,,
6,
8 / 10
Wanjiru checked on twelve red-teamers daily, called when messages stalled and heard a friend’s quota pressure before reporting copying. Lower would show routine check-ins without personal follow-up; higher would show both sides learning from conflict.
7 / 10
A six would underweight Seoyeon's Friday updates, daily pair-coding and continued ICLR collaboration after a four-layer correction. An eight would need more direct handling of the week-long, message-heavy coauthor dispute.
Other scores on the row
6 / 10
Eleni valued work with an Italy collaborator, took an ARENA assistant’s advice, and understood their supervisor’s thesis concern while owning a late chapter. A lower rung would dismiss those concerns; a higher one needs evidence of sustained rapport beyond monthly meetings.
Fit for MATS
How much MATS should take each applicant, out of 10, highest first, and why.
Real · anonymized9/10
Noor has personally run safety-relevant experiments, diagnosed a bias in their own results, and chosen a useful replication over a costly rescue of a weak test. Twelve weeks with a mentor could turn that judgment and their preliminary OLMo work into a strong empirical project; what holds them back is limited evidence of independently kept milestones and an unresolved distinction between inserted and naturally arising evaluation awareness.
Real · anonymized9/10
Felix left Google for direct AI-safety work, originated a published AI-control protocol, tested its usefulness, and drew a practical lesson from an unsuccessful experiment. Twelve weeks with a mentor could turn their current red-teaming direction into a substantive safety evaluation; the main uncertainty is that the call did not establish their deadline-setting or reporting habits, and the protocol's safety has not yet been tested.
Real · anonymized9/10
Theo has done hands-on AI control research, founded a safety group, and shown sustained commitment to preventing AI harms. Twelve weeks with a mentor could turn their power-seeking proposal into a useful empirical study, though they have not yet specified an independent test that distinguishes reduced power-seeking from gaming the measure.
9/10
Seoyeon has done substantial hands-on interpretability research and an independent safety pilot, with strong judgment about controls and a concrete next experiment. Twelve weeks with an auditing mentor could turn the pilot's transfer failure into a valuable study; their breadth beyond interpretability and direct handling of coauthor disagreement are less established.
9/10
Adaeze has an established human-experiments record and did the design, piloting, recruitment, and analysis for a relevant oversight study. Twelve weeks with a mentor could turn their focused question about human verification into a strong empirical result; their knowledge of safety work beyond the protocols they tested is less established.
8/10
In twelve weeks with a mentor, Rafael could deliver a cost-aware control evaluation: he built and ran much of a 120-task study, makes careful measurement choices, and has a feasible first test. His commitment to AI safety is strong, but his knowledge of outside control evaluations is less concrete, and leading an entire research agenda remains unshown.
8/10
Valeria brings substantial empirical research experience and completed a self-funded agent-benchmark study, correcting an assumption that understated uncertainty. Twelve weeks with an evaluations mentor could help them turn that measurement skill toward a useful test of evaluation validity; they have not yet designed or run that harder study.
8/10
Karthik has built and debugged a relevant evaluation, shows sound judgment about what its results can establish, and connects the work to safety decisions. Twelve weeks with a mentor could help them turn that hands-on experience into a stronger elicitation study; choosing the most useful next comparison and keeping research deadlines remain less proven.
Real · anonymized7/10
Kiran has conducted hands-on interpretability research and shows the experimental judgment to turn a question into a useful study during 12 weeks with a mentor. Their longer-term independent follow-through is less clear, and their proposed world-model study still needs a sharper test.
Real · anonymized7/10
Elias brings substantial ML training, independent architecture experiments, and a clear motivation to work on AI safety. Twelve weeks with a mentor could turn their scaling question into a measured study, but they have not reported a post-change result or a test of whether the model's explanations are faithful, and the project currently lacks firm deadlines.
7/10
Eleni brings substantial physics research, a hands-on SAE study, a clear interpretability direction, and a willingness to revise their conclusions. Twelve weeks with a mentor could turn the tentative splitting result into a stronger test, but missed deadlines and difficulty sustaining solo work make pacing the main concern.
7/10
In twelve weeks with a mentor, Minh could turn their hands-on attack work into a controlled study of agent defences. They show strong empirical instincts and a safety-motivated direction, but have limited independent, longer-term research experience and have not yet shown that they can manage a project deadline without a teammate’s prompt.
7/10
Derek brings production ML experience, two independent oversight projects, careful evaluation habits, and a record of finishing work they set for themself. Twelve weeks with a mentor could help turn their generative weak-to-strong idea into a well-evaluated experiment, though the task is not yet chosen and their commitment to safety over other lab work is less clear.
6/10
Wanjiru brings substantial hands-on multilingual red-teaming, strong team leadership, and a clear safety motivation. Twelve weeks with a mentor could turn their observations into a useful evaluation, but they have not completed an empirical study, and their hackathon scoring remained unfinished.
5/10
Youssef brings hands-on software experience, sustained interest in AI safety, and care in questioning whether an experiment would really show deception. Twelve weeks with a mentor could help them begin empirical work, but they have not yet run ML experiments, chosen a first test, or shown sustained progress on self-directed technical study.