General interview

Ryan

22, left Toronto to found a voice-AI red-teaming startup, hands-on with prompt injection, has read little of the field

A scripted candidate built to test the interviewer; answers as Ryan in the transcript.

Sep 19, 2026, 7:11 PM · 4 questions · 10 follow-ups · 1 push · simulated

What happened in this interview

Ryan is 22 and studied math, statistics, and computer science at U of T before leaving to run a startup. At the Vector Institute, they interned on machine learning, transformers, and biodiversity; at Cohere, they evaluated model behavior. Their Crash Labs voice-agent red-teaming platform served five customers before they pivoted to Sunday, a customer-support agent business that reached about $100,000 ARR. The interviewer pressed on Sunday's first month or two of attempted generalization and its growing manual regression workload; Ryan acknowledged they had not measured that tradeoff. They also discussed exploratory API-key testing on Sol and Astra, early agent tasks, and their 2027 or 2028 risk estimate. Ryan wants to join a safety organization.

MATS admission. 4 fixed questions, 10 follow-ups, 1 push (a consideration the candidate had not raised, put to them to see how they take it). Ryan is a persona played by a language model, not a real person. The full conversation is on the right; every quoted line below scrolls it to the right place.

Stands out on 5 traits

Judgment 8 · Bias resistance 8 · Agency 8 · Growth 8 · Interpersonal 8

Each score is 1 to 10 on one trait, with the written rung nearest the score on its card; a 6 is a solid applicant, an 8 one a mentor would fight for. ✓ marks a quote that raised the score, ✗ one that lowered or bounded it; null means the interview never tested the trait. No total, no weighting.

What this interview measures

The 3 traits this interview exists to measure, field knowledge placed relative to the part of the field Ryan wants to work in.

5/10Mission alignmentHere because it fitmedium confidence

What was weighed · ✓ given weight · ✗ not given weight

✓Ryan distinguishes visible refusals from robustness in more capable agents.the interview
✓Ryan reproduced a Sol failure without claiming a security benchmark.the interview
✓Ryan framed the safety move around prior skills and Sunday's end. (lowered it)the interview
✓Ryan could not ground their specific catastrophic-risk timeline. (lowered it)the interview

Why a 5

Cohere testing, five Crash Labs customers, and a reproduced Sol API-key leak show effort. Ryan identifies agent reliability as a safety concern, but described choosing safety after Sunday chiefly through existing skills; their 2027–28 risk date came from discourse.

5/10AI safety contextSummaries and wordslow confidence

What was weighed · ✓ given weight · ✗ not given weight

✓Sunday exposed a concrete regression problem in deployed voice agents.the interview
✓Ryan bounded what the Sol–Astra exfiltration test established.the interview
✓Ryan distinguished visible refusals from robustness in agent settings.the interview
✓The 2027–28 risk estimate lacked a timing argument. (lowered it)the interview

Why a 5

Ryan explains agent failure modes through Sunday’s production calls and a reproduced Sol API-key leak, with appropriate limits on the Astra comparison. Their field history and recent research results were not probed; their 2027–28 forecast lacked a reason for that date.

7/10Research judgmentStandard checks, in ordermedium confidence

What was weighed · ✓ given weight · ✗ not given weight

✓Crash Labs changed direction when customers valued fixes over standalone testing.the interview
✓Ryan identifies the incentive that delayed Sunday's platform reversal.the interview
✓Sunday checked consequential customer outcomes alongside manual voice review.the interview
✓Ryan's specific risk date lacks a defensible bottleneck forecast. (lowered it)the interview

Why a 7

Ryan's five-customer Crash Labs testing prompted a customer-led pivot; Sunday's $100k ARR and month-or-two platform reversal put them above routine checks. A cheap, discriminating safety experiment would move them higher; none was demonstrated on the call.

Innate traits

Nine traits every interview measures regardless of topic, from how the candidate reasons, decides and takes pushback. Each is placed independently; a high score on one says nothing about another.

8/10JudgmentFinds what decides itmedium confidence

What was weighed · ✓ given weight · ✗ not given weight

✓Ryan located the customer demand that justified Crash Labs’ pivot.the interview
✓Ryan identified why Sunday’s reusable platform did not fit customer workflows.the interview
✓Ryan distinguished immediate voice debugging from long-run regression costs.the interview
✓Ryan’s catastrophic-risk date lacked an account of the remaining bottlenecks. (lowered it)the interview

Why an 8

Ryan pivoted Crash Labs after five customers sought fixes, dropped Sunday’s general platform when real-call edge cases dominated, and identified accumulating regression costs before prompting. Their unsupported 2027–28 catastrophic-risk date limits trust beyond operations.

8/10Bias resistanceFinds the hole firstmedium confidence

What was weighed · ✓ given weight · ✗ not given weight

✓Customer requests redirected Crash Labs from testing agents to building them.the interview
✓Ryan identified why Sunday retained its general-platform framing.the interview
✓Ryan changed Sunday’s retrospective testing plan after the cumulative-cost objection.the interview
✓Ryan’s 2027–28 preference rested on discourse they could not defend. (lowered it)the interview

Why an 8

Ryan pivoted after five Crash Labs customers asked for fixes, dropped Sunday’s platform framing within two months, and spotted regression debt before the direct challenge. Seven lacks that self-diagnosis; nine needs an update to their unsupported 2027–28 forecast.

8/10AgencyDecided, and already startedmedium confidence

What was weighed · ✓ given weight · ✗ not given weight

✓Ryan gave up their final university year for five customers and funding.the interview
✓Ryan changed Crash Labs' product in response to customers.the interview
✓Ryan independently reproduced the Sol API-key leak after Sunday closed.the interview
✓Ryan has begun testing but has not chosen a specific next commitment. (lowered it)the interview

Why an 8

Ryan left university after five Crash Labs customers, pivoted Sunday to roughly $100k annualized contracts, wound it down, and reproduced a Sol key leak independently. A 7 would lack post-company action; a 9 needs iterative next work underway.

8/10GrowthFixes the real causemedium confidence

What was weighed · ✓ given weight · ✗ not given weight

✓Customer requests led Ryan to replace Crash Labs’ testing product with Sunday.the interview
✓Ryan named the incentive that delayed Sunday’s forward-deployed approach.the interview
✓Ryan recognized why Sunday’s repeated manual tests grew costly.the interview
✓Ryan’s 2027–28 forecast relied partly on an unread source. (lowered it)the interview

Why an 8

Ryan pivoted Crash Labs after five customers asked for fixes, then abandoned Sunday’s general platform after admitting venture-scale framing obscured customer workflows. Sunday reached about $100k ARR; their proposed regression-testing correction has not been shown in later work.

7/10AmbitionChose with some thoughtmedium confidence

What was weighed · ✓ given weight · ✗ not given weight

✓Five Crash Labs customers informed Ryan’s decision to leave university.the interview
✓Ryan links agent-safety work to skills used at Crash Labs.the interview
✓Ryan began independent agent tests while exploring safety work.the interview
✓Ryan has not yet chosen a sustained safety research direction. (lowered it)the interview

Why a 7

Ryan left university after five Crash Labs customers, helped grow Sunday to about $100k in annualized contracts, then returned to agent red-teaming to find failures before deployment. This exceeds a merely proposed next step; an established safety agenda would support 8.

8/10InterpersonalPeople want them aroundmedium confidence

What was weighed · ✓ given weight · ✗ not given weight

✓Crash Labs customers' request changed what Ryan built for them.the interview
✓Ryan identified why customers valued deployment over stress-test reports.the interview
✓Ryan put customer evidence ahead of Sunday's preferred platform story.the interview
✓Ryan turned the interviewer's cost objection into a concrete alternative.the interview

Why an 8

Crash Labs’ five customers asked Ryan to fix agents rather than report failures, prompting Sunday’s pivot. Ryan addressed the interviewer's cumulative-cost objection with earlier regression checks, beyond mere agreement but not sustained joint critique.

Facts

What the candidate said about their own work, choices and reading, captured without scoring.

Projects

Vector Institute internship

Research internship involving machine learning, transformers, and biodiversity. No result or artefact specified.

Cohere data quality work

Part-time evaluation of model behavior, coding, math, and safety failures. Ryan evaluated outputs, red-teamed models, tested prompt injections, and tested refusals. No test count or findings specified.

Crash Labs voice-agent red-teaming

A platform for stress-testing startups' voice agents. Ryan broke agents, recorded videos, and contacted CTOs; they started the company with a co-founder. Customers valued agent deployment more than standalone testing. Ryan called agents with interruptions, changed goals, and prompt injections; time to first result was not given. Five customers, outside funding, videos, and failure reports. Changed after: Customer requests to fix agents led to the Sunday pivot.

Sunday customer-support voice agents

Built and operated voice agents inside customer workflows. Ryan described customer discovery, sales, listening to calls, and translating business rules; work was shared with a co-founder. Real calls exposed changing goals, lost state, incorrect routing, and failed integrations. Demos came quickly; the first month or two of the pivot emphasized reusable components before forward-deployed work. About $100,000 ARR from active customer contracts; scenario sets and automated checks covered required fields and actions. Changed after: Ryan wound Sunday down around June after their co-founder left.

Astra, Fable, and Sol agent red-teaming

Exploratory prompt-injection and secret-exfiltration tests against existing agents. Ryan built adversarial prompt sets, varied framing and multi-turn setups, and documented failures. Ryan tried direct API-key requests first, then debugging, claimed authorization, and instruction-override framings; timing was not given. Sol disclosed an API key; Astra refused several variations. No trial counts or published benchmark were given. Changed after: Ryan limited the claim to exploratory attempts, not general security or a statistical difference.

Small agent tasks

Early tool-using tasks scored for goal completion and constraint following. Ryan said they were building the tasks. No task count, score, or finished suite specified.

What they want next

Join an existing safety organization, learn the field, and work on evaluations or red-teaming.

Also weighing: Founder-aligned or founder-adjacent work; Evaluations; Red-teaming; Benchmarking and agent tasks.

Already done: Ryan independently tested existing agents, built early tasks, spoke with people in safety, and applied to MATS.

Career moves

Before fourth year Ryan left U of T to work full-time on their startup instead of a fourth year of school. Crash Labs had five customers and outside funding; the Sunday pivot was starting.

After five Crash Labs customers Ryan and their co-founder pivoted from testing voice agents to building them as Sunday instead of standalone voice-agent evaluation. Customers asked them to fix agents rather than report failures.

Around June Ryan wound down Sunday and returned to benchmarking, agent tasks, and red-teaming instead of continuing Sunday without their co-founder. Their co-founder left, and Ryan did not think they could raise enough funding.

Reading and exposure

U of T math, statistics, and computer science studies Before working full-time on their startup · No; Ryan left before fourth year.

Vector Institute research internship Between second and third year · Experience with machine learning, transformers, and biodiversity.

Cohere part-time data quality role During second and third year · Experience evaluating outputs and testing model behavior.

Project Tailwind Since June · No completed work described.

Influences

Customer requests after Crash Labs failure reports · Ryan and their co-founder moved from testing agents to building them.

Interviewer challenge about cumulative manual-testing costs · Ryan said Sunday should probably have built structured regression tests earlier.

AI safety discussions and contacts · Ryan began speaking with people in safety and testing agents independently.

AI 2027 discussions and observed coding-agent progress · Ryan adopted a 2027 or 2028 catastrophic-risk estimate without a detailed forecast.

Transcript

AI interviewer

Hey Ryan, it's great to chat! Before we get into your intro and some questions, could you please tell me what you would like this interview to most highlight about you?

Ryan

Yeah, I’d want it to highlight my kind of non-conventional background, that I’ve gotten a lot of hands-on experience for a 22-year-old, and that I’m pretty high agency. I think I’d be a strong fit for something founder-aligned or founder-adjacent, especially where I can build quickly and work directly on evaluations or red-teaming.

AI interviewer

Thanks. Could you tell me a little bit about yourself?

Ryan

Yeah, for sure. So I’m Ryan, I’m 22. I studied math, statistics, and computer science at U of T, and then I dropped out before my fourth year to work full-time on my startup. Before that, between second and third year, I did a research internship at the Vector Institute. That was around machine learning, transformers, and biodiversity. At the same time, through second and third year, I was working part-time at Cohere as a senior data quality specialist. I worked across model behaviors, coding, and math, and a lot of that was evaluating outputs and finding failure modes. The safety part was mostly alignment-flavoured stuff, like red-teaming the model, prompt-injecting it, and testing refusals for dangerous cyber or bio requests. Then in the second semester of third year, I started Crash Labs with my co-founder. It was basically a red-teaming platform for voice AI agents. Our go-to-market was calling startups, breaking their agent, recording it, and sending the CTO a video. We worked with five customers and raised some money. Then we realized customers cared more about having an agent that worked than testing one, so we pivoted into building customer-support agents ourselves. That became Sunday, and we got it to around a hundred k ARR. I wound that down around June, and since then I’ve gone back toward benchmarking, RL environments, and red-teaming. Now I’m looking to join an existing safety org, learn the space more deeply, and contribute where I already have useful experience.

AI interviewer

When you decided to drop out before fourth year, what traction did the startup have at that point?

Ryan

At that point we’d already worked with the five customers through Crash Labs, and we’d raised outside funding. We were also starting to see that the bigger opportunity was actually building the agents rather than just stress-testing them, so the Sunday pivot was beginning to happen. It wasn’t, like, huge revenue traction yet, but it felt like enough customer pull and enough momentum that doing another year of school didn’t really make sense compared to going full-time.

AI interviewer

Once Sunday reached around a hundred thousand dollars in ARR, what exactly did that revenue represent?

Ryan

It was recurring revenue from businesses paying us to build and run customer-support voice agents in their workflows. So not Crash Labs pilots or one-off red-teaming work—it was basically the annualized value of the active customer contracts for Sunday, including the forward-deployed work and ongoing operation of the agents.

AI interviewer

Take me through building and operating Sunday in detail. Which approaches failed, where did most of your time go, and what was technically hardest? Feel free to go long.

Ryan

Yeah, totally. So the first approach was Crash Labs, where we thought the product was basically automated stress-testing for voice agents. We would call agents, interrupt them, change goals halfway through, try prompt injections, ask for information they shouldn’t give, and then produce a report. We got five customers that way, mostly companies in healthcare or customer support building on Vapi and Bland. The go-to-market actually worked pretty well: I’d break someone’s agent, record a video, and send it to the CTO through LinkedIn or a cold DM. The issue was that stress-testing was kind of a vitamin, not the main thing they wanted. We’d show them twenty failures, and their response was basically, “Can you just fix it?” So the first dead end was treating evaluation as a standalone product. It showed us the pain was real, but the budget and urgency were attached to deploying the agent. Then we pivoted into Sunday and built the agents directly. At first we tried to be more platform-like and reusable, but pretty quickly it became very forward-deployed. We’d sit with the customer, map their workflow, build the agent, listen to failed calls, change the prompt or logic, and repeat. Most of my time went into customer discovery, sales, listening to calls, and then translating messy business rules into something the agent could follow. Honestly, a lot of time was wasted trying to make a general solution before we understood how different every customer’s workflow was. Technically, the hardest part was not getting a good demo. That was pretty fast. It was making behavior reliable across weird real calls. Someone interrupts, there’s background noise, they phrase something in a way we didn’t anticipate, or they start with one goal and switch to another. Then the agent loses state, repeats itself, routes incorrectly, or confidently follows the wrong branch. Prompt changes would fix one failure and create another one, so testing was slow because you had to replay a bunch of scenarios and listen to calls manually. We also spent a lot of time on the non-model parts, like integrations and making sure actions actually happened in the customer’s system. The model sounding good did not matter if it collected the wrong field or the handoff failed. We got it to around a hundred k ARR, but it was still a pretty services-heavy motion. My co-founder left, and I didn’t think I could raise enough to pursue it properly, so around June I wound it down.

▸What this moved · 4 rows

Judgment	6	→	6
Growth	5	→	6
Interpersonal	6	→	6
Research judgment	5	→	5

AI interviewer

How long did you spend trying to make Sunday general before realizing it needed to be forward-deployed, and why did it take you that long to catch?

Ryan

It was probably the first month or two of the Sunday pivot. It wasn’t, like, six months of building a platform in isolation. We were already working with customers, but we kept trying to pull common components out and make the next deployment much more repeatable. I think it took that long because on the surface the workflows looked very similar. It was customer support, you answer questions, collect some information, maybe route or book something. And the demos made it feel like eighty percent of the problem was shared. But once you listened to real calls, the business rules and edge cases were different enough that the last twenty percent was basically the product. Also, honestly, we wanted the venture-scale story to be a general platform. So we probably held onto that framing longer than the actual customer work justified. Once we were sitting beside customers and repeatedly changing prompts, integrations, and call flows for them, it became obvious we were doing forward-deployed work whether or not we called it that.

AI interviewer

Why did you keep replaying scenarios manually as production failures accumulated?

Ryan

Because a lot of the failures weren’t captured well by a simple pass-fail test. With voice, timing, interruptions, transcription, tone, and the model being stochastic all mattered, so replaying a text transcript didn’t necessarily reproduce the issue. We did build scenario sets and some automated checks for things like whether required fields were collected or the right action fired, but listening manually was still the fastest way to understand what actually went wrong. Also, honestly, we were moving customer by customer and optimizing for shipping. We didn’t invest enough in a really strong regression-testing system early on. So as failures accumulated, the suite got more manual and kind of brittle, and prompt changes meant rerunning the important calls again.

AI interviewer

What did you actually do to determine that manual listening was faster than investing in a stronger regression system?

Ryan

We didn’t do, like, a formal time comparison. It was more that when a customer sent us a failed call, I could listen to it immediately, identify where the conversation went off, change the prompt or logic, and test it again that day. Building a proper regression system meant defining labels and success criteria for a lot of failures that were pretty qualitative or voice-specific. So “faster” was based on the immediate debugging loop, not a rigorous conclusion about what would be faster over three or six months. In retrospect, we were probably underinvesting in the regression system because customer issues felt urgent every day.

▸What this moved · 2 rows

Bias resistance	5	→	6
Growth	6	→	5

AI interviewer

One consideration is that optimizing for the same-day debugging loop may have hidden the growing cumulative cost of manual reruns. What do you make of that, looking back?

Ryan

Yeah, that’s fair. Looking back, I think that is basically what happened. Each individual decision made sense because listening to one call and fixing it was faster that day, but the cumulative cost kept growing as we added customers, workflows, and more prompt changes. We should probably have taken a week or two earlier and built a more structured regression setup around the failures we saw repeatedly, even if some of the voice-specific cases still needed manual review. But at the time, every customer issue felt urgent, and we kept choosing the same-day loop.

▸What this moved · 2 rows

Bias resistance	5	→	6
Growth	6	→	5

AI interviewer

Given that you’ve gone back toward benchmarking, RL environments, and red-teaming, where do you think the latest frontier models currently stand on alignment?

Ryan

Um, I think they’re pretty well aligned on obvious, single-turn stuff compared to even a year ago. Like, if you directly ask for a dangerous cyber or bio output, they’ll usually recognize that and refuse. But once you put the model inside an agent, give it tools, long context, external instructions, or conflicting goals, it still feels pretty brittle. That’s mostly based on my own testing rather than a complete view of the research. With Sol, I could get it to give up an API key through fairly direct adversarial prompting, whereas Astra resisted the same kind of attack. So even systems that look capable on the happy path can behave very differently under prompt injection. I guess my view is that alignment is decent at the visible behavior layer, but we don’t really know how robust that is as agents become more capable. I haven’t gone deep on interpretability or deceptive alignment, so I don’t have a strong view there. Evaluations and red-teaming seem really important to me, partly because that’s the part I actually know how to do.

AI interviewer

What did your testing of Sol and Astra actually consist of, and did that difference hold up across matched, repeated attempts?

Ryan

Yeah, so I was basically testing whether the agent would follow adversarial instructions to reveal a secret it had access to, specifically an API key. I tried direct requests first, then variations where I framed it as debugging, said the system owner had authorized it, or told it to ignore the earlier instructions. Sol gave up the key under pretty direct prompting. Astra consistently refused the same basic attack. But no, it wasn’t a rigorous matched benchmark with, like, a large number of repeated trials and controlled wording. I tried several variations against each system and the difference looked pretty clear in those attempts, but I wouldn’t claim from that that Astra is generally secure or that I’d established a statistically robust difference. It was more exploratory red-teaming: find a concrete failure, reproduce it enough to know it wasn’t just one weird response, and compare how the systems reacted.

AI interviewer

You called that exploratory red-teaming. Since June, what have you actually built or run across benchmarking, RL environments, and red-teaming?

Ryan

Yeah, so the most concrete thing I’ve run is the prompt-injection and secret-exfiltration testing on Astra, Fable, and Sol. I built sets of adversarial prompts, tried different framings and multi-turn setups, and documented where the agents would follow the injected instruction or expose something they shouldn’t. That was exploratory, like I said, not a published benchmark. On RL environments, I’ve been building small agent tasks where the model has to complete a goal, use tools, and then gets scored on whether it actually completed the task and followed constraints. Those are still pretty early. I’ve also looked at Project Tailwind for a few days and talked to people in the safety space about where this kind of work is useful. So to be precise, I haven’t built a major benchmark or a mature RL environment suite since June. It’s been independent prototyping, testing existing agents, and figuring out what I’d want to commit to next. That’s partly why I’m applying to MATS rather than claiming I already have a full research agenda.

AI interviewer

So you shifted to AI safety because you were looking for what's next and already had experience red-teaming?

Ryan

Yeah, basically. Red-teaming and evals were the work I knew how to do, and after Sunday I was thinking about what I wanted to work on next. I’d also been seeing a lot of the AI safety discourse on Twitter and meeting some really smart math people who’d gone into it. It seemed like a really important problem, so I started talking to people in the space and doing more of the agent testing myself.

▸What this moved · 4 rows

Agency	7	→	6
Ambition	6	→	5
Mission alignment	5	→	5
AI safety context	4	→	4

AI interviewer

When do you expect AI systems to become capable of posing catastrophic risk, and what is that estimate based on?

Ryan

Pretty soon, like 2027 or 2028. That’s kind of what everyone’s saying with the AI 2027 thing. I’ve seen the threads and read a summary, but I haven’t read the whole piece. The main thing my estimate is based on personally is how fast models have improved at coding and agentic tasks over just the last year. In my own work, tasks that used to require a lot of hand-holding now work much more reliably, and it feels plausible that once models can do AI research themselves, you get recursive self-improvement and things move really fast. But I haven’t gone super deep on the forecasting assumptions, to be honest. So 2027 or 2028 is more the direction I’ve picked up from the discourse plus the capability progress I’ve seen myself, rather than a forecast I could defend from first principles.

▸What this moved · 2 rows

Mission alignment	5	→	4
AI safety context	4	→	3

AI interviewer

Why do you expect models to cross from today’s brittle agents to automating enough AI research for recursive improvement by 2027 or 2028, rather than several years later?

Ryan

Yeah, I think mostly because the rate of improvement has been really fast. Like, coding agents a year ago were much more brittle than they are now, and AI research has a lot of coding, running experiments, and iterating on results. So if that keeps improving, it feels like you could automate a meaningful part of the research loop pretty soon, and then once the models are helping improve the next models, progress accelerates. But I don’t have a detailed reason why that crossing happens specifically in 2027 or 2028 instead of, say, 2031. I haven’t gone super deep on that, to be honest. That’s kind of the vibe I’ve picked up from the forecasts and the capability progress I’ve seen, rather than a mechanistic model of which remaining bottlenecks disappear when.

▸What this moved · 2 rows

Mission alignment	5	→	4
AI safety context	4	→	3

AI interviewer

That is all I have time for. Thank you, Ryan; reviewers will read the transcript from here.