Rubric

The research and technical work they have done themselves, how well they do it hands-on, and why they chose their problems and methods.

“How good is their taste for what to focus on and which ablations and experiments to run?”

1. 2Restates the plan. The problem is whatever was handed over and the plan is whatever came with it. Nothing in how they talk suggests that one question is worth more of the month than another, or that one check is cheaper or more telling than another; a result is a thing to get, not a thing that would change what comes next.
2. 4Builds first, checks later. A reasonable problem and a reasonable plan, and the plan starts by building. The checks arrive at the end if at all, nothing in the plan could stop it early, and a flaw pointed out gets more data or a bigger model rather than a different design. They can tell a good result from a bad one but not a cheap test from an expensive one.
3. 6Standard checks, in order. Inside their own area they know the checks that matter and run through them in a sensible order, with a cheap first step and a rough sense of when to stop. The order is the one their field teaches rather than one built for this scenario, the fix is the textbook fix, and the plan bends to the checks only a little: the build still comes first and the stop rule is a feeling more than a condition.
4. 8Cheap and decisive first. The plan is built around the check that decides, and the build waits on it. They know what the month can and cannot answer and shrink the claim to fit; the stop rule is a result they can name, and the cost of each step is part of how they choose it. Being scooped is good news because the part their setup answers is still open.
5. 10Rewrites the question. Somewhere on the call they do something the plan did not invite: they see that the question as posed is not the one worth answering, or cannot be answered with what is available, and replace it with the smaller or truer one that can; or they find a cheap test that separates two explanations nobody had separated. You would fund the plan as they described it, with nothing added. One such moment is enough; it needs no record behind it.

How much of the field they know, with views of their own.

“How well do they know the field? How much they take in (what they read, how much and how regularly, and how widely), how deep and current they are in their own area, how much they know of the other promising techniques and where each stands, how well they hold the arguments for and against, and whether they know what work has the most leverage and where the field should go.”

1. 2Names and a mood. The field is a few names and a mood picked up from podcasts and threads, and they do not read it. Asked what is wrong with how models are trained today, they say it is moving fast and nobody understands these things.
2. 4Summaries and words. They keep up through threads, newsletters and other people's summaries. Definitions come and the argument does not: they can say what reward hacking means but not why anyone expects it, what the usual objection is or what the field has found.
3. 6Current in their own corner. They read the papers and posts of their own area regularly and can say what its recent results show and where they stop. Beyond it, the field is the map as taught: the main approaches by name and roughly what each is for, the standard arguments in the community's words.
4. 8Deep in their area, a working map of the rest. Deep and current in their own area, and they read beyond it: they know the other promising techniques and roughly where each stands, hold the main arguments for and against with reasons of their own, and have a view of where the field should go. A 7 keeps up in their own domain and reads some of the forums, without a good grasp of the research across the field or of all the arguments.
5. 10Knows the field and what matters in it. They have gone really deep in their own field and know a lot about the general field: what the other most promising techniques are, the arguments for and against most things, and which work has the most leverage. They are well informed about what they want to work on and why, and can take part in the field's argument.

Why they work on AI safety, read off their reasons and their story, never asked outright.

“Why do they do what they do? Build a picture of what drives them: the work being absorbing, wanting to grow and do something meaningful, a worthy goal, a calling to serve the world, a conviction that the risk from AI is real. Then place how much the world going well, and AI going well in particular, is what moves them.”

1. 2A job or a puzzle. The work is interesting, or well paid, or the next step on a ladder, and that is the whole story. The world beyond them comes up only when you bring it in, and then in words that belong to someone else.
2. 4Here because it fit. They came because their experience or skills transferred, or the program was there, and nothing they have said or done since shows that the problem itself moves them.
3. 6Fascinated by the work. The research is intellectually absorbing to them and they are good at it, but they do not really believe in the risk, and the big picture of what the work is for rarely comes up unprompted.
4. 8A worthy goal, worked for. They find the research absorbing, want to grow and do something meaningful with their life, think AI safety is a worthy goal to work towards, and have put in a lot of effort and given things up for it. Concern about the risk is there but is not what drives them; a 7 is the same picture with less effort or less shown of why.
5. 10A calling. They believe they have a higher calling to serve the world, or are so convinced that AI carries an existential risk that they cannot picture doing anything else. Someone who keeps thinking hard about the most ambitious thing they could work on and goes from moonshot to moonshot at what would help the world most reads here too.

How much of their path they decided rather than accepted, and how hard they drive their own work.

“How much of their path did they decide rather than accept, and is the next move already moving without waiting on anyone's permission?”

1. 2Takes what is offered. Things happen to them. Each step was whatever was in front of them, and the next step belongs to a committee, a manager or a program. Asked what they would do otherwise, they have not thought about it, because there was never an otherwise.
2. 4Hard work, assigned frame. They work hard and it shows: extras taken on, promotions earned, groups joined. But the frame was always drawn by someone else, and it has not occurred to them that the frame itself was a choice. The next step is to apply and see what happens.
3. 6Stepped off the default. Somewhere they left the path laid out for them and it cost them something real by their own means, or they started something nobody asked for and kept at it after the novelty wore off. The next step exists as a plan, told as something that might happen rather than something already moving.
4. 8Decided, and already started. Read as a whole, their path looks decided rather than drifted: at the forks they can say what the easy option was and why they did not take it. The next move is theirs, chosen because it fits their situation rather than for prestige, and the first action has already happened. They are not waiting on anyone to begin. With a short record this shows in one stretch of life told the same way: a thing they wanted, the easier thing they passed up, and the first move already made.
5. 10Already underway. You cannot picture them waiting. The next thing is in motion whether or not anyone says yes; the fear is named and treated as a constraint rather than a reason; the rejected case is the same work with a different funding line. Their life may be the evidence, but in fifteen minutes it can show entirely in how they talk about what comes next: names, half-built things, what came back.

How they run their own work: urgency, pacing, who set the milestones.

“When the work is theirs to run, do they run it: drive it with their own urgency, pace it and keep their own deadlines, want the control and do well with it, and understand from how it went how they work?”

1. 2Waits to be run. The work moves when someone sets it and checks on it. No pace of their own to describe, deadlines are other people's, and when it slipped the reason was someone else. Nothing in how they tell it says they wanted the thing to go well beyond their part.
2. 4Does their part. Keeps to the deadlines they were given and reports back when asked. The pace was the job's. Control is not something they talk about wanting or using, and what they took from it is about what happened to them rather than how they work.
3. 6Runs their own piece. Runs their own part without being chased: sets their own milestones and mostly keeps them, says how they paced it and why, checks in with the people who need to know. Has a view on what they would change about how they worked, if not yet a lesson about their own habits.
4. 8Takes the whole thing on. Takes a whole problem on themselves and drives it at their own urgency, fastest when the experiments are theirs to control, and says so. Clears the bottlenecks in the way one after another. Knows how they pace themselves, and the takeaways are about how they work and what they now do differently.
5. 10Carries it alone, for a long time. Keeps driving the same aim for a year or more with nobody setting the deadlines, through pivot after pivot and without money coming in, and has drawn lessons about their own way of working from each turn.

How they work with the people around them and under stress.

“How are they with the people they work with, and under stress: open-minded in a conflict, warm, caring and friendly, easy to be around, in touch with others often, and at ease under someone else's authority?”

1. 2Others are the problem. The people in their stories are obstacles or incompetent, with no reasons of their own. A conflict is something to win. Nothing in how they tell it is warm, and someone with authority over them is someone to get around.
2. 4Civil, at a distance. Gets along, works alone where they can, and has few touch points with others. In a conflict they hold their line or go quiet; the other side's reasons do not appear. Little warmth or humour in the telling, and uneasy when someone else leads.
3. 6A good colleague. Friendly, checks in with people when it matters, disagrees in the open and can say what the other side wanted. The people they describe seem glad to work with them, and they can follow someone else's lead while doing it differently in their head.
4. 8People want them around. Warm, caring and easy company in how they tell it and in what they choose to say others say of them; still close to the people after the work ended. In a conflict they can say where the other side was right and change course quickly once they see it, even after holding their own idea hard. Touches base often, and can work under someone else's lead while saying plainly where they disagree.
5. 10Makes the people around them better. The warmth, care and ease run through every story and through what others chose to say of them. A conflict becomes something both sides learn from, they are open to being wrong in the middle of it, and they are trusted by the people above them and the people they lead.