“Do AI safety talent programmes work? Nobody has run the study that would tell us.” by Laura Thomas-Walters

Headline finding: I audited 17 AI Safety Talent programmes. Zero of 17 have published any comparison group, rejected-applicant follow-up, matched control or randomisation. Not one. Every programme that mentions a counterfactual does it by asking participants to self-report.

Background

At least 70 million dollars has gone into AI safety talent programmes so far, but we still can’t really say much about their impact. Notably, Kairos has just raised 50 million dollars on a self-described "shallow retrospective" that looked at "around 80 people". I decided to do a deep dive into how AI Safety Talent programmes self-evaluate their impact.

Disclaimer, these programmes are small, fast, and generally run by people with no evaluation training and no slack. The oldest one was only established in 2018 (AI Safety Camp, as far as I can tell). Most last less than 6 months. When timelines are genuinely short and urgent, then time spent evaluating is time that could be spent building. I am not criticising any individual programme.

Having said that, there are retrospective evaluation designs that could be run cheaply without slowing anything down. And a field that funds talent pipelines at this scale without knowing whether they work is not moving [...]

---

Outline:

(00:34) Background

(01:52) The audit

(04:52) Thoughts on the counterfactual impact of talent programmes

(08:37) Proposed design

(12:04) Conclusion

The original text contained 4 footnotes which were omitted from this narration.

---

First published:
September 7th, 2026

Source:
https://forum.effectivealtruism.org/posts/S5yBGo4fnqm2zKMKj/do-ai-safety-talent-programmes-work-nobody-has-run-the-study

---

Narrated by TYPE III AUDIO.