flâneur

How I select alignment research projects — AI Alignment Forum

alignmentforum.org · 8,037 words · saved by 1 readers

Recently, I was interviewed by Henry Sleight and Mikita Balesni about how I select alignment research projects. Below is the slightly cleaned up transcript for the YouTube video. Henry Sleight: How about you two introduce yourselves? Ethan Perez: I'm Ethan. I'm a researcher at Anthropic and do a lot of external collaborations with other people, via the Astra Fellowship and SERI MATS. Currently my team is working on adversarial robustness, and we recently did the sleeper agents paper. So, basically looking at we can use RLHF or adversarial training or current state-of-the-art alignment safety training techniques to train away bad behavior. And we found that in some cases, the answer is no: that they don't train away hidden goals or backdoor behavior and models. That was a lot of my focus in the past, six to twelve months. Mikita Balesni: Hey, I'm Mikita. I work at Apollo. I'm a researcher doing evals for scheming. So trying to look for whether models can plan to do something bad later.

x How I select alignment research projects — AI Alignment Forum Practice & Philosophy of Science Research Taste AI Practical Frontpage 14 How I select alignment research projects by Ethan Perez , Henry Sleight , Mikita Balesni 10th Apr 2024 29 min read 4 14 Youtube Video Recently, I was interviewed by Henry Sleight and Mikita Balesni about how I select alignment research projects. Below is the slightly cleaned up transcript for the YouTube video. Introductions Henry Sleight : How about you two introduce yourselves? Ethan Perez : I'm Ethan. I'm a researcher at Anthropic and do a lot of external

saved by

related reading