flâneur

LLMs for Alignment Research: a safety priority? — AI Alignment Forum

alignmentforum.org · 4,855 words · saved by 1 readers

A recent short story by Gabriel Mukobi illustrates a near-term scenario where things go bad because new developments in LLMs allow LLMs to accelerate capabilities research without a correspondingly large acceleration in safety research. This scenario is disturbingly close to the situation we already find ourselves in. Asking the best LLMs for help with programming vs technical alignment research feels very different (at least to me). LLMs might generate junk code, but you can keep pointing out the problems with the code, and the code will eventually work. This can be faster than doing it myself, in cases where I don't know a language or library well; the LLMs are moderately familiar with everything. When I try to talk to LLMs about technical AI safety work, however, I just get garbage. I think a useful safety precaution for frontier AI models would be to make them more useful for safety research than capabilities research. This extends beyond applying AI technology to accelerate safety

x LLMs for Alignment Research: a safety priority? — AI Alignment Forum Cyborgism AI Frontpage 60 LLMs for Alignment Research: a safety priority? by abramdemski 4th Apr 2024 13 min read 25 60 A recent short story by Gabriel Mukobi illustrates a near-term scenario where things go bad because new developments in LLMs allow LLMs to accelerate capabilities research without a correspondingly large acceleration in safety research. This scenario is disturbingly close to the situation we already find ourselves in. Asking the best LLMs for help with programming vs technical alignment research feels very

related reading