Representation Engineering: a New Way of Understanding Models | CAIS

safe.ai · 735 words · saved by 1 readers

Interpreting and controlling models has long been a significant challenge. Our research ‘Representation Engineering: A Top-Down Approach to AI Transparency’ explores a new way of understanding traits like honesty, power seeking, and morality in LLMs. We show that these traits can be identified live at the point of output, and they can also be controlled. This method differs from mechanistic approaches which focus on bottom-up interpretations of node to node connections. In contrast, representation engineering looks at larger chunks of representations and higher-level mechanisms to understand models. Overall, we believe that this ‘top-down’ method makes exciting progress towards model transparency, paving the way for more research and exploration into understanding and controlling AI. Why understanding and controlling AI is important Transparency and honesty are important features of models - as AI becomes more powerful, capable and autonomous, it is increasingly important that they are

Representation Engineering: a New Way of Understanding Models | CAIS About About AI risk Resources Resources Contact Careers Donate Our Work Resources AI Risk Contact Careers Donate Careers Donate Representation Engineering: a New Way of Understanding Models BLOG AI Risks April 17, 2024 5 min read View as PDF Author: Izzy Barrass, Long Phan Related Posts: A Significant Increase in Digital Labor Automation Submit Your Toughest Questions for Humanity's Last Exam Reading the minds of LLMs Interpreting and controlling models has long been a significant challenge. Our research ‘Representation Engin

Representation Engineering: a New Way of Understanding Models | CAIS

related reading