flâneur

Safety Alignment Should Be Made More Than Just a Few Tokens Deep

arxiv.org · 28,774 words · saved by 1 readers

This is experimental HTML to improve accessibility. We invite you to report rendering errors. Use Alt+Y to toggle on accessible reporting links and Alt+Shift+Y to toggle off. Learn more about this project and help improve conversions. HTML conversions sometimes display errors due to content that did not convert correctly from the source. This paper uses the following packages that are not yet supported by the HTML conversion tool. Feedback on these issues are not necessary; they are known and are being worked on. Authors: achieve the best HTML results from your LaTeX submissions by following these best practices. The safety alignment of current Large Language Models (LLMs) is vulnerable. Relatively simple attacks, or even benign fine-tuning, can jailbreak aligned models. We argue that many of these vulnerabilities are related to a shared underlying issue: safety alignment can take shortcuts, wherein the alignment adapts a model’s generative distribution primarily over only its very fi

Safety Alignment Should Be Made More Than Just a Few Tokens Deep Xiangyu Qi Princeton University xiangyuqi@princeton.edu &Ashwinee Panda Princeton University ashwinee@princeton.edu &Kaifeng Lyu Princeton University klyu@cs.princeton.edu &Xiao Ma Google DeepMind xmaa@google.com &Subhrajit Roy Google DeepMind subhrajitroy@google.com &Ahmad Beirami Google DeepMind beirami@google.com &Prateek Mittal Princeton University pmittal@princeton.edu &Peter Henderson Princeton University peter.henderson@princeton.edu Abstract The safety alignment of current Large Language Models (LLMs) is vulnerable. Relat

related reading