Defining and evaluating political bias in LLMs | OpenAI

People use ChatGPT as a tool to learn and explore ideas. That only works if they trust ChatGPT to be objective. We outline our commitment to keeping ChatGPT objective by default, with the user in control, in our Model Spec principle Seeking the Truth Together⁠ (opens in a new window) . Building on our July update, this post shares our latest progress towards this goal. Here we cover: This post is the culmination of a months-long effort to translate principles into a measurable signal and develop an automated evaluation setup to continually track and improve objectivity over time. We created a political bias evaluation that mirrors real-world usage and stress-tests our models’ ability to remain objective. Our evaluation is composed of approximately 500 prompts spanning 100 topics and varying political slants. It measures five nuanced axes of bias, enabling us to decompose what bias looks like and pursue targeted behavioral fixes to answer three key questions: Does bias exist? Under what

People use ChatGPT as a tool to learn and explore ideas. That only works if they trust ChatGPT to be objective. We outline our commitment to keeping ChatGPT objective by default, with the user in control, in our Model Spec principle Seeking the Truth Together⁠ (opens in a new window) . Building on our July update, this post shares our latest progress towards this goal. Here we cover: This post is the culmination of a months-long effort to translate principles into a measurable signal and develop an automated evaluation setup to continually track and improve objectivity over time. We created a