[2303.17548] Whose Opinions Do Language Models Reflect?
Language models (LMs) are increasingly being used in open-ended contexts, where the opinions reflected by LMs in response to subjective queries can have a profound impact, both on user satisfaction, as well as shaping the views of society at large. In this work, we put forth a quantitative framework to investigate the opinions reflected by LMs -- by leveraging high-quality public opinion polls and their associated human responses. Using this framework, we create OpinionsQA, a new dataset for evaluating the alignment of LM opinions with those of 60 US demographic groups over topics ranging from abortion to automation. Across topics, we find substantial misalignment between the views reflected by current LMs and those of US demographic groups: on par with the Democrat-Republican divide on climate change. Notably, this misalignment persists even after explicitly steering the LMs towards particular demographic groups. Our analysis not only confirms prior observations about the left-leaning tendencies of some human feedback-tuned LMs, but also surfaces groups whose opinions are poorly reflected by current LMs (e.g., 65+ and widowed individuals). Our code and data are available at this https URL.
Whose Opinions Do Language Models Reflect? Shibani Santurkar Esin Durmus Faisal Ladhak Stanford Stanford Columbia University shibani@stanford.edu esindurmus@cs.stanford.edu faisal@cs.columbia.edu arXiv:2303.17548v1 [cs.CL] 30 Mar 2023 Cinoo Lee Percy Liang…
saved by
related reading
- [2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USarxiv.org
- [2305.18290] Direct Preference Optimization: Your Language Model is Secretly a Reward Modelarxiv.org
- 2308.03958arxiv.org
- [2203.02155] Training language models to follow instructions with human feedbackarxiv.org
- Discovering Language Model Behaviors with Model-Written Evaluations — LessWronglesswrong.com
- [2302.08582] Pretraining Language Models with Human Preferencesarxiv.org
- Assessing Political Bias in Language Models | Stanford HAIhai.stanford.edu
- Training language models to follow instructions with human feedback.pdfproceedings.neurips.cc
- [2511.08579] Training Language Models to Explain Their Own Computationsarxiv.org
- [2604.21751] Why are all LLMs Obsessed with Japanese Culture? On the Hidden Cultural and Regional Biases of LLMsarxiv.org
- Probing Persona-Dependent Preferences in Language Modelsarxiv.org
- [2605.27288] It's Not Always Sycophancy: Measuring LLM Conformity as a Function of Epistemic Uncertaintyarxiv.org