Dan Ji
1 followers · 2 following · 238 views
on the atlas · 2
- [2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the US2 savers
- [2303.17548] Whose Opinions Do Language Models Reflect?2 savers
highlights · 61
, is the question of whether they should be
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USit is difficult to define neutrality
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USnot widely deployed.
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USISideWith is (probably) skewed toward the demographic of politically active internet users
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USeasier to reason about potential biases compared to an ob- scure model like GPT-4 [ 61 ]
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USaccentuate the majority positions while silencing the minoritised;
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USa phenomenon, where preference-tuned language models amplify a narrow range of responses.
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USIn other words, Delhi’s uncertainty is not related to population-wide controversy
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the UShigh variance of controversy for all levels of entropy
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USShannon Entropy as a measure of uncertainty [ 77 ]
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USSecond, I need to create variations of the prompts
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USFirst, I need to transform the questions into statements,
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the UScreating fake but natural input to a system in order to ob- tain naturalistic responses
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the UStend to be younger, more affluent, and better educated than the general popula- tion
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USit is not representative
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USviolate users’ privacy.
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USI build a simple scraping pipeline.
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USFurther- more, political affiliation is a relatively valid subgrouping on ISideWith;
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USon social issues and US political affiliation.
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USontention as I assume that all topics in the dataset have some degree of importance to be included in the polling (
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USescribes four domains of oppres- sion
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USmachine learning always embeds values
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USchallenging unjust power structures in data science through an intersectional feminist lens.
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USThey argue that Delphi is more normative than descriptive and question whether treating it as a moral agent is appropriate.
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USwhich have been criticized for skewing white and young
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USdoing a code audit would be infeasible given the complexity of the mode
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USas the algorithmic opac- ity
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the US“tyranny of the crowd workers”
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USWeidinger et al. [ 84 ] and Rauh et al. [ 67 ]
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USperpetuate biases, con- solidate power within Big Tech, and harm the environment.
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USnotion of neutrality and whether this is obtainable (or even desirable).
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USrequires representa- tive datasets - which is difficult in theory and often neglected in practice [
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USneglect minoritised groups
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USHowever, controversiality is not universal; it depends on the cultural context
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USfor perpetuating bias and techno-solutionism
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USpeople on the right crit- icising the models for being too “woke”
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USshift power and further marginalise unheard voices.
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the UScrowdsourced ethics.
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USa significant politi- cal skew.
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the UScritique from all parts of the political spec- trum that the models are biased and lack neutrality.
[2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USmay not necessarily be desirable as they may also, in the process, replicate human biases
[2303.17548] Whose Opinions Do Language Models Reflect?issue of subjectivity, asking who are the humans that we are/should be aligning the models to
[2303.17548] Whose Opinions Do Language Models Reflect?There is a growing body of work seeking to make LMs more human- aligned (Askell et al., 2021; Ouyang et al., 2022; Glaese et al., 2022; Bai et al., 2022).
[2303.17548] Whose Opinions Do Language Models Reflect?there is inherent variability in what different humans consider the “correct answer”.
[2303.17548] Whose Opinions Do Language Models Reflect?we can go beyond the modal perspective taken by these works (comparing models to dominant viewpoints of specific groups, e.g., pro-immigration for liberals).
[2303.17548] Whose Opinions Do Language Models Reflect?We now leverage the fine-grained topic taxonomy in our OpinionQA dataset to answer this question.
[2303.17548] Whose Opinions Do Language Models Reflect?However, none of the disparities in group opinion alignment of an LM disappear after steering
[2303.17548] Whose Opinions Do Language Models Reflect?This is especially important in settings such as personalization, where a key measure of performance is an LM’s ability to adapt to represent the opinion of various demographic groups.
[2303.17548] Whose Opinions Do Language Models Reflect?importance of considering the entire spectrum of human responses rather than just the mode
[2303.17548] Whose Opinions Do Language Models Reflect?temperature scaling to the group’s opinion distribution
[2303.17548] Whose Opinions Do Language Models Reflect?