Hidden Bias
Models trained on real-world data can encode real-world bias. Hiding information about protected classes doesn't always fix things — sometimes it can even hurt.
Hidden Bias Explorables Hidden Bias Models trained on real-world data can encode real-world bias. Hiding information about protected classes doesn't always fix things — sometimes it can even hurt. Modeling College GPA Let's pretend we're college admissions officers trying to predict the GPA students will have in college (in these examples we'll use simulated data). One simple approach: predict that students will have the same GPA in college as they did in high school. This is at best a very rough approximation, and it misses a key feature of this data set: students usually have better grades i
Explore this link on the map →related reading
- Subliminal Learning: Language Models Transmit Behavioral Traits via Hidden Signals in Dataalignment.anthropic.com
- Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
- Statistical Fairness - Turing Commonsalan-turing-institute.github.io
- Can a Model Be Differentially Private and Fair?pair.withgoogle.com
- Race and Gender Bias As An Example of Unfaithful Chain of Thought in the Wild — LessWronglesswrong.com
- Should We Train Against (CoT) Monitors? — LessWronglesswrong.com
- Models don’t seem to be dishonest in the way humans are — LessWronglesswrong.com
- Teaching Models to Dream of Better Monitors through Evaluation Conditioned Training — LessWronglesswrong.com
- A Course in Machine Learningciml.info
- MAI-Thinking-1: Building a Hill-Climbing Machinemicrosoft.ai
- Papers · Nikhil Garggargnikhil.com
- Auditing language models for hidden objectives — LessWronglesswrong.com