[2407.20371] Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval

Abstract:Artificial intelligence (AI) hiring tools have revolutionized resume screening, and large language models (LLMs) have the potential to do the same. However, given the biases which are embedded within LLMs, it is unclear whether they can be used in this scenario without disadvantaging groups based on their protected attributes. In this work, we investigate the possibilities of using LLMs in a resume screening setting via a document retrieval framework that simulates job candidate selection. Using that framework, we then perform a resume audit study to determine whether a selection of Massive Text Embedding (MTE) models are biased in resume screening scenarios. We simulate this for nine occupations, using a collection of over 500 publicly available resumes and 500 job descriptions. We find that the MTEs are biased, significantly favoring White-associated names in 85.1\% of cases and female-associated names in only 11.1\% of cases, with a minority of cases showing no statistically significant differences. Further analyses show that Black males are disadvantaged in up to 100\% of cases, replicating real-world patterns of bias in employment settings, and validate three hypotheses of intersectionality. We also find an impact of document length as well as the corpus frequency of names in the selection of resumes. These findings have implications for widely used AI tools that are automating employment, fairness, and tech policy.

View PDF HTML (experimental) Abstract:Artificial intelligence (AI) hiring tools have revolutionized resume screening, and large language models (LLMs) have the potential to do the same. However, given the biases which are embedded within LLMs, it is unclear whether they can be used in this scenario without disadvantaging groups based on their protected attributes. In this work, we investigate the possibilities of using LLMs in a resume screening setting via a document retrieval framework that simulates job candidate selection. Using that framework, we then perform a resume audit study to…

saved by

• Lailah Nabegu

related reading

• Thoughts on AI in academiatheinfinitesimal.substack.com
• Race and Gender Bias As An Example of Unfaithful Chain of Thought in the Wild — LessWronglesswrong.com
• [2306.13000] Apolitical Intelligence? Auditing Delphi's responses on controversial political issues in the USarxiv.org
• [2606.06614] Re-Centering Humans in LLM Personalizationarxiv.org
• Thoughts on the job market in the age of LLMsinterconnects.ai
• Predicting Empirical AI Research Outcomes with Language Modelsarxiv.org
• [2303.17548] Whose Opinions Do Language Models Reflect?arxiv.org
• [2604.21751] Why are all LLMs Obsessed with Japanese Culture? On the Hidden Cultural and Regional Biases of LLMsarxiv.org
• Language Models can Solve Computer Tasksarxiv.org
• [2601.10160] Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignmentarxiv.org
• Lost in Simulation: LLM-Simulated Users are Unreliable Proxies for Human Users in Agentic Evaluationsarxiv.org
• Foundational Challenges in Assuring Alignment and Safety of Large Language Modelsarxiv.org