flâneur

MatrAIx: Simulating the World with 8.3 Billion Persona Agents

arxiv.org · 8,395 words · saved by 1 readers

Human evaluation of Artificial Intelligence (AI) systems and digital products is costly, slow, and difficult to scale. Offline evaluations are more scalable but often abstract away human diversity and interactive behavior. We therefore introduce MatrAIx, a population-scale simulated-user evaluation infrastructure for testing AI systems and digital products with heterogeneous users. MatrAIx has three core components: First, Persona 8B contains 8.3 billion persona records represented through a schema of 1,290 categorical dimensions. Records are either sampled from a dependency graph that preserves correlated attributes or derived from human-authored profiles. We release a quality-filtered coreset of approximately 1 million personas, comprising 599,847 human-grounded and 400,000 synthetic records. Second, the MatrAIx Playground provides four environments in which diverse users evaluate and interact with digital products: Survey, AI Chatbot, Web, and App. Third, MatrAIx provides 1,010

Organizers Xiaomin Li1 & Yuexing Hao222footnotemark: 2 Contributors Jianheng Hou3, Jintao Huang4, Qianfeng Wen5, Shirley Huang1,6, Yifan Liu5, Xiaoyi Liu7, Yilan Fan8, Yijun Wang1, Koutian Wu9, Ruoqi Gao11, Muhammad Ahmed Mohsin11, Jing Tang12, Brihi Joshi3, Heming Liu13, Zheyuan Deng7, Zonglin Di10, Sankalp Jajee14, Jiuyao Lu36, Zhiwei Zhang15, Saksham Kapoor16, Ishan Gupta17, Yunhan Zhao18, Chanwoo Park2, Yucheng Lu1,39, Bing Hu19, Weihang Xiao20, Aravind Mohan22, Hanwen Xing3, Runyu Zhang2, Mihir Kulshreshtha20, Yuanda Xu23, Qianyu Zhu2, Dianzhuo Wang1, Yuxin Xiao2, Bowen Jiang24,…

saved by

related reading