Systematic Review of the Demographic Representativeness of LLMs

Code repository for the ACL 2025 Findings paper: "Missing the Margins: A Systematic Literature Review on the Demographic Representativeness of LLMs."

Data

All the annotatated data can be found in the data folder. Additonally the list of papers can be found in this repository categorized by LLM usage contexts here on Zotero: https://www.zotero.org/groups/6070711/demographic_representativeness_of_llms/library

Analysis

• descriptive and rq1 assesses the relationship between contexts, evaluation, and demographics (Sections 4.1 and 4.2)
• rq2 and deeper analysis assess the relation between 'conclusion on representativeness' and other variables (Sections 4.3 and 4.4)
• temporal has the plot for Figure 5

If you use our dataset or code, please cite our paper: Sen, I., Lutz, M., Rogers, E., Garcia, D., & Strohmaier, M. (2025). Missing the Margins: A Systematic Literature Review on the Demographic Representativeness of LLMs. To appear in ACL Findings 2025