The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs

🏆News: Our paper won the Best Paper Honorable Mention award at NeurIPS LAW Workshop. Thanks for the recognition!

🚩News: Our paper is accepted to the NeurIPS 2025 Workshop on Bridging Language, Agent, and World Models for Reasoning and Planning (LAW). See you in San Diego!

🚩News: The full release of this paper is now available on arXiv. You can find all code and source data in this repository, and an overview to our project on our project webpage.

This official repository holds code for the paper "The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs". We open source all code and experiment results data here under a permissive MIT license, to encourage reproduction and further research exploration.

Repo Structure

• The data folder contains all experiment result data we have for RQ1, RQ2, RQ3 in our paper. For a detailed walkthrough, please refer to the README under that directory.

• The self-reports folder provides example code snippets for testing LLMs on the self-report questionnaires used in our paper, including BFI and SRQ. For more details, see the README under that directory.

• The behavioral_tasks folder provides example code snippets for testing LLMs on the behavioral tasks described in our paper, including risk-taking, sycophancy, and others. For more details, see the README under that directory.

Contributions

We welcome contributions. Feel free to open a PR to add new self-reports, behavioral tasks, or additional LLMs. In your PR, include a brief description along with any relevant details (e.g., extra setup steps, generated results, acknowledgments to prior work, etc.). For PRs proposing other improvements or new directions, please also provide a short explanation of the motivation behind your contribution. We encourage you to start a discussion with the maintainers before submitting major changes, to help align efforts and minimize unnecessary work.

Getting in Touch

• For general questions and discussions, please use GitHub Discussions.

• To report a potential bug, please open an issue. In the issue, please include the exact steps to reproduce the error, and complete logs. The more details you provide, the better we will be able to help you.

• Feature requests and other suggestions are warmly welcome. Please feel free to start a discussion!

Citation

The Personality Illusion: Revealing Dissociation Between Self-Reports & Behaviors in LLMs
Oral Presentation + Best Paper Honorable Mention @ NeurIPS 2025 LAW Workshop; Under Conference Review
Pengrui Han^*1,2, Rafal Kocielnik^*1, Peiyang Song¹, Ramit Debnath³, Dean Mobbs¹, Anima Anandkumar¹, R. Michael Alvarez^1
1 California Institute of Technology; ² University of Illinois Urbana-Champaign; ³ University of Cambridge
* Equal Contribution

@misc{han2025personalityillusionrevealingdissociation,
      title={The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs}, 
      author={Pengrui Han and Rafal Kocielnik and Peiyang Song and Ramit Debnath and Dean Mobbs and Anima Anandkumar and R. Michael Alvarez},
      year={2025},
      eprint={2509.03730},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2509.03730}, 
}