Probing into the Fairness of Large Language Models: A Case Study of ChatGPT

TY - GEN

T1 - Probing into the Fairness of Large Language Models

T2 - 58th Annual Conference on Information Sciences and Systems, CISS 2024

AU - Li, Yunqi

AU - Zhang, Lanjing

AU - Zhang, Yongfeng

N1 - Publisher Copyright: © 2024 IEEE.

PY - 2024

Y1 - 2024

N2 - Understanding and addressing unfairness in LLMs are crucial for responsible AI deployment. However, there is a limited number of quantitative analyses and in-depth studies regarding fairness evaluations in LLMs, especially when applying LLMs to high-stakes fields. This work aims to fill this gap by providing a systematic evaluation of the effectiveness and fairness of LLMs using ChatGPT as a study case. We focus on assessing ChatGPT's performance in high-takes fields including education, criminology, finance and healthcare. To conduct a thorough evaluation, we consider both group fairness and individual fairness metrics. We also observe the disparities in ChatGPT's outputs under a set of biased or unbiased prompts. This work contributes to a deeper understanding of LLMs' fairness performance, facilitates bias mitigation and fosters the development of responsible AI systems. Code and data are open-sourced on GitHub.

AB - Understanding and addressing unfairness in LLMs are crucial for responsible AI deployment. However, there is a limited number of quantitative analyses and in-depth studies regarding fairness evaluations in LLMs, especially when applying LLMs to high-stakes fields. This work aims to fill this gap by providing a systematic evaluation of the effectiveness and fairness of LLMs using ChatGPT as a study case. We focus on assessing ChatGPT's performance in high-takes fields including education, criminology, finance and healthcare. To conduct a thorough evaluation, we consider both group fairness and individual fairness metrics. We also observe the disparities in ChatGPT's outputs under a set of biased or unbiased prompts. This work contributes to a deeper understanding of LLMs' fairness performance, facilitates bias mitigation and fosters the development of responsible AI systems. Code and data are open-sourced on GitHub.

KW - ChatGPT

KW - Fairness

KW - Large Language Model

UR - http://www.scopus.com/inward/record.url?scp=85190625609&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85190625609&partnerID=8YFLogxK

U2 - 10.1109/CISS59072.2024.10480206

DO - 10.1109/CISS59072.2024.10480206

M3 - Conference contribution

T3 - 2024 58th Annual Conference on Information Sciences and Systems, CISS 2024

BT - 2024 58th Annual Conference on Information Sciences and Systems, CISS 2024

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 13 March 2024 through 15 March 2024

ER -