TY - GEN
T1 - Artificial Intelligence Bias on English Language Learners in Automatic Scoring
AU - Guo, Shuchen
AU - Wang, Yun
AU - Yu, Jichao
AU - Wu, Xuansheng
AU - Ayik, Bilgehan
AU - Watts, Field M.
AU - Latif, Ehsan
AU - Liu, Ninghao
AU - Liu, Lei
AU - Zhai, Xiaoming
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
PY - 2025/7
Y1 - 2025/7
N2 - This study investigated potential scoring biases and disparities toward English Language Learners (ELLs) when using automatic scoring systems for middle school students’ written responses to science assessments. We specifically focus on examining how unbalanced training data with ELLs contributes to scoring bias and disparities. We fine-tuned BERT with four datasets: responses from (1) ELLs, (2) non-ELLs, (3) a mixed dataset reflecting the real-world proportion of ELLs and non-ELLs (unbalanced), and (4) a balanced mixed dataset with equal representation of both groups. The study analyzed 21 assessment items: 10 items with about 30,000 ELL responses, five items with about 1,000 ELL responses, and six items with about 200 ELL responses. Scoring accuracy (Acc) was calculated and compared to identify bias using Friedman tests. We measured the Mean Score Gaps (MSGs) between ELLs and non-ELLs and then calculated the differences in MSGs generated through both the human and AI models to identify the scoring disparities. We found that no AI bias and distorted disparities between ELLs and non-ELLs were found when the training dataset was large enough (ELL≈30,000 and ELL≈1,000), but concerns could exist if the sample size is limited (ELL≈200).
AB - This study investigated potential scoring biases and disparities toward English Language Learners (ELLs) when using automatic scoring systems for middle school students’ written responses to science assessments. We specifically focus on examining how unbalanced training data with ELLs contributes to scoring bias and disparities. We fine-tuned BERT with four datasets: responses from (1) ELLs, (2) non-ELLs, (3) a mixed dataset reflecting the real-world proportion of ELLs and non-ELLs (unbalanced), and (4) a balanced mixed dataset with equal representation of both groups. The study analyzed 21 assessment items: 10 items with about 30,000 ELL responses, five items with about 1,000 ELL responses, and six items with about 200 ELL responses. Scoring accuracy (Acc) was calculated and compared to identify bias using Friedman tests. We measured the Mean Score Gaps (MSGs) between ELLs and non-ELLs and then calculated the differences in MSGs generated through both the human and AI models to identify the scoring disparities. We found that no AI bias and distorted disparities between ELLs and non-ELLs were found when the training dataset was large enough (ELL≈30,000 and ELL≈1,000), but concerns could exist if the sample size is limited (ELL≈200).
KW - AI bias
KW - AI disparities
KW - Artificial Intelligence
KW - Automatic scoring
KW - English language learners
KW - Science assessment
UR - https://www.scopus.com/pages/publications/105012031864
U2 - 10.1007/978-3-031-98462-4_34
DO - 10.1007/978-3-031-98462-4_34
M3 - Conference article published in proceeding or book
AN - SCOPUS:105012031864
SN - 9783031984617
T3 - Lecture Notes in Computer Science
SP - 268
EP - 275
BT - Artificial Intelligence in Education - 26th International Conference, AIED 2025, Proceedings
A2 - Cristea, Alexandra I.
A2 - Walker, Erin
A2 - Lu, Yu
A2 - Santos, Olga C.
A2 - Isotani, Seiji
PB - Springer Science and Business Media Deutschland GmbH
T2 - 26th International Conference on Artificial Intelligence in Education, AIED 2025
Y2 - 22 July 2025 through 26 July 2025
ER -