Paper Number
ECIS2026-2700
Paper Type
SP
Abstract
AI-enabled hiring systems are widely adopted, yet their fairness remains uncertain. New York City’s Local Law 144 mandates annual bias audits to increase transparency. However, the effectiveness of these audits remains unclear. An analysis of LL144 audit reports reveals demographic missingness, from under 3% to over 50%, which reduces the applicant pool used for fairness calculation and undermines the metrics. Using institutional theory, we argue that such limitations reflect symbolic compliance, while stewardship theory highlights the potential for deeper accountability. We propose leveraging audit outputs as red-teaming inputs to stress-test fairness robustness and strengthen AI governance through improved data quality and oversight.
Recommended Citation
Ogbanufe, Obi, "Towards Using AI Bias Audits As Inputs For Red Teaming and Performance" (2026). ECIS 2026 Proceedings. 9.
https://aisel.aisnet.org/ecis2026/resp_AI/resp_AI/9
Towards Using AI Bias Audits As Inputs For Red Teaming and Performance
AI-enabled hiring systems are widely adopted, yet their fairness remains uncertain. New York City’s Local Law 144 mandates annual bias audits to increase transparency. However, the effectiveness of these audits remains unclear. An analysis of LL144 audit reports reveals demographic missingness, from under 3% to over 50%, which reduces the applicant pool used for fairness calculation and undermines the metrics. Using institutional theory, we argue that such limitations reflect symbolic compliance, while stewardship theory highlights the potential for deeper accountability. We propose leveraging audit outputs as red-teaming inputs to stress-test fairness robustness and strengthen AI governance through improved data quality and oversight.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.