A Survey on Bias and Fairness in Machine Learning

Ninareh Mehrabi; Fred Morstatter; Nripsuta Saxena; Kristina Lerman; Aram Galstyan

doi:10.1145/3457607

journal article Jul 13, 2021

A Survey on Bias and Fairness in Machine Learning

Ninareh Mehrabi Fred Morstatter Nripsuta Saxena Kristina Lerman Aram Galstyan

ACM Computing Surveys Vol. 54 No. 6 pp. 1-35 · Association for Computing Machinery (ACM)

View at Publisher Save 10.1145/3457607

Abstract

With the widespread use of artificial intelligence (AI) systems and applications in our everyday lives, accounting for fairness has gained significant importance in designing and engineering of such systems. AI systems can be used in many sensitive environments to make important and life-changing decisions; thus, it is crucial to ensure that these decisions do not reflect discriminatory behavior toward certain groups or populations. More recently some work has been developed in traditional machine learning and deep learning that address such challenges in different subdomains. With the commercialization of these systems, researchers are becoming more aware of the biases that these applications can contain and are attempting to address them. In this survey, we investigated different real-world applications that have shown biases in various ways, and we listed different sources of biases that can affect AI applications. We then created a taxonomy for fairness definitions that machine learning researchers have defined to avoid the existing bias in AI systems. In addition to that, we examined different domains and subdomains in AI showing what researchers have observed with regard to unfair outcomes in the state-of-the-art methods and ways they have tried to address them. There are still many future directions and solutions that can be taken to mitigate the problem of bias in AI systems. We are hoping that this survey will motivate researchers to tackle these issues in the near future by observing existing work in their respective fields.

Topics

No keywords indexed for this article. Browse by subject →

References

166

[1]

Agarwal Alekh (2019)

[2]

10.1609/aaai.v33i01.33011418

[3]

Alipourfard Nazanin (2018)

[4]

Alipourfard Nazanin (2018)

[5]

10.1145/3306618.3314243

[6]

Julia Angwin Jeff Larson Surya Mattu and Lauren Kirchner. 2019. Machine bias: There’s software used across the country to predict future criminals. and it’s biased against blacks. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing. Julia Angwin Jeff Larson Surya Mattu and Lauren Kirchner. 2019. Machine bias: There’s software used across the country to predict future criminals. and it’s biased against blacks. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.

[7]

A. Asuncion and D. J. Newman. 2007. UCI Machine Learning Repository. Retrieved from http://www.ics.uci.edu/$∖sim$mlearn/{MLR}epository.html. A. Asuncion and D. J. Newman. 2007. UCI Machine Learning Repository. Retrieved from http://www.ics.uci.edu/$∖sim$mlearn/{MLR}epository.html.

[8]

Backurs Arturs (2019)

[9]

10.1145/3209581

[10]

10.1145/2872427.2883083

[11]

Bellamy Rachel K. E. (2018)

[12]

Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science

Emily M. Bender, Batya Friedman

Transactions of the Association for Computational... 10.1162/tacl_a_00041

[13]

Misha Benjamin Paul Gagnon Negar Rostamzadeh Chris Pal Yoshua Bengio and Alex Shee. 2019. Towards standardization of data licenses: The Montreal data license. Misha Benjamin Paul Gagnon Negar Rostamzadeh Chris Pal Yoshua Bengio and Alex Shee. 2019. Towards standardization of data licenses: The Montreal data license.

[14]

Berk Richard (2017)

[15]

10.1177/0049124118782533

[16]

Bickel Peter J. (1975)

[17]

R. D. P. Binns. 2018. Fairness in machine learning: Lessons from political philosophy. J. Mach. Learn. Res. (2018). R. D. P. Binns. 2018. Fairness in machine learning: Lessons from political philosophy. J. Mach. Learn. Res. (2018).

[18]

10.1080/01621459.1972.10482387

[19]

Bogen Miranda (2018)

[20]

Bolukbasi Tolga

[21]

10.18653/v1/n19-3002

[22]

Bose Avishek (2019)

[23]

Brunet Marc-Etienne (2019)

[24]

Buolamwini Joy (2018)

[25]

10.1007/s10618-010-0190-x

[26]

Caliskan Aylin (2017)

[27]

Calmon Flavio

[28]

Capdevila Manel (2005)

[29]

Chaney Allison J. B.

[30]

10.1145/3287560.3287594

[31]

Chen Xingyu (2019)

[32]

10.1609/aaai.v33i01.33017801

[33]

S. Chiappa and W. S. Isaac. 2019. A causal Bayesian networks viewpoint on fairness. In Privacy and Identity Management. Fairness Accountability and Transparency in the Age of Big Data. Privacy and Identity 2018. IFIP Advances in Information and Communication Technology E. Kosta J. Pierson D. Slamanig S. Fischer-Hübner S. Krenn (Eds. ) Vol. 547. Springer Cham. S. Chiappa and W. S. Isaac. 2019. A causal Bayesian networks viewpoint on fairness. In Privacy and Identity Management. Fairness Accountability and Transparency in the Age of Big Data. Privacy and Identity 2018. IFIP Advances in Information and Communication Technology E. Kosta J. Pierson D. Slamanig S. Fischer-Hübner S. Krenn (Eds. ) Vol. 547. Springer Cham.

[34]

Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments

Alexandra Chouldechova

Big Data 10.1089/big.2016.0047

[35]

Chouldechova Alexandra (2018)

[36]

Chouldechova Alexandra (2018)

[37]

Chuang John S. (2009)

[38]

10.1080/07388940500339183

[39]

Cohen Lee (2019)

[40]

United States. (1987)

[41]

10.1145/3097983.3098095

[42]

Creager Elliot (2019)

[43]

10.1089/big.2016.0048

[44]

10.24963/ijcai.2017/654

[45]

10.1073/pnas.1018033108

[46]

Dressel Julia (2018)

[47]

Dheeru Dua and Casey Graff. 2017. UCI Machine Learning Repository. Retrieved from http://archive.ics.uci.edu/ml. Dheeru Dua and Casey Graff. 2017. UCI Machine Learning Repository. Retrieved from http://archive.ics.uci.edu/ml.

[48]

10.1145/2090236.2090255

[49]

Dwork Cynthia (2018)

[50]

10.1145/3278721.3278733

Showing 50 of 166 references

Cited By

3,466

An agentic AI marketplace for prelitigation analyses with ZKP-integrated ethical verifications

Chibuzor Udokwu, Vimal Dwivedi · 2026

Frontiers in Blockchain

From adoption to augmentation: a TCCM analysis of generative AI in higher education

Priya Chugh, Vishu Jain · 2026

Quality & Quantity

A differentiable and uncertainty-aware mutual information regularizer for bias mitigation

Alessandro Incremona, Andrea Pozzi · 2026

Neurocomputing

Artificial intelligence in clinical trials: A comprehensive review of opportunities, challenges, and future directions

David B. Olawade, Sandra Chinaza Fidelis · 2026

International Journal of Medical In...

ARTIFICIAL INTELLIGENCE IN EARLY ONCOLOGY SCREENING: A SYSTEMATIC REVIEW OF ETHICAL IMPLICATIONS, HEALTH EQUITY, AND DIGITAL ACCESS DISPARITIES

Karolina Magda Leszczyńska, Anna Krzysztofik · 2026

International Journal of Innovative...

Investigating the effect of bias mitigation in machine learning algorithms

Clodagh Maclean-Milner, Madhushi Bandara’s · 2026

Procedia Computer Science

Unfair Inequality in Education: A Benchmark for AI-Fairness Research

Joseph Giovanelli, Matteo Magnini · 2026

Scientific Data

Virtual medicine: medical AI in human health and diseases

Chen Zhang, Fu-Xiao Wang · 2026

Military Medical Research

Still unsafe: what’s holding us back on online safety for women

Ángel Pavón Pérez, Tracie Farrell · 2026

AI and Ethics

Bias-Free? An Empirical Study on Ethnicity, Gender, and Age Fairness in Deepfake Detection

Aditi Panda, Tanusree Ghosh · 2026

ACM Computing Surveys

Maximal Transparency for Online Recommender Systems

Katherine Dormandy, Luis M. Rodriguez-R · 2026

Philosophy & Technology

Multimodal Agent AI: A Survey of Recent Advances and Future Directions

Yu-Zhu Sun, He-Li Sun · 2025

Journal of Computer Science and Tec...

Fair Influence Maximization in Social Networks: A Community-Based Evolutionary Algorithm

Kaicong Ma, Xinxiang Xu · 2025

IEEE Transactions on Emerging Topic...

On the Data Quality and Imbalance in Machine Learning-based Design and Manufacturing—A Systematic Review

Jiarui Xie, Lijun Sun · 2025

Engineering

Retrieval-Augmented Generation (RAG)

Michael Klesel, H. Felix Wittmann · 2025

Business & Information Systems...

Algorithmic discrimination: a grounded conceptualization

Matthew Bui, Charlton McIlwain · 2025

Information, Communication & So...

Understanding and Mitigating Gender Bias in Information Retrieval Systems

Shirin Seyedsalehi, Amin Bigdeli · 2025

Foundations and Trends® in Informat...

Fair Clustering Ensemble With Equal Cluster Capacity

Peng Zhou, Rongwen Li · 2025

IEEE Transactions on Pattern Analys...

Biases in AI: acknowledging and addressing the inevitable ethical issues

Bjørn Hofmann · 2025

Frontiers in Digital Health

Bias recognition and mitigation strategies in artificial intelligence healthcare applications

Fereshteh Hasanzadeh, Colin B. Josephson · 2025

npj Digital Medicine

Metrics

3,466

Citations

166

References

Details

Published: Jul 13, 2021
Vol/Issue: 54(6)
Pages: 1-35
License: View

Authors

USC-ISI

USC-ISI

USC-ISI

USC-ISI

USC-ISI

Funding

Defense Advanced Research Projects Agency Award: Agreement No. HR0011890019.

Cite This Article

Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, et al. (2021). A Survey on Bias and Fairness in Machine Learning. ACM Computing Surveys, 54(6), 1-35. https://doi.org/10.1145/3457607

A Survey on Bias and Fairness in Machine Learning

You May Also Like