Bridging the Gap Between Ethics and Practice

Ben Shneiderman

doi:10.1145/3419764

journal article Oct 16, 2020

Bridging the Gap Between Ethics and Practice

Ben Shneiderman

ACM Transactions on Interactive Intelligent Systems Vol. 10 No. 4 pp. 1-31 · Association for Computing Machinery (ACM)

View at Publisher Save 10.1145/3419764

Abstract

This article attempts to bridge the gap between widely discussed ethical principles of Human-centered AI (HCAI) and practical steps for effective governance. Since HCAI systems are developed and implemented in multiple organizational structures, I propose 15 recommendations at three levels of governance: team, organization, and industry. The recommendations are intended to increase the reliability, safety, and trustworthiness of HCAI systems: (1) reliable systems based on sound software engineering practices, (2) safety culture through business management strategies, and (3) trustworthy certification by independent oversight. Software engineering practices within teams include audit trails to enable analysis of failures, software engineering workflows, verification and validation testing, bias testing to enhance fairness, and explainable user interfaces. The safety culture within organizations comes from management strategies that include leadership commitment to safety, hiring and training oriented to safety, extensive reporting of failures and near misses, internal review boards for problems and future plans, and alignment with industry standard practices. The trustworthiness certification comes from industry-wide efforts that include government interventions and regulation, accounting firms conducting external audits, insurance companies compensating for failures, non-governmental and civil society organizations advancing design principles, and professional organizations and research institutes developing standards, policies, and novel ideas. The larger goal of effective governance is to limit the dangers and increase the benefits of HCAI to individuals, organizations, and society.

Topics

No keywords indexed for this article. Browse by subject →

References

127

[1]

A. Abdul , J. Vermeulen , D. Wang , B. Y. Lim , and M. Kankanhalli . 2018. Trends and trajectories for explainable, accountable and intelligible systems: An HCI research agenda . Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--18 . A. Abdul, J. Vermeulen, D. Wang, B. Y. Lim, and M. Kankanhalli. 2018. Trends and trajectories for explainable, accountable and intelligible systems: An HCI research agenda. Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--18.

[2]

S. Alsheibani , C. Messom , and Y. Cheung . 2019. Towards an artificial intelligence maturity model: From science fiction to business facts . Proceedings of the 23rd Pacific Asia Conference on Information Systems. Association for Information Systems. Retrieved from http://www.pacis2019.org/wd/Submissions/PACIS 2019 _paper_146.pdf. S. Alsheibani, C. Messom, and Y. Cheung. 2019. Towards an artificial intelligence maturity model: From science fiction to business facts. Proceedings of the 23rd Pacific Asia Conference on Information Systems. Association for Information Systems. Retrieved from http://www.pacis2019.org/wd/Submissions/PACIS2019_paper_146.pdf.

[3]

S. Amershi , A. Begel , C. Bird , R. DeLine , H. Gall , E. Kamar , N. Nagappan , B. Nushi , and T. Zimmermann . 2019a. Software engineering for machine learning: A case study . In Proceedings of the IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP’19) . IEEE, 291--300. S. Amershi, A. Begel, C. Bird, R. DeLine, H. Gall, E. Kamar, N. Nagappan, B. Nushi, and T. Zimmermann. 2019a. Software engineering for machine learning: A case study. In Proceedings of the IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP’19). IEEE, 291--300.

[4]

S. Amershi , D. Weld , M. Vorvoreanu , A. Fourney , B. Nushi , P. Collisson , and E. Horvitz . 2019b. Guidelines for human-AI interaction . In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13 . S. Amershi, D. Weld, M. Vorvoreanu, A. Fourney, B. Nushi, P. Collisson, and E. Horvitz. 2019b. Guidelines for human-AI interaction. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13.

[5]

10.1145/3209581

[6]

10.1111/j.1468-0009.2012.00679.x

[7]

10.1147/jrd.2019.2942287

[8]

J. C. Berry J. T. Davis T. Bartman C. C. Hafer L. M. Lieb N. Khan and R. J. Brilli. 2016. Improved safety culture and teamwork climate are associated with decreases in patient harm and hospital mortality across a hospital system. J. Patient Safety (Jan. 2016). Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/26741790. J. C. Berry J. T. Davis T. Bartman C. C. Hafer L. M. Lieb N. Khan and R. J. Brilli. 2016. Improved safety culture and teamwork climate are associated with decreases in patient harm and hospital mortality across a hospital system. J. Patient Safety (Jan. 2016). Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/26741790. 10.1097/pts.0000000000000251

[9]

O. Biran and C. Cotton . 2017. Explanation and justification in machine learning: A survey . In Proceedings of the International Joint Conference on Artificial Inteeligence Workshop on Explainable AI (XAI’17) . O. Biran and C. Cotton. 2017. Explanation and justification in machine learning: A survey. In Proceedings of the International Joint Conference on Artificial Inteeligence Workshop on Explainable AI (XAI’17).

[10]

10.6028/jres.121.015

[11]

E. Breck , N. Polyzotis , S. Roy , S. E. Whang , and M. Zinkevich . 2019. Data validation for machine learning . In Proceedings of the Conference on Systems and Machine Learning (SysML’19) . Retrieved from https://www.sysml.cc/doc/ 2019 /167.pdf. E. Breck, N. Polyzotis, S. Roy, S. E. Whang, and M. Zinkevich. 2019. Data validation for machine learning. In Proceedings of the Conference on Systems and Machine Learning (SysML’19). Retrieved from https://www.sysml.cc/doc/2019/167.pdf.

[12]

10.1109/mts.2018.2826059

[13]

B. G. Buchanan and E. H. Shortliffe (Eds.). 1985. Rule-based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project. Addison-Wesley Publishing Company. B. G. Buchanan and E. H. Shortliffe (Eds.). 1985. Rule-based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project. Addison-Wesley Publishing Company.

[14]

J. Buolamwini and T. Gebru . 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification . Proc. Mach. Learn. Res. 81 , ( 2018 ), 77--91. J. Buolamwini and T. Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. Proc. Mach. Learn. Res. 81, (2018), 77--91.

[15]

Calo Ryan . 2016. Robots in American law . University of Washington School of Law Research Paper. Retrieved from https ://papers.ssrn.com/sol3/papers.cfm?abstract_id=2737598. Calo Ryan. 2016. Robots in American law. University of Washington School of Law Research Paper. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2737598.

[16]

N. Campbell . 2007 . The evolution of flight data analysis . Proceedings of Australian Society of Air Safety Investigators. Retrieved from https://asasi.org/papers/2007/The_Evolution_of_Flight_Data_Analysis_Neil_Campbell.pdf. N. Campbell. 2007. The evolution of flight data analysis. Proceedings of Australian Society of Air Safety Investigators. Retrieved from https://asasi.org/papers/2007/The_Evolution_of_Flight_Data_Analysis_Neil_Campbell.pdf.

[17]

Canadian Government. 2019. Responsible use of artificial intelligence (AI). Retrieved from https://www.canada.ca/en/government/system/digital-government/modern-emerging-technologies/responsible-use-ai.html. Canadian Government. 2019. Responsible use of artificial intelligence (AI). Retrieved from https://www.canada.ca/en/government/system/digital-government/modern-emerging-technologies/responsible-use-ai.html.

[18]

10.1016/j.ijinfomgt.2018.07.001

[19]

10.1136/bmjqs-2018-008370

[20]

10.1145/3282878

[21]

H. F. Cheng , R. Wang , Z. Zhang , F. O'Connell , T. Gray , F. M. Harper , and H. Zhu . 2019. Explaining decision-making algorithms through UI: Strategies to help non-expert stakeholders . In Proceedings of the CHI Conference on Human Factors in Computing Systems ACM, 1--12 . H. F. Cheng, R. Wang, Z. Zhang, F. O'Connell, T. Gray, F. M. Harper, and H. Zhu. 2019. Explaining decision-making algorithms through UI: Strategies to help non-expert stakeholders. In Proceedings of the CHI Conference on Human Factors in Computing Systems ACM, 1--12.

[22]

10.5555/16244.16245

[23]

J. Couzin-Frankel . 2019. Medicine contends with how to use artificial intelligence. Science 354, 6446 ( 2019 ), 1119--1120. J. Couzin-Frankel. 2019. Medicine contends with how to use artificial intelligence. Science 354, 6446 (2019), 1119--1120.

[24]

P. R. Daugherty and H. J. Wilson . 2018 . Human + Machine: Reimagining Work in the Age of AI . Harvard Business Press . P. R. Daugherty and H. J. Wilson. 2018. Human+ Machine: Reimagining Work in the Age of AI. Harvard Business Press.

[25]

10.1007/s11704-018-8900-4

[26]

F. Doshi-Velez and B. Kim . 2017 . Towards a rigorous science of interpretable machine learning . Arxiv Preprint Arxiv : 1702 . 08608 . F. Doshi-Velez and B. Kim. 2017. Towards a rigorous science of interpretable machine learning. Arxiv Preprint Arxiv:1702.08608.

[27]

10.1145/3301402

[28]

Techniques for interpretable machine learning

Mengnan Du, Ninghao Liu, Xia Hu

Communications of the ACM 10.1145/3359786

[29]

10.1145/3185517

[30]

10.1109/ms.2019.2921037

[31]

10.1109/issre.2000.885887

[32]

S. M. Erickson , J. Wolcott , J. M. Corrigan , and P. Aspden (Eds.). 2004 . Patient Safety: Achieving a New Standard for Care . National Academies Press , Washington, DC . S. M. Erickson, J. Wolcott, J. M. Corrigan, and P. Aspden (Eds.). 2004. Patient Safety: Achieving a New Standard for Care. National Academies Press, Washington, DC.

[33]

European Commission . 2020a. White paper on artificial intelligence—A European approach to excellence and trust , Brussels . Retrieved from https://ec.europa.eu/info/sites/info/files/commission-white-paper-artificial-intelligence-feb 2020 _en.pdf. European Commission. 2020a. White paper on artificial intelligence—A European approach to excellence and trust, Brussels. Retrieved from https://ec.europa.eu/info/sites/info/files/commission-white-paper-artificial-intelligence-feb2020_en.pdf.

[34]

European Commission . 2020b. The assessment list for trustworthy artificial intelligence (ALTAI) for self-assessment, independent high-level expert group on artificial intelligence , Brussels . Retrieved from https://ec.europa.eu/digital-single-market/en/news/assessment-list-trustworthy-artificial-intelligence-altai-self-assessment. European Commission. 2020b. The assessment list for trustworthy artificial intelligence (ALTAI) for self-assessment, independent high-level expert group on artificial intelligence, Brussels. Retrieved from https://ec.europa.eu/digital-single-market/en/news/assessment-list-trustworthy-artificial-intelligence-altai-self-assessment.

[35]

10.1126/science.aaz4795

[36]

J. Fjeld N. Achten H. Hilligoss A. Nagy and M. Srikumar. 2020. Principled artificial intelligence: Mapping consensus in ethical and rights-based approaches to principles for AI. Berkman Klein Center Research Publication. Retrieved from https://cyber.harvard.edu/publication/2020/principled-ai. J. Fjeld N. Achten H. Hilligoss A. Nagy and M. Srikumar. 2020. Principled artificial intelligence: Mapping consensus in ethical and rights-based approaches to principles for AI. Berkman Klein Center Research Publication. Retrieved from https://cyber.harvard.edu/publication/2020/principled-ai. 10.2139/ssrn.3518482

[37]

P. Fraser , J. Moultrie , and M. Gregory . 2002. The use of maturity models/grids as a tool in assessing product development capability . In Proceedings of the IEEE International Engineering Management Conference. IEEE, 244--249 . P. Fraser, J. Moultrie, and M. Gregory. 2002. The use of maturity models/grids as a tool in assessing product development capability. In Proceedings of the IEEE International Engineering Management Conference. IEEE, 244--249.

[38]

S. A. Friedler , C. Scheidegger , S. Venkatasubramanian , S. Choudhary , E. P. Hamilton , and D. Roth . 2019. A comparative study of fairness-enhancing interventions in machine learning . In Proceedings of the Conference on Fairness, Accountability, and Transparency, ACM, 329--338 . https://doi.org/10.1145/3287560.3287589 10.1145/3287560.3287589 S. A. Friedler, C. Scheidegger, S. Venkatasubramanian, S. Choudhary, E. P. Hamilton, and D. Roth. 2019. A comparative study of fairness-enhancing interventions in machine learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency, ACM, 329--338. https://doi.org/10.1145/3287560.3287589

[39]

Bias in computer systems

Batya Friedman, Helen Nissenbaum

ACM Transactions on Information Systems 10.1145/230538.230561

[40]

10.1145/3125780

[41]

European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”

Bryce Goodman, Seth Flaxman

AI Magazine 10.1609/aimag.v38i3.2741

[42]

D. R. Grossi . 1999 . Aviation Recorder Overview . In Proceedings of the International Symposium on Transportation Recorders. 153--164 . D. R. Grossi. 1999. Aviation Recorder Overview. In Proceedings of the International Symposium on Transportation Recorders. 153--164.

[43]

10.1016/s0925-7535(00)00014-x

[44]

10.1016/j.ssci.2016.08.010

[45]

10.1073/pnas.1807184115

[46]

10.1007/s00778-017-0486-1

[47]

10.1109/mis.2017.54

[48]

10.1109/mis.2017.3121544

[49]

F. Hohman , A. Head , R. Caruana , R. DeLine , and S. M. Drucker . 2019. Gamut: A design probe to understand how data scientists understand machine learning models . In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13 . F. Hohman, A. Head, R. Caruana, R. DeLine, and S. M. Drucker. 2019. Gamut: A design probe to understand how data scientists understand machine learning models. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13.

[50]

10.1109/tvcg.2019.2934659

Showing 50 of 127 references

Cited By

592

Impact of Interactive Interface Explainability on User Acceptance of AI-Generated Logos

Ja Kyoung Min · 2026

Journal of Digital Contents Society

The Organization–AI–User Responsibility Triangle: Public Understandings of AI and Expectations for Organizational Responses in AI-Service-Failure Crises

Ke Huang, Fang Wu · 2025

Emerging Media

Enhancing Security and Privacy in Occluded Face Recognition: A Human‐Centered GAN‐Based Approach for Masked Identities in High‐Security Environments

Fatima Aslam, Adil Afzal · 2025

IET Image Processing

The Road to Intelligent Cities

João Carlos N. Bittencourt, Thiago C. Jesus · 2025

Smart Cities

Responsible AI Pattern Catalogue: A Collection of Best Practices for AI Governance and Engineering

Qinghua Lu, Liming Zhu · 2024

ACM Computing Surveys

Understanding human–technology interaction: evolving boundaries

Pierpaolo Magliocca, Rossella Canestrino · 2024

European Journal of Innovation Mana...

The role of artificial intelligence and digital technologies in dam engineering: Narrative review and outlook

M. Amin Hariri-Ardebili, Golsa Mahdavi · 2023

Engineering Applications of Artific...

A Review of Trustworthy and Explainable Artificial Intelligence (XAI)

Vinay Chamola, Vikas Hassija · 2023

IEEE Access

Who needs explanation and when? Juggling explainable AI and user epistemic uncertainty

Jinglu Jiang, Surinder Kahai · 2022

International Journal of Human-Comp...

Metrics

592

Citations

127

References

Details

Published: Oct 16, 2020
Vol/Issue: 10(4)
Pages: 1-31
License: View

Authors

B

Ben Shneiderman

University of Maryland, College Park, MD

Cite This Article

Ben Shneiderman (2020). Bridging the Gap Between Ethics and Practice. ACM Transactions on Interactive Intelligent Systems, 10(4), 1-31. https://doi.org/10.1145/3419764

Bridging the Gap Between Ethics and Practice

You May Also Like