Abstract
This article attempts to bridge the gap between widely discussed ethical principles of Human-centered AI (HCAI) and practical steps for effective governance. Since HCAI systems are developed and implemented in multiple organizational structures, I propose 15 recommendations at three levels of governance: team, organization, and industry. The recommendations are intended to increase the reliability, safety, and trustworthiness of HCAI systems: (1) reliable systems based on sound software engineering practices, (2) safety culture through business management strategies, and (3) trustworthy certification by independent oversight. Software engineering practices within teams include audit trails to enable analysis of failures, software engineering workflows, verification and validation testing, bias testing to enhance fairness, and explainable user interfaces. The safety culture within organizations comes from management strategies that include leadership commitment to safety, hiring and training oriented to safety, extensive reporting of failures and near misses, internal review boards for problems and future plans, and alignment with industry standard practices. The trustworthiness certification comes from industry-wide efforts that include government interventions and regulation, accounting firms conducting external audits, insurance companies compensating for failures, non-governmental and civil society organizations advancing design principles, and professional organizations and research institutes developing standards, policies, and novel ideas. The larger goal of effective governance is to limit the dangers and increase the benefits of HCAI to individuals, organizations, and society.
Topics

No keywords indexed for this article. Browse by subject →

References
127
[1]
A. Abdul , J. Vermeulen , D. Wang , B. Y. Lim , and M. Kankanhalli . 2018. Trends and trajectories for explainable, accountable and intelligible systems: An HCI research agenda . Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--18 . A. Abdul, J. Vermeulen, D. Wang, B. Y. Lim, and M. Kankanhalli. 2018. Trends and trajectories for explainable, accountable and intelligible systems: An HCI research agenda. Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--18.
[2]
S. Alsheibani , C. Messom , and Y. Cheung . 2019. Towards an artificial intelligence maturity model: From science fiction to business facts . Proceedings of the 23rd Pacific Asia Conference on Information Systems. Association for Information Systems. Retrieved from http://www.pacis2019.org/wd/Submissions/PACIS 2019 _paper_146.pdf. S. Alsheibani, C. Messom, and Y. Cheung. 2019. Towards an artificial intelligence maturity model: From science fiction to business facts. Proceedings of the 23rd Pacific Asia Conference on Information Systems. Association for Information Systems. Retrieved from http://www.pacis2019.org/wd/Submissions/PACIS2019_paper_146.pdf.
[3]
S. Amershi , A. Begel , C. Bird , R. DeLine , H. Gall , E. Kamar , N. Nagappan , B. Nushi , and T. Zimmermann . 2019a. Software engineering for machine learning: A case study . In Proceedings of the IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP’19) . IEEE, 291--300. S. Amershi, A. Begel, C. Bird, R. DeLine, H. Gall, E. Kamar, N. Nagappan, B. Nushi, and T. Zimmermann. 2019a. Software engineering for machine learning: A case study. In Proceedings of the IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP’19). IEEE, 291--300.
[4]
S. Amershi , D. Weld , M. Vorvoreanu , A. Fourney , B. Nushi , P. Collisson , and E. Horvitz . 2019b. Guidelines for human-AI interaction . In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13 . S. Amershi, D. Weld, M. Vorvoreanu, A. Fourney, B. Nushi, P. Collisson, and E. Horvitz. 2019b. Guidelines for human-AI interaction. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13.
[8]
J. C. Berry J. T. Davis T. Bartman C. C. Hafer L. M. Lieb N. Khan and R. J. Brilli. 2016. Improved safety culture and teamwork climate are associated with decreases in patient harm and hospital mortality across a hospital system. J. Patient Safety (Jan. 2016). Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/26741790. J. C. Berry J. T. Davis T. Bartman C. C. Hafer L. M. Lieb N. Khan and R. J. Brilli. 2016. Improved safety culture and teamwork climate are associated with decreases in patient harm and hospital mortality across a hospital system. J. Patient Safety (Jan. 2016). Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/26741790. 10.1097/pts.0000000000000251
[9]
O. Biran and C. Cotton . 2017. Explanation and justification in machine learning: A survey . In Proceedings of the International Joint Conference on Artificial Inteeligence Workshop on Explainable AI (XAI’17) . O. Biran and C. Cotton. 2017. Explanation and justification in machine learning: A survey. In Proceedings of the International Joint Conference on Artificial Inteeligence Workshop on Explainable AI (XAI’17).
[11]
E. Breck , N. Polyzotis , S. Roy , S. E. Whang , and M. Zinkevich . 2019. Data validation for machine learning . In Proceedings of the Conference on Systems and Machine Learning (SysML’19) . Retrieved from https://www.sysml.cc/doc/ 2019 /167.pdf. E. Breck, N. Polyzotis, S. Roy, S. E. Whang, and M. Zinkevich. 2019. Data validation for machine learning. In Proceedings of the Conference on Systems and Machine Learning (SysML’19). Retrieved from https://www.sysml.cc/doc/2019/167.pdf.
[13]
B. G. Buchanan and E. H. Shortliffe (Eds.). 1985. Rule-based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project. Addison-Wesley Publishing Company. B. G. Buchanan and E. H. Shortliffe (Eds.). 1985. Rule-based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project. Addison-Wesley Publishing Company.
[14]
J. Buolamwini and T. Gebru . 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification . Proc. Mach. Learn. Res. 81 , ( 2018 ), 77--91. J. Buolamwini and T. Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. Proc. Mach. Learn. Res. 81, (2018), 77--91.
[15]
Calo Ryan . 2016. Robots in American law . University of Washington School of Law Research Paper. Retrieved from https ://papers.ssrn.com/sol3/papers.cfm?abstract_id=2737598. Calo Ryan. 2016. Robots in American law. University of Washington School of Law Research Paper. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2737598.
[16]
N. Campbell . 2007 . The evolution of flight data analysis . Proceedings of Australian Society of Air Safety Investigators. Retrieved from https://asasi.org/papers/2007/The_Evolution_of_Flight_Data_Analysis_Neil_Campbell.pdf. N. Campbell. 2007. The evolution of flight data analysis. Proceedings of Australian Society of Air Safety Investigators. Retrieved from https://asasi.org/papers/2007/The_Evolution_of_Flight_Data_Analysis_Neil_Campbell.pdf.
[17]
Canadian Government. 2019. Responsible use of artificial intelligence (AI). Retrieved from https://www.canada.ca/en/government/system/digital-government/modern-emerging-technologies/responsible-use-ai.html. Canadian Government. 2019. Responsible use of artificial intelligence (AI). Retrieved from https://www.canada.ca/en/government/system/digital-government/modern-emerging-technologies/responsible-use-ai.html.
[21]
H. F. Cheng , R. Wang , Z. Zhang , F. O'Connell , T. Gray , F. M. Harper , and H. Zhu . 2019. Explaining decision-making algorithms through UI: Strategies to help non-expert stakeholders . In Proceedings of the CHI Conference on Human Factors in Computing Systems ACM, 1--12 . H. F. Cheng, R. Wang, Z. Zhang, F. O'Connell, T. Gray, F. M. Harper, and H. Zhu. 2019. Explaining decision-making algorithms through UI: Strategies to help non-expert stakeholders. In Proceedings of the CHI Conference on Human Factors in Computing Systems ACM, 1--12.
[23]
J. Couzin-Frankel . 2019. Medicine contends with how to use artificial intelligence. Science 354, 6446 ( 2019 ), 1119--1120. J. Couzin-Frankel. 2019. Medicine contends with how to use artificial intelligence. Science 354, 6446 (2019), 1119--1120.
[24]
P. R. Daugherty and H. J. Wilson . 2018 . Human + Machine: Reimagining Work in the Age of AI . Harvard Business Press . P. R. Daugherty and H. J. Wilson. 2018. Human+ Machine: Reimagining Work in the Age of AI. Harvard Business Press.
[26]
F. Doshi-Velez and B. Kim . 2017 . Towards a rigorous science of interpretable machine learning . Arxiv Preprint Arxiv : 1702 . 08608 . F. Doshi-Velez and B. Kim. 2017. Towards a rigorous science of interpretable machine learning. Arxiv Preprint Arxiv:1702.08608.
[28]
Techniques for interpretable machine learning

Mengnan Du, Ninghao Liu, Xia Hu

Communications of the ACM 10.1145/3359786
[32]
S. M. Erickson , J. Wolcott , J. M. Corrigan , and P. Aspden (Eds.). 2004 . Patient Safety: Achieving a New Standard for Care . National Academies Press , Washington, DC . S. M. Erickson, J. Wolcott, J. M. Corrigan, and P. Aspden (Eds.). 2004. Patient Safety: Achieving a New Standard for Care. National Academies Press, Washington, DC.
[33]
European Commission . 2020a. White paper on artificial intelligence—A European approach to excellence and trust , Brussels . Retrieved from https://ec.europa.eu/info/sites/info/files/commission-white-paper-artificial-intelligence-feb 2020 _en.pdf. European Commission. 2020a. White paper on artificial intelligence—A European approach to excellence and trust, Brussels. Retrieved from https://ec.europa.eu/info/sites/info/files/commission-white-paper-artificial-intelligence-feb2020_en.pdf.
[34]
European Commission . 2020b. The assessment list for trustworthy artificial intelligence (ALTAI) for self-assessment, independent high-level expert group on artificial intelligence , Brussels . Retrieved from https://ec.europa.eu/digital-single-market/en/news/assessment-list-trustworthy-artificial-intelligence-altai-self-assessment. European Commission. 2020b. The assessment list for trustworthy artificial intelligence (ALTAI) for self-assessment, independent high-level expert group on artificial intelligence, Brussels. Retrieved from https://ec.europa.eu/digital-single-market/en/news/assessment-list-trustworthy-artificial-intelligence-altai-self-assessment.
[36]
J. Fjeld N. Achten H. Hilligoss A. Nagy and M. Srikumar. 2020. Principled artificial intelligence: Mapping consensus in ethical and rights-based approaches to principles for AI. Berkman Klein Center Research Publication. Retrieved from https://cyber.harvard.edu/publication/2020/principled-ai. J. Fjeld N. Achten H. Hilligoss A. Nagy and M. Srikumar. 2020. Principled artificial intelligence: Mapping consensus in ethical and rights-based approaches to principles for AI. Berkman Klein Center Research Publication. Retrieved from https://cyber.harvard.edu/publication/2020/principled-ai. 10.2139/ssrn.3518482
[37]
P. Fraser , J. Moultrie , and M. Gregory . 2002. The use of maturity models/grids as a tool in assessing product development capability . In Proceedings of the IEEE International Engineering Management Conference. IEEE, 244--249 . P. Fraser, J. Moultrie, and M. Gregory. 2002. The use of maturity models/grids as a tool in assessing product development capability. In Proceedings of the IEEE International Engineering Management Conference. IEEE, 244--249.
[38]
S. A. Friedler , C. Scheidegger , S. Venkatasubramanian , S. Choudhary , E. P. Hamilton , and D. Roth . 2019. A comparative study of fairness-enhancing interventions in machine learning . In Proceedings of the Conference on Fairness, Accountability, and Transparency, ACM, 329--338 . https://doi.org/10.1145/3287560.3287589 10.1145/3287560.3287589 S. A. Friedler, C. Scheidegger, S. Venkatasubramanian, S. Choudhary, E. P. Hamilton, and D. Roth. 2019. A comparative study of fairness-enhancing interventions in machine learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency, ACM, 329--338. https://doi.org/10.1145/3287560.3287589
[39]
Bias in computer systems

Batya Friedman, Helen Nissenbaum

ACM Transactions on Information Systems 10.1145/230538.230561
[41]
European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”

Bryce Goodman, Seth Flaxman

AI Magazine 10.1609/aimag.v38i3.2741
[42]
D. R. Grossi . 1999 . Aviation Recorder Overview . In Proceedings of the International Symposium on Transportation Recorders. 153--164 . D. R. Grossi. 1999. Aviation Recorder Overview. In Proceedings of the International Symposium on Transportation Recorders. 153--164.
[49]
F. Hohman , A. Head , R. Caruana , R. DeLine , and S. M. Drucker . 2019. Gamut: A design probe to understand how data scientists understand machine learning models . In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13 . F. Hohman, A. Head, R. Caruana, R. DeLine, and S. M. Drucker. 2019. Gamut: A design probe to understand how data scientists understand machine learning models. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 1--13.

Showing 50 of 127 references

Metrics
592
Citations
127
References
Details
Published
Oct 16, 2020
Vol/Issue
10(4)
Pages
1-31
License
View
Cite This Article
Ben Shneiderman (2020). Bridging the Gap Between Ethics and Practice. ACM Transactions on Interactive Intelligent Systems, 10(4), 1-31. https://doi.org/10.1145/3419764
Related

You May Also Like

The MovieLens Datasets

F. Maxwell Harper, Joseph A. Konstan · 2015

2,567 citations

Modeling User Preferences in Recommender Systems

Gawesh Jawaheer, Peter Weller · 2014

129 citations

Co-design of Human-centered, Explainable AI for Clinical Decision Support

Cecilia Panigutti, Andrea Beretta · 2023

88 citations