Measurement of Trust in Automation: A Narrative Review and Reference Guide

Spencer C. Kohn; Ewart J. de Visser; Eva Wiese; Yi-Ching Lee; Tyler H. Shaw

doi:10.3389/fpsyg.2021.604977

journal article Open Access Oct 19, 2021

Measurement of Trust in Automation: A Narrative Review and Reference Guide

Spencer C. Kohn Ewart J. de Visser Eva Wiese Yi-Ching Lee

Tyler H. Shaw

Frontiers in Psychology Vol. 12 · Frontiers Media SA

View at Publisher Save 10.3389/fpsyg.2021.604977

Abstract

With the rise of automated and autonomous agents, research examining Trust in Automation (TiA) has attracted considerable attention over the last few decades. Trust is a rich and complex construct which has sparked a multitude of measures and approaches to study and understand it. This comprehensive narrative review addresses known methods that have been used to capture TiA. We examined measurements deployed in existing empirical works, categorized those measures into self-report, behavioral, and physiological indices, and examined them within the context of an existing model of trust. The resulting work provides a reference guide for researchers, providing a list of available TiA measurement methods along with the model-derived constructs that they capture including judgments of trustworthiness, trust attitudes, and trusting behaviors. The article concludes with recommendations on how to improve the current state of TiA measurement.

Topics

No keywords indexed for this article. Browse by subject →

References

169

[1]

Aasman "Operator effort and the measurement of heart-rate variability" Hum. Factors (1987) 10.1177/001872088702900204

[2]

Akash "A classification model for sensing human trust in machines using EEG and GSR" ACM Trans. Interact. Intell. Syst. (TiiS) (2018) 10.1145/3132743

[3]

Alarcon "Exploring the differential effects of trust violations in human-human and human-robot interactions" Appl. Ergon. (2021) 10.1016/j.apergo.2020.103350

[4]

Alarcon "Trustworthiness perceptions of computer code: A heuristic-systematic processing model" (2018) 10.24251/hicss.2018.671

[5]

Bahner "Misuse of automated decision aids: Complacency, automation bias and the impact of training experience" Int. J. Hum. Comput. Stud. (2008) 10.1016/j.ijhcs.2008.06.001

[6]

Beller "Improving the driver–automation interaction: An approach using automation uncertainty" Hum. Factors (2013) 10.1177/0018720813482327

[7]

Bellucci "Neural signatures of trust in reciprocity: A coordinate-based meta-analysis" Hum. Brain Mapp. (2017) 10.1002/hbm.23451

[8]

Berberian "The out-of-the-loop brain: a neuroergonomic approach of the human automation interaction" Annu. Rev. Control. (2017) 10.1016/j.arcontrol.2017.09.010

[9]

Trust, Reciprocity, and Social History

Joyce Berg, John Dickhaut, Kevin McCabe

Games and Economic Behavior 1995 10.1006/game.1995.1027

[10]

The Predictive Validity of Multiple-Item versus Single-Item Measures of the Same Constructs

Lars Bergkvist, John R. Rossiter

Journal of Marketing Research 2007 10.1509/jmkr.44.2.175

[11]

Bergkvist "Tailor-made single-item measures of doubly concrete constructs" Int. J. Advert. (2009) 10.2501/s0265048709200783

[12]

Bethel "Survey of psychophysiology measurements applied to human-robot interaction" RO-MAN 2007-The 16th IEEE International Symposium on Robot and Human Interactive Communication. (2007) 10.1109/roman.2007.4415182

[13]

Biros "The influence of task load and automation trust on deception detection" Group Decis. Negot. (2004) 10.1023/b:grup.0000021840.85686.57

[14]

Chancey "The role of trust as a mediator between system characteristics and response behaviors" Hum. Factors (2015) 10.1177/0018720815582261

[15]

Chancey "Trust and the compliance–reliance paradigm: The effects of risk, error bias, and reliability on trust and dependence" Hum. Factors (2017) 10.1177/0018720816682648

[16]

Human–Agent Teaming for Multirobot Control: A Review of Human Factors Issues

Jessie Y. C. Chen, Michael J. Barnes

IEEE Transactions on Human-Machine Systems 2014 10.1109/thms.2013.2293535

[17]

Chien "Towards the development of an inter-cultural scale to measure trust in automation" (2014) 10.1007/978-3-319-07308-8_4

[18]

Trusting Automation: Designing for Responsivity and Resilience

Erin K. Chiou, John D. Lee

Human Factors: The Journal of the Human Factors an... 2021 10.1177/00187208211009995

[19]

Chita-Tegmark "Can you trust your trust measure" (2021) 10.1145/3434073.3444677

[20]

Christensen "Lack of association between human plasma oxytocin and interpersonal trust in a prisoner’s dilemma paradigm" PLoS One (2014) 10.1371/journal.pone.0116172

[21]

Colquitt "Trust, trustworthiness, and trust propensity: A meta-analytic test of their unique relationships with risk taking and job performance" J. Appl. Psychol. (2007) 10.1037/0021-9010.92.4.909

[22]

Das "The risk-based view of trust: A conceptual framework" J. Bus. Psychol. (2004) 10.1023/b:jobu.0000040274.23551.1b

[23]

de Melo "People’s biased decisions to trust and cooperate with agents that express emotions" (2013)

[24]

de Visser "Learning from the slips of others: neural correlates of trust in automated agents" Front. Hum. Neurosci. (2018) 10.3389/fnhum.2018.00309

[25]

de Visser "Interpersonal trust as a dynamic belief?" (2012)

[26]

de Visser "A little anthropomorphism goes a long way: effects of oxytocin on trust, compliance, and team performance with automated agents" Hum. Factors (2017) 10.1177/0018720816687205

[27]

de Visser "Almost human: anthropomorphism increases trust resilience in cognitive agents" J. Exp. Psychol. Appl. (2016) 10.1037/xap0000092

[28]

de Visser "A Neuroergonomic perspective on human-automation etiquette and trust" (2010) 10.1201/ebk1439835012-c21

[29]

de Visser (2011)

[30]

de Visser "Towards a theory of longitudinal trust calibration in human–robot teams" Int. J. Soc. Robot. (2020) 10.1007/s12369-019-00596-x

[31]

Delgado "Perceptions of moral character modulate the neural systems of reward during the trust game" Nat. Neurosci. (2005) 10.1038/nn1575

[32]

Desai "Modeling trust to improve human-robot interaction. doctoral dissertation" (2012)

[33]

Desai "Impact of robot failures and feedback on real-time trust" (2013) 10.1109/hri.2013.6483596

[34]

Desmet "How social is error observation? The neural mechanisms underlying the observation of human and machine errors" Soc. Cogn. Affect. Neurosci. (2014) 10.1093/scan/nst002

[35]

DiSalvo "All robots are not created equal: The design and perception of humanoid robot heads" (2002) 10.1145/778712.778756

[36]

Dong "A preliminary study on human trust measurements by EEG for human-machine interactions" (2015) 10.1145/2814940.2814993

[37]

Drnec "From trust in automation to decision neuroscience: applying cognitive neuroscience methods to understand and improve interaction decisions involved in human automation interaction" Front. Hum. Neurosci. (2016) 10.3389/fnhum.2016.00290

[38]

Dzindolet "The role of trust in automation reliance" Int. J. Hum. Comput. Stud. (2003) 10.1016/s1071-5819(03)00038-7

[39]

Dzindolet "The perceived utility of human and automated aids in a visual detection task" Hum. Factors (2002) 10.1518/0018720024494856

[40]

Ezer "Age-related differences in reliance behavior attributable to costs within a human-decision aid system" Hum. Factors (2008) 10.1518/001872008x375018

[41]

Fedota "Neuroergonomics and human error" Theor. Issues Ergon. Sci. (2010) 10.1080/14639220902853104

[42]

Fine "Secrecy, trust, and dangerous leisure: Generating group cohesion in voluntary organizations" Soc. Psychol. Q. (1996) 10.2307/2787117

[43]

Geels-Blair "Using system-wide trust theory to reveal the contagion effects of automation false alarms and misses on compliance and reliance in a simulated aviation task" Int. J. Aviat. Psychol. (2013) 10.1080/10508414.2013.799355

[44]

Gold "Trust in automation–before and after the experience of take-over scenarios in a highly automated vehicle" Procedia Manuf. (2015) 10.1016/j.promfg.2015.07.847

[45]

Goodyear "An fMRI and effective connectivity study investigating miss errors during advice utilization from human and machine agents" Soc. Neurosci. (2017) 10.1080/17470919.2016.1205131

[46]

Goodyear "Advice taking from humans and machines: An fMRI and effective connectivity study" Front. Hum. Neurosci. (2016) 10.3389/fnhum.2016.00542

[47]

Gramann "Trends in neuroergonomics" Front. Hum. Neurosci. (2017) 10.3389/fnhum.2017.00165

[48]

Güth "An experimental analysis of ultimatum bargaining" J. Econ. Behav. Organ. (1982) 10.1016/0167-2681(82)90011-7

[49]

Gutzwiller "Positive bias in the ‘Trust in Automated Systems Survey’? An examination of the Jian et al. (2000) scale" Proc. Hum. Factors Ergon. Soc. Annu. Meet. (2019) 10.1177/1071181319631201

[50]

Hancock "A meta-analysis of factors affecting trust in human-robot interaction" Hum. Factors (2011) 10.1177/0018720811417254

Showing 50 of 169 references

Cited By

239

Exploring the relationship between operator experience, propensity to trust automation and perceived system trustworthiness of uncrewed Air Vehicles

Ben Grindley, Tom Cherrett · 2026

Ergonomics

Research Automation in Healthcare: Scoping Review with ☸️SAIMSARA

· 2026

SAIMSARA Journal

How do we assess the trustworthiness of AI? Introducing the trustworthiness assessment model (TrAM)

Nadine Schlicker, Kevin Baum · 2025

Computers in Human Behavior

Trust and reliance on AI — An experimental study on the extent and costs of overreliance on AI

Artur Klingbeil, Cassandra Grützner · 2024

Computers in Human Behavior

Metrics

239

Citations

169

References

Details

Published: Oct 19, 2021
Vol/Issue: 12
License: View

Authors

Department of Psychology, George Mason University, 4400 University Drive, MS 3F5, Fairfax, VA 22030, USA

T

Tyler H. Shaw

Cite This Article

Spencer C. Kohn, Ewart J. de Visser, Eva Wiese, et al. (2021). Measurement of Trust in Automation: A Narrative Review and Reference Guide. Frontiers in Psychology, 12. https://doi.org/10.3389/fpsyg.2021.604977

Measurement of Trust in Automation: A Narrative Review and Reference Guide

You May Also Like