Investigating AI Teammate Communication Strategies and Their Impact in Human-AI Teams for Effective Teamwork

Rui Zhang; Wen Duan; Christopher Flathmann; Nathan McNeese; Guo Freeman; Alyssa Williams

doi:10.1145/3610072

journal article Sep 28, 2023

Investigating AI Teammate Communication Strategies and Their Impact in Human-AI Teams for Effective Teamwork

Rui Zhang

Wen Duan

Christopher Flathmann

Nathan McNeese

Guo Freeman

Alyssa Williams

Proceedings of the ACM on Human-Computer Interaction Vol. 7 No. CSCW2 pp. 1-31 · Association for Computing Machinery (ACM)

View at Publisher Save 10.1145/3610072

Abstract

Recently, AI is integrating into teams to collaborate with humans as a teammate with the goal of achieving unprecedented team outcomes. Much of the coordination between humans and AI teammates relies on human-AI communication, which is challenging due to AI's limitations on natural language communication. Thus, it is essential to identify and develop effective communication strategies for AI teammates in human-AI teams to facilitate the coordination process. Through interviews with 60 participants who collaborated with an AI teammate in a multiplayer online game, in this paper, we explore communication strategies that humans expect AI teammates to apply to support human-AI coordination and collaboration in dyadic teaming environments, and how the AI teammate's communication can impact teaming processes. Our findings highlight four communication strategies AI teammates should apply to support their coordination with humans in dyadic teaming environments. We also find that AI teammates' proactive communication with humans could facilitate the development of human trust and situation awareness, whereas AI lacking such proactive communication is often not perceived as a teammate. Our study extends the current CSCW/HCI research on human-AI communication in teaming environments by shedding light on how communication should be structured in dyadic human-AI teams for effective and smooth collaboration.

Topics

No keywords indexed for this article. Browse by subject →

References

126

[1]

10.1108/00220411211277064

[2]

10.1145/3290605.3300233

[3]

10.1145/3411764.3445256

[4]

10.1007/s42454-021-00026-2

[5]

10.1609/hcomp.v7i1.5285

[6]

10.1145/3411764.3445717

[7]

10.1145/3359206

[8]

10.1145/3514257

[9]

10.1177/1071181319631278

[10]

Kathy Charmaz. 2014. Constructing grounded theory. sage.

[11]

10.1145/3243503

[12]

10.1007/s12369-021-00834-1

[13]

10.1145/1518701.1519059

[14]

Nancy J Cooke Mustafa Demir and Nathan McNeese. 2016. Synthetic teammates as team players: Coordination of human and synthetic teammates. Technical Report. Cognitive Engineering Research Institute Mesa United States.

[15]

Nancy J Cooke, Jamie C Gorman, Christopher W Myers, and Jasmine L Duran. 2013. Interactive team cognition. Cognitive science, Vol. 37, 2 (2013), 255--285.

[16]

10.1037/1089-2699.7.3.179

[17]

10.1002/job.2213

[18]

Nils Dahlb"ack, Arne Jönsson, and Lars Ahrenberg. 1993. Wizard of Oz studies-why and how. Knowledge-based systems, Vol. 6, 4 (1993), 258--266.

[19]

10.1080/00140139.2018.1457725

[20]

10.1109/thms.2018.2877482

[21]

Mustafa Demir, Nathan J McNeese, and Nancy J Cooke. 2016. Team communication behaviors of the human-automation teaming. In 2016 IEEE international multi-disciplinary conference on cognitive methods in situation awareness and decision support (CogSIMA). IEEE, 28--34.

[22]

10.1016/j.cogsys.2016.11.003

[23]

10.1016/j.ijhcs.2020.102436

[24]

Wen Duan, Nathan J McNeese, and Rui Zhang. 2023. Communication in Human-AI Teaming. In Group Communication: An Advanced Introduction, Torsten Reimer, Ernest Park, and Joseph Bonito (Eds.). Routledge/Taylor and Francis.

[25]

10.1145/3359273

[26]

10.1145/3479579

[27]

Mica R Endsley. 1995. Measurement of situation awareness in dynamic systems. Human factors, Vol. 37, 1 (1995), 65--84.

[28]

10.1177/154193120004400120

[29]

10.1145/3491102.3517734

[30]

10.1609/hcomp.v8i1.7462

[31]

10.1177/0018720810369807

[32]

10.1080/10447318.2018.1456150

[33]

10.1145/3024969.3025079

[34]

10.1145/289444.289502

[35]

10.1145/358916.358947

[36]

10.1145/2702123.2702498

[37]

Cristina B Gibson and Jennifer A Manuel. 2003. Building trust. Virtual teams that work (2003), 59--86.

[38]

Luis R Gomez-Mejia David B Balkin Robert L Cardy and Kenneth P Carson. 2007. Managing human resources. Pearson/Prentice Hall Upper Saddle River NJ.

[39]

10.1177/0093650209351468

[40]

Camilla Göras, Karolina Olin, Maria Unbeck, Karin Pukk-H"arenstam, Anna Ehrenberg, Mesfin Kassaye Tessma, Ulrica Nilsson, and Mirjam Ekstedt. 2019. Tasks, multitasking and interruptions among the surgical team in an operating room: a prospective observational study. BMJ open, Vol. 9, 5 (2019), e026410.

[41]

10.1163/9789004368811_003

[42]

10.1177/1071181319631307

[43]

10.1007/978-3-031-02218-0

[44]

10.1177/1461444819858691

[45]

10.1002/pon.949

[46]

Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139--183.

[47]

Allyson I Hauptman, Wen Duan, and Nathan J Mcneese. 2022. The Components of Trust for Collaborating With AI Colleagues. In Companion Publication of the 2022 Conference on Computer Supported Cooperative Work and Social Computing. 72--75.

[48]

10.1146/annurev.psych.56.091103.070250

[49]

10.1145/3442188.3445923

[50]

10.1080/07421222.1998.11518185

Showing 50 of 126 references

Cited By

69

A Mixed Methods Approach to Analyzing the Role of AI Teammates in Transition Phases

Beau G. Schelble, Rohit Mallick · 2026

Proceedings of the ACM on Human-Com...

Dittos: Personalized, Embodied Agents That Participate in Meetings When You Are Unavailable

Joanne Leong, John Tang · 2024

Proceedings of the ACM on Human-Com...

Metrics

69

Citations

126

References

Details

Published: Sep 28, 2023
Vol/Issue: 7(CSCW2)
Pages: 1-31
License: View

Authors

R

Rui Zhang