journal article
Open Access
Aug 06, 2025
Are Robots More Engaging When They Respond to Joint Attention? Findings from a Turn-Taking Game with a Social Robot
Abstract
Joint attention, the capacity of two or more individuals to focus on a common event simultaneously, is fundamental to human–human interaction, enabling effective communication. When considering the field of social robotics, emulating this capability might be necessary for promoting natural interactions and thus improving user engagement. Responding to joint attention (RJA), defined as the ability to react to external attentional cues by aligning focus with another individual, plays a critical role in promoting mutual understanding. This study examines how RJA impacts user engagement during human–robot interaction. The participants play a turn-taking game against a social robot under two conditions: with our RJA system active and with the system inactive. Auditory and visual stimuli are introduced to simulate real-world dynamics, testing the robot’s ability to detect and follow the user’s focus of attention. We use a twofold approach to evaluate the system’s impact on the user’s experience during the interaction. On the one hand, we use head pose telemetry to quantify attentional aspects of engagement, including measures of distraction and focus during the interaction. On the other hand, we use a post-experimental questionnaire incorporating the User Engagement Scale Short Form to assess engagement. The results regarding telemetry data reveal reduced distraction and improved attentional consistency, highlighting the system’s ability to maintain attention on the current task effectively. Furthermore, the questionnaire responses show that RJA significantly enhances self-reported engagement when the system is active. We believe these findings confirm the value of attentional mechanisms in promoting engaging human–robot interactions.
Topics
No keywords indexed for this article. Browse by subject →
References
64
[1]
Picton, T.W., Campbell, K.B., Baribeau-Braun, J., and Proulx, G.B. (2022). The neurophysiology of human attention: A tutorial review. Attention and Performance VII, Routledge.
10.4324/9781003310228-29
[2]
Burgoon, J.K., Manusov, V., and Guerrero, L.K. (2021). Nonverbal Communication, Routledge.
10.4324/9781003095552
[3]
Moll "What We Do and Don’t Know About Joint Attention" Topoi (2024) 10.1007/s11245-023-09961-y
[4]
Flom, R., Lee, K., and Muir, D. (2017). Gaze-Following: Its Development and Significance, Psychology Press.
[5]
Tomasello "Joint attention to mental content and the social origin of reasoning" Synthese (2021) 10.1007/s11229-019-02327-1
[6]
Moore, C., Dunham, P.J., and Dunham, P. (2014). Joint Attention: Its Origins and Role in Development, Psychology Press.
10.4324/9781315806617
[7]
Huang, C.M., and Thomaz, A.L. (2010, January 11–13). Joint attention in human-robot interaction. Proceedings of the 2010 AAAI Fall Symposium Series, Arlington, VA, USA.
[8]
Mundy, P. (2021). RJA/IJA (Initiating/responding to joint attention). Encyclopedia of Autism Spectrum Disorders, Springer.
10.1007/978-3-319-91280-6_605
[9]
Cho, Y. (2015). The Relationship Between CJA, IJA, RJA of Joint Attention and Language in Infancy. [Ph.D. Thesis, Seoul National University Graduate School].
[10]
Cipolletta "Intersubjectivity, joint action and sociality" J. Constr. Psychol. (2022)
[11]
Yuill, N., and Yuill, N. (2021). Engagement and Joint Attention. Technology to Support Children’s Collaborative Interactions: Close Encounters of the Shared Kind, Springer.
10.1007/978-3-030-75047-3
[12]
Paliga "Human–cobot interaction fluency and cobot operators’ job performance. The mediating role of work engagement: A survey" Robot. Auton. Syst. (2022) 10.1016/j.robot.2022.104191
[13]
Avelino "Break the ice: A survey on socially aware engagement for human–robot first encounters" Int. J. Soc. Robot. (2021) 10.1007/s12369-020-00720-2
[14]
Alam "Social robots in education for long-term human-robot interaction: Socially supportive behaviour of robotic tutor for creating robo-tangible learning environment in a guided discovery learning interaction" ECS Trans. (2022) 10.1149/10701.12389ecst
[15]
Breazeal, C., Dautenhahn, K., and Kanda, T. (2016). Social robotics. Springer Handbook of Robotics, Springer.
10.1007/978-3-319-32552-1_72
[16]
Woertman, S.E. (2024). Joint Attention in Human-Robot Interaction. [Bachelor’s Thesis, University of Twente].
[17]
Admoni "Social eye gaze in human-robot interaction: A review" J.-Hum.-Robot. Interact. (2017) 10.5898/jhri.6.1.admoni
[18]
Zabidi "Application of Robots in Improving Joint Attention and Imitation Skills for Children with Autism: A Comprehensive Review" J. Mech. Eng. Res. Dev. (2022)
[19]
Skantze "Turn-taking, feedback and joint attention in situated human–robot interaction" Speech Commun. (2014) 10.1016/j.specom.2014.05.005
[20]
Castillo "Implementation of a Biologically Inspired Responsive Joint Attention System for a Social Robot" Adv. Intell. Syst. (2025) 10.1002/aisy.202400650
[21]
García-Martínez, J., Gamboa-Montero, J.J., Castillo, J.C., and Castro-González, Á. (2024). Analyzing the Impact of Responding to Joint Attention on the User Perception of the Robot in Human-Robot Interaction. Biomimetics, 9.
10.3390/biomimetics9120769
[22]
A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form
Heather L. O’Brien, Paul Cairns, Mark Hall
International Journal of Human-Computer Studies
2018
10.1016/j.ijhcs.2018.01.004
[23]
Sorrentino "From the Definition to the Automatic Assessment of Engagement in Human–Robot Interaction: A Systematic Review" Int. J. Soc. Robot. (2024) 10.1007/s12369-024-01146-w
[24]
Nagai "A constructive model for the development of joint attention" Connect. Sci. (2003) 10.1080/09540090310001655101
[25]
Boucenna, S., Gaussier, P., and Hafemeister, L. (2011, January 24–27). Development of joint attention and social referencing. Proceedings of the 2011 IEEE International Conference on Development and Learning (ICDL), Frankfurt am Main, Germany.
10.1109/devlrn.2011.6037317
[26]
Scassellati, B. (1998, January 6–10). Imitation and mechanisms of joint attention: A developmental structure for building social skills on a humanoid robot. Proceedings of the International Workshop on Computation for Metaphors, Analogy, and Agents, University of Aizu, Aizuwakamatsu, Japan.
10.1007/3-540-48834-0_11
[27]
Brooks, R.A., Breazeal, C., Marjanović, M., Scassellati, B., and Williamson, M.M. (1998, January 6–10). The Cog project: Building a humanoid robot. Proceedings of the International Workshop on Computation for Metaphors, Analogy, and Agents, University of Aizu, Aizuwakamatsu, Japan.
10.1007/3-540-48834-0_5
[28]
Ito, M., and Tani, J. (2004, January 20–22). Joint attention between a humanoid robot and users in imitation game. Proceedings of the International Conference on Development and Learning (ICDL), La Jolla, CA, USA.
[29]
Ishida, T. (November, January 31). Development of a small biped entertainment robot QRIO. Proceedings of the Micro-Nanomechatronics and Human Science, and The Fourth Symposium Micro-Nanomechatronics for Information-Based Society, Nagoya, Japan.
[30]
Chew, J.Y., and Wang, X. (2024, January 11–15). Joint Attention Estimation during Multi-party Facilitation Using Multi-Modal Fusion. Proceedings of the Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, Boulder, CO, USA. HRI ’24.
10.1145/3610978.3640669
[31]
Huang, C.M., and Thomaz, A.L. (August, January 31). Effects of responding to, initiating and ensuring joint attention in human-robot interaction. Proceedings of the 2011 Ro-Man, Atlanta, GA, USA.
10.1109/roman.2011.6005230
[32]
Westlund "Children use non-verbal cues to learn new words from robots as well as people" Int. J.-Child-Comput. Interact. (2017)
[33]
Scassellati "Improving social skills in children with ASD using a long-term, in-home social robot" Sci. Robot. (2018) 10.1126/scirobotics.aat7544
[34]
Pereira, A., Oertel, C., Fermoselle, L., Mendelson, J., and Gustafson, J. (2019, January 4–8). Responsive joint attention in human-robot interaction. Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China.
10.1109/iros40897.2019.8968130
[35]
Cao "Robot-assisted joint attention: A comparative study between children with autism spectrum disorder and typically developing children in interaction with NAO" IEEE Access (2020) 10.1109/access.2020.3044483
[36]
Mishra, C., and Skantze, G. (September, January 29). Knowing where to look: A planning-based architecture to automate the gaze behavior of social robots. Proceedings of the 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Napoli, Italy.
10.1109/ro-man53752.2022.9900740
[37]
Teow, K. (2024). Joint Attention in Human-Robot Interaction, a Case of Eye Gaze-Leading Task. [Bachelor’s Thesis, University of Twente].
[38]
Mazzola, C., Morocutti, L., Pedrizzi, H., Poslon, X.D., Lucny, A., Wingert, S.M., Nagai, Y., Rea, F., and Sciutti, A. (2024, January 20–23). Sketch it for the Robot! How Child-Like Robots’ Joint Attention Affects Humans’ Drawing Strategies. Proceedings of the 2024 IEEE International Conference on Development and Learning (ICDL), Austin, TX, USA.
10.1109/icdl61372.2024.10644384
[39]
Toms "What is user engagement? A conceptual framework for defining user engagement with technology" J. Am. Soc. Inf. Sci. Technol. (2008) 10.1002/asi.20801
[40]
O’Brien, H., and Cairns, P. (2016). Why Engagement Matters: Cross-Disciplinary Perspectives of User Engagement in Digital Media, Springer.
10.1007/978-3-319-27446-1
[41]
Toms "The development and evaluation of a survey to measure user engagement" J. Am. Soc. Inf. Sci. Technol. (2010) 10.1002/asi.21229
[42]
Hall "Perception of own and robot engagement in human–robot interactions and their dependence on robotics knowledge" Robot. Auton. Syst. (2014) 10.1016/j.robot.2013.09.012
[43]
Rich, C., Ponsler, B., Holroyd, A., and Sidner, C.L. (2010, January 2–5). Recognizing engagement in human-robot interaction. Proceedings of the 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Osaka, Japan.
10.1109/hri.2010.5453163
[44]
Sidner "Explorations in engagement for humans and robots" Artif. Intell. (2005) 10.1016/j.artint.2005.03.005
[45]
Prendinger, H., Lester, J., and Ishizuka, M. (2008, January 1–3). Estimating User’s Conversational Engagement Based on Gaze Behaviors. Proceedings of the Intelligent Virtual Agents, Tokyo, Japan. Lecture Notes in Computer Science.
10.1007/978-3-540-85483-8
[46]
Ivaldi "Robot initiative in a team learning task increases the rhythm of interaction but not the perceived engagement" Front. Neurorobotics (2014) 10.3389/fnbot.2014.00005
[47]
Sanghvi, J., Castellano, G., Leite, I., Pereira, A., McOwan, P.W., and Paiva, A. (2011, January 8–11). Automatic analysis of affective postures and body motion to detect engagement with a game companion. Proceedings of the 6th International Conference on Human-Robot Interaction, Lausanne, Switzerland. HRI ’11.
10.1145/1957656.1957781
[48]
Thrun, S., Brooks, R., and Durrant-Whyte, H. (2007, January 26–29). How Social Robots Will Help Us to Diagnose, Treat, and Understand Autism. Proceedings of the Robotics Research, Hiroshima, Japan. Tracts in Advanced Robotics.
[49]
Boucenna "Learning of Social Signatures Through Imitation Game Between a Robot and a Human Partner" IEEE Trans. Auton. Ment. Dev. (2014) 10.1109/tamd.2014.2319861
[50]
Anzalone "Evaluating the Engagement with Social Robots" Int. J. Soc. Robot. (2015) 10.1007/s12369-015-0298-7
Showing 50 of 64 references
Metrics
1
Citations
64
References
Details
- Published
- Aug 06, 2025
- Vol/Issue
- 15(15)
- Pages
- 8684
- License
- View
Authors
Cite This Article
Jesús García-Martínez, Juan José Gamboa-Montero, Álvaro Castro-González, et al. (2025). Are Robots More Engaging When They Respond to Joint Attention? Findings from a Turn-Taking Game with a Social Robot. Applied Sciences, 15(15), 8684. https://doi.org/10.3390/app15158684
Related
You May Also Like
Machine Learning and Deep Learning Methods for Intrusion Detection Systems: A Survey
Hongyu Liu, Bo Lang · 2019
892 citations
Deep Residual Learning for Image Recognition: A Survey
Muhammad Shafiq, Zhaoquan Gu · 2022
819 citations
Diversity of Synthetic Dyes from Textile Industries, Discharge Impacts and Treatment Methods
Houda Ben Slama, Ali Chenari Bouket · 2021
660 citations
Fe3O4 Nanoparticles: Structures, Synthesis, Magnetic Properties, Surface Functionalization, and Emerging Applications
Minh Dang Nguyen, Hung-Vu Tran · 2021
629 citations
An Overview of Variants and Advancements of PSO Algorithm
Meetu Jain, Vibha Saihjpal · 2022
594 citations