LLM-based Conversational AI Therapist for Daily Functioning Screening and Psychotherapeutic Intervention via Everyday Smart Devices

Jingping Nie; Hanya (Vera) Shao; Yuang Fan; Qijia Shao; Haoxuan You; Matthias Preindl; Xiaofan Jiang

doi:10.1145/3712299

journal article Jan 23, 2025

LLM-based Conversational AI Therapist for Daily Functioning Screening and Psychotherapeutic Intervention via Everyday Smart Devices

Jingping Nie

Hanya (Vera) Shao Yuang Fan Qijia Shao Haoxuan You Matthias Preindl

Xiaofan Jiang

ACM Transactions on Computing for Healthcare · Association for Computing Machinery (ACM)

View at Publisher Save 10.1145/3712299

Abstract

Despite the global mental health crisis, access to screenings, professionals, and treatments remains high. In collaboration with licensed psychotherapists, we propose a
C
onversational
AI
T
herapist with psychotherapeutic
I
nterventions (CaiTI), a platform that leverages large language models (LLM)s and smart devices to enable better mental health self-care. CaiTI can screen the day-to-day functioning using natural and psychotherapeutic conversations. CaiTI leverages reinforcement learning to provide personalized conversation flow. CaiTI can accurately understand and interpret user responses. When the user needs further attention during the conversation, CaiTI can provide conversational psychotherapeutic interventions, including cognitive behavioral therapy (CBT) and motivational interviewing (MI). Leveraging the datasets prepared by the licensed psychotherapists, we experiment and microbenchmark various LLMs’ performance in tasks along CaiTI's conversation flow and discuss their strengths and weaknesses. With the psychotherapists, we implement CaiTI and conduct 14-day and 24-week studies. The study results, validated by therapists, demonstrate that CaiTI can converse with users naturally, accurately understand and interpret user responses, and provide psychotherapeutic interventions appropriately and effectively. We showcase the potential of CaiTI LLMs to assist the mental therapy diagnosis and treatment and improve day-to-day functioning screening and precautionary psychotherapeutic intervention systems.

Topics

No keywords indexed for this article. Browse by subject →

References

104

[1]

2022. Mental Health Tests, Quizzes, Self-Assessments, & Screening Tools. https://www.psycom.net/quizzes. [Online; accessed 16-August-2022].

[2]

2022. Speech-to-Text. https://cloud.google.com/speech-to-text. [Online; accessed 24-April-2022].

[3]

2023. 3 things to know before talking to ChatGPT about your mental health. https://mashable.com/article/how-to-chat-with-chatgpt-mental-health-therapy.

[4]

2023. Using ChatGPT as a therapist? https://www.reddit.com/r/ChatGPTPro/comments/126rtvb/using_chatgpt_as_a_therapist/

[5]

2023. What is the Alexa Skills Kit? https://developer.amazon.com/en-US/docs/alexa/ask-overviews/what-is-the-alexa-skills-kit.html

[6]

Gregory A Aarons, Mark G Ehrhart, Joanna C Moullin, Elisa M Torres, and Amy E Green. 2017. Testing the leadership and organizational change for implementation (LOCI) intervention in substance abuse treatment: a cluster randomized trial study protocol. Implementation Science 12, 1 (2017), 1–11. 10.1186/s13012-017-0562-3

[7]

Alexander T Adams, Elizabeth L Murnane, Phil Adams, Michael Elfenbein, Pamara F Chang, Shruti Sannon, Geri Gay, and Tanzeem Choudhury. 2018. Keppi: A tangible user interface for self-reporting pain. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13.

[8]

Adaeze Adigwe, Noé Tits, Kevin El Haddad, Sarah Ostadabbas, and Thierry Dutoit. 2018. The emotional voices database: Towards controlling the emotion dimension in voice generation systems. arXiv preprint arXiv:1806.09514 (2018).

[9]

Brad A Alford Aaron T Beck and John V Jones Jr. 1997. The integrative power of cognitive therapy. 10.1891/0889-8391.11.4.309

[10]

Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. 2023. Palm 2 technical report. arXiv preprint arXiv:2305.10403 (2023).

[11]

Anthropic. 2024. Introducing the next generation of Claude. https://www.anthropic.com/news/claude-3-family. Accessed: 2024-03-06.

[12]

Combined Pharmacotherapies and Behavioral Interventions for Alcohol Dependence

Raymond F. Anton, Stephanie S. O’Malley, Domenic A. Ciraulo et al.

JAMA 2006 10.1001/jama.295.17.2003

[13]

Martin M Antony, Deborah Roth Ledley, and Richard G Heimberg. 2005. Improving outcomes and preventing relapse in cognitive-behavioral therapy. Guilford Press.

[14]

Hal Arkowitz and Henny A Westra. 2004. Integrating motivational interviewing and cognitive behavioral therapy in the treatment of depression and anxiety. Journal of Cognitive Psychotherapy 18, 4 (2004), 337–350. 10.1891/jcop.18.4.337.63998

[15]

Judith S Beck and AT Beck. 2011. Cognitive behavior therapy: Basic and beyond. New York: Guilford (2011).

[16]

Lisa J Bible, Kristin A Casper, Jennifer L Seifert, and Kyle A Porter. 2017. Assessment of self-care and medication adherence in individuals with mental health conditions. Journal of the American Pharmacists Association 57, 3 (2017), S203–S210. 10.1016/j.japh.2017.02.023

[17]

Simone Borsci, Alessio Malizia, Martin Schmettow, Frank Van Der Velde, Gunay Tariverdiyeva, Divyaa Balaji, and Alan Chamberlain. 2022. The Chatbot usability scale: The design and pilot of a usability scale for interaction with AI-based conversational agents. Personal and ubiquitous computing 26, 1 (2022), 95–119.

[18]

Menna Brown, Noelle O’Neill, Hugo van Woerden, Parisa Eslambolchilar, Matt Jones, Ann John, et al. 2016. Gamification and adherence to web-based mental health interventions: a systematic review. JMIR mental health 3, 3 (2016), e5710.

[19]

Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. 2023. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712 (2023).

[20]

David D Burns and Aaron T Beck. 1999. Feeling good: The new mood therapy. (1999).

[21]

Yusuf Cay. 2023. How to use ChatGPT as your therapist or mental coach? https://chatgptplus.blog/use-chatgpt-as-your-therapist-or-mental-coach/

[22]

Pooja Chandrashekar. 2018. Do mental health mobile apps work: evidence and recommendations for designing high-efficacy mental health mobile apps. Mhealth 4 (2018).

[23]

Sue J Channon, Michelle V Huws-Thomas, Stephen Rollnick, Kerenza Hood, Rebecca L Cannings-John, Carol Rogers, and John W Gregory. 2007. A multicenter randomized controlled trial of motivational interviewing in teenagers with diabetes. Diabetes care 30, 6 (2007), 1390–1395. 10.2337/dc06-2260

[24]

Siyuan Chen, Mengyue Wu, Kenny Q Zhu, Kunyao Lan, Zhiling Zhang, and Lyuchun Cui. 2023. LLM-empowered Chatbots for Psychiatrist and Patient Simulation: Application and Evaluation. arXiv preprint arXiv:2305.13614 (2023).

[25]

Shu Ming Chen, Debra Creedy, Huey-Shyan Lin, and Judy Wollin. 2012. Effects of motivational interviewing intervention on self-management, psychological and glycemic outcomes in type 2 diabetes: a randomized controlled trial. International journal of nursing studies 49, 6 (2012), 637–644.

[26]

David M Clark, Anke Ehlers, Freda McManus, Ann Hackmann, Melanie Fennell, Helen Campbell, Teresa Flower, Clare Davenport, and Beverley Louis. 2003. Cognitive therapy versus fluoxetine in generalized social phobia: a randomized placebo-controlled trial. Journal of consulting and clinical psychology 71, 6 (2003), 1058. 10.1037/0022-006x.71.6.1058

[27]

Whitney Clausen, Shinobu Watanabe-Galloway, M Bill Baerentzen, and Denise H Britigan. 2016. Health literacy among people with serious mental illness. Community mental health journal 52, 4 (2016), 399–405.

[28]

Gerald Corey. 2013. Theory and practice of counseling and psychotherapy. Cengage learning.

[29]

Ruixuan Dai, Thomas Kannampallil, Seunghwan Kim, Vera Thornton, Laura Bierut, and Chenyang Lu. 2023. Detecting Mental Disorders with Wearables: A Large Cohort Study. In Proceedings of the 8th ACM/IEEE Conference on Internet of Things Design and Implementation. 39–51.

[30]

Aniruddha Deb, Neeva Oza, Sarthak Singla, Dinesh Khandelwal, Dinesh Garg, and Parag Singla. 2023. Fill in the blank: Exploring and enhancing LLM capabilities for backward reasoning in math word problems. arXiv preprint arXiv:2310.01991 (2023).

[31]

Benjamin D Dickstein, Kristen H Walter, Jeremiah A Schumm, and Kathleen M Chard. 2013. Comparing response to cognitive processing therapy in military veterans with subthreshold and threshold posttraumatic stress disorder. Journal of Traumatic Stress 26, 6 (2013), 703–709. 10.1002/jts.21869

[32]

DSM-IV-TR. 2000. Diagnostic and statistical manual of mental disorders. American Psychiatric Association.

[33]

Conclusiones El ASRS. 2009. Adult ADHD Self-Report Scale (ASRS-v1. 1) symptom checklist in patients with substance use disorders. Actas Esp Psiquiatr 37, 6 (2009), 299–305.

[34]

Brynjar Emilsson, Gisli Gudjonsson, Jon F Sigurdsson, Gisli Baldursson, Emil Einarsson, Halldora Olafsdottir, and Susan Young. 2011. Cognitive behaviour therapy in medication-treated adults with ADHD and persistent symptoms: a randomized controlled trial. BMC psychiatry 11, 1 (2011), 1–10.

[35]

Explodingtopics. 2023. How Many People Own Smartphones? (2024-2029). https://explodingtopics.com/blog/smartphone-stats

[36]

Edna B Foa, Michael R Liebowitz, Michael J Kozak, Sharon Davies, Rafael Campeas, Martin E Franklin, Jonathan D Huppert, Kevin Kjernisted, Vivienne Rowan, Andrew B Schmidt, et al. 2005. Randomized, placebo-controlled trial of exposure and ritual prevention, clomipramine, and their combination in the treatment of obsessive-compulsive disorder. American Journal of psychiatry 162, 1 (2005), 151–161.

[37]

Baoyan Fu and XinXin Fu. 2022. Distributed Simulation System for Athletes’ Mental Health in the Internet of Things Environment. Computational Intelligence and Neuroscience 2022 (2022).

[38]

Madelyn S Gould, Jimmie Lou Harris Munfakh, Keri Lubell, Marjorie Kleinman, and Sarah Parker. 2002. Seeking help from the internet during adolescence. Journal of the American Academy of Child & Adolescent Psychiatry 41, 10 (2002), 1182–1189. 10.1097/00004583-200210000-00007

[39]

Diagnostic and Statistical Manual of Mental Disorders, 4th ed. (DSM-IV)

SAMUEL B. GUZE

American Journal of Psychiatry 1995 10.1176/ajp.152.8.1228

[40]

Katherine A Halmi, W Stewart Agras, Scott Crow, James Mitchell, G Terence Wilson, Susan W Bryson, and Helena C Kraemer. 2005. Predictors of treatment acceptance and completion in anorexia nervosa: implications for future study designs. Archives of general psychiatry 62, 7 (2005), 776–781.

[41]

Christine A Helfrich, Glenn T Fujiura, and Violet Rutkowski-Kmitta. 2008. Mental health disorders and functioning of women in domestic violence shelters. Journal of interpersonal violence 23, 4 (2008), 437–453. 10.1177/0886260507312942

[42]

The White House. 2022. Reducing the Economic Burden of Unmet Mental Health Needs. https://www.whitehouse.gov/cea/written-materials/2022/05/31/reducing-the-economic-burden-of-unmet-mental-health-needs/

[43]

Zifan Jiang, Salman Seyedi, Emily Lynn Griner, Ahmed Abbasi, Ali Bahrami Rad, Hyeokhyen Kwon, Robert O Cotes, and Gari D Clifford. 2023. Multimodal mental health assessment with remote interviews using facial, vocal, linguistic, and cardiovascular patterns. medRxiv (2023), 2023–09.

[44]

Subbarao Kambhampati, Karthik Valmeekam, Lin Guan, Mudit Verma, Kaya Stechly, Siddhant Bhambri, Lucas Saldyt, and Anil Murthy. 2024. LLMs can’t plan, but can help planning in LLM-modulo frameworks. arXiv preprint arXiv:2402.01817 (2024).

[45]

Angela Kertes, Henny A Westra, Lynne Angus, and Madalyn Marcus. 2011. The impact of motivational interviewing on client experiences of cognitive behavioral therapy for generalized anxiety disorder. Cognitive and Behavioral Practice 18, 1 (2011), 55–69. 10.1016/j.cbpra.2009.06.005

[46]

Mina J Kian, Mingyu Zong, Katrin Fischer, Abhyuday Singh, Anna-Maria Velentza, Pau Sang, Shriya Upadhyay, Anika Gupta, Misha A Faruki, Wallace Browning, et al. 2024. Can an LLM-Powered Socially Assistive Robot Effectively and Safely Deliver Cognitive Behavioral Therapy? A Study With University Students. arXiv preprint arXiv:2402.17937 (2024).

[47]

The PHQ-9

Kurt Kroenke, Robert L. Spitzer, Janet B. W. Williams

Journal of General Internal Medicine 2001 10.1046/j.1525-1497.2001.016009606.x

[48]

Kaylee Payne Kruzan, Jonah Meyerhoff, Theresa Nguyen, Madhu Reddy, David C Mohr, and Rachel Kornfield. 2022. “I Wanted to See How Bad it Was”: Online Self-screening as a Critical Transition Point Among Young Adults with Common Mental Health Conditions. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–16.

[49]

Tin Lai, Yukun Shi, Zicong Du, Jiajie Wu, Ken Fu, Yichao Dou, and Ziqi Wang. 2023. Psy-llm: Scaling up global mental health psychological services with ai-based large language models. arXiv preprint arXiv:2307.11991 (2023).

[50]

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019).

Showing 50 of 104 references

Metrics

18

Citations

104

References

Details

Published: Jan 23, 2025

Authors

J

Jingping Nie