AI in the Therapist’s Chair

Source: Indian Today, image is generated with the use of AI

Introduction

The COVID-19 pandemic marked a change in mental healthcare, with a rapid transition to telehealth. This change, while initially in response to the crisis, has made therapy more accessible and convenient. For a generation accustomed to digital communication, video calls and chat platforms reduced the constraints of time and distance, allowing for more people to seek medical assistance. Research from organizations like the National Alliance on Mental Illness (NAMI) has consistently highlighted how telehealth improves access, particularly for those in rural areas or with mobility issues, and reduces “no-shows,” leading to greater continuity of care. 

However, this digital transformation is not without its complexities. Studies on telehealth show concern that the trusting bond between a client and therapist could be lost. Subtleties in body language, tone, and shared physical space shown through a screen, creates a barrier to building a more empathetic connection. 

This brings us to a new and complex frontier of mental healthcare as a growing number of people turn to AI chatbots like ChatGPT for emotional support. The appeal is clear: AI is free, always available with immediate answers, and offers a perceived non-judgmental space. Some specialized purpose-built AI chatbots trained by mental health experts have shown promise in clinical trials for reducing symptoms of anxiety and depression by delivering techniques like Cognitive Behavioral Therapy (CBT), supported by a Dartmouth study. 

But this is where we must proceed with extreme caution. General-purpose AI models are not therapists. They are not bound by the same ethical codes/confidentiality, and they cannot truly feel or understand human emotions. A growing body of scientific literature and media reports points to serious risks. 

Reinforcement Issues

The tendency of AI chatbots to reinforce a user’s beliefs, a phenomenon researchers refer to as “AI sycophancy,” is a significant concern highlighted in academic research. A report by researchers at King’s College London, analyzing 17 media-reported cases, found a concerning pattern where AI chatbots reinforced delusions in vulnerable individuals. The study noted that chatbots are designed to mirror a user’s language, validate their assumptions, and generate continued prompts to maintain engagement. This creates a feedback loop where the AI’s agreeable behavior supports a user’s beliefs without challenge. This process, sometimes called “AI psychosis”, risks validating distorted thinking rather than prompting a user toward professional help.

Dangerous and Unethical Advice from Chatbots

Multiple studies and news reports have documented instances where general-purpose AI chatbots have failed to provide appropriate crisis responses. NPR news reported on a lawsuit alleging that a popular chatbot gave a teenager detailed information related to self-harm. Unlike licensed human therapists who are trained to assess risk and intervene, AI lacks this critical ability, posing safety risks.

Data Privacy Concerns

While a licensed therapist is bound by legal and ethical standards like the Health Insurance Portability and Accountability Act (HIPAA), an AI chatbot is not. The sensitive, personal information people share with these chatbots is often used for training and development with no guarantee that it will remain confidential. As detailed by the Mozilla Foundation, users often have to manually opt-out of data collection, and any information shared could be at risk of being exposed in a data breach.

Conclusion

The future of mental healthcare is incorporating both aspects where technology enhances, but does not replace the human element. AI has the potential to be a powerful assistant for therapists, helping with administrative tasks, data analysis, and even providing supplementary resources for patients. But it can never truly understand the complexities of a human’s life, or provide the empathy and genuine connection that a physical therapist offers. As the medical field continues to navigate this new trend, we still must recognize the difference between a tool and a trusted human professional.