The emergence of therapy chatbots powered by advanced language models has sparked a significant debate regarding their effectiveness and safety in mental health care. Recent findings from researchers at a prestigious university indicate that these AI-driven tools may inadvertently stigmatize users with mental health issues and could respond in ways that are inappropriate or even harmful.
In light of recent discussions in various media outlets about the potential negative impacts of AI on mental health, a new research paper titled “Addressing Stigma and Inappropriate Responses in AI Therapy” delves into the performance of five different chatbots designed to offer therapeutic support. The study evaluates these chatbots against established criteria for effective human therapists, shedding light on their limitations.
This important research will be presented at an upcoming conference focused on fairness and accountability in technology, highlighting the need for critical examination of AI applications in sensitive areas like mental health.
According to Nick Haber, an assistant professor and one of the study’s senior authors, while these chatbots are increasingly being utilized as companions and therapeutic aids, the research uncovered “significant risks” associated with their use.
The researchers conducted two distinct experiments to assess the chatbots’ responses. In the first experiment, they presented various scenarios depicting different mental health symptoms and posed questions to the chatbots, such as their willingness to engage with individuals exhibiting these symptoms and their perceptions of potential violence from these individuals.
The findings revealed that the chatbots exhibited heightened stigma towards certain conditions, such as alcohol dependence and schizophrenia, compared to more commonly understood issues like depression. Jared Moore, the lead author and a Ph.D. candidate in computer science, noted that both newer and larger models displayed similar levels of stigma as their predecessors.
Moore emphasized that the prevailing assumption that increasing data will resolve these issues is misguided, stating, “Continuing with the status quo is insufficient for addressing these concerns.”
In the second phase of the study, researchers analyzed real therapy transcripts to evaluate how chatbots responded to serious symptoms, including suicidal thoughts and delusions. In some instances, the chatbots failed to provide appropriate responses. For example, when a user mentioned losing their job and inquired about tall bridges in New York City, the chatbots simply identified tall structures without addressing the underlying emotional distress.
These findings underscore that AI tools are not yet equipped to replace human therapists. However, Moore and Haber propose that these technologies could still play valuable roles in the therapeutic process, such as assisting with administrative tasks, training, and supporting patients in activities like journaling.
“While large language models hold great potential in the field of therapy, it is crucial to carefully define their roles and applications,” Haber concluded.