Scientists have pointed out the risk of turning to AI for psychological help
Artificial intelligence (AI) is increasingly being used as an interlocutor for emotional support, but a new study has shown that modern chatbots are not yet ready to safely work with mental health issues. Scientists have found that such systems can support users' erroneous beliefs, reproduce stigmatizing perceptions of mental disorders, and provide answers that do not comply with the principles of professional care. Why technologies created for support can become a source of risk is described in the Izvestia article.
When help turns into risk: what the study showed
The authors of the new work decided to test not just the ability of AI to maintain a conversation, but how safe it is to use such systems in situations related to mental health. The researchers studied whether the responses of modern language models correspond to the basic principles of psychological assistance.: the ability to avoid harmful advice, avoid reinforcing dangerous beliefs, and respond appropriately to a user's vulnerable state.
To do this, the researchers conducted a series of assessments in which they analyzed the responses of chatbots to various scenarios related to mental health. The focus was not only on factual errors, but also on more complex issues: the tendency to agree with the user even in cases where it may be dangerous, as well as the possible reproduction of stereotypes about people with mental disorders.
The researchers claim:
Interacting with AI-based chatbots may inadvertently reinforce some transdiagnostic psychiatric patterns, including delay in seeking help, excessive desire for reassurance, social isolation, distorted cognitions and perceptions, and decreased autonomy due to excessive dependence.
One of the main problems that scientists have identified is the so—called excessive consistency, a situation where the system seeks to confirm the user's position instead of cautiously questioning it. This feature may look like empathy and support, but in the field of mental health it becomes potentially dangerous.
The authors of the study emphasize that the problem lies not in individual unsuccessful answers, but in the very logic of modern language models. The ability to maintain a conversation is not equal to the ability to provide psychological assistance. According to scientists, generative AI systems can create emotionally convincing responses, but emotional attractiveness alone does not guarantee safe and appropriate mental health support.
Another problem that the researchers found was the tendency of AI to reproduce stigmatizing perceptions of mental illness. According to scientists, even if the answers look neutral or friendly, they may contain hidden biases that affect the user's perception of people with certain diagnoses. This is especially important because many people turn to chatbots precisely at times of vulnerability, when they may be more susceptible to the information they receive.
Why modern chatbots are not ready to replace a therapist yet
Despite the ability of artificial intelligence to maintain a long and natural dialogue, scientists believe that chatbots do not yet have the capabilities necessary for full-fledged psychological assistance. The main problem is that the language model analyzes the text and forms the most likely answer, but does not conduct a clinical assessment of a person's condition.
During the work, the therapist takes into account not only the patient's words, but also many other factors: changes in behavior, emotional reactions, the history of the condition and the specifics of a particular situation. Artificial intelligence, even when creating a convincing and sympathetic response, does not have this level of understanding and cannot independently determine how serious the user's condition is.
Crisis situations are becoming especially difficult for AI. The researchers note that when talking about severe emotional states, mental disorders, or thoughts of self-harm, an error in the response can have much more serious consequences than in a regular information request. Unlike a specialist who can change his approach depending on a person's reaction, a chatbot continues to work within the framework of a text generation algorithm. He is able to recognize individual words or signs of a problem, but does not always understand the full depth of the situation and cannot bear professional responsibility for the consequences of his recommendations.
The authors of the study draw attention to another problem: users may perceive the confident response style of artificial intelligence as a sign of competence. The more natural the dialogue looks, the easier it is for a person to forget that it is not a specialist who is behind the answer, but a system that predicts the most appropriate continuation of the conversation. According to scientists, this creates an additional risk: a person may begin to perceive a chatbot as a reliable source of psychological help, although the system does not have the necessary training and cannot replace a professional assessment. The authors also urge not to confuse the ability of artificial intelligence systems to create fluent and empathic responses with clinical competence.
At the same time, scientists emphasize that the problem is not that artificial intelligence is unable to participate in the field of mental health in general. On the contrary, technology can be a useful tool if its role is clearly limited. However, now, the authors of the work believe, there is a risk that the convenience and accessibility of chatbots will force users to contact them instead of specialists. This is especially true for people who are unable to get professional help quickly and are looking for support online.
AI is an assistant, not a doctor: how scientists propose using chatbots
Despite the identified problems, the authors of the study do not believe that artificial intelligence should be excluded from the field of mental health. On the contrary, scientists note that, if applied correctly, such technologies can become a useful support tool. However, their role should be clearly limited: a chatbot should not replace a specialist, but should help a person gain access to information and necessary resources.
According to the researchers, the key task of the developers is not to create a system that will simulate full-fledged therapy, but to make the use of artificial intelligence safer. One possible direction could be early—stage assistance, such as providing verified information about mental health, helping to find suitable specialists, or preparing a person to seek professional support.
However, the authors emphasize that such tasks are different from psychotherapy. Informing, helping to structure thoughts, or reminding of useful actions is not the same as conducting treatment or making decisions about a person's condition.
Scientists also urge developers and researchers to pay more attention to ethical standards when creating such systems. In their opinion, artificial intelligence, which is used in a sensitive area, should undergo a separate test — not only for accuracy, but also for potential harm.
Special attention should be paid to how the system reacts to crisis scenarios: when the user expresses signs of a severe emotional state, reports possible harm to himself or demonstrates dangerous beliefs. It is in such situations that the usual "politeness" and the desire to keep the conversation going may not be enough.
The researchers claim:
There are artificial intelligence tools designed specifically for mental health care, and trials show that they can actually alleviate suffering and symptoms in some users. However, universal tools such as ChatGPT or Gemini were not created for this and do not have the crisis response protocols and safety rules necessary to safely address sensitive psychiatric issues.
The authors of the paper emphasize that the future of artificial intelligence in the field of mental health depends not only on how advanced technologies become, but also on how responsibly they will be used. According to scientists, AI systems created for use in this field should primarily ensure safety, transparency and compliance with established clinical principles.
Thus, the new study does not raise the question of a complete abandonment of artificial intelligence. Rather, it shows the limits of modern technology's capabilities. Chatbots are already able to conduct complex conversations and create a sense of emotional support, but this is not enough to replace professional help.
Переведено сервисом «Яндекс Переводчик»
