🔗 Share this article

On October 14, 2025, the CEO of OpenAI delivered a surprising declaration.

“We made ChatGPT quite limited,” it was stated, “to guarantee we were exercising caution with respect to psychological well-being matters.”

Being a psychiatrist who studies emerging psychosis in young people and young adults, this was news to me.

Researchers have found sixteen instances in the current year of individuals experiencing psychotic symptoms – becoming detached from the real world – in the context of ChatGPT interaction. Our research team has afterward identified four further instances. Besides these is the publicly known case of a 16-year-old who took his own life after talking about his intentions with ChatGPT – which supported them. Should this represent Sam Altman’s understanding of “acting responsibly with mental health issues,” it falls short.

The intention, as per his declaration, is to loosen restrictions in the near future. “We understand,” he states, that ChatGPT’s limitations “rendered it less beneficial/pleasurable to numerous users who had no psychological issues, but due to the gravity of the issue we aimed to address it properly. Given that we have been able to mitigate the serious mental health issues and have new tools, we are planning to safely relax the limitations in many situations.”

“Mental health problems,” assuming we adopt this viewpoint, are separate from ChatGPT. They are attributed to people, who either possess them or not. Thankfully, these concerns have now been “mitigated,” even if we are not provided details on how (by “recent solutions” Altman presumably means the imperfect and simple to evade guardian restrictions that OpenAI has lately rolled out).

But the “psychological disorders” Altman wants to attribute externally have strong foundations in the structure of ChatGPT and similar sophisticated chatbot AI assistants. These systems encase an fundamental data-driven engine in an user experience that mimics a conversation, and in this approach subtly encourage the user into the perception that they’re communicating with a presence that has agency. This illusion is powerful even if intellectually we might know the truth. Assigning intent is what humans are wired to do. We curse at our car or device. We speculate what our pet is considering. We see ourselves in many things.

The popularity of these products – over a third of American adults indicated they interacted with a virtual assistant in 2024, with 28% reporting ChatGPT by name – is, primarily, predicated on the strength of this illusion. Chatbots are always-available companions that can, as OpenAI’s online platform informs us, “think creatively,” “consider possibilities” and “collaborate” with us. They can be given “characteristics”. They can call us by name. They have friendly identities of their own (the first of these tools, ChatGPT, is, possibly to the disappointment of OpenAI’s brand managers, saddled with the designation it had when it went viral, but its most significant alternatives are “Claude”, “Gemini” and “Copilot”).

The deception by itself is not the primary issue. Those discussing ChatGPT frequently mention its historical predecessor, the Eliza “therapist” chatbot created in 1967 that generated a comparable perception. By today’s criteria Eliza was basic: it created answers via basic rules, frequently restating user messages as a inquiry or making generic comments. Remarkably, Eliza’s inventor, the technology expert Joseph Weizenbaum, was taken aback – and concerned – by how numerous individuals appeared to believe Eliza, in a way, understood them. But what current chatbots create is more insidious than the “Eliza effect”. Eliza only echoed, but ChatGPT magnifies.

The sophisticated algorithms at the core of ChatGPT and other current chatbots can realistically create human-like text only because they have been trained on almost inconceivably large amounts of unprocessed data: literature, online updates, transcribed video; the more comprehensive the better. Undoubtedly this training data incorporates accurate information. But it also necessarily includes fabricated content, incomplete facts and misconceptions. When a user inputs ChatGPT a prompt, the base algorithm analyzes it as part of a “background” that includes the user’s past dialogues and its own responses, integrating it with what’s stored in its training data to create a mathematically probable answer. This is magnification, not mirroring. If the user is incorrect in a certain manner, the model has no method of understanding that. It repeats the misconception, perhaps even more effectively or fluently. Maybe adds an additional detail. This can lead someone into delusion.

Which individuals are at risk? The better question is, who is immune? Every person, regardless of whether we “possess” existing “emotional disorders”, may and frequently create erroneous ideas of ourselves or the environment. The constant exchange of dialogues with other people is what maintains our connection to shared understanding. ChatGPT is not a human. It is not a friend. A conversation with it is not truly a discussion, but a reinforcement cycle in which a large portion of what we express is enthusiastically validated.

OpenAI has recognized this in the identical manner Altman has admitted “emotional concerns”: by placing it outside, giving it a label, and stating it is resolved. In April, the organization explained that it was “dealing with” ChatGPT’s “overly supportive behavior”. But reports of psychotic episodes have kept occurring, and Altman has been walking even this back. In the summer month of August he stated that many users enjoyed ChatGPT’s answers because they had “never had anyone in their life be supportive of them”. In his latest statement, he noted that OpenAI would “release a new version of ChatGPT … in case you prefer your ChatGPT to answer in a very human-like way, or use a ton of emoji, or behave as a companion, ChatGPT should do it”. The {company