Less than a year ago, Illinois state legislator Bob Morgan heard from a group of social workers. They asked him to look into artificial intelligence therapy bots. 

Morgan said he heard “story after story of new apps and new examples of AI therapy bots that are really providing bad advice — and sometimes dangerous advice” to individuals dealing with substance abuse, psychosis, suicidal ideation, and other life-or-death issues. 

In one particular example from a therapist, Morgan said a chatbot told a person with an addiction to take more drugs "because it felt good in the moment.” 

So, the state representative got to work drafting a bill that bans therapists from using AI other than for administrative purposes, like notetaking or scheduling. The law also says chatbots cannot diagnose or treat mental illness — or market themselves as if they do. 

“We're stepping in and saying, if you're an AI bot pretending to be a therapist, that is inappropriate, and we're going to shut that down,” Morgan said. 

Illinois is not the first state to pass legislation regarding the use of AI in psychotherapy. Utah and Nevada have passed laws this year to rein in the use of chatbots and other tools in mental health treatment. 

This “patchwork approach” by the states is likely to continue, according to Vaile Wright, senior director of health care innovation at the American Psychological Association. 

Wright said a federal approach is preferred because “then you would have some uniformity and greater specificity across the different states that could have better outcomes.”

But such regulation is unlikely, given efforts this summer by the U.S. House of Representatives to ban states from regulating artificial intelligence for a decade as part of President Donald Trump’s One Big Beautiful Bill Act. The U.S. Senate eventually voted to strike that provision from the bill. 

Federal laws aside, Wright said AI therapy bans like Illinois’ don’t address one of the biggest issues plaguing her field: people going to generative AI platforms like ChatGPT and Character AI for mental health support. 

“They call themselves companions; they say they'll help with your loneliness. But when you read the fine print, they very clearly say, ‘We are not a therapeutic aid,’” Wright said. 

She added that the business model for these platforms is to keep visitors on by being validating and reinforcing. 

“Basically, they're telling you exactly what you want to hear. This is the antithesis of therapy,” she said.

However, Wright does see a future where mental health chatbots are “rigorously tested, rooted in psychological science, co-created with experts, and they’ll have humans monitoring the interactions,” she said. 

Such tools — if federally regulated — could help fill a void in the U.S.’s growing mental health crisis, Wright added.

Until then, licensed psychologists like Michelle Kalnasy Powell say they will watch and wait.

“I am skeptical about AI,” she said, adding that Illinois’ ban is a good starting point, but it may not go far enough.

Kalnasy Powell currently uses AI for billing, but not for taking notes during sessions with patients. Some of her peers use dictation software that complies with patient confidentiality laws. 

“Even then, I question when I read the terms of service,” she said. “It’s kind of like sending a session off into the ether. What are you doing with that content? Is it really deleted?” 

She added that her work is very personal and vulnerable. 

“It is a privilege and an honor to be able to hear people's stories, both the joy, the happiness and the sorrow,” she said. “[If] that would somehow wind up out there in a way that none of us could predict, but wind up harming the client — I'm just not willing to risk it.”