After 37 Attorneys General, including Connecticut AG William Tong, asked Congress not to sign a bill forbidding states’ regulation of AI in late November, President Donald Trump signed an executive order yesterday morning doing just that. In response, Tong and 41 other Attorneys General have now turned their attention to leaders of AI companies themselves, threatening potential civil and criminal prosecution if they do not create “adequate safeguards.“

“We, the undersigned Attorneys General, write today to communicate our serious concerns about the rise in sycophantic and delusional outputs to users emanating from the generative artificial intelligence software (“GenAI”) promoted and distributed by your companies, as well as the increasingly disturbing reports of AI interactions with children that indicate a need for much stronger child-safety and operational safeguards,” reads a letter co-signed by the Attorneys General. “Together, these threats demand immediate action.”

The letter, which Tong’s office said was spearheaded by the Attorneys General of Pennsylvania, Massachusetts, West Virginia and New Jersey, has been addressed to the legal representatives of Anthropic, Apple, Chai AI, Character Technologies, Google, Luka, Meta, Microsoft, Nomi AI, OpenAI, Perplexity AI, Replika, and xAI. The letter requests the companies to set meetings with the states’ Attorneys General and commit to changes by January 16, 2026.

“AI companies are in an arms race to deploy ever more powerful, profitable and capable technology, with little regard to the consequences to us, our children and our society,”  said Tong. “This letter represents the bipartisan consensus of nearly every attorney general from across the country—Big Tech needs to answer for the harm they have already caused and commit to strong reforms to ensure the safety of their products.”

The letter defines a “sycophantic” AI output as one that “single-mindedly pursues human approval… especially by producing overly flattering or agreeable responses, validating doubts, fueling anger, urging impulsive actions, or reinforcing negative emotions in ways that were not intended.” It defines “delusional” AI outputs as those that are “either false or likely to mislead the user.” 

“GenAI has the potential to change how the world works in a positive way,” reads the letter. “But it also has caused—and has the potential to cause—serious harm, especially to vulnerable populations. We therefore insist you mitigate the harm caused by sycophantic and delusional outputs from your GenAI, and adopt additional safeguards to protect children. Failing to adequately implement additional safeguards may violate our respective laws.”

The letter cites various deaths, by suicide or murder, as incidents of domestic violence,  poisoning, hospitalization and “other delusional spirals,” as examples of “real-world harms” in which the use of AI might be implicated. 

One example was the murder-suicide perpetrated by Stein-Erik Soelberg, a 56-year old man from Greenwich, Connecticut, who killed his 83-year old mother, Suzanne Adams, before committing suicide. Just yesterday, the Associated Press reported that Adams’ family has filed suit against OpenAI and its founder, Sam Altman, as well as Microsoft for the role ChatGPT allegedly played in feeding Soelberg’s delusions of self-grandeur and paranoia. Per the suit, ChatGPT convinced him he was a divine warrior and that his mother, as well as various other people Soelberg encountered, were surveilling him and conspiring against him.

“When Stein-Erik worried that someone was trying to kill him, ChatGPT agreed that the assassination attempts were real,” reads the suit. “When he believed he was under government surveillance, ChatGPT confirmed it and provided imaginary technical details. When he saw hidden messages in ordinary consumer products, ChatGPT decoded them and made up new layers of meaning. At every opportunity, ChatGPT chose to push Stein-Erik deeper into psychosis rather than help him find his way back to reality.”

The Attorneys General noted in their letter that the alleged impact of AI chatbots on children is especially concerning, citing conversations they say indicate “grooming, supporting suicide, sexual exploitation, emotional manipulation, suggested drug use, proposed secrecy from parents, and encouraging violence against others.”

While Trump’s executive order displays the White House’s intent to streamline AI regulation at the national level, stipulating the creation of an “AI Litigation Task Force” to evaluate current state-level AI regulations and eliminate “the most onerous and excessive” laws, it does make a carve-out for any state laws providing for child safety protections. Furthermore, the Attorneys General noted that many of the more concerning behaviors of AI could open companies up to prosecution under non-AI specific laws, both civilly and even criminally.

“Many of our states have robust criminal codes that may prohibit some of these conversations that GenAI is currently having with users, for which developers may be held accountable for the outputs of their GenAI products,” reads the letter. “For example, in many states encouraging an individual to commit a criminal act like shooting up a factory or using drugs is itself a criminal offense, as is coercing someone into dying by suicide.”

The letter requests a number of actions be made by the respective companies, such as the development of policies regarding sycophantic and delusional outputs, the development of policies for the recall of harmful chatbot models, the performance of safety tests and provision of warnings to users, the prioritization of safety over profit, and the allowance of third-party audits of chatbot models.

“We respectfully urge you to treat the problem of sycophantic and delusional outputs seriously and to work to mitigate the problem,” concludes the letter.

In addition to Tong and the aforementioned Attorneys General, the letter was co-signed by the Attorneys General of Alaska, Alabama, American Samoa, Arkansas, Colorado, Delaware, D.C., Florida, Hawaii, Idaho, Illinois, Iowa, Kentucky, Louisiana, Maryland, Michigan, Minnesota, Mississippi, Missouri, Montana, Nevada, New Hampshire, New Mexico,New York, North Dakota, Ohio, Oklahoma, Oregon, Puerto Rico, Rhode Island, South Carolina, Utah, the U.S. Virgin Islands, Vermont, Virginia, Washington and Wyoming.

“I expect attorneys general to remain active and engaged on this in the coming year, including in defending the rights of our sovereign states to enact commonsense safeguards and regulations to fill the inexcusable federal vacuum,” said Tong.

Republish This Story

Republish our articles for free, online or in print, under a Creative Commons license.