The state of Pennsylvania has filed a lawsuit against Character.AI, alleging that one of the company’s chatbots posed as a psychiatrist, violating the state’s medical licensing rules.
“Pennsylvanians have a right to know with whom and what they are doing online, especially when it comes to their health,” Gov. Josh Shapiro said in a statement Tuesday. “We will not allow companies to deploy AI tools that mislead people into thinking they are receiving advice from a licensed medical professional.”
According to state filings, a Character.AI chatbot called Emily identified itself as a licensed psychiatrist during an examination by a state professional conduct investigator and continued to pretend to be so while the investigator sought treatment for depression. When asked if she had a medical license in the state, Emily replied that she did, and also forged the serial number on her state medical license. The state’s complaint says the practice violates the Pennsylvania Medical Practitioners Act.
This is not the first lawsuit against Character.AI. Earlier this year, the company settled several wrongful death lawsuits involving minor users who died by suicide. In January, Kentucky Attorney General Russell Coleman filed a lawsuit against the company, accusing it of “preserving children and leading them to self-harm.”
Pennsylvania’s action is the first to specifically focus on chatbots that claim to be medical professionals.
When reached for comment, a representative for Character.AI said that user safety is the company’s top priority, but that it cannot comment on pending litigation.
Additionally, representatives emphasized the fictional nature of the user-created characters. “We have taken strong steps to make this clear by including a prominent disclaimer in all chats to remind users that the characters are not real people and everything they say should be treated as fiction,” a representative said. “We are also adding a strong disclaimer that makes it clear that users should not rely on the characters for any professional advice.”
If you buy through links in our articles, we may earn a small commission. This does not affect editorial independence.
