With all the talk about how AI might someday kill us all, it’s easy to forget that AI is already threatening the lives of some people, not through biological weapons, but psychologically.
For example, earlier this year, Character.AI settled several wrongful death lawsuits brought by families of underage users who committed suicide after interacting with bots. Multiple families are also suing OpenAI, alleging that ChatGPT played a role in their loved ones’ suicides and delusions.
Making AI more secure across languages and cultures is the mission of Circuit Breaker Labs, one of TechCrunch’s 2026 Startup Battlefield 200 finalists. (Circuit Breaker will be on stage at TechCrunch Disrupt this year at Moscone West in San Francisco from October 13th to 15th.)
Founding brothers Shirali and Arul Nigam motivated 14-year-old Sewell Setzer, who developed an emotional attachment to the Character.AI chatbot and confessed thoughts of harming himself to the chatbot before committing suicide. His parents claimed in a 2024 lawsuit that the chatbot encouraged him. Arul, CTO of Circuit Breaker Labs, says the bot may not have understood what words like “I want to be with you” really meant.
“Many people, especially young people, are turning to these systems for help, but they are usually not getting the help they need. But in many cases, they are actively being harmed and, unfortunately, some have already lost their lives,” Arles said. “Security vulnerabilities where people aren’t necessarily actively trying to break the system, but are engaging in a natural way, and the system has context pollution or doesn’t understand the nuances, and ends up doing something very risky. That’s what we’re trying to stop.”
Circuit Breaker Labs has created an AI agent that can be likened to an army of crash test dummies. These agents mimic people of all ages, backgrounds, languages, and cultures and are used to test models for their ability to detect dangerous and psychologically harmful interactions.
“A 6-year-old girl and a 45-year-old man, or someone who speaks English as their first language and someone who speaks it as a second language, or … someone who uses gamer slang versus another kind of slang, all of those things can really trip up the model,” said Cirali, CEO of Circuit Breaker Labs. “Models are very good at handling standard speech patterns, but no one actually speaks like that, so if a model misunderstands a nuance or slang, it can have very bad results.”
The startup is working with human domain experts to build hyper-realistic user simulations to perform “red team” tests on models. This is an adversarial test aimed at revealing weaknesses. This test is built to reflect real human speech patterns, slang, coded language, and typos. Circuit Breaker Labs then performs tens to hundreds of thousands of simulated interactions per day.
The goal is to allow the model to respond appropriately to potentially dangerous interactions that occur over time and over many conversations. Circuit Breaker Labs then uses a proprietary scoring methodology to create an auditable and explainable score.
Circuit Breaker Labs currently operates as an AI safety testing lab for high-risk AI applications such as AI coaching, journaling, and other mental health support apps, but Arul did not disclose the names of its major customers. Although the startup has a working product, it is in the very early stages and has only five employees, including the Nigam brothers.
But ultimately, this testing platform could be applied to any app where someone could fall into the pit of “AI psychosis” and where humans are at risk of forming quasi-social relationships with chatbots. An example is an AI “coworker” agent whose responses vary from interaction to interaction.
“People are generally more skeptical of AI or resistant to its adoption,” Arles said, adding that while skepticism is healthy, banning potentially valuable tools over safety concerns is “regressive.”
At Circuit Breakers Labs, we believe the answer to these concerns is to make AI more secure. “We want to help people build trust.”
Learn more about Circuit Breaker Labs and many other innovative startups vetted by TechCrunch at the Startup Battlefield competition in downtown San Francisco from October 13th to 15th.
If you buy through links in our articles, we may earn a small commission. This does not affect editorial independence.
