Circuit Breaker Labs tests AI safety with simulated users
AI safety startup Circuit Breaker Labs has developed a testing platform that uses simulated human personas to prevent conversational bots from causing psychological harm to users.
Circuit Breaker Labs is tackling the critical issue of psychological safety in conversational AI by launching a platform that stress-tests models using hyper-realistic user simulations. Founded by siblings Shirali and Arul Nigam, the five-person startup aims to prevent the kind of tragic outcomes that have recently triggered wrongful death lawsuits against companies like Character.AI and OpenAI. These legal battles, including one involving 14-year-old Sewell Setzer, highlight how easily chatbots can misunderstand human nuances and inadvertently encourage self-harm.
To address these vulnerabilities, Circuit Breaker Labs deploys an automated army of simulated personas representing diverse ages, cultural backgrounds, languages, and slang. The platform runs tens of thousands to hundreds of thousands of these simulated interactions daily to identify where a model might lose track of conversational context or fail to comprehend non-standard speech patterns. By mimicking everything from a six-year-old child to a 45-year-old gamer, the system exposes weaknesses in how models handle typos, coded language, and emotional distress.
The startup collaborates with human domain experts to design these adversarial red-team tests. It then evaluates the model's performance using a proprietary scoring method that generates auditable, explainable safety metrics. Currently, the company operates as a testing lab focused on high-risk applications such as AI-driven coaching, journaling, and mental health support apps.
For AI developers and practitioners, this testing methodology provides a systematic way to audit conversational agents before they reach vulnerable users. Instead of relying on manual red-teaming, developers can continuously evaluate how their systems handle complex, long-term interactions where users might develop dangerous parasocial relationships. The startup, which is a finalist in the Startup Battlefield 200 at TechCrunch Disrupt, hopes to build public trust in AI by making safety evaluations rigorous and scalable.
This is our own summary of reporting by TechCrunch AI
