AI agents are all the rage, but they’re not as capable as their purveyors believe.
When we attempted to benchmark agents tasked with complex tasks, we found that agents were typically not reliable enough to function without close supervision. LiveBench 2.0 shows that Claude Fable 5.2 and ChatGPT-6 Astra succeeded at 82.9 and 75.6 out of 100 tasks, respectively, and in an effort by AI infrastructure company EON to test agents in a simulated corporate environment, the most successful model tested, Claude Opus 4.8, succeeded at 76.8%. We found that it performed the task with an accuracy of .
Now, a startup known as Plurai hopes to push that number up using the same technology that’s putting self-driving cars on the road: simulations with synthetic data based on the company’s proprietary digital systems aimed at preparing models to tackle difficult edge cases.
Co-founders Ilan Kadar and Elad Levi previously worked at self-driving technology developer Nexar. Kadar collaborated with Waymo to simulate and evaluate end-to-end deep learning-powered software.
“The moment we saw the wave of AI agents, we saw the same pattern: It’s very easy to build a demo, but it’s[hard]to get something reliable and cost-effective,” Kadar told TechCrunch. “We saw the same patterns, so we also wanted to build this simulation layer.”
On Wednesday, the startup announced it had raised $13 million in seed funding to make that happen. This round was led by Team8, with participation from U&I, Velocity, and Mercer.
Similar to an end-to-end model trained to drive a car, AI agents are probabilistic systems operating in unpredictable environments. Agents responsible for booking travel may ignore spending policies when pressed by customers. Alternatively, if an agent is tasked with querying a single database, it can reliably provide an answer, but combining information from multiple datasets tends to be overwhelming. Long conversations with customer service bots can overwhelm agents with too much context and lead them in the wrong direction.
To solve this, Plurai creates a digital twin of a company’s electronic systems, including knowledge bases and tools that agents can access in production. It then maps all possible flows of the agent and uses that graph to prepare a model that includes evaluation and remediation for possible edge cases. The platform works with existing agent tools such as Claude Code, Codex, Copilot, and Cursor.
But Kadal says the company’s real promise is to use the data it collects to post-train small, zero-gravity models tailored to individual customer needs. Tests conducted in collaboration with Nvidia and Nebius have shown that the company’s models perform better than lower-cost, mid-tier Frontier models.
“You can take a large language model, take a task, use simulation to break it down into smaller tasks, train an (open-weight) model, use simulation to verify that the quality is better than or equal to the frontier model, and you can save an order of magnitude or two,” he said.
Lia Miyala, a Google DeepMind researcher who backed Plurai as an angel investor, said the company’s advantage lies in understanding failure modes in production and generating synthetic data to tune models. Plurai’s approach could be a real advantage as companies seek to optimize their AI spend. Miyara said that’s because IT leaders see that “the frontier-ready model won’t deliver with my particular data, and these costs aren’t sustainable over the long term.”
Plurai hopes the similarities between AI agents in physical and digital environments will help make predictions. Although powerful general-purpose AI models are giving robots attractive capabilities, they are not yet reliable enough to be widely deployed. Meanwhile, robots equipped with AI models built for specific tasks are now operating in the field, working for customers and collecting valuable real-world data.
“My view is that no matter how smart OpenAI and Anthropic models are, and they will get smarter, they will not be able to understand your business or your policies,” Kadar said.
If you make a purchase through links in our articles, we may earn a small commission. This does not affect editorial independence.
