OpenAI made its Decisions API model widely available to users for the first time, entered the market for fast and cheap intelligence, and recently helped grow TypeSafe, a startup founded by former Frontier Labs researchers, into a $7.6 billion company.
TypeSafe’s Jev model has created a category of deep learning models that act as super classifiers. Given a set of options, the model can quickly make a choice and provide a confidence rating for that decision. Software developers quickly realized that these decision models, or System One models, could replace LLMs in a variety of workflows, allowing them to operate more quickly and at dramatically lower cost.
Nikunj Handa, product lead for APIs at OpenAI, told TechCrunch that his colleagues looked at Jev and said, “Wow, this has a million use cases.” Handa’s team did their best to deliver their own version. Rather than building a specially tailored decision-making model, the team took ChatGPT Luna and reconfigured it to speed up inference and lower cost.
“The big advantage of the Decisions API is speed, which has gone from 1-3 seconds to 100, 120, 180 milliseconds,” Handa says. “Maybe in the future we’ll get special training to make our models even better in ways like this. But for now, it’s just plain old Luna getting the job done.”
Announced at the company’s developer day on September 29, the Decisions API became available to all customers on Thursday. OpenAI added new audio and visual modalities to the public version and targeted inference resources geographically to ensure lower latency in regions around the world.
In initial trials with customers such as Box, Datadog, Ramp, and Speak, engineers used the API to replace OpenAI’s LLM with higher throughput. Andrew Hsu, CTO of Speak, which develops AI language tutors, said the Decisions API reduced the time between actions from 1.4 seconds to 0.13 seconds. OpenAI also used the product to improve the responsiveness of its Dots voice assistant.
Still, it’s not clear what constitutes a good decision-making model other than “what works for me.” Optimizations across intelligence, cost, coordination, and speed appear to vary by workload. TypeSafe CEO Diogo Almeida doesn’t trust benchmarks and encourages people to try the models in their own work, but even TypeSafe released its own ranking last week to coincide with the release of its latest Jev model, which includes new features like support for images and smart if statements.
Public benchmarks such as the AIM System One Index and Artificial Analysis Decision Index tend to favor D1 Ultra, a new decision model released by Liquid last week, and Base Lab’s Elephant-1. The latest Jev model remains competitive in both indices at 7th and 3rd place, while OpenAI’s decision-making model has dropped to 8th and 14th place, respectively. However, TypeSafe’s products are currently the most widely asked about on OpenRouter.
However, the most interesting Jev benchmark is JevMan. This is a benchmark for a decision model to play Pac-Man. The latest Jev model is at the top of that ranking. Felix Wunderlich, chief growth officer at Opper AI, which built the benchmark, said the classic game is “an interesting test because it’s a nonstop series of high-stakes decisions. At every branch point, the model has to decide where to go within a few hundred milliseconds.”
Where OpenAI’s more capable models could potentially have an advantage is in the multimodality of image classification: “Luna is very powerful and shows better calibration,” Handa told TechCrunch, adding that OpenAI customers also want the Decisions API to be more flexible when using images as output.
Handa also noticed further speed improvements that enabled new use cases. “Can we make this something like 60 milliseconds? And can you imagine calling an API for every single frame, like a video?”
As these products become faster and cheaper, we can hope that they will live up to the name of Jeb, the economist who explained how resources can become more ubiquitous by being used more efficiently. Or, as Handa sees it, we’ll see an expansion of a “crazy set of use cases” that were hidden until decision models revealed them.
If you make a purchase through links in our articles, we may earn a small commission. This does not affect editorial independence.
