Nvidia’s era of unparalleled market dominance is not over, but challengers and alternatives are emerging from all directions.
ZML, the buzzy French AI startup backed by Turing Award winner Yann LeCun, has released inference performance software that allows it to run a variety of open source large-scale language models on a variety of chips, including Nvidia, AMD, Google TPUs, Apple Metal, and Intel Arc.
With its newly launched LLM inference server, ZML/LLMD, the company’s ambition is to break down existing silos and make different chips available for AI use cases at the maximum speed available, and in some cases faster, ZML founder Steve Moerin told TechCrunch.
As AI becomes more integrated into our work and daily lives, the importance of optimizing inference (a.k.a. handling prompts) has outpaced rain models, but it often feels patchy behind the scenes, with software and architectural barriers leading to vendor lock-in, Morin said.
The promise of the best performance across a variety of chips is a technological feat, but it also has the potential to be a market disruptor as concerns about AI-related costs grow.
ZML wants to give enterprises and clouds the option to use a combination of cheaper and/or lower power chips. “The idea is to give people back the power to build their own systems and achieve real efficiency gains that will enable widespread adoption,” Morin said.
Such software assistance could help new AI chip makers (many of which happen to be European), Morin said, citing Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA as examples. But what’s more important to him than their region of origin is that ZML can work with them on “something that hasn’t been done before anywhere in the world.”
That doesn’t mean Morin is bearish on Nvidia. He doesn’t, and that’s partly because of the existing supply. He told TechCrunch that ZML has good relationships with AI chip giants gearing up for the rise of inference.
Inference is an area of intense investment, and this trend has been hailed as the “inference gold rush.” So ZML has competitors such as Baseten, which was recently valued at $13 billion. Inferact, by the creators of the open source project vLLM. RadixArk, the commercial company behind SGLang, is no different.
Both vLLM and SGLang partially compete with LLMD, but Morin’s ambitions for ZML are broader in scope. “We have reached a stage where we co-design the silicon,” he said. He also credited ZML’s lean team of 20 people for allowing the Paris-based startup to plan more releases and move quickly.
It also helped that this small team was well-funded relative to its size. Thanks to his track record as VP of Engineering at Zenly, which Snapchat acquired for nine figures in 2017, Morin has raised $20 million from venture firms including Harry Stebbings’ 20VC, >commit, AALVC, Drysdale Ventures, Xavier Niel’s Kima Ventures, Kindred Capital, LocalGlobe, and Puzzle Ventures.
Unlike ZML’s first public project, an inference-centric ML framework released in 2024 and updated in March, ZML/LLMD is not open source. However, it is released as a free product for the purpose of learning how to use it. “We want to measure where we are most effective (and then generate revenue) without foolishly stunting growth because we were too greedy from the beginning,” Morin said.
It is too early to tell when ZML/LLMD will become a paid product and what its implementation will look like. But the startup’s cap table confirms the attention of other founders, including Dagger and Docker founder Solomon Hykes, Hugging Face’s Clément Delangue and Julien Chaumond, and LeCun, now at AMI Labs. This is also proof that European AI startups can be built from home. “We couldn’t do ZML anywhere other than Paris,” Morin said.
If you buy through links in our articles, we may earn a small commission. This does not affect editorial independence.
