ZML launches LLMD free inference server for many AI chips

ZML has released ZML/LLMD, a free LLM inference server designed to run open-source large language models across a wide set of chips, including Nvidia, AMD, Google TPU, Apple Metal and Intel Arc. The Paris-based startup is not open source with this release; it is launching LLMD as a free product to learn about usage before monetizing.

What is ZML/LLMD and which chips does it support?

ZML/LLMD is an inference-performance server that allows a variety of open-source large language models to run on a variety of chips, explicitly listing Nvidia, AMD, Google TPU, Apple Metal and Intel Arc as supported targets. The release is intended to break software and architecture silos that lead to vendor lock-in and to let different chips operate at their "maximum available speed, and sometimes faster," founder Steeve Morin told TechCrunch.

The product is not open source, unlike ZML’s earlier inference-focused ML framework released in 2024 and updated in March, but ZML is distributing LLMD for free initially to measure usage. Morin framed the effort as giving people back the ability to "create their own system," which he tied to achieving efficiency gains that could help AI be more widely disseminated.

How is ZML positioned in the inference market and who backs it?

ZML is a small, well-funded startup: Morin credited a lean team of 20 people for moving quickly, and the company has raised $20 million from investors including Harry Stebbings’ 20VC, >commit, AALVC, Drysdale Ventures, Xavier Niel’s Kima Ventures, Kindred Capital, LocalGlobe, and Puzzle Ventures. The startup counts high-profile founders among its stakeholders, including Solomon Hykes, Clément Delangue and Julien Chaumond from Hugging Face, and Yann LeCun, now with AMI Labs.

ZML faces competition in inference tooling from companies such as Baseten (recently valued at $13 billion), Inferact from the creators of vLLM, and RadixArk, the commercial company behind SGLang. The article notes that projects like vLLM and SGLang partially compete with LLMD, but Morin describes ZML’s ambitions as broader, including co-designing silicon with chip partners.

How does ZML connect to alternative chipmakers?

ZML positions LLMD as a way to expand chip choice for inference and to help emerging AI chipmakers. Morin cited several European chipmakers by name as potential collaborators or beneficiaries, including Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud and VSORA. He argued ZML can work with those companies "on things that haven’t been done before anywhere in the world," and said the startup maintains a good relationship with Nvidia even as it seeks to enable a mix of chips in clouds and enterprises.

Why it matters

ZML’s free LLMD foreshadows a shift in inference tooling from vendor-tied stacks toward multi‑chip flexibility, which could lower cost and energy consumption for deployments by letting operators pick less costly or more efficient silicon. The company’s approach — a proprietary, free-to-start server aimed at broad hardware support — also tests a route to growth that relies on usage data before monetization, rather than immediate open-sourcing or upfront licensing.

The presence of high-profile backers and a $20 million raise for a 20-person team signals that investors view inference optimization as high leverage, while the named competitors and partial overlaps with open-source projects show the market is already crowded and fast-moving.

What to watch

Watch two concrete signals: whether ZML converts LLMD into a paid product and when, and whether major clouds or enterprises adopt a mixed-chip approach using LLMD. The company said it intends to learn from usage before deciding a revenue path, and it has more releases planned.