LFM2.5-24B-A2B: Built for fast on-device agents
liquid.aiWow, those are seriously impressive numbers for how intelligent this model seems to be!
One thing that this article doesn't clarify, and that I guess I really should look into, is what exactly AutomationBench tests for as compared to other benchmarks. The standout performance on AutomationBench is interesting to see, but I don't know what it really means because the model seems to score relatively close to other local models on every other benchmark.
It's possible that they may have generally capped out for some reason... but, I'd rather be bullish on a new American AI lab making it to first place among western open-weight models that can run on a common consumer device. (Closest in size that's better across that range of benchmarks is Inkling-Small I think, at 276B params, unless I'm forgetting someone.)
What does it matter what country an open weight model comes from? Please don't say you tried asking about tienamen square to a model 2 years ago and this was so scary you couldn't use it like 85% of people with this take have said...
Edit: I was wrong, I looked at the wrong model
very nice.
I can't speak to the intelligence of it yet.
these small agentic-focused models are a lot of fun to play with if you haven't yet.
That seems pretty decent. How many tokens/s do you need for a coding agent to feel like Claude or codex?
it's kind of a loaded question, this model gets a lot of tok/s but reasoning is forced on; the time-to-answer is really a combination of things like "how much does this thing want to over-reason" and "how much does this thing like tool use?" .
When running locally usually I have to trade off speed for a better model. I can run decently big models at about 2tk/s, or a pretty dumb one at 40-100tk/s
the results i've gotten so far have been a little braindead though. still playing around. bad luck with traversing filesystems remotely so far.
it seems to want to make concluding decisions very fast regardless of reasoning levels.
(The resulting pelican was underwhelming.)
I've been messing around low computing power hardware to run models locally and to me 100 is already plenty fast. You most likely won't notice a huge difference between 184 and 250, unless you're running some serious multi-age agent circus.
If Qwen had released a Qwen3.8 MoE of similar size, with a naive extrapolation its capabilities would roughly be similar to this model. So basically someone has caught up to Qwen (albeit Qwen from 1-2 months ago) in parameter efficiency.
At least for my use cases, Qwen 3.6 35B-A3B appears better than this new release. An MoE for Qwen 3.8 would be even better than that!
but i'm prepared to be surprised, maybe they'll conjure up something standalone useful this time
https://blogs.nvidia.com/blog/local-ai-rtx-spark-laptops-sof...
https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-1...
And their own benchmarks seem to show it well behind of Qwen 3.8 flash next on pretty much everything except generation speed.
Which is not going to compare well to the 3.8 Flash because it's much smaller; designed to fit quite a large model into quite a small space.
Will be interesting to see how it compares to Muse Glimmer, Qwen 3.8 27B and the older Qwen 3.6 35B (and how its image analysis performance compares to Gemma 4 12B QAT, which is still IMO the most interesting open weights model released all year).
Is that Qwen 3.8 27B? Because yes, you'd expect a plain 27B model to be better than a 24B-A2B MoE model all else being equal.
(All else isn't equal though of course - Qwen models are very, very good at coding)
Gemma-4 26BA4B would be a more similar comparison.
fast is fast (18tk/s on a Intel Core 7 155H laptop), but quality is something else
Same thing with Qwen3.6 35B A3B produced mangled garbage, but still, it didn't have errors in the code.
Exact same prompt in ChatGPT produces a half-decent model (ok, but exactly not the thing I asked) so long as animations are not involved.
Just also wanted to note that this model has a questionably-enforceable non-commercial use clause in its license. I really wish they had consulted with an IP lawyer, it’s very confusing.
But bonsai is basically trash on real agentic coding work. I've tried both bonsai models, all the quants, messed with the hyperparameters, tried to fine tune (didn't succeed), and it just kept looping or failing to finish tasks. Qwen3.6 35b-a3b is my current go to for agentic work for any systems under 24gb vram, although honestly I'm not sure the dense qwen3.8 27b is meaningfully better. The main difference I see is it does a lot more testing attempts and burns a ton of tokens + compactions, so every task it takes 1.2-2x as long as 35b-a3b does. But the quality of the results is comparable.
I've experimented with kolibri1 (78B-a3.5b), but despite the similar size in active parameters it is much slower. Even though the experts fit completely in main memory, as for the other models. I haven't tuned llama.cpp for it as much, so maybe there is still something to gain?
> What are signals in Angular?
> How does that relate to RxJS?
This was enough to loop Bonsai 2.
I do run the gpt-oss-20B fast on my 16GB vram, but it’s, limited.
The cost over the diff from the moment of beginning training to the end. It's iterative from a prior model so it starts there and becomes this release after spending occurs I presume.
For example, is liquid spending millions in energy in this case and the cost is amortized over successive training runs over the initial datacenter spend? What if it's just eating cloud costs and has no dedicated hardware, that sort of thing. I don't particularly mean this model and liquid specifically. Just for this class of smaller fast on-device range of model what is the order of the training cost in real terms?
Maybe its short sighted to ask it as if it's a singular training run and is perhaps the end state of many variations of testing thus all of the R&D training runs would have to be included to to inform what became the final release. Seems like this should be taken into consideration. If anyone happens to know it'd be appreciated!
The license doesn't really make much sense to me, because if you're already at $10m revenue, you can probably train your own (better, more tailor-suited) model, or at the very least raise money to do so.
I get ~45 tokens/s generation and ~680 tokens/s prompt processing.
The problem is ollama is not caching the prompt, I get -> forcing full prompt re-processing due to lack of cache data
So this wouldn't work in its current state in an agentic loop.
Which makes for a dangerous combination, because it will delete prod, and it will do so at 250t/s.
The actually sane default is to treat it like a junior trainee you would never trust with rights to anything critical. Use existing operating system facilities, namely user accounts and file permissions, to make everything it doesn't need protected from writes, and everything it shouldn't blithely broadcast to the world like private data and keys, also protected against reads.
You spin up your amazing startup in a day, full product and site. It's got all the users embedded in a js file, but who cares. Until people find out. Then you task the agent with fixing it, it deletes prod. You start over again.
It's beautiful, if Disney make movie about this I want to be in the credits.
When used for coding, how do you divide the work between agents? Use a relatively fast model, like this one, as the main agent that calls out to other subagents that write the actual code? But then you'd need multiple models loaded, and why not just use the coding model to do everything? Do you use hosted cloud models for coding and code review, and this local one for orchestration? I don't see the benefit of that workflow either.
I've used local models to write some scripts etc, I like the idea of local computation instead of buying tokens from the cloud, but I don't know what the various agentic models are for, if not code generation.
I see it as a niche now but it will hopefully grow in the future. The amount of information these servers are collecting about you for potential use against you is staggering and makes adware spy tech look like the good old days of privacy.
Strongest and easiest use case now would be just to hook one of these up to a search engine tool and ask your random questions to it rather than adding to OpenAI or Gemini's database about you.
When I am wrapping up work for the day, I usually set Qwen3.8-27B loose on some coding task. It's very good about continuing its agent loop until it can plausibly claim success. A lot of this work I might end up throwing away, but pretty often there's some piece of it that's worth keeping. Because it was "agentic", I end up with some useful pieces for little effort.
A less agentic model would need me involved the whole time, and while my being there would probably steer it towards initially better results, I'm not working overnight anyway. So, autonomous/agentic but not-so-smart and not-always-correct is still valuable to me.
Anyway, I've found that it's reasonably easy to constrain a "highly agentic" model to suit interactive usage, but back when they were less agentic it was annoying to get them to be autonomous. So I think more agentic is better, provided the instruction-adherence is strong enough.
But Qwen3.8-27B is a known reasonably good coding model that's dense and needs more resources than the smallest local models. What about this one, would you leave a 24B-A2B model coding overnight?
I haven't tried this particular model, but I've used smaller models for real tasks, so I wouldn't be surprised if there is something.
Perhaps there are some system prompt tweaks that could get it in line
Is that a trigger or a grey area? The company will profit from my work, and my work would be empowered by this model, is that a "use for commercial purposes" or not?