The current push for on-device AI has hit a persistent wall where developers must choose between model intelligence and execution speed. For those deploying on edge hardware, the typical workflow involves taking a high-precision model and squeezing it down through post-training quantization to fit into limited RAM. While this makes the model portable, it often introduces a noticeable degradation in reasoning capabilities, leaving a gap between the theoretical power of the architecture and the actual user experience on a smartphone or a single-board computer.

The Precision Mechanics of LFM2.5 QAD

To bridge this gap, the LFM research team has released the Q4_0 checkpoints for the LFM2.5 model series, utilizing a technique known as Quantization-Aware Distillation (QAD). Unlike standard post-training quantization (PTQ), which compresses a model after it has been fully trained, QAD integrates the quantization process into a distillation framework. This allows the smaller, quantized model to learn more effectively from the high-precision teacher model, resulting in a significantly higher retention of the original BF16 baseline performance. The results across the LFM2.5 lineup show a retention rate ranging from 96.5% to 97.4%.

Specifically, the LFM2.5-1.2B-Instruct model achieved the highest fidelity at 97.4% of its BF16 performance. This was followed by the LFM2.5-230M at 97.1%, the LFM2.5-2.6B at 96.6%, and the LFM2.5-350M at 96.5%. To validate these numbers, the team employed a rigorous suite of benchmarks focusing on reasoning, instruction following, and agentic capabilities, including GPQA Diamond, MMLU-Pro, IFEval, IFBench, Multi-IF, and BFCLv4. For mathematical proficiency, the team used GSM8K for the 230M and 350M variants, while the 1.2B-Instruct and 2.6B models were tested against AIME25, with final scores derived from the average of five repeated measurements. These checkpoints are now available via Hugging Face and are fully compatible with `llama.cpp` and any runtime supporting GGUF Q4_0 artifacts.

Breaking the Trade-off on Arm Architecture

The real-world impact of QAD becomes evident when moving from synthetic benchmarks to actual hardware. The research team tested the LFM2.5 QAD Q4_0 checkpoints across four distinct target devices: a MacBook Pro, a NucBox EVO-X2 (GPU), a Samsung Galaxy S26 Ultra, and a Raspberry Pi 5. The primary tension in these environments is the Arm CPU's memory bandwidth, which often bottlenecks decode throughput. By using QAD, the team managed to decouple the relationship between bit-depth and quality.

For the 230M and 350M models, the QAD Q4_0 checkpoints delivered a decode throughput increase of 4% to 33% while maintaining a quality level comparable to Q5_K_M (5-bit quantization). Similarly, the 1.2B and 2.6B models saw throughput gains of 3% to 14% while preserving the quality of Q4_K_M (4-bit quantization). When compared to high-performance PTQ alternatives, such as the Unsloth UD-Q4_K_XL checkpoints, the 230M and 1.2B QAD models performed at a nearly identical quality level. This means users on low-power Arm CPU devices, such as the Raspberry Pi 5 or the Galaxy S26 Ultra, can now achieve the reasoning quality of a 5-bit model with the speed and memory footprint of a 4-bit model.

This shift suggests that the industry is moving away from simple compression and toward intelligent distillation to unlock the full potential of edge silicon.

This optimization marks a critical step toward making high-fidelity, low-latency AI agents viable on consumer-grade mobile hardware.