What happened
On 22 September 2026, Hugging Face announced that Transformers can run quantized GGUF files through `from_pretrained(..., gguf_file=...)`. The library could previously import such a file by dequantizing the weights. On supported hardware, the new integration uses ggml kernels and keeps weights packed during computation. The team's initial focus is Apple Silicon and the Qwen3.5 family. According to the published instructions, users currently need Transformers from its `main` branch and a compatible version of `kernels`, rather than the next stable release.
For a small technical team, the practical benefit is that one GGUF file can be evaluated in familiar Python workflows: compare quantization quality on the company's own documents, use existing evaluation scripts, and, if appropriate, run `transformers serve` with an OpenAI-compatible interface. The article gives a Qwen3.5-4B Q4_K_M file size of 2.74 GB versus 8.42 GB for the BF16 variant. These are file sizes, not a promise of identical working-memory consumption.
The limits
This is not a reason to migrate a production server away from llama.cpp automatically. Hugging Face itself still recommends llama.cpp when efficient local inference is the main priority. The optimized path is currently limited by hardware and architecture. If a compatible kernel is unavailable, Transformers warns that it falls back to dequantization, increasing memory use. The published measurements were made on a MacBook Pro M2 Max with 32 GB of unified memory, and the comparison conditions are not identical: the llama.cpp figure measures decoding, whereas the Transformers figure includes processing of a short input. They do not establish equal speed for Windows, Linux, or other model families.
For a business in Russia, a sensible next test is not buying hardware on the strength of a headline. Run a set of 30–50 representative tasks on an existing Mac and record answer quality, peak memory, time to first token, and the cost of human review. Check the particular model's licence and commercial-use rights separately; GGUF format support does not change them.
Photo: SimonWaldherr / Wikimedia Commons, CC BY-SA 4.0; square crop of the original. The cropped image is available under the same CC BY-SA 4.0 terms. It shows a MacBook Pro M2 Max, the same class of device used for the published measurements.