llama.cpp merges FP8 KV-cache support
4.4/10Significance
The llama.cpp project merged pull request 28392 on Nov 10, 2026, adding OCP-FP8-based KV-caches. Run-time static per-tensor scales are supported but optional.
What changed
The change has 24 commits and new end-to-end tests for conversion, loading and context shifts. The author disclosed pairing with Codex on it.
Why it matters
Smaller KV-caches can let local models run longer contexts in less memory.
What remains uncertain
The pull request page gives no speed or memory measurements in the facts we confirmed.
What to watch
Watch for the feature appearing in tagged releases.
Source
Why this ranks here
A useful open source change with limited wider impact.
- Impact4
- Reach5
- Novelty4
- Commitment5
- Coverage quality4
- Future potential5