Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Banaxi-Tech 
posted an update about 17 hours ago
Post
1,322
We're introducing KV1 1.2B.
Its our first KV1 model using a 2 bit KV cache, based on LFM 2.5 1.2B.

KV1 1.2B retains 75.8% of original intelligence.

More details: klein-ml/blog

Model klein-ml/KV1-1.2B

2 bit is a lot of quantization, how does the model hold up on long conversations and benchmarks, im curious