it is possible to quantize kv cache in strata ? :(In llama.cpp and Unsloth’s llama.cpp, quantizing the KV cache for this model will increase its size instead of decreasing it
· Sign up or log in to comment