In partnership with Bench Labs

What is "FWKV"???
FWKV (call it Feed-forward Weighted Key Value, or Floored Weighted Key Value, its up to you) is a efficient language model architecture proposed on May 13, 2026 by FlameF0X.

It started as a fun side project inspired by RWKV. Then I (We, FlameF0X) realised that for a 50 million parameter language model is fast, like, faster than any Transformers base Dense or MoE 50m model without needing a custom kernel to make it run faster.

Now, this architecture was not intended to be similar with RWKV intentionally, it was accidentally because of the idea of keeping it simple. That lead to its RWKV like design. (The name is intentional -ish, since it uses FFNN and WKV and I had no idea how to name it, so FWKV was the name)

Why should I use FWKV?
You should not use it, yet, since its still in its early stages and its very unstable. But its fast.

About this organisation
We do not self proclaim as a lab as the rest of the small or "inovative" ai/ml organisation. This is a small side project on the idea to make language models very simple while trying to achive RWKV like language modeling quality.

Uh, I made it so I can keep track of this project much more easier.

The flagship models of this project are gonna be:

• Myosotis — a 100 million parameter language model for quick and simple questions.
• Cyanus — a 500 million parameter model for everyday tasks.
• Delphinium — a one billion parameter model for more complex tasks.

We don't gatekeep models behind the gate system of Huggingface, even if they are just checkpoints, we allow you to use any model and checkpoints.