Basically the title.
For me, a dense model around 20b, with an additional engram table of 10-20b, or a sparse MoE around 80b would be amazing.
Basically the title.
For me, a dense model around 20b, with an additional engram table of 10-20b, or a sparse MoE around 80b would be amazing.
I have an M4 Pro 24GB and I still use Gemma 26B-A4B as my primary model. Beats every other model I’ve ever tried locally on most stuff except coding and tool use, so a Qwen 4 at this size would be super interesting