Hey guys,

What’s currently the best LLM for low-VRAM machines with only 6 GB VRAM? I’ve got 32GB RAM as well.

I’m experimenting a little with SillyTavern and I’m curious which model gets the most out of my setup. Should be multilingual and suitable for “casual chatting”.

I know I will probably not get very far with this, but I’m still interested in how far we’ve already come.

(Using KoboldCPP if that matters).

~sp3ctre

  • PetteriPano@lemmy.worldEnglish
    8·
    3 days ago

    I’m running gemma-4-e4b on my 8GB machine. I’ll drop down to e2b on CPU. It’s probably the best you’ll get. 140 languages, vision, decent at agentic work. Not great at code.