I’m a noob to local LLMs. I want to use an LLM to create Python and Bash functions from well-defined specs written by me. I’ve used Claude’s Sonnet for this mostly.

I’m a light user of LLMs and almost never hit the request limit on Claude.

It seems like cloud LLMs are in a price war right now and last-gen capabilities are bottoming out in cost? Is that correct?

I pay about $0.12 per kWh, probably going up as more AI data centers get built.

Hardware I have:

  • <16GB VRAM - AMD BC-250 APU - “16 GB total with approximately 12 GiB assigned to GPU UMA and 4 GiB left for the OS” (hardware unlocking changes by the day)
  • 8GB RAM - 11th gen Intel laptop with Xe graphics
  • 8GB RAM - Pixel 7a
  • 32 GB DDR3 - 2nd gen intel - doubt this does anything

Hardware I’m eventually selling:

  • 8GB VRAM RTX 3070 + 32 GB DDR5 + 1TB NVMe SSD - AMD Ryzen 5 7600X CPU - putting this here in case it’s substantially better than the BC-250

Among this hardware, I should look for a model that fits on the BC-250?

  • BeefAndPoultry@lemmus.orgEnglish
    8·
    13 days ago

    8GB VRAM RTX 3070 + 32 GB DDR5 + 1TB NVMe SSD - AMD Ryzen 5 7600X CPU - putting this here in case it’s substantially better than the BC-250

    Yes this can run Qwen 3.6 35b-a3b pretty nicely! And they might be releasing an updated version of that soon. Your BC-250 only has 16GB total which is not enough for 35b.

    I also have 32GB RAM and 8GB VRAM, my computer is a little slower than yours, see my guide: https://lemmus.org/post/24235317

    For the BC-250 you might try smaller models like Ling 3.0 Tiny, Ornith 1.5 9b, or Gemma 4 12b QAT

    For your 8GB RAM devices, you can run Gemma 4 e4b QAT, Qwen 3.5 4b, or maybe Ling 3.0 Tiny

    I’m a noob to local LLMs.

    Use Unsloth Desktop or llama.cpp. Then you can connect Zoo Code to it, that’s a VSCode extension which I like for programming with my local LLMs.