Sabata

On the 1080ti I run gemma4:e4b to to do RAG and vision jobs. Not sure the quants and I just run it in Ollama that auto adjust the model because I’m lazy. Not 100% sure but I think mixture of experts load the parts that are needed so there easier on memory. Offloading to CPU memory or swap is a big performance hit.

I tried to run a 300b model on the main PC once, but it took half an hour to get a few words out of it. You can go past the VRAM on the card but sacrifice speed.

These are some reasons I stopped using Windows in the first place. I don’t need an evil corpo telling me what settings I must use so they can hide what there doing.

so Secure Boot and TPM 2.0 need to be on

Nah, fuck off and stick to Windows.

Spy device from evil data broker and spying company found to be spying.

Probably not, or if it runs it may take hours or days to get a full output. VRAM on the GPU really decides what will run reasonably. Try an 8 billion parameter phone sized model and see if its faster than sending a letter, then keep moving up until your setup can’t handle it.

8B models are a bit too dumb to use any tools reliably but should handle a basic conversation.
llama cpp will allow you to manually load a model and tweak settings.

unsloth will give you a basic setup with some tools, manage model downloads, and settings.

I run a 5 bit quantized Qwen 3.8 27b at home on a gaming PC with an RX 7900 XTX and RAG memory using Gemma MoE (i forget what one)running on an old 1080 ti on my Valheim server.

It takes 3-15 minutes to run depending on how many tool calls the model makes and that’s mostly memory or web scrapes since that runs on the slower hardware. I mostly just use it as a discord bot to look stuff up for me and to fetch the news.

“I wrote it down and glued it to the screen and its not fixed. Do you even know how to do your job?”

Making up demon names is fun and whimsical. Neth’sin the Ethereal Tormentor likes chin scratches, will rip strangers to shreds, and sometimes defiles laundry.

Because Samsung likes knowing when the wash is done from across the world.

I’d still say valid military target, just in case.

I just bought a hoodie and mug from them. I been on the email list for years. Solid trolling on their part.

Too tight of a schedule to cast that counter spell.

Another citizen, a pastor named Rick Bick, rushed to the podium. “Let’s just reverse that curse that was just put on our city,” he said. “That ending with hail Satan and hail dark god. That’s not welcome here.”

“Mayor, that should be done at the end,” city councilor Cara Galloway said.

“It needs to be done now, OK.” Rick said. “I appreciate what Ms. Galloway is saying but that was a curse put on our city when he said ‘Hail Satan’ and the dark god and most of you know that. And we need to just stop and reverse it.”

“So what I’m gonna do now…I’m just gonna invite us into a 30 second time of prayer — time of silence.”

“No,” Galloway said. “We are at the consent agenda. I’m sorry, but it’s time to move forward. The invocation happened.”

“As mayor of this city…I’m going to ask right now that we do a moment of silence to reflect on whatever it is we need to reflect on at this time. And then we will move to the consent agenda,” mayor Benjamin Foote said.

I should start giving incantations, that reaction is gold.

You don’t have to worry about the The Great Desk of Half Assembled Electronics and Forgotten Projects taking up half a room.

It designates a room with the right stuff in it as a guild for a mood buff. Dwarfs may petition you to build one.

Quick guild: Dig hole, live in hole, die in hole.

How did they know I like anime and tech? Couldn’t have been all the body pillows and hardware?

Uranium Fuel Rod Blast for sure.

Its got to be at least a 3.6.

Fuck ya, I want to smell like a nuclear reactor.