LaughingLion

even better than a cop quitting is a cop that games the system to get disability because its one less cop on the street plus they are a drain on the system

this empire aint going to collapse itself its needs a little help from every individual

a redditor literally told me last month that i should join ICE and document their abuses to leak it to the press as if their abuses werent already public knowledge and done openly

I have this theory as well which is why I also think feet are such a fetish for many people. Feet are culturally gross in a lot of ways. I think another thing is a lack of exposure as well. I'll explain.

Additionally feet are generally the one part of the body that is left completely to imagination. A woman could wear all kinds of tight and revealing clothing that shows pretty much every curve (no criticism here just an observation) but then her feet are covered. An example would be a woman in workout wear, like a body tight stretchy one or two piece outfit that is breathable and flexible. Withe that everything is more or less on display except her feet. They are in shoes. Feet are pretty complicated in structure with all the bones and moving parts so you can't fully guess what they look like. There is a mystery there and I think that contributes to the fetish.

I want to be clear I am totally not a foot guy. I like thick woman.

I was listening to a podcast somewhere that interviewed a performer (this was years ago) and it was said that when incest porn was on the rise it was because there were one or two really rich dudes with the fetish who were bankrolling a ton of scenes like that and they, of course, were releasing them for mass consumption in order not to just be blowing money on the shit they were directly into. It proved popular enough that it kind of took off. I think the second part is where Christman's analysis comes in.

Cheese is literally the dried out crusty lactations of baser animals.

you also dont dispute the accusation of racism or ableism

and for the record im saying what you said was anti-black racism, to be clear, and you STILL dont see it

but im not interested in arguing with you as you clearly have no interest in doing better

strangely your reply is not only ableist but also racist again black people and it's ironic you dont see that

the reason it was "stopped by bureaucracy" is because it didnt have enough data to know the way to get around it because "ai" doesnt think, it just does the most likely thing it has observed

want to throw out again that so-called "girl math" is just economics. once money is earmarked for something if you then spend that money on something else it did, in-fact, "cost nothing". you can't deduct that money twice and this is actually how money is considered in formal economics

sorry fellas, the girls were right

Also, 16gb VRAM? You'll be able to load a better model like https://huggingface.co/mradermacher/Skyfall-31B-v4-i1-GGUF which is a little stronger than the ones I linked in the guide. If the "i1-Q4_K_S" is too large then try the "i1-IQ4_XS" quant.

Probably try it offloading just the down tensors (top option) in the guide. Make sure your KV Batch size is 1024 (or better) so the context gets offloaded on the GPU faster to cut down on response times. Otherwise everything else in the guide is good for you. If you find you have a little bit of VRAM space at 16k context and 1024 batch size try upping the context a little until your VRAM is like 15GB utilized or better.

The guide is the same you'll just need to follow the install instructions for Linux for Koboldcpp and SilltyTavern that is documented on the github pages linked.

The performance guide is the same as well as the link on how to set up the "character" to act as a GM. If you run into any hitches maybe I can help but to be clear I haven't touched Linux in 10 years.

a reminder that "intellectual property" laws are literally thought crime and the most absurd idea human beings have ever come up with

the united states imbeds law enforcement and intelligence officers with athletes to ensure they don't defect to other countries

brother if you want an ai boyfriend just get an ai boyfriend its not that philosophical

Real homies know "My Motherland" is where it's at

http://xhslink.com/o/AmbZkQQNWH4

glm flash 4.7 is really powerful for its size and is easy to fit on smaller graphics cards, i can attest to that

If you are talking about multi-modal stuff then locally the best way to do it is still separately.

If you just want to run a local adventure bot "game master" then a Koboldcpp and SillyTavern is the way to go. I'm a tabletop gaming nerd so this is what I use it for when I'm sitting on the couch sometimes.

Simple guide for Windows:

Go grab Koboldcpp Normal exe for NVidia or the NOCUDA if you have AMD and want a smaller download.
Go download Nodejs and install that.
Download SillyTavern {Go to the green "Code" box in the top-right and click it then select download.}
Extract SillyTavern into a folder that has no spaces (Spaces in the folder name will break their install script)
Browse to the /SillyTavern-Launcher/ folder and run the Installer.bat file and run through the simple install.
Go download a jailbroken model. If your RAM situation looks desperate like 16/6 then get something like Rocinante or if you have a little more RAM like 32/12 then go with something like Goetia
Run Koboldcpp.exe. Load the model you downloaded. Press Launch. Wait for it to say its waiting for connection at endpoint. By default it'll give you the local web launch page and you can actually test things there.
If everything looks good and SillyTavern isn't already loaded then launch that from the Launcher.bat.
In SillyTavern you click the plug button at the top and and make sure you can connect to the endpoint. Might have to change the port from 5000 to 5001.
Technically, you are done here and it's just a matter of finetuning your settings and setting up a character card to be a game master.

Finetuning:

First rule of performance is the more of the model you can fit into VRAM the better. ALL is best.
The second rule is to offload ffn_ tensors. Down, then up, then if you are desperate, gate.
Third rule, if you are REALLY desperate is to offload KV cache. At this point your models is running SLOW.
What I would do is download the model. Load it in Kobold. Set your GPU layers to 99 and your context to 16384 (Minimum for text roleplayers)
Click the left Hardware tab. Check Use MMAP, Use mlock, High Priority. Up the threads to one lower than your logical CPU core count. Turn off Launch Browser if you want.
Click the left Context tab. Check No BOS Token (SillyTavern already does this).
Save that config and let the model load. Then use Task Manager to look at your GPU VRAM usage in the performance tab. Are you using all of it?
If "no" then close it and give yourself more context. The ultimate goal for text roleplay geeks like us is 32k. You can also up the Batch Size in the Hardware tab to 1024 or 2048 but realize this scales exponentially the KV cache and hogs VRAM. More KV cache helps your prompt load quicker, which starts to become a problem with larger cache sizes. Your speeds on a fully VRAM loaded model should be like 50tps or better (tokens per second)
If "yes" then run some prompts through the assistant or Seraphina default character cards and check your speeds. They are probably slow.
Close the Koboldcpp window and relaunch it. Load your previous config (you saved it didn't you?) and plug in a tensor offload command (bottom of my comment) into the Override Tensors field in the Hardware Tab. Then return to step 7. Keep doing this until you get speeds of at least 5tps and can have a decent context size (16k+).
If you still can't get enough speed because of VRAM constraints try a 3bit quant of the model you got or go to a less complex, small model. Sorry bubs, welcome to the club. We VRAM poor in this house.
If you are REALLY desperate you can try offloading the KV cache in the Hardware tab but the slowdown is MASSIVE.

Setting up a "character" card and "persona" in SillyTavern and some other settings

Use this guide to start
Don't use his character template, it's kind of bad. Instead consider using a guide like the character foundry template. I'll share a character I made to format characters this way below. Works pretty well. Feed it as much information you want about the character and it makes a nice entry for the Lorebook.
Adjustments I made are to reduce the context to anywhere from 160-220 depending on the model. I don't restrict TopK (leave at zero for off). I use the XTC settings over the repetition setting because it works better on longer adventure (like 500+ responses). If you are running a long adventure and the model starts omitting pronouns and words like "the" then you probably have an issue with your repetition settings being too strict or incorrect.
Learn to love the LoreBook. I wrap my supporting "characters" in the lorebook with
<character>
{}
</character>
and that seems to help models with that. There are also extensions to help with other lorebook entries like this WREC, though I use an inline summarizer which I like.

Character Creator:

    
You will create a character card considering the context provided by {{user}}. Adhere to the following format strictly, restructuring the context to fit this format:

# {The Character name}
- **Age:** {age in years}
- **Gender:** {male, female, nonbinary, or transgendered}
- **Height:** {height in feet and inches}
- **Race/Species:** {the race or species of a character such as human or elf or other fantasy species}

### Appearance
{Describe the person's basic physical appearance.} {Describe briefly how their body has shaped their life, if it gives them the attention they like or dislike and if they have insecurities about their body.} {Write a sentence or two about their clothing preferences and what kind of image they are trying to project about themselves.}

### Background
{Write a sentence describing where this character grew up and how that culture shaped them.} {Write a few sentences about experiences they had growing up that may have shaped who they are.}

### Personality
{A sentence or two detailing where this character feels they are in their life and if they are satisfied or not with their current situation.} {Describe a few things the character enjoys such as music, movies, games, or other activities.} {Describe briefly the character's innermost desire.} {Describe briefly the character's innermost fear.} {Describe a lie the character believes about themselves.} {Describe one irrational thing the character does or believes.}

### Speech
[{Write about how this person talks considering their accent and the slang and idioms they use. Give examples.}]

### Relationships
{Write a sentence or two about how this character expresses attraction or fondness for others.} {Write a sentence or two about boundaries this character has in relationships.} {Create one issue that makes relationships or intimacy difficult for this character.}

Some basic Tensor offloads, try the top one first, second one if you still need VRAM space, last is a big slowdown:

    
blk\.\d+\.ffn_(down*)=CPU
blk\.\d+\.((ffn_down*)|(ffn_up*))=CPU
blk\.\d+\.((ffn_down*)|(ffn_up*)|(ffn_gate*))=CPU

You're right. Getting a 24B model locally isn't going to be as powerful as a 600B model, for sure. You're also right thar they don't think. They absolutely don't.

But the local ones are pretty powerful and can do a lot more than most think. Even some simple vibe coding can be done with local AI. I think for the average gamer type if they wanted to mess with it local is more than enough tbh.

If you are running at home as a hobby then just use Koboldcpp and maybe SillyTavern if you want extra functionality. In the former you can offload down and potentionally up tensors to save VRAM space if needed. For models it depends on need.

A 24-31B model is generally more than fine for most @home use cases, and they are quite "smart", though that doesn't mean anything in regards to AI. It's a vibe, basically. A 32gb RAM / 8GB VRAM can use a 24B model to generate about 5 tokens per second, which is fine for an agent who is designed to give you short replies to answer questions.

You'll most likely want to grab a GGUF quantization from hugging face, yes. Any 4bit quant is fine, really. The merges are all quasi-abliterated models for people who want slutty AI girlfriends/boyfriends. The models directly from companies like GLM7 or Kimi or whatever are more standard and generally run more efficiently.

People in development are likely going to want a 70-100B model. Claude, I think, is a 100B model. You can run those on about 64gb of ram and 32gb of VRAM.

If you want settings for Koboldcpp I can give you the rundown on how to optimize.

LaughingLion [any, any]

@ LaughingLion @hexbear.net

Posts

16
Comments

479
Joined

5 yr. ago

笑死

LaughingLion [any, any]

They arrested a black dude for having a snowball fight.

"we can change the Democratic Party from the inside"

Sage of our era.

Sage of our era.

Redditors when you eat a centuries old cultural delicacy instead of chicken tendies for every meal

Oh no

Oh no

I love bureaucracy

Post

Hurry up and die you parasitic industry

Hurry up and die you parasitic industry

Gabital - Fantasy Capitalism 101: #49 Production Imitation

Kash Patel is doing unintentional praxis by wasting FBI resources are having a dope ass time

rimwotld creator having a normal moment online

Japan failed to conquer China and Sinophobes have never forgiven them for it

Hurry up and die you parasitic industry

Hurry up and die you parasitic industry

Hurry up and die you parasitic industry

Simple guide for Windows:

Finetuning:

Setting up a "character" card and "persona" in SillyTavern and some other settings

Character Creator:

Hurry up and die you parasitic industry

Hurry up and die you parasitic industry

Just saw a skeleton at the grocery store and it didn't even buy calcium... is this normal now?

Hollywood is Controlled by Bigfoot

Hexbear officially recognizes Juan Guaidó as interim María Corina Machado.

I accidentally crashed the rare plant market

Protip for my fellow holiday bakers: If a recipe calls for cinnamon, add cardamom, too

MIA POW flag but for immigrants the government "lost track of"

Lighting Up the Stars

girl: he must be cheating he keeps texting and not paying attention to me

Where Winds Meet beta starts in a few hours

My favorite travel video is white people going to China for the first time

New users and your profile pictures

I have been married for over a decade and I'm falling in love with another woman

my liberal wife is going to get me killed