Thanks! I’m experimenting with my laptop with 16GB RAM and no GPU/VRAM. I installed llama.cpp and am testing Gemma 7b Q5 but it’s not answering prompts correctly. It’s analyzing the prompt and not answering the question, or it gives me a poem haha. Trying to figure it out.
Any lightweight model you recommend for just chat experimenting for now? Can they connect to the internet?
deleted by creator
Thanks! I’m experimenting with my laptop with 16GB RAM and no GPU/VRAM. I installed llama.cpp and am testing Gemma 7b Q5 but it’s not answering prompts correctly. It’s analyzing the prompt and not answering the question, or it gives me a poem haha. Trying to figure it out.
Any lightweight model you recommend for just chat experimenting for now? Can they connect to the internet?
deleted by creator
thanks! i’ll try that one out. i’m experimenting with gemma v2 2B for now.