SocialistVibes01@lemmy.ml to Linux@lemmy.mlEnglish · 2 months agoWhich specs are as low as reasonable possible for local LLM models? Do you recommend some distro in particular?message-squaremessage-square19linkfedilinkarrow-up132arrow-down111file-text
arrow-up121arrow-down1message-squareWhich specs are as low as reasonable possible for local LLM models? Do you recommend some distro in particular?SocialistVibes01@lemmy.ml to Linux@lemmy.mlEnglish · 2 months agomessage-square19linkfedilinkfile-text
minus-squarewraekscadu@vargar.orglinkfedilinkarrow-up2arrow-down1·2 months agoDepends on how big your local LLM model is. You can easily run 2-3B param models on a potato. BUT, the model is shit.
Depends on how big your local LLM model is. You can easily run 2-3B param models on a potato. BUT, the model is shit.