VITecNet@programming.dev to Linux@lemmy.ml · 10 个月前GNOME AI Virtual Assistant "Newelle" Reaches Version 1.0 Milestonewww.phoronix.comexternal-linkmessage-square27linkfedilinkarrow-up167arrow-down112cross-posted to: linux@programming.dev
arrow-up155arrow-down1external-linkGNOME AI Virtual Assistant "Newelle" Reaches Version 1.0 Milestonewww.phoronix.comVITecNet@programming.dev to Linux@lemmy.ml · 10 个月前message-square27linkfedilinkcross-posted to: linux@programming.dev
minus-squarefelsiq@piefed.ziplinkfedilinkEnglisharrow-up9·10 个月前Did you use a heavily quantized version? Those models are much smaller than the state of the art ones to begin with, and if you chop their weights from float16 to float2 or something it reduces their capabilities a lot more
Did you use a heavily quantized version? Those models are much smaller than the state of the art ones to begin with, and if you chop their weights from float16 to float2 or something it reduces their capabilities a lot more