autoround (optimized for intel but works on amd) integer quantization provides good CPU performance, and good accuracy benchmarks.
autoround (optimized for intel but works on amd) integer quantization provides good CPU performance, and good accuracy benchmarks.
github.com
GitHub - intel/auto-round: 🎯An accuracy-first, highly efficient quantization toolkit for LLMs, designed to minimize quality degradation across Weight-Only Quantization, MXFP4, NVFP4, GGUF, and adaptive schemes.
