What it does
The skill applies activation-aware weight quantization to compress large language models to 4-bit precision. The publisher states a 3x speedup with minimal accuracy loss, and says the method preserves accuracy better than GPTQ while running inference faster. It also notes that the approach won a Best Paper Award at MLSys 2024.
The publisher suggests reaching for it when deploying models in the 7B to 70B parameter range on GPUs with limited memory, when faster inference than GPTQ is needed, or when working with instruction-tuned and multimodal models.
The resource is released under the MIT license. The source does not describe setup steps or runtime requirements.
Safety
Our team has not reviewed this resource. Only automated rules checked it; resources with high-risk signals are not published. These are signals, not a guarantee that installing it is safe. Try it in a sandbox project first.
- Runs code or changes an assistant's behaviour
Sources
- AI Templates (aitmpl) first seen 13 Nov 2026 · last seen 18 Nov 2026 · its page on the source · figures reported by the source: aitmpl downloads 26
optimization-awq
Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preservation, or for instruction-tuned and multimodal models. MLSys 2024 Best Paper Award winner.
v1.0.0As shown on www.aitmpl.com when we last read it