What it does
The skill applies post-training 4-bit quantization to LLMs. The publisher describes it as keeping accuracy loss small, citing perplexity degradation below 2%. It integrates with transformers and PEFT for QLoRA fine-tuning.
The publisher points to deploying large models, such as 70B and 405B parameter models, on consumer GPUs. It is also described for cases that need about a 4× reduction in memory use, or faster inference, with a stated 3-4× speedup compared with FP16.
The supplied facts do not describe setup steps or runtime requirements. The resource is released under the MIT license.
Safety
Our team has not reviewed this resource. Only automated rules checked it; resources with high-risk signals are not published. These are signals, not a guarantee that installing it is safe. Try it in a sandbox project first.
- Runs code or changes an assistant's behaviour
Sources
- AI Templates (aitmpl) first seen 13 Nov 2026 · last seen 17 Nov 2026 · its page on the source · figures reported by the source: aitmpl downloads 28
optimization-gptq
Post-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or for faster inference (3-4× speedup) vs FP16. Integrates with transformers and PEFT for QLoRA fine-tuning.
v1.0.0As shown on www.aitmpl.com when we last read it