What it does
The skill runs LLM inference on CPU, Apple Silicon and consumer GPUs that do not need NVIDIA hardware. It supports GGUF quantization from 1.5 to 8 bits, which the publisher describes as reducing memory use. The publisher reports a 4-10x speedup over PyTorch on CPU.
It suits edge deployment, M1, M2 and M3 Macs, AMD and Intel GPUs, and setups where CUDA is unavailable. The publisher is davila7 and the skill is released under the MIT license.
Safety
Our team has not reviewed this resource. Only automated rules checked it; resources with high-risk signals are not published. These are signals, not a guarantee that installing it is safe. Try it in a sandbox project first.
- Runs code or changes an assistant's behaviour
Sources
- AI Templates (aitmpl) first seen 13 Nov 2026 · last seen 18 Nov 2026 · its page on the source · figures reported by the source: aitmpl downloads 64
inference-serving-llama-cpp
Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.
v1.0.0As shown on www.aitmpl.com when we last read it