What it does
This skill guides the optimization of large language model inference with NVIDIA TensorRT. It covers model quantization (FP8 and INT4), in-flight batching and scaling across multiple GPUs.
It is intended for production deployment on NVIDIA GPUs such as the A100 and H100, and for cases where faster inference than PyTorch is needed. The publisher states speedups of 10-100x over PyTorch.
The publisher's description does not list setup steps. It is distributed under the MIT license as part of the claude-code-templates collection.
Limitations and requirements
Requires NVIDIA GPUs; the publisher names the A100 and H100. The 10-100x speedup over PyTorch is the publisher's own figure.
Safety
Our team has not reviewed this resource. Only automated rules checked it; resources with high-risk signals are not published. These are signals, not a guarantee that installing it is safe. Try it in a sandbox project first.
- Runs code or changes an assistant's behaviour
Sources
- AI Templates (aitmpl) first seen 13 Nov 2026 · last seen 18 Nov 2026 · its page on the source · figures reported by the source: aitmpl downloads 60
inference-serving-tensorrt-llm
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
v1.0.0As shown on www.aitmpl.com when we last read it