What it does
The skill provides PyTorch-native distributed pretraining of large language models with torchtitan. It combines 4D parallelism (FSDP2, TP, PP and CP) with Float8, torch.compile and distributed checkpointing.
It is intended for pretraining Llama 3.1, DeepSeek V3 or custom models at scale, from 8 to 512+ GPUs.
Safety
Our team has not reviewed this resource. Only automated rules checked it; resources with high-risk signals are not published. These are signals, not a guarantee that installing it is safe. Try it in a sandbox project first.
- Runs code or changes an assistant's behaviour
Sources
- AI Templates (aitmpl) first seen 13 Nov 2026 · last seen 18 Nov 2026 · its page on the source · figures reported by the source: aitmpl downloads 30
model-architecture-torchtitan
Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.
v1.0.0As shown on www.aitmpl.com when we last read it