What it does
This skill evaluates large language models on more than 60 academic benchmarks, including MMLU, HumanEval, GSM8K, TruthfulQA and HellaSwag. It supports models served through HuggingFace, vLLM and APIs.
It is suited to benchmarking model quality, comparing models, reporting academic results and tracking progress during training. The source material does not describe further setup steps.
Safety
Our team has not reviewed this resource. Only automated rules checked it; resources with high-risk signals are not published. These are signals, not a guarantee that installing it is safe. Try it in a sandbox project first.
- Runs code or changes an assistant's behaviour
Sources
- AI Templates (aitmpl) first seen 13 Nov 2026 · last seen 17 Nov 2026 · its page on the source · figures reported by the source: aitmpl downloads 98
evaluation-lm-evaluation-harness
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
v1.0.0As shown on www.aitmpl.com when we last read it