Meta-Llama-3-70B-Instruct

Meta-Llama-3-70B-Instruct is Meta's instruction-tuned 70-billion-parameter large language model from the Llama 3 family, released in April 2024. Built on an optimized Transformer architecture with Grouped-Query Attention (GQA), it was pretrained on over 15 trillion tokens and fine-tuned with supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) for dialogue and assistant use cases. The model supports an 8K token context window and is designed primarily for English text generation, including code.

On standard benchmarks, Llama 3 70B Instruct ranks among the strongest open-weight chat models of its generation. It scores 82.0 on MMLU (5-shot), 93.0 on GSM8K (8-shot, chain-of-thought), 50.4 on MATH (4-shot, CoT), 81.7 on HumanEval (0-shot), and 39.5 on GPQA (0-shot). On the harder MMLU-Pro benchmark it achieves 56.2. These results substantially outperform Llama 2 70B and compete with several proprietary systems on reasoning, math, and code tasks.

The model is released under the Meta Llama 3 Community License, with weights available on Hugging Face (~140 GB). It is intended for commercial and research applications including chat assistants, content generation, and code assistance, subject to Meta's acceptable use policy. Meta also provides safety tooling such as Llama Guard 2 and Code Shield to support responsible deployment.