Meta-Llama-3-8B

Meta-Llama-3-8B is the 8-billion-parameter base (pretrained) model in Meta's Llama 3 family, released in April 2024. It is an autoregressive language model built on an optimized Transformer architecture with Grouped-Query Attention (GQA), pretrained on over 15 trillion tokens of publicly available data with a knowledge cutoff of March 2023. The model accepts text-only input and generates text and code, and is intended for commercial and research use as a foundation for fine-tuning and downstream adaptation.

On standard benchmarks, the base 8B model achieves 66.6% on MMLU (5-shot), 35.36% on MMLU-Pro, 61.1% on BIG-Bench Hard (3-shot CoT), 78.6% on ARC-Challenge (25-shot), 45.9% on AGIEval English, and 58.4 F1 on DROP (3-shot). It substantially outperforms Llama 2 models of comparable size across knowledge, reasoning, and reading comprehension tasks. An instruction-tuned variant (Meta-Llama-3-8B-Instruct) is also available for dialogue and assistant use cases.

The model is released under the Meta Llama 3 Community License, allowing free commercial use subject to license terms (including attribution requirements and restrictions on using outputs to train competing LLMs). Weights are available on Hugging Face in BF16 format (~16 GB). The model supports an 8K token context window and can be run with Transformers, vLLM, or the original llama3 codebase.