Llama-4-Scout-17B-16E

Llama 4 Scout (17Bx16E) is Meta's natively multimodal mixture-of-experts foundation model, released in April 2025 as part of the Llama 4 herd. It activates 17 billion parameters per token from a total of 109 billion across 16 expert modules, using early-fusion architecture to jointly process multilingual text and images. The model was pretrained on approximately 40 trillion tokens of multimodal data with a knowledge cutoff of August 2024, supporting 12 languages including English, Arabic, Hindi, Spanish, and French.

A defining capability is its industry-leading 10-million-token context window enabled by iRoPE positional encoding, extending far beyond Llama 3's 128K limit. This supports multi-document summarization, large codebase reasoning, and long-horizon user activity analysis. On standard benchmarks, the pretrained Scout scores 79.6 on MMLU, 58.2 on MMLU-Pro, 50.3 on MATH, and 67.8 on MBPP, outperforming all prior Llama generations while fitting on a single H100 GPU via INT4 quantization.

Llama 4 Scout is released under the Llama 4 Community License for commercial and research use. The BF16 weights total approximately 218 GB and are designed for adaptation via fine-tuning for natural language generation, visual recognition, image reasoning, and captioning tasks. Developers can deploy with on-the-fly INT4 or INT8 quantization to minimize VRAM requirements without significant quality degradation.