Llama-3.2-1B-Instruct

Llama-3.2-1B-Instruct is Meta's smallest instruction-tuned model in the Llama 3.2 family, designed for efficient on-device and edge deployment. With 1.23 billion parameters and a 128K token context window, it targets multilingual dialogue, summarization, agentic retrieval, and lightweight instruction-following tasks where larger models are impractical.

The model was pretrained on up to 9 trillion tokens of publicly available data, using knowledge distillation from Llama 3.1 8B and 70B teacher models via logit-level targets combined with pruning. Post-training followed the Llama 3.1 recipe with multiple rounds of supervised fine-tuning (SFT), rejection sampling, and direct preference optimization (DPO). It uses an optimized Transformer architecture with Grouped-Query Attention (GQA) and shared embeddings for improved inference efficiency.

On standard benchmarks, the model achieves 49.3 on MMLU, 44.4 on GSM8K, and 59.5 on IFEval, outperforming many comparably sized open and closed chat models. It supports eight languages (English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai) and is released under the Llama 3.2 Community License for both research and commercial use.