Llama-3.2-1B

Llama 3.2 1B is Meta's smallest open-weight text-only language model, released in September 2024 as part of the Llama 3.2 family. With 1.23 billion parameters and Grouped-Query Attention (GQA), it is designed for edge and mobile deployment where larger models are impractical. The model supports a 128K token context window and was pretrained on up to 9 trillion tokens of multilingual data, with knowledge distilled from Llama 3.1 8B and 70B models during training.

As a base (pretrained) model, Llama 3.2 1B is intended for fine-tuning and adaptation rather than direct conversational use. It officially supports eight languages—English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai—and can be extended to additional languages via fine-tuning under the Llama 3.2 Community License. On standard benchmarks, it achieves 32.2 on MMLU (5-shot), 23.3 on AGIEval, 32.8 on ARC-Challenge, and 28.0 F1 on DROP.

Meta also provides quantized variants (SpinQuant and QLoRA) that reduce model size by roughly 50% and improve inference speed by 2–3x on mobile hardware, making this model well-suited for on-device applications such as summarization, query rewriting, and lightweight agentic tasks with limited compute budgets.