gemma-2-2b

Gemma 2 2B is the smallest model in Google's second-generation Gemma family of open-weight decoder-only language models. Built on the same research stack as Gemini, it is a text-to-text model designed for question answering, summarization, reasoning, and general text generation. At roughly 2 billion parameters (about 2.6B including embeddings), it targets deployment on laptops, desktops, and modest cloud instances where larger models are impractical.

Architecturally, Gemma 2 applies interleaved local sliding-window attention (4,096 tokens) and global attention (8,192 tokens), grouped-query attention (GQA), logit soft-capping, and pre/post RMSNorm. The 2B variant was trained with knowledge distillation from a larger teacher on 2 trillion tokens of web, code, and math data, rather than standard next-token prediction alone. The model supports an 8,192-token context window and is released under the Gemma Terms of Use with open pre-trained weights.

On standard benchmarks, Gemma 2 2B substantially outperforms Gemma 1 2B across MMLU, GSM8K, BBH, and coding tasks, delivering strong capability for its size. An instruction-tuned variant (gemma-2-2b-it) is also available for chat and assistant use cases. Gemma 2 2B was released on July 31, 2024, alongside ShieldGemma and Gemma Scope.