tiny-aya-base

Tiny Aya Base is a 3.35-billion-parameter pretrained multilingual foundation model released by Cohere Labs as part of the Tiny Aya family. Trained across 70+ languages spanning European, Asian, African, and Middle Eastern language families, it is designed to deliver balanced multilingual representations with particular strength in lower-resourced languages. As the base pretrained checkpoint, it serves as the starting point for downstream fine-tuning, instruction tuning, and regional specialization rather than direct conversational use.

The model uses an optimized decoder-only Transformer architecture with 36 layers, alternating sliding-window attention (4,096-token window with RoPE) and periodic global attention layers for full-sequence interaction. It supports an 8,000-token input context and 8,000-token maximum output. Weights are released under the CC-BY-NC license on Hugging Face, enabling local deployment on consumer hardware with modest compute requirements.

Tiny Aya Base is one of five variants in the Tiny Aya release alongside the instruction-tuned Global model and three region-specialized models (Earth for Africa/West Asia, Fire for South Asia, Water for Europe/Asia-Pacific). The family targets efficient on-device multilingual AI without cloud dependency, making it suitable for translation, summarization, cross-lingual NLP research, and adaptation in multilingual regions where larger models are impractical.