aya-23-8B

Aya-23-8B is an open-weights instruction-tuned multilingual language model released by CohereLabs (Cohere For AI). It pairs a high-quality Cohere Command-family pretrained backbone with the Aya Collection instruction data, targeting depth over breadth by focusing on 23 languages that were included during pretraining. The model supports Arabic, Chinese, Czech, English, French, German, Hebrew, Hindi, Indonesian, Italian, Japanese, Korean, Persian, Polish, Portuguese, Romanian, Russian, Spanish, Turkish, Ukrainian, and Vietnamese, among others.

Architecturally, Aya-23-8B is an 8-billion-parameter decoder-only Transformer using grouped-query attention, RoPE positional embeddings, SwiGLU activations, and a 256K-token BPE vocabulary. It was instruction fine-tuned with an 8,192-token context window on a multilingual mixture spanning human annotations, synthetic data, and translated ShareGPT/Dolly prompts. On the official Aya 23 evaluation suite, it leads comparable sub-13B models on multilingual MMLU (48.2% average across 14 languages), MGSM mathematical reasoning (36.6% average across 7 languages), and held-out discriminative tasks (67.6% average on XCOPA, XStoryCloze, and XWinograd), while also delivering strong translation and summarization quality.

Released under the CC-BY-NC license for research use, Aya-23-8B is designed for multilingual chat, translation, summarization, and general text generation on consumer-grade hardware. It consistently outperforms prior multilingual baselines such as Aya-101-13B, Mistral-7B-Instruct, and Gemma-1.1-7B-it in both automatic benchmarks and human/LLM preference evaluations across many non-English languages, making it a practical choice for multilingual applications that cannot fit larger models.