AI21-Jamba-Mini-1.5

AI21-Jamba-Mini-1.5 is an instruction-tuned open-weight language model from AI21 Labs, released in August 2024 as part of the Jamba 1.5 family. It features 12 billion active parameters out of 52 billion total in a hybrid Transformer-Mamba mixture-of-experts (MoE) architecture—the first non-pure-Transformer design scaled to competitive quality with leading models in its size class. The model supports a 256K-token context window with confirmed effective long-context performance on the RULER benchmark, and is optimized for enterprise use cases including function calling, structured JSON output, grounded generation, and multilingual support across nine languages.

Built on AI21's Jamba architecture combining interleaved attention and Mamba SSM layers with MoE feed-forward modules, Jamba 1.5 Mini delivers up to 2.5× faster inference and roughly 10× lower KV-cache memory usage compared to similarly sized Transformer models at long contexts. It was post-trained on conversational, skill-specific, and long-context data with a knowledge cutoff of March 5, 2024. Weights are released under the Jamba Open Model License, with API access also available through AI21 Studio.

On standard benchmarks, Jamba 1.5 Mini achieves 69.7 on MMLU (CoT), 42.5 on MMLU-Pro, 75.8 on GSM8K, 62.8 on HumanEval, and 85.7 on ARC Challenge—performance comparable to LLaMA 3.1 8B and Gemma 2 9B while offering superior long-context throughput and latency. Recommended deployment uses vLLM with FlashAttention2 and Mamba kernels; AI21's ExpertsInt8 quantization enables single-GPU deployment on an 80GB GPU.