Jamba-tiny-random
About this model
This is a tiny, dummy version of Jamba, used for debugging and experimentation over the Jamba architecture.
It has 128M parameters (instead of 52B), and is initialized with random weights and did not undergo any training.
Technical Specs
- Parameters: 0.128B
- Architecture: Hybrid Transformer-Mamba MoE
- Context Window: 262,144 tokens
- Input Modalities: text
Hardware Requirements
- VRAM: 0.3 GB
- Compute: Any GPU with 1GB+ VRAM; ~0.3GB in FP16, runs on CPU
Related Models
More Open Source →
Description: The NVIDIA DeepSeek-V4-Pro-NVFP4 model is the quantized version of the DeepSeek-V4-Pro model, which is a Mixture-of-Experts (MoE) language model with 1.6 trillion total parameters and 49 billion activated parameters. For more information, please check here. The NVIDI…
Description: The NVIDIA GLM-5.2 NVFP4 model is the quantized version of ZAI’s GLM-5.2 model, which is an auto-regressive language model that uses an optimized transformer architecture. GLM-5.2 is a Mixture-of-Experts (MoE) model for reasoning and coding that uses sparse attention…
👋 Join our WeChat or Discord community. 📖 Check out the GLM-5.2 blog and GLM-5 Technical report . 📍 Use GLM-5.2 API services on Z.ai API Platform. 🔜 Try GLM-5.2 here .
👋 Join our WeChat or Discord community. 📖 Check out the GLM-5.2 blog and GLM-5 Technical report . 📍 Use GLM-5.2 API services on Z.ai API Platform. 🔜 Try GLM-5.2 here .