A new hybrid Mamba-2 machine learning architecture operates with 360 million active parameters to achieve high computational efficiency. The 2.3 billion parameter model, named Rigel, achieves results close to Meta's Llama-3.2-3B performance while using less than 1% of the floating point operations required for the training of the latter, according to researcher Mayank Mish.

The model was developed without a dedicated compute cluster, relying on a single codebase to run across Nvidia H100, A100, and V100 GPUs as well as Google TPU v5p and v6e hardware. The use of a Mixture-of-Experts design allows the model to keep only a fraction of its total parameters active for any single calculation.