Open 3B language models from AMD
Instella, from AMD, is the high-performance 3B language models. ResearchRAIL license for model weights, MIT license for code. Trained on MI300X.
Instella-3B-SFT and Instella-3B-Instruct variants provide supervised fine-tuning and direct preference optimization (DPO) alignment. Users can use these variants for instruction-following, conversational tasks, and problem-solving without relying on closed-source models or commercial APIs.
Instella demonstrates end-to-end training on 128 AMD Instinct MI300X GPUs using ROCm, FlashAttention-2, and FSDP. Users can replicate the training pipeline, adapt hyperparameters, and leverage published configurations to train custom models on AMD hardware, validating AMD GPU scalability for AI workloads.
Instella provides comprehensive benchmark results across MMLU, GSM8k, ARC, and other standard tasks, outperforming comparable fully open models by 8% on average. Users can use published performance metrics to evaluate Instella against Llama-3.2-3B, Gemma-2-2B, and Qwen-2.5-3B for their specific use case.
Instella releases all artifacts including model weights, training hyperparameters, datasets, and code on Hugging Face and GitHub. Users can fork, modify, and contribute improvements to the model, datasets, and training pipeline, fostering community-driven innovation in open language models.
Instella-3B is a fully open 3-billion-parameter model with released weights, training code, and datasets. Users can download, modify, and deploy it locally without licensing restrictions, enabling cost-effective inference on modest hardware while maintaining full transparency and control over model behavior.