Stability AI, renowned for its stable diffusion text-to-image generative AI technology, has unveiled its latest addition, the Stable LM 2 1.6B – one of its smallest models to date. This follows the recent launch of Stable Code 3B earlier in the week, marking the second model release for Stability AI in 2024. Stable LM, introduced by Stability AI in April 2023 with 3 billion and 7 billion parameter models, is a text content generation LLM.
The new Stable LM model, although compact, packs a powerful punch, aiming to break down barriers and encourage more developers to engage in the generative AI ecosystem. This model is designed to handle multilingual data in seven languages, including English, Spanish, German, Italian, French, Portuguese, and Dutch. Leveraging recent algorithmic advancements in language modeling, Stability AI seeks to strike a delicate balance between speed and performance.
Carlos Riquelme, Head of the Language Team at Stability AI, acknowledged the general trend favoring larger models but highlighted the potential for recent smaller models to outperform older, larger counterparts as they implement improved algorithms and access higher quality data.
The merits of the smaller Stable LM 2 1.6B are evident in its performance compared to other small language models with under 2 billion parameters, including Microsoft’s Phi-2 (2.7B), TinyLlama 1.1B, and Falcon 1B. Impressively, it even outshines some larger models, including Stability AI’s own earlier Stable LM 3B model.
While celebrating the model’s achievements, Stability AI acknowledges certain drawbacks due to its size. The company cautions about potential issues like high hallucination rates or the use of potentially toxic language.
Performance Meets Precision: Stability AI’s Latest in Generative AI – Stable LM 2 1.6B
Stability AI’s shift towards smaller yet more powerful LLM options has been ongoing, with the release of the StableLM Zephyr 3B model in December 2023, offering enhanced performance with a smaller footprint than its initial iteration.
Riquelme emphasized the transparency and data-centric approach in the new model release.
Stability AI trains the Stable LM 2 models on more data, integrating multilingual documents in six additional languages. Riquelme emphasized the significance of presenting data in a specific order during training, suggesting that focusing on different types of data at various stages can provide benefits.
In a move to empower developers, Stability AI offers the new models with pre-trained and fine-tuned options, including a format described as the “last model checkpoint before the pre-training cooldown.” Riquelme expressed the goal of providing tools and artifacts for individual developers to innovate and build upon the existing model.
The decision to make the model available before the last stage of training is an attempt to enhance its malleability, allowing developers to specialize it for different tasks or datasets. While uncertain about the outcome, Stability AI believes in the capability of individuals to leverage these tools and models in innovative and surprising ways.
