Announcing StableCode

Stability AI has just announced the release of StableCode, its very first LLM generative AI product for coding. This product is designed to assist programmers with their daily work while also providing a great learning tool for new developers ready to take their skills to the next level.

 

StableCode offers a unique way for developers to become more efficient by using three different models to help in their coding. The base model was first trained on a diverse set of programming languages from the stack-dataset (v1.2) from BigCode and then trained further with popular languages like Python, Go, Java, Javascript, C, markdown and C++.  In total, we trained our models on 560B tokens of code on our HPC cluster. 

After the base model had been established, the instruction model was then tuned for specific use cases to help solve complex programming tasks. ~120,000 code instruction/response pairs in Alpaca format were trained on the base model to achieve this result. 

StableCode is the ideal building block for those wanting to learn more about coding, and the long-context window model is the perfect assistant to ensure single and multiple-line autocomplete suggestions are available for the user. This model is built to handle a lot more code at once (2-4X more than previously-released open models with a context window of 16,000 tokens), allowing the user to review or edit the equivalent of up to five average-sized Python files at the same time, making it the ideal learning tool for a beginner who wants to rise to bigger challenges.

And here is how we compare to other models of a similar number of parameters and number of tokens trained on. We use the standard pass@1 and pass@10 metrics using the popular HumanEval benchmark.

Stability AI aims to make technology more accessible, and StableCode is a significant step toward this goal. People of every background will soon be able to create code to solve their everyday problems and improve their lives using AI, and we’d like to help make this happen. We hope that StableCode will help the next billion software developers learn to code while providing fairer access to technology all over the world.