Stability AI has just announced the release of StableCode, its very first LLM generative AI product for coding. This product is designed to assist programmers with their daily work while also providing a great learning tool for new developers ready to take their skills to the next level.
“a colorful parrot with glasses typing away at a computer, flat icon, vector” – SDXL 0.9
StableCode offers a unique way for developers to become more efficient by using three different models to help in their coding. The base model was first trained on a diverse set of programming languages from the stack-dataset (v1.2) from BigCode and then trained further with popular languages like Python, Go, Java, Javascript, C, markdown and C++. In total, we trained our models on 560B tokens of code on our HPC cluster.
After the base model had been established, the instruction model was then tuned for specific use cases to help solve complex programming tasks. ~120,000 code instruction/response pairs in Alpaca format were trained on the base model to achieve this result.
Code for using StableCode Instruct to generate a response to a given instruction.
StableCode is the ideal building block for those wanting to learn more about coding, and the long-context window model is the perfect assistant to ensure single and multiple-line autocomplete suggestions are available for the user. This model is built to handle a lot more code at once (2-4X more than previously-released open models with a context window of 16,000 tokens), allowing the user to review or edit the equivalent of up to five average-sized Python files at the same time, making it the ideal learning tool for a beginner who wants to rise to bigger challenges.
StableCode completing a relatively complex python file utilizing the Pytorch deep learning library (gray text shows StableCode’s prediction).
And here is how we compare to other models of a similar number of parameters and number of tokens trained on. We use the standard pass@1 and pass@10 metrics using the popular HumanEval benchmark.
benchmark scores of stablecode.
HumanEval Benchmark Comparison with models of similar size(3B).
Stability AI aims to make technology more accessible, and StableCode is a significant step toward this goal. People of every background will soon be able to create code to solve their everyday problems and improve their lives using AI, and we’d like to help make this happen. We hope that StableCode will help the next billion software developers learn to code while providing fairer access to technology all over the world.
Speech foundation models, such as HuBERT and its variants, are pre-trained on large amounts of…
This post was co-written with Vishal Singh, Data Engineering Leader at Data & Analytics team…
At Definity, a leading Canadian P&C insurer with a history spanning over 150 years, we…
Don't expect to hear a lot about better framerates and raytracing at the Nvidia GTC…
The team working at the Social Security Administration appears to be among the largest DOGE…
Many companies invest heavily in hiring talent to create the high-performance library code that underpins…