Stable Diffusion v2.1 and DreamStudio Updates 7-Dec 22

Stable Diffusion v2.1 Release

We’re happy to bring you the latest release of Stable Diffusion, Version 2.1. We promised faster releases after releasing Version 2,0 and we’re delivering only a few weeks later. The Version 2 model line trained up using a brand new text encoder (OpenCLIP), developed by LAION, that gives us a deeper range of expression than version 1.

Within a few days of releasing SD v2, people started getting fantastic results as they learned some new ways to prompt, and you’ll be happy to discover that 2.1 supports the new prompting style and brings back many of the old prompts too! The differences are more data, more training, and less restrictive filtering of the dataset.

When we set out to train SD 2 we worked hard to give the model a much more diverse and wide-ranging dataset and we filtered it for adult content using LAION’s NSFW filter. The dataset delivered a big jump in image quality when it came to architecture, interior design, wildlife, and landscape scenes. But the filter dramatically cut down on the number of people in the dataset and that meant folks had to work harder to get similar results generating people.

We listened to our users and adjusted the filters. The filter still stripped out adult content, but was less aggressive, which cut down the number of false positives it detected. We fine-tuned the SD 2.0 model with this updated setting, giving us a model which captures the best of both worlds. It can render beautiful architectural concepts and natural scenery with ease, and yet still produce fantastic images of people and pop culture too. The new release delivers improved anatomy and hands and is much better at a range of incredible art styles than SD 2.0.

The model also has the power to render non-standard resolutions. That helps you do all kinds of awesome new things, like work with extreme aspect ratios that give you beautiful vistas and epic widescreen imagery.

 

DreamStudio Updates

Lots of people have noticed that “negative prompts” worked wonders with 2.0 and they work even better in 2.1.

Negative prompts are the opposites of a prompt; they allow the user to tell the model what not to generate. Negative prompts often eliminate unwanted details like mangled hands or too many fingers or out of focus and blurry images.

You can easily give negative prompts a try in DreamStudio right now by appending “| <negative prompt>: -1.0” to the prompt. For instance, appending “| disfigured, ugly:-1.0, too many fingers:-1.0” occasionally fixes the issue of generating too many fingers.

Users can prompt the model to have more or less of certain elements in a composition, such as certain colors, objects or properties, using weighted prompts. Starting with a standard prompt and then refining the overall image with prompt weighting to increase or decrease compositional elements gives users greater control over image synthesis.

For example:

Open-Source Release

At Stability we know open is the future of AI and we’re committed to developing current and future versions of Stable Diffusion in the open. Expect more models and more releases to come fast and furious and some amazing new capabilities as generative AI gets more and more powerful in the new year.

For more details about accessing the model, please check out the release notes on the Stability AI GitHub.

Also, you can find the weights and model cards here.

View our ongoing project the Stable Diffusion Prompt Book online here.

Visit beta.dreamstudio.ai to create a DreamStudio account.

Join our 100k+ member community on Discord.

Above: the negative prompt is used to reinforce the visual fidelity and style of cinematic science-fiction concept art.

---

We are hiring researchers and engineers who are excited to work on the next generation of open source Generative AI models! If you’re interested in joining Stability AI, please reach out to careers@stability.ai, with your CV and a short statement about yourself.