SDXL Gets Boost from NVIDIA TensorRT

The Stability AI team is proud to announce a collaboration with NVIDIA that will significantly enhance the speed of our popular text-to-image generative AI product, Stable Diffusion XL. This marks a substantial improvement in the speed and efficiency of our SDXL, thereby drawing us closer to our mission of building the foundation to activate humanity’s potential.

The key to this success is the integration of NVIDIA TensorRT, a high-performance, state-of-the-art performance optimization framework. We are proud to host the TensorRT versions of SDXL and make the open ONNX weights available to users of SDXL globally. 

We have seen a double of performance on NVIDIA H100 chips after integrating TensorRT and the converted ONNX model, generating high-definition images in just 1.47 seconds. With further optimizations such as 8-bit precision, we are confident we can collaboratively increase both speed and accessibility. 

Next, let’s look deeper at the performance benchmark for measuring latency and throughput to compare baseline (non-optimized) vs. NVIDIA’s TensorRT (optimized) model on A10, A100, and H100 GPU accelerators. For latency, the NVIDIA TensorRT (optimized) model is 13%, 26%, and 41% faster than the Baseline (non-optimized model) on A10, A100, and H100 GPU accelerators, respectively. For throughput, the NVIDIA TensorRT (optimized) model is 20%, 33%, and 70% better than the Baseline (non-optimized model) for A10, A100, and H100 GPU accelerators, respectively.

Latency performance comparison

Throughput performance comparison

The significance of this speed improvement extends far beyond just our company. Accelerating the performance of our text-to-image generative AI model has broader implications for democratizing generative AI. As the speed of AI models increases, they become more accessible and affordable, enabling more individuals and organizations to harness the power of generative AI. This empowers creators, researchers, and innovators to explore new frontiers in AI-powered applications, ultimately unleashing the untapped potential of generative AI for a more inclusive and innovative world.

We are incredibly excited to work with NVIDIA to make our models train and run as quickly as possible. Together, we are accelerating our mission of activating humanity’s potential and NVIDIA’s mission of making a positive impact on the world through the use of technology and the power of innovation. 

We can’t wait to see what you create.