Stability AI has recently launched Stable Diffusion XL 1.0 (SDXL), an advanced open weights AI image synthesis model. This next-generation version excels at generating unique images from text descriptions, boasting greater detail and higher resolution compared to its predecessors in the Stable Diffusion series.
Similar to the release of Stable Diffusion 1.4 last August as an open-source project, SDXL also allows anyone with the appropriate hardware and technical expertise to freely download the SDXL files and run the model locally on their own machines.
The advantage of local operation means that users don’t need to pay for access to the SDXL model and reduces concerns regarding censorship. Moreover, the weights files, containing essential neural network data that enables the model to function, can be fine-tuned by hobbyists in the future to generate specific types of imagery.
For instance, while the default model of Stable Diffusion 1.5 can produce a wide range of images, it may not perform as effectively with more niche subjects. To address this limitation, hobbyists have fine-tuned SD 1.5, creating custom models and later LoRA models. These adaptations significantly enhance Stable Diffusion’s ability to generate specific aesthetics, such as Disney-style art, Anime art, landscapes, customized adult content, images of renowned actors or characters, and more.
Stability AI anticipates that this community-driven development trend will continue with SDXL, allowing users to expand its rendering capabilities far beyond the capabilities of the base model.
Upgrades under the hood
SDXL, similar to other latent diffusion image generators, commences with random noise and “recognizes” images within the noise, guided by a text prompt, gradually refining the image. However, what sets SDXL apart is its utilization of a “three times larger UNet backbone” compared to previous Stable Diffusion models, featuring more model parameters, enabling it to achieve its impressive results. In simpler terms, the SDXL architecture conducts more extensive processing to produce the final image.
For image generation, SDXL adopts an “ensemble of experts” architecture, which employs a latent diffusion process. This methodology involves training a single initial model, which is then divided into specialized models specifically tailored for different stages of the generation process, ultimately enhancing image quality. In this case, SDXL comprises a base model, and an optional “refiner” model can be used after the initial generation to further enhance image appearance.
Additionally, SDXL incorporates two distinct text encoders to interpret the written prompts, effectively identifying associated imagery encoded in the model weights. Users can provide different prompts to each encoder, leading to novel and high-quality combinations of concepts. A noteworthy example shared on Twitter involved combining an elephant and an octopus using this technique.
Moreover, SDXL showcases improvements in image detail and size. While Stable Diffusion 1.5 was trained on 512×512 pixel images (an optimal image generation size but lacking in small feature details), Stable Diffusion 2.x increased the size to 768×768 pixels. Now, Stability AI suggests generating 1024×1024 pixel images with Stable Diffusion XL, resulting in even greater detail compared to a similarly-sized image produced by SD 1.5.
You can see the 3 ways to use Stable Diffusion AI to create amazing images