Room 5 · Transformers · 2022

Images from Text

Images from Text (DALL·E 2, Midjourney and Stable Diffusion, 2022): From noise to an image that matches the description.

In 2022 diffusion models reached the public. They learn to remove noise little by little: they start from pure noise and clean it up until it forms an image that matches the text they were given.

The idea behind diffusion models came from physics. In 2015 Jascha Sohl-Dickstein and colleagues proposed training a model to reverse a gradual process of adding noise to data. In 2020 Jonathan Ho, Ajay Jain and Pieter Abbeel showed that the approach could generate high-quality images, and progress accelerated.

Here is the idea. Take a training image and add a little random noise, then a little more, step by step, until nothing but static remains. A neural network is trained to undo one step at a time: given a noisy image, predict the noise so it can be removed. To generate a new image, you start from pure noise and apply the network many times, revealing an image gradually. To make it follow a description, the model is also given a representation of the text at every step, learned from huge collections of images paired with captions.

In 2022 several systems brought this to the public. OpenAI announced DALL·E 2 in April. Midjourney opened a public beta in July. In August Stability AI, working with researchers from Ludwig Maximilian University of Munich and Runway, released Stable Diffusion with open weights, based on the latent diffusion method from the paper cited here. Because it worked in a compressed version of the image, it could run on a home computer with a good graphics card.

Within months, millions of people were generating images by typing a sentence. The tools also opened conflicts that are still unresolved: artists and stock photo companies sued over the use of their images in training data, and generated images raised new questions about misinformation and consent. You can follow those debates in our Culture section.

The same technique now generates video, audio and 3D objects.

Ask your AI: describe an image in one sentence for a diffusion model, then ask which words in your sentence carry the most weight and why.

Did you know? Stable Diffusion was released with open weights in August 2022, and anyone could run it on their own computer.

Source: arxiv.org/abs/2112.10752

See it in the museum →

← Learning from Human Preferences · ChatGPT →

All of Room 5: Transformers

Cite this: Nazareno, I. (2026). Images from Text (2022). New Mind Museum. Retrieved November 17, 2026, from https://newmindmuseum.com/museum/images-from-text-2022/