Room 5 · Transformers · 2022
Images from Text
Images from Text (DALL·E 2, Midjourney and Stable Diffusion, 2022): From noise to an image that matches the description.
In 2022 diffusion models reached the public. They learn to remove noise little by little: they start from pure noise and clean it up until it forms an image that matches the text they were given.
The idea behind diffusion models came from physics. In 2015 Jascha Sohl-Dickstein and colleagues proposed training a model to reverse a gradual process of adding noise to data. In 2020 Jonathan Ho, Ajay Jain and Pieter Abbeel showed that the approach could generate high-quality images, and progress accelerated.
Here is the idea. Take a training image and add a little random noise, then a little more, step by step, until nothing but static remains. A neural network is trained to undo one step at a time: given a noisy image, predict the noise so it can be removed. To generate a new image, you start from pure noise and apply the network many times, revealing an image gradually. To make it follow a description, the model is also given a representation of the text at every step, learned from huge collections of images paired with captions.
In 2022 several systems brought this to the public. OpenAI announced DALL·E 2 in April. Midjourney opened a public beta in July. In August Stability AI, working with researchers from Ludwig Maximilian University of Munich and Runway, released Stable Diffusion with open weights, based on the latent diffusion method from the paper cited here. Because it worked in a compressed version of the image, it could run on a home computer with a good graphics card.
Within months, millions of people were generating images by typing a sentence. The tools also opened conflicts that are still unresolved: artists and stock photo companies sued over the use of their images in training data, and generated images raised new questions about misinformation and consent. You can follow those debates in our Culture section.
The same technique now generates video, audio and 3D objects.
Ask your AI: describe an image in one sentence for a diffusion model, then ask which words in your sentence carry the most weight and why.
Source: arxiv.org/abs/2112.10752
See it in the museum →← Learning from Human Preferences · ChatGPT →
Cite this: Nazareno, I. (2026). Images from Text (2022). New Mind Museum. Retrieved November 17, 2026, from https://newmindmuseum.com/museum/images-from-text-2022/