Learning path · Level 2 · Tutorial 5

How Diffusion Models Generate Images

Explain diffusion-based image generation and recognize why a visually convincing result may still misrepresent relationships, lettering, or recurring details. No image-generation tool is required.

  • Level: Beginner
  • Duration: 14 min
  • 8 steps
Start tutorial Back to the path
How Diffusion Models Generate Images

What you will go through

  1. Learning to Denoise See how deliberately adding noise to examples gives a diffusion model a training task: learning predictions that support denoising. 2 min
  2. From Random Noise to an Image Follow a simplified diffusion generation attempt from random noise to a generated image, and distinguish it from training. 2 min
  3. How Guidance Steers the Image Learn how conditioning from a text description or supported reference image steers denoising without guaranteeing exact compliance. 2 min
  4. Why the Same Description Can Look Different See why two images can differ even under the same guidance. 2 min
  5. Checking Counts and Spatial Relationships Compare illustrated outputs with a requested scene to identify count and spatial mismatches. 2 min
  6. When Lettering Looks Right but Is Wrong Check rendered lettering against the exact requested word and describe a mismatch precisely. 2 min
  7. Keeping Details Consistent Across Images 2 min
  8. Explain the Process and Check the Result Bring together learned denoising, random starts, conditioning, and concrete checks for image-output failures. 2 min

What you will learn

  • Order the stages of a simplified noise-to-image example and explain what changes during denoising.
  • Distinguish learning to denoise during training from using that learned ability during generation.
  • Explain how a text description or reference input can guide an image without guaranteeing exact compliance.
  • Identify spatial, text-rendering, and consistency failures in worked examples.
  • Explain why similar guidance can produce different images and why these limitations vary across models.

Before you start

  • How Models Learn from Data
  • How Neural Networks Learn Layered Patterns

Step 1 of 8

Learning to Denoise

See how deliberately adding noise to examples gives a diffusion model a training task: learning predictions that support denoising.

Seeing noise as corruption

A clean image becomes harder to recognize

Diffusion is a common approach to image generation, not the only kind of image-model architecture. During training, a diffusion model practices working from deliberately corrupted examples.

Here, noise means random visual disturbance added to an image representation. As more noise is added, recognizable structure becomes harder to see. The visible pictures below are a simplified way to show the idea.

Illustrative example

Three versions of the same red mug on a desk: clean, partly obscured by random noise, and almost fully obscured by dense noise.

Illustrative example: clean representation → add noise → noisy representation.

Learning from known corruption

What the model practices

Training uses examples at different noise levels. For each example, the original representation and the added noise are known. That provides a learning signal: the model can compare its prediction with the known added noise and adjust what it learns.

Often, the model learns to predict the added noise. Those predictions can then support denoising—reducing noise from a representation. It is learning a useful pattern across many corrupted examples, not storing a fixed sequence of finished pictures.

Tip

A useful simplification

A noisy picture is only a teaching illustration. Many diffusion models work on compressed image representations rather than directly on visible pixels. The central idea is the same: training supplies deliberately corrupted examples and feedback about the noise that was added.

Example

The learning signal

Known clean representation + known added noise → compare with the model's prediction → adjust the model during training.

A later image-making attempt is not simply a hidden finished photograph waiting to be revealed.

Practice: identify the training task

Check your understanding

Which statement best describes the diffusion model's training task?

Step 2 of 8

From Random Noise to an Image

Follow a simplified diffusion generation attempt from random noise to a generated image, and distinguish it from training.

From noise toward structure

A new image begins as noise

In generation (inference), a text-to-image diffusion system often begins with a random noise representation—not with a hidden original photograph.

The trained model repeatedly makes a denoising prediction. Each prediction is used to change the current representation. Across many updates, visible image structure can emerge from noise.

This is not a fixed process of adding one object at a time. An update may affect broad shapes, colors, edges, or fine details together.

Selected denoising snapshots

Illustrative example: Start shows random noise; Update 1 shows faint scene structure; Update 2 shows clearer structure with remaining noise; Final image shows a completed landscape.

Illustrative example — Start → Update 1 → Update 2 → Final image.

Tip

Important limitation

These are selected illustrative snapshots, not a claim that generation always uses two updates. Real generation uses repeated denoising updates, and a later update does not guarantee that every detail improves.

Use learned ability without retraining

What changes—and what stays fixed

The model uses an ability learned earlier during training. During an ordinary generation attempt, that learned ability is applied to the developing representation; the model is not retrained for each image.

So there are two different things that can change: training changes the model itself, while generation changes the image representation being developed.

Training and generation

Compare the two processes.

Training and generation change different things.
ProcessWhat changesWhat stays fixed
TrainingThe model's learned parametersA particular generation attempt is not being developed
GenerationThe current image representationThe model's learned parameters

Check your understanding

Order the process

Put the overall process in order. Include both the training change and the changes during one generation attempt.

  1. Generation Start: random noise is the current representation.
  2. Generation result: a generated image.
  3. Training: examples adjust the model's learned parameters.
  4. Generation: repeated denoising updates change the current representation, not the learned parameters.

Step 3 of 8

How Guidance Steers the Image

Learn how conditioning from a text description or supported reference image steers denoising without guaranteeing exact compliance.

Conditioning provides direction

Guidance during denoising

Conditioning is information supplied to a model to steer denoising toward a requested kind of image. It provides direction while the developing image is being updated; it is not a label attached after the image is finished.

A text description can guide requested objects, their appearance, and their relationship throughout generation. It influences the result, but it does not function as an exact rule checker.

Conditioning steers repeated updates

A diagram shows a text description guiding several denoising stages toward a red mug on the viewer's left of a blue notebook.

The text description supplies guidance throughout denoising. The arrows show influence, not a guarantee that every requested detail will be exact.

Guidance is not enforcement

Two possible sources of conditioning

A system may also support a reference image as conditioning. When supported, a reference image can guide aspects such as appearance or composition. Its role depends on the particular system.

Not every image-generation system accepts a reference image, and a supported reference image still does not turn the request into a set of enforced rules.

Example

Requested is not guaranteed

Illustrative example

Text description: One red mug to the left of one blue notebook on a wooden desk. The mug has a white handle and the word TEA.

This description steers generation toward those requested properties. It does not promise that every property will appear exactly as requested.

If a particular system supports a reference image, that image can provide additional guidance. The model still generates through denoising; the reference image is guidance, not a guaranteed copy or a universal feature.

Guidance versus a guarantee

Check your understanding

Which statement is accurate?

Step 4 of 8

Why the Same Description Can Look Different

See why two images can differ even under the same guidance.

One description can allow many images

Same guidance, different starts

A text description can guide generation, but it does not specify every visual choice. Consider this same description for two separate attempts:

One red mug to the left of one blue notebook on a wooden desk. The mug has a white handle and the word TEA.

Both attempts can use that identical conditioning and still begin from different random noise representations. As denoising develops each start, the resulting images can differ in choices the description leaves open, such as lighting, viewpoint, wood grain, or the notebook’s angle.

A new attempt is generation, not new training: it changes a developing image representation, not the model’s learned parameters.

Two generation attempts

Two different random-noise patterns each develop through arrows into a plausible red mug and blue notebook scene with different visual details.

Illustrative example. Run 1 produces Image A; Run 2 produces Image B. The text description, model, and generation settings are the same. The random start differs.

Isolate the source of variation

Read a controlled comparison

To identify one cause of variation, compare a controlled pair. Hold the text description, model, and generation settings constant, then change only the random start. That comparison supports a limited conclusion: the changed start can lead to a changed output.

In other comparisons, a different model or different generation settings can also affect the result. Do not attribute every difference to randomness unless the comparison controlled those other factors.

What the pair isolates

Use the controls to decide what the pair demonstrates.

A controlled pair changes one factor while holding the others fixed.
FactorIn the illustrative pairInterpretation
Text descriptionHeld constantThe guidance did not change
ModelHeld constantA model difference does not explain this pair
Generation settingsHeld constantA settings difference does not explain this pair
Random startChangedIt can account for different outputs
Separate comparisonModel or settings may changeThese can also cause variation

Quick check

Interpret the pair

Run 1 and Run 2 use the same text description, model, and generation settings, but produce different images. What is the best-supported explanation?

Apply the idea

Explain the controlled pair

Illustrative example: Run 1 produces Image A and Run 2 produces Image B. The text description, model, and generation settings were held constant.

Write 2–3 sentences explaining why the outputs can differ. State what changed, what did not need to change, and whether the second generation means the model learned from the first one.

Write at least 120 characters (0/120).

Step 5 of 8

Checking Counts and Spatial Relationships

Compare illustrated outputs with a requested scene to identify count and spatial mismatches.

Turn a description into checks

Check the requested facts

A convincing image can still miss a stated arrangement. Check the request as separate facts, not as a general impression.

Requested scene:
One red mug to the left of one blue notebook on a wooden desk. The mug has a white handle and the word TEA.

For this step, check three facts from the viewer’s perspective:

  • Is there 1 mug?
  • Is there 1 notebook?
  • Is the mug to the viewer’s left of the notebook?

Recognizable objects and an attractive wooden desk do not prove that these facts are correct.

Viewer perspective

A red mug is on the viewer's left of a blue notebook on a wooden desk.

Read left and right from the viewer’s perspective: the mug is left of the notebook.

Inspect counts and positions

Two polished but incorrect results

Left panel has one mug to the right of one notebook. Right panel has two mugs and one notebook.

Illustrative example. Image A is the left panel; Image B is the right panel.

Example

What the comparison shows

Image A has the requested object types and counts, but its 1 mug is on the viewer’s right of the 1 notebook. That is a spatial mismatch.

Image B includes a mug on the viewer’s left, but it shows 2 mugs instead of 1. That is a count mismatch.

A result can have both kinds of mismatch. Check quantity and position independently.

Practice: identify the mismatch

Classify the results

Match each worked result to its classification. Use the stated count and viewer-perspective position as evidence.

Tap an item and then its matching pair.

Step 6 of 8

When Lettering Looks Right but Is Wrong

Check rendered lettering against the exact requested word and describe a mismatch precisely.

Why word-like lettering can still be wrong

Lettering needs its own check

Lettering is part of the picture

When a diffusion model creates lettering inside an image, it renders visual shapes that resemble letters. It does not automatically insert verified typed text as a text editor would.

So a label can look convincingly word-like while still containing a missing, substituted, repeated, or distorted letter. Check the requested word character by character instead of judging whether the label merely resembles a word.

Compare the visible characters

A side-by-side mug-label comparison: one label reads TEA and the other reads TFA, with the differing middle letter emphasized.

A label can look like a familiar word yet fail an exact character-by-character check.

Tip

Exact match, not visual plausibility

The requested lettering is TEA. A result satisfies this detail only when the visible characters match T, then E, then A, in that order.

Text-rendering performance varies across models and versions. A lettering mistake is evidence about this particular result, not proof that every diffusion model always produces unreadable text.

Worked lettering check

Read the result literally

Requested scene:

One red mug to the left of one blue notebook on a wooden desk. The mug has a white handle and the word TEA.

For lettering, record what is actually visible. Do not silently replace an incorrect observed label with the requested word.

Illustrative example

Image B shows one red mug with a white handle to the viewer's left of one blue notebook on a wooden desk. The mug lettering visibly reads TAE.

Illustrative example — Image B. The scene is polished and the objects are correctly recognizable, but the observed mug lettering is TAE.

Example

Separate checks reveal specific failures

The scene can satisfy some requested details while failing another. Here, inspect the lettering separately from the mug color, handle, notebook, desk, and arrangement.

A precise report quotes the observed letters exactly and then compares them with the requested word.

Your lettering inspection

Report the mismatch precisely

Look back at Image B. Compared with the requested word TEA, what is the exact visible lettering discrepancy? Quote the observed lettering and state what differs.

Write at least 20 characters (0/20).

Step 7 of 8

Keeping Details Consistent Across Images

Identity needs more than the right object type

A category match is not an identity match

Separate generation attempts do not automatically preserve the identity of a recurring subject. A result can match the broad category—such as a red mug—while changing details that would make it a different mug.

Use the recurring scene description: One red mug to the left of one blue notebook on a wooden desk. The mug has a white handle and the word TEA.

That description specifies 1 mug, 1 notebook, color, position, handle color, and lettering. If a project also needs the same mug in several images, distinctive details such as handle shape, proportions, or a small marking may need to remain stable too. The description can guide those details, but it does not guarantee them.

Compare the recurring mug

Side-by-side desk scenes: both have a red mug to the left of a blue notebook, but the mug handle shape and gold star marking differ while the lighting also changes.

Illustrative example. Image A and Image B keep the basic scene, but the recurring mug's details are not all the same.

Check the details that were meant to recur

Separate incidental variation from identity changes

Not every difference is a consistency failure. First ask which properties were meant to remain fixed for the recurring subject, and which properties were not specified.

What changed—and how to interpret it

Compare Image A and Image B as two attempts intended to show the same recurring mug.

Illustrative example. The intended purpose determines which differences matter.
PropertyImage AImage BInterpretation
Desk lightingWarmCoolUsually incidental if lighting was not fixed
Mug colorRedRedBroad category remains aligned
HandleThick, rounded, whiteThin, angular, whiteDetail change if the same mug was intended
Gold star markingPresentAbsentIdentity-detail change if the mark was meant to recur
Lettering"TEA""TEA"Requested lettering is unchanged

Apply the consistency check

Make a limited consistency claim

Image A and Image B were intended to show the same recurring mug. Both are red and show "TEA", but Image B has a thin angular white handle instead of a thick rounded one and lacks the gold star marking. The desk lighting also differs. Which conclusion is best?

Step 8 of 8

Explain the Process and Check the Result

Bring together learned denoising, random starts, conditioning, and concrete checks for image-output failures.

Connect the ideas

One connected process

A diffusion model learns during training from examples that have been deliberately corrupted with noise. That learned ability is then used during generation: a random noise representation is repeatedly updated toward an image.

Conditioning supplies guidance throughout those updates. Here, the text description is: One red mug to the left of one blue notebook on a wooden desk. The mug has a white handle and the word TEA.

The text description can steer the result, while the random start can contribute to variation. Neither one means that every requested detail is guaranteed.

Training and generation, connected

A two-panel diagram. The left panel shows training corruption: clean representation → add noise → noisy representation. The right panel shows generation: random noise progresses through Start, Update 1, Update 2, and Final image under guidance from a text description.

Training learns denoising from corrupted examples. Generation applies that ability to a changing representation; these are selected illustrative snapshots, not a fixed number of updates.

Order the generation stages

Sequence check

Put this simplified generation sequence in order.

  1. Random noise
  2. Generated image
  3. Denoising steps

Inspect a worked pair

A final pair to inspect

Illustrative example

Both outputs use the same text description:
One red mug to the left of one blue notebook on a wooden desk. The mug has a white handle and the word TEA.

For this comparison, assume the same tall mug with a round white handle should recur in both images. Use the observable evidence below rather than relying on whether small lettering is rendered clearly in the illustration.

Image A and Image B

Illustrative example. Left panel, Image A: 1 blue notebook is on the viewer's left and 1 tall red mug is on the viewer's right. The mug has a round white handle and observed lettering "TEA". Right panel, Image B: 1 short wide red mug with an angular white handle is on the viewer's left of 2 blue notebooks. Its observed lettering is "TFA".

The illustration supports the comparison. The evidence table records the observations to use for the case check.

Inspection evidence

Raw observable evidence for the illustrative pair:

Compare each observation with the requested description and with the intended recurring mug.
OutputObjects and viewer-perspective positionLetteringMug detail
Image A1 blue notebook left; 1 red mug right"TEA"Tall mug; round white handle
Image B1 red mug left of 2 blue notebooks"TFA"Short wide mug; angular white handle

Apply the full check

Final case response

Using the generation sequence and the illustrative pair on the previous screens, write a concise case response. Include:

  1. The generation stages in order, using arrows.
  2. One distinction between training and generation.
  3. The role of conditioning and the role of a random start.
  4. One spatial or count mismatch, the lettering mismatch (quote the observed lettering exactly), and one cross-image consistency mismatch.

Do not treat a visually convincing result as proof that every requirement was met.

Write at least 250 characters (0/250).

Takeaway

Summary

Key takeaway

  • Training learns denoising; generation repeatedly updates a representation from a random start.
  • Conditioning steers generation but does not guarantee every requested property.
  • Check counts, viewer-perspective relationships, lettering, and recurring details separately.
  • Different outputs can be plausible while differing in compliance; capabilities vary across models.

Tutorial complete

Congratulations! You completed: How Diffusion Models Generate Images

You can now explain the basic diffusion process and evaluate whether a convincing image actually satisfies its requested details.

20 XP

Download the App now to have access to + 5000 free courses, exercises, certificates and lots of content without paying anything!

  • 100% free online courses from start to finish

    Thousands of online courses in video, ebooks and audiobooks.

  • More than 60 thousand free exercises

    To test your knowledge during online courses

  • Valid free Digital Certificate with QR Code

    Generated directly from your cell phone's photo gallery and sent to your email

Cursa app on the ebook screen, the video course screen and the course exercises screen, plus the course completion certificate