The Denoising Schedule and the Step Count

Doubling the step count exactly doubles the time and does not come close to doubling the quality. The first half of that sentence is arithmetic. The second half has a specific cause — the discretisation error of the solver falls below the network’s own prediction error — and once you know the cause you can find your own crossover instead of copying somebody else’s number.

The short answer

Step count is the number of times the denoising network is evaluated along the trajectory from noise to image. It is a numerical integration parameter: the underlying process is continuous, and you are choosing how coarsely to approximate it. More steps means a closer approximation to the trajectory the model defines, up to the point where the approximation error is smaller than the model’s own error and further refinement is refining towards a target that is itself wrong.

What this does not mean: more steps do not add detail, do not increase resolution, and do not make the model smarter. They make the sample land closer to where that model, that prompt and that seed were always going to put it.

Time is exactly linear, and here is the arithmetic

Every step is one forward pass through the same network on a tensor of the same shape. There is no per-step variation and no warm-up after the first. So:

total_time  =  fixed_overhead  +  steps × passes_per_step × time_per_pass

fixed_overhead     text encoding + VAE decode + transfer, paid once
passes_per_step    2 with classifier-free guidance, 1 without
time_per_pass      set by model size, resolution and GPU

Worked example, all four numbers stated as assumptions:
  time_per_pass    = 55 ms   (measure yours; see the harness below)
  passes_per_step  = 2       (guidance on)
  fixed_overhead   = 300 ms  (encode + decode)

  20 steps →  0.3 + 20 × 2 × 0.055  =  2.5 s
  30 steps →  0.3 + 30 × 2 × 0.055  =  3.6 s
  50 steps →  0.3 + 50 × 2 × 0.055  =  5.8 s
  80 steps →  0.3 + 80 × 2 × 0.055  =  9.1 s

Going from 20 to 80 steps costs 6.6 extra seconds per image, every image, forever. On a service generating ten thousand images a day that is 18 GPU-hours daily. The full conversion from steps to money is worked in the GPU-seconds behind one generated image.

Steps are not the unit of cost

The unit that actually predicts time is the number of function evaluations, usually written NFE: how many times the denoising network runs in total. Step count and NFE come apart in two common cases.

Comparing two samplers at equal step count is therefore usually comparing them at unequal cost. Compare at equal NFE, or the comparison says nothing.

Why the quality curve flattens

Sampling is numerical integration of an ordinary differential equation whose right-hand side is the network’s prediction. Numerical integration has a known error structure, and it explains the shape of the curve without anybody having to run a grid.

A first-order method such as Euler or DDIM accumulates global error proportional to the step size h. Halving the step size — that is, doubling the step count — halves the error. A second-order method accumulates error proportional to h², so doubling the step count quarters it. Either way the error falls smoothly towards zero as steps increase.

But zero error means exactly reproducing the trajectory that model defines, and the model’s own prediction has error too. Once the discretisation error is well below the network error, extra steps are computing a more accurate approximation to an inaccurate target. Nothing visible improves. That is the flattening, and it is why the crossover point depends on:

• The solver’s order. A second-order multistep solver reaches the flattening point in far fewer steps than Euler, which is the single largest factor.
• The model. A better-trained network has lower prediction error, so it keeps benefiting from finer integration for longer.
• The guidance scale. High guidance produces a stiffer trajectory that is harder to integrate, so high-CFG generations often need more steps to look stable.
• Stochastic samplers do not flatten in the same way. An ancestral sampler injects fresh noise at every step, so the output keeps changing with step count rather than converging. It has no fixed point to approach.

The schedule matters as much as the count

Step count says how many evaluations. The schedule says where along the noise trajectory they are placed, and the placement is not uniform. Because the signal-to-noise ratio changes by orders of magnitude across the trajectory, evenly spaced timesteps waste evaluations in regions where nothing is changing.

Karras-style schedules place steps according to a power law in the noise level, concentrating them where the trajectory curves most. Swapping the schedule at a fixed step count changes the result, often more than swapping the solver does. When a comparison of two samplers does not state the schedule, it has not controlled for the larger variable.

Finding your own flattening point

The honest way to locate the flattening point is convergence testing: generate a high-step reference with a deterministic sampler and a fixed seed, then measure how far lower-step generations sit from it. When the distance stops falling meaningfully, you have found your point. This measures convergence, not aesthetics, which is the part that can be measured without a human panel.

# steps_sweep.py — find where extra steps stop changing the image.
# Requires: diffusers, torch, numpy, pillow.
# The pipeline class and argument names should be checked against the
# version of diffusers you have installed; they have changed before.

import numpy as np, torch
from diffusers import DiffusionPipeline

MODEL   = "<your checkpoint>"
PROMPT  = "a lighthouse on a rocky shore at dusk, long exposure"
SEED    = 12345
CFG     = 5.0
REF_STEPS = 200
SWEEP     = [4, 8, 12, 16, 20, 25, 30, 40, 60]

pipe = DiffusionPipeline.from_pretrained(MODEL, torch_dtype=torch.float16).to("cuda")

def gen(steps):
    g = torch.Generator(device="cpu").manual_seed(SEED)   # CPU generator: portable
    out = pipe(PROMPT, num_inference_steps=steps, guidance_scale=CFG, generator=g)
    return np.asarray(out.images[0], dtype=np.float32) / 255.0

ref = gen(REF_STEPS)
prev = None
print(f"{'steps':>6}  {'RMSE vs ref':>12}  {'change vs prev':>14}")
for s in SWEEP:
    img  = gen(s)
    rmse = float(np.sqrt(((img - ref) ** 2).mean()))
    delta = "" if prev is None else f"{prev - rmse:14.5f}"
    print(f"{s:6d}  {rmse:12.5f}  {delta}")
    prev = rmse

# Read it like this: the flattening point is the first step count where the
# "change vs prev" column drops to roughly the size of the differences you
# cannot see. RMSE is a crude perceptual proxy — if you have lpips installed,
# swap it in for a measure that tracks visible difference more closely.

Use a deterministic sampler for this. An ancestral sampler will show RMSE that never converges, because the reference and the test are drawing different noise at every step, and you will conclude wrongly that more steps always help.

Step-distilled models are a different regime

A family of published techniques trains a model to take much larger steps than the original trajectory allows: progressive distillation halves the step count repeatedly by training a student to match two teacher steps in one; consistency models train a network to map any point on the trajectory directly to its endpoint; adversarial distillation adds a discriminator so that few-step outputs stay on the image manifold.

The practical signature of such a checkpoint is that it is documented with a specific step count — one, four, eight — and that raising the step count beyond it makes results worse rather than better, because the trajectory it was trained to follow is not the original one. These models usually also have guidance baked in, so the guidance scale parameter either does nothing or means something different. Read the model card; the general advice on this page does not apply to them.