Faster diffusion model training with Min-SNR
Min-SNR improves diffusion model training through multi-task learning and loss weighting per task, making convergence 3.4× faster and image quality better.
In the diffusion process, an initial noisy image is denoised step by step until a clean, high-quality image emerges. That initial noisy image is random Gaussian noise. Denoising can take anywhere from just 15 steps to thousands of them. As a rule, more steps yield a better image, although beyond a very large number of steps the gain in quality becomes negligible. Because diffusion generates iteratively, training is slow.. The authors of the paper Efficient Diffusion Training via Min-SNR Weighting Strategy show that changing the training objective to use multi-task learning greatly accelerates model convergence, reaching 3.4× faster convergence than earlier methods while also improving the FID.
Slow Convergence
The standard approach to denoising diffusion models converges slowly, so training a model takes many GPU hours. The authors of Min-SNR examined the training process and found some interesting results: optimizing the denoising function for one particular noise level (training timestep) can be detrimental to other timesteps.

Figure 1. The diffusion-denoising timesteps were grouped into clusters of 100 timesteps each. Every cluster was fine-tuned directly, only on the denoising levels it contained. Loss improved for the cluster itself and for neighbouring timesteps, but fine-tuning hurt steps that were further away. Source
Figure 1 shows a pre-trained diffusion model being fine-tuned exclusively on selected denoising levels. Although loss improves for the fine-tuned timesteps and those near them, model performance suffers on other, more distant timesteps. The researchers’ hypothesis is that different steps have different needs. The denoising strength differs at every step of the diffusion process, and different timesteps pull optimization in conflicting directions. Given that, optimizing all diffusion steps directly with equal strength leads to many gradient steps in sub-optimal directions, which considerably lengthens training time and model convergence. The proposed remedy is a multi-task learning approach.
Multi-Task Learning
The goal of multi-task learning is to optimize several different tasks directly. Various approaches have been suggested, including:
- A feature extractor shared across tasks
- Learning what should be shared across tasks
- Sharing parameters at a fine-grained level
- Weighting the loss per task to prevent conflicting gradients
A diffusion model already has a module shared by all the tasks – the entire U-Net model. The obvious next step is to combine other techniques with the shared model. Multi-task learning with deep neural networks: a survey names per-task loss weighting as the most promising direction for MTL, and that’s the direction the Min-SNR authors take.
If every denoising level is treated as its own task, Multi-Task Learning can be applied to diffusion model training. Some tasks in the diffusion process are more difficult than others. The early denoising levels, for instance, consist of almost pure noise, so the right denoising direction isn’t clear, whereas the final denoising steps already contain a well-formed image and mostly reconstruct the input. Treating each noise level as a separate task therefore lets us give them different levels of importance and weight the gradient update accordingly. Optimizing all tasks together makes it possible to select the weights properly. They’re picked so that a gradient update at one level doesn’t damage the other levels. What remains is finding those optimal weights.
Min-SNR
Min-SNR determines the weight strength from the Signal-to-Noise level of each noise level. Intuitively, SNR measures how much useful information (signal) the image contains.
SNR(t) = αt2 /σt2
xt = αtx0 + σtϵ
αt = sqrt(1 − σt2)
, where ϵ is noise sampled from the Gaussian distribution N(0, I), σt is the magnitude of noise added to the clean data x0 at timestep t, and αt can either be learned through reparameterization or kept fixed as hyperparameters.

Figure 2. The top plot shows SNR pre-computed for 100 steps. Once the formula min{SNR(t), γ} is applied, it can serve to weight the loss function of a diffusion model that outputs the signal x. These days, though, most diffusion models predict the added noise instead. The bottom plot shows the pre-computed loss weight for a diffusion model that predicts noise ϵ. Time t=100 is the first step of the diffusion process, and t=0 is the last.
Note that SNR can be computed in advance for every timestep. SNR is low for the early denoising steps and high for the final denoising levels. This makes intuitive sense: the final denoising layers improve image quality only a little, mostly reconstructing the input, whereas the early denoising levels are very noisy, with no clear direction for improvement. SNR weighting makes sure that these uncertain early forward steps are updated slowly, while the last step, whose update direction is clearer, is updated more strongly.
Figure 3. Noise level across timesteps.
min-SNR = min{SNR(t), γ}. Setting y = 5 prevents the SNR weight from becoming too large. As the paper’s authors demonstrate, and as Figure 4 shows, min-SNR makes the model converge x3.4 times faster than a model trained with every noisy step equally likely, while also improving the FID score.
Since most diffusion models nowadays predict noise rather than the signal, the snr weight has to be inverted:
wt = min{SNR(t),γ} / SNR(t) = min{ γ / SNR(t) , 1}.
, where wt denotes the loss weight for timestep t.
y = 5
loss_weight = np.stack([snr, y * np.ones_like(snr)], axis=1).min(axis=1) / snr

Figure 4. Convergence speed and FID score compared across training strategies. In this setup, min-SNR with y = 5 gives the best results. Source
Summary
This blog post showed that standard diffusion training is slow because gradient updates for different noise levels point in conflicting directions, so updating one noise level can hurt the others. Treating every timestep as a separate training task lets us use multi-task learning methods to try to prevent gradient conflicts. Weighting the loss of each task can effectively balance harder and easier noise levels and make sure one noise level doesn’t damage the others. The paper’s authors demonstrate that min-SNR is an effective loss weighting strategy: it makes convergence 3.4x times faster and improves the FID at the same time.