The parameter 'predictor_order' and 'corrector_order' of SA-Solver

[eeyrw]

The SA Solver works very well on stable diffusion. But what's the true affect with different order for predictor and corrector? I tried both set 'predictor_order = 3' and 'corrector_order = 4' and 'predictor_order = 2' and 'corrector_order = 2' and seems not quite different. By the way, my classifier-free gudiance scale ranges from 7.5~9. Is it a big scale or small to the SA Solver? Higher order gets higer accuracy?

[lawrence-cj]

Cc: @scxue

[scxue]

Glad to hear that SA-Solver generates good samples. The 'predictor_order' and 'corrector_order' are related to the 'step' that the Stochastic Adams-Bashforth method and Stochastic Adams-Moulton method used. Specifically, it is related to the number of the former model evaluations (model output at former timesteps) that are used in the Lagrange interpolation. For example, DDIM and DDPM only use the current timestep model output, thus DDIM and DDPM correspond to the first-order discretization of diffusion ODE and diffusion SDE. SA-Solver interpolates the former model evaluations to get higher accuracy (for the ODE case, our solver is similar to UniPC). So with enough NFEs, the results of orders (3,4) and (2,2) are quite similar. My experience is for limited NFEs (like < 20), (3,4) is better for smaller guidance scales, while (2,2) is better for larger guidance scales.

For the classifier-free guidance scale, my experience is that it will be harder to converge in small steps if a larger cfg scale is employed (which means that a larger cfg scale should be accompanied by more NFEs). The different scales result in different styles of images. Usually SA-Solver with a bit smaller cfg scale will looks better. But just try different settings and choose your favorite.

BTW, the SA-Solver's repo will be released soon, if you have any other questions, do not hesitate to reach out to me!

[eeyrw]

I see. It is about convergence speed. Considering the step number 50 I used is fairly large, so there is not obvious difference between different orders. According my quick trials (the scheduler implementation from huggingface space app), 25 steps seems a suitable number for SD1.5 as same as Pixart-α demo used.