SYCL error out of resources

[paoletto]

ggml_extend.hpp:1000 - vae compute buffer size: 2496.00 MB(VRAM)
Native API failed. Native API returns: 40 (UR_RESULT_ERROR_OUT_OF_RESOURCES)

This happens on the second run of stable-diffusion-cpp-python.
First "run" with as many generations as needed completes succesfully.

Not sure if it's my fault that i should destroy the pipeline somehow and not just let the script end.
Alternately probably a bug of intel drivers (i'm on 22.04, i could try to upgrade to 24.04 and see if that makes any diff)

using 0.2.1 version of the bindings. 0.2.2 do not work due to a different bug (leejet/stable-diffusion.cpp#542)