[WWB]: Add ImageText-to-Image pipeline validation #1373
Conversation
|
is image to image sensitive to random |
the problem came from the difference in the resolutions of generated images by HF and GenAI libs. |
github-actions
bot
removed
the
category: tokenizers
|
I am getting good accuracy convergence for big models, e.g. SD-XL but cannot make tests working with GenAI for any dummy model (I tried several). Waiting for fixes from OV. |
…nvino.genai into ak/wwb_inpainting
|
@ilya-lavrenov, CI is passed with non-dummy model. |
|
WWB tests on master: WWB tests on current PR: WWB tests with dummy random model: Even if we switch to dummy model once the related issue is fixed by CPU team, execution time is still very huge, can it be optimized? why did im2im give 3x to execution time even with dummy models? |
| prompt, | ||
| image=image_data, | ||
| num_inference_steps=num_inference_steps, | ||
| strength=0.8, |
There was a problem hiding this comment.
do you run optimum / diffusers with the same strength value?
I just don't see where reference image2timage is called..
There was a problem hiding this comment.
Good point. It is 0.8 by default in Diffusers but it is better to set it explicitly.
There was a problem hiding this comment.
Actually, it is already set to 0.8 as well in the default_gen_image_fn of im2im pipeline.
| @@ -65,6 +67,7 @@ def test_image_model_types(model_id, model_type, backend): | |||
| @pytest.mark.parametrize( | |||
| ("model_id", "model_type"), | |||
| [ | |||
| ("dreamlike-art/dreamlike-anime-1.0", "image-to-image"), | |||
There was a problem hiding this comment.
maybe echarlaix/tiny-random-latent-consistency can work here instead of non-working echarlaix/tiny-random-stable-diffusion-xl ?
There was a problem hiding this comment.
I tried several of them including the one you mentioned and they do not work due the known bug.
There was a problem hiding this comment.
do you need torch model in these tests or openvino only enough?
We have several preconverted models here https://huggingface.co/collections/OpenVINO/image-generation-67697d9952fb1eee4a252aa8 (I requested dreamlike models too, but current status is waiting approval)
among trained but still small is https://huggingface.co/segmind/tiny-sd
but not sure that used in model scheduler currently supported
There was a problem hiding this comment.
also for speedup testing, you can expose num_infeence_steps parameter in cli (I suppose even with streigh=0.8, it runs 40 steps
There was a problem hiding this comment.
do you need torch model in these tests or openvino only enough? We have several preconverted models here https://huggingface.co/collections/OpenVINO/image-generation-67697d9952fb1eee4a252aa8 (I requested dreamlike models too, but current status is waiting approval)
among trained but still small is https://huggingface.co/segmind/tiny-sd but not sure that used in model scheduler currently supported
Yes, I need a Torch model and I tried "segmind/tiny-sd" but it didn't work OOB.
And, yes num_infeence_steps is already exposed and I am planning to use it in the test to reduce the test time if I don't find a dummy model that works.
CVS-159223