Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for IP Adapter Full Face #2252

Merged
merged 3 commits into from
Nov 13, 2023

Conversation

sdbds
Copy link
Collaborator

@sdbds sdbds commented Nov 12, 2023

Closes #2250
just support for IP new model
tested with 1.1.416
image

@sdbds sdbds requested a review from lllyasviel November 12, 2023 07:44
@huchenlei huchenlei self-requested a review November 12, 2023 18:38
@huchenlei
Copy link
Collaborator

I will add some unittests for this once the other test PR landed.

@huchenlei huchenlei force-pushed the ipadapter_full_model branch from 1698c50 to 5d9a811 Compare November 13, 2023 01:59
@huchenlei
Copy link
Collaborator

Interesting. Those unittests pass locally.

@huchenlei
Copy link
Collaborator

huchenlei commented Nov 13, 2023

@sdbds Seems like we have issue running IPAdapter on CPU.

2023-11-13 02:31:04,524 - ControlNet - �[0;32mINFO�[0m - Loading model: ip-adapter-full-face_sd15 [852b9843]
2023-11-13 02:31:04,548 - ControlNet - �[0;32mINFO�[0m - Loaded state_dict from [/home/runner/work/sd-webui-controlnet/sd-webui-controlnet/stable-diffusion-webui/extensions/sd-webui-controlnet/models/ip-adapter-full-face_sd15.safetensors]
2023-11-13 02:31:04,773 - ControlNet - �[0;32mINFO�[0m - ControlNet model ip-adapter-full-face_sd15 [852b9843] loaded.
2023-11-13 02:31:04,776 - ControlNet - �[0;32mINFO�[0m - Loading preprocessor: ip-adapter_clip_sd15
2023-11-13 02:31:04,776 - ControlNet - �[0;32mINFO�[0m - preprocessor resolution = -1
*** Error running process: /home/runner/work/sd-webui-controlnet/sd-webui-controlnet/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py
    Traceback (most recent call last):
      File "/home/runner/work/sd-webui-controlnet/sd-webui-controlnet/stable-diffusion-webui/modules/scripts.py", line 619, in process
        script.process(p, *script_args)
      File "/home/runner/work/sd-webui-controlnet/sd-webui-controlnet/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py", line 981, in process
        self.controlnet_hack(p)
      File "/home/runner/work/sd-webui-controlnet/sd-webui-controlnet/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py", line 970, in controlnet_hack
        self.controlnet_main_entry(p)
      File "/home/runner/work/sd-webui-controlnet/sd-webui-controlnet/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py", line 808, in controlnet_main_entry
        detected_map, is_image = preprocessor(
      File "/home/runner/work/sd-webui-controlnet/sd-webui-controlnet/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/utils.py", line 75, in decorated_func
        return cached_func(*args, **kwargs)
      File "/home/runner/work/sd-webui-controlnet/sd-webui-controlnet/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/utils.py", line 63, in cached_func
        return func(*args, **kwargs)
      File "/home/runner/work/sd-webui-controlnet/sd-webui-controlnet/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/global_state.py", line 35, in unified_preprocessor
        return preprocessor_modules[preprocessor_name](*args, **kwargs)
      File "/home/runner/work/sd-webui-controlnet/sd-webui-controlnet/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/processor.py", line 350, in clip
        from annotator.clipvision import ClipVisionDetector
      File "/home/runner/work/sd-webui-controlnet/sd-webui-controlnet/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/clipvision/__init__.py", line 82, in <module>
        clip_vision_h_uc = torch.load(clip_vision_h_uc)['uc']
      File "/home/runner/work/sd-webui-controlnet/sd-webui-controlnet/stable-diffusion-webui/modules/safe.py", line 108, in load
        return load_with_extra(filename, *args, extra_handler=global_extra_handler, **kwargs)
      File "/home/runner/work/sd-webui-controlnet/sd-webui-controlnet/stable-diffusion-webui/modules/safe.py", line 156, in load_with_extra
        return unsafe_torch_load(filename, *args, **kwargs)
      File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/torch/serialization.py", line 809, in load
        return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
      File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/torch/serialization.py", line 1172, in _load
        result = unpickler.load()
      File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/pickle.py", line 1213, in load
        dispatch[key[0]](self)
      File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/pickle.py", line 1254, in load_binpersid
        self.append(self.persistent_load(pid))
      File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/torch/serialization.py", line 1142, in persistent_load
        typed_storage = load_tensor(dtype, nbytes, key, _maybe_decode_ascii(location))
      File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/torch/serialization.py", line 1116, in load_tensor
        wrap_storage=restore_location(storage, location),
      File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/torch/serialization.py", line 217, in default_restore_location
        result = fn(storage, location)
      File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/torch/serialization.py", line 182, in _cuda_deserialize
        device = validate_cuda_device(location)
      File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/torch/serialization.py", line 166, in validate_cuda_device
        raise RuntimeError('Attempting to deserialize object on a CUDA '
    RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.

Maybe the solution is #2248

@huchenlei huchenlei force-pushed the ipadapter_full_model branch from 6ec74a0 to b878287 Compare November 13, 2023 02:44
@huchenlei huchenlei merged commit 711203e into Mikubill:main Nov 13, 2023
1 check passed
@sdbds sdbds deleted the ipadapter_full_model branch November 14, 2023 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Request]: IP Adapter Full Face Support
2 participants