ControlNet Reference Error with Flux

[MarianSijanin]

Latest ComfyUI and ComfyUI-Advanced-ControlNet. Workflow included.
NB, I use Flux-Dev NF4.

Flux-reference-error.json

ComfyUI Error Report

Error Details

• Node Type: KSampler
• Exception Type: AttributeError
• Exception Message: 'NoneType' object has no attribute 'get'

Stack Trace

  File "/home/marian/ComfyUI/execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/marian/ComfyUI/execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/marian/ComfyUI/execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)

  File "/home/marian/ComfyUI/execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/marian/ComfyUI/nodes.py", line 1430, in sample
    return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/marian/ComfyUI/nodes.py", line 1397, in common_ksampler
    samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/marian/ComfyUI/custom_nodes/ComfyUI-Advanced-ControlNet/adv_control/sampling.py", line 206, in acn_sample
    model.model.diffusion_model.forward = reference_injections.diffusion_model_orig_forward.__get__(model.model.diffusion_model, type(model.model.diffusion_model))
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

System Information

• ComfyUI Version: v0.2.2-44-g56e8f5e
• Arguments: main.py --listen=0.0.0.0
• OS: posix
• Python Version: 3.12.3 (main, Sep 11 2024, 14:17:37) [GCC 13.2.0]
• Embedded Python: false
• PyTorch Version: 2.4.1+cu124

Devices

• Name: cuda:0 NVIDIA GeForce RTX 3060 : cudaMallocAsync
  • Type: cuda
  • VRAM Total: 12519473152
  • VRAM Free: 7158459570
  • Torch VRAM Total: 5268045824
  • Torch VRAM Free: 82275506

Logs

2024-09-22 12:38:52,470 - root - INFO - Total VRAM 11940 MB, total RAM 32005 MB
2024-09-22 12:38:52,471 - root - INFO - pytorch version: 2.4.1+cu124
2024-09-22 12:38:52,471 - root - INFO - Set vram state to: NORMAL_VRAM
2024-09-22 12:38:52,471 - root - INFO - Device: cuda:0 NVIDIA GeForce RTX 3060 : cudaMallocAsync
2024-09-22 12:38:53,127 - root - INFO - Using pytorch cross attention
2024-09-22 12:38:54,018 - root - INFO - [Prompt Server] web root: /home/marian/ComfyUI/web
2024-09-22 12:38:54,020 - root - INFO - Adding extra search path checkpoints /home/marian/stable-diffusion-webui/models/Stable-diffusion
2024-09-22 12:38:54,020 - root - INFO - Adding extra search path configs /home/marian/stable-diffusion-webui/models/Stable-diffusion
2024-09-22 12:38:54,020 - root - INFO - Adding extra search path vae /home/marian/stable-diffusion-webui/models/VAE
2024-09-22 12:38:54,020 - root - INFO - Adding extra search path loras /home/marian/stable-diffusion-webui/models/Lora
2024-09-22 12:38:54,020 - root - INFO - Adding extra search path loras /home/marian/stable-diffusion-webui/models/LyCORIS
2024-09-22 12:38:54,020 - root - INFO - Adding extra search path upscale_models /home/marian/stable-diffusion-webui/models/ESRGAN
2024-09-22 12:38:54,020 - root - INFO - Adding extra search path upscale_models /home/marian/stable-diffusion-webui/models/RealESRGAN
2024-09-22 12:38:54,020 - root - INFO - Adding extra search path upscale_models /home/marian/stable-diffusion-webui/models/SwinIR
2024-09-22 12:38:54,020 - root - INFO - Adding extra search path embeddings /home/marian/stable-diffusion-webui/embeddings
2024-09-22 12:38:54,020 - root - INFO - Adding extra search path hypernetworks /home/marian/stable-diffusion-webui/models/hypernetworks
2024-09-22 12:38:54,020 - root - INFO - Adding extra search path controlnet /home/marian/stable-diffusion-webui/models/ControlNet
2024-09-22 12:38:54,317 - root - INFO - 
Import times for custom nodes:
2024-09-22 12:38:54,317 - root - INFO -    0.0 seconds: /home/marian/ComfyUI/custom_nodes/websocket_image_save.py
2024-09-22 12:38:54,317 - root - INFO -    0.0 seconds: /home/marian/ComfyUI/custom_nodes/ComfyUI-Adaptive-Guidance
2024-09-22 12:38:54,317 - root - INFO -    0.0 seconds: /home/marian/ComfyUI/custom_nodes/ComfyUI-Advanced-ControlNet
2024-09-22 12:38:54,317 - root - INFO -    0.0 seconds: /home/marian/ComfyUI/custom_nodes/ComfyUI-Manager
2024-09-22 12:38:54,317 - root - INFO -    0.0 seconds: /home/marian/ComfyUI/custom_nodes/ComfyUI_bitsandbytes_NF4
2024-09-22 12:38:54,317 - root - INFO - 
2024-09-22 12:38:54,325 - root - INFO - Starting server

2024-09-22 12:38:54,325 - root - INFO - To see the GUI go to: http://0.0.0.0:8188
2024-09-22 12:39:00,342 - root - INFO - got prompt
2024-09-22 12:39:00,525 - root - INFO - model weight dtype torch.bfloat16, manual cast: None
2024-09-22 12:39:00,526 - root - INFO - model_type FLUX
2024-09-22 12:39:00,577 - root - INFO - Using pytorch attention in VAE
2024-09-22 12:39:00,579 - root - INFO - Using pytorch attention in VAE
2024-09-22 12:39:01,458 - root - INFO - Requested to load AutoencodingEngine
2024-09-22 12:39:01,459 - root - INFO - Loading 1 new model
2024-09-22 12:39:01,496 - root - INFO - loaded completely 0.0 159.87335777282715 True
2024-09-22 12:39:02,011 - root - INFO - Requested to load FluxClipModel_
2024-09-22 12:39:02,012 - root - INFO - Loading 1 new model
2024-09-22 12:39:02,662 - root - INFO - loaded completely 0.0 4777.53759765625 True
2024-09-22 12:39:03,659 - root - ERROR - !!! Exception during processing !!! 'NoneType' object has no attribute '__get__'
2024-09-22 12:39:03,663 - root - ERROR - Traceback (most recent call last):
  File "/home/marian/ComfyUI/custom_nodes/ComfyUI-Advanced-ControlNet/adv_control/sampling.py", line 162, in acn_sample
    module = model.model.diffusion_model.input_blocks[i]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/marian/ComfyUI/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1729, in __getattr__
    raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'Flux' object has no attribute 'input_blocks'. Did you mean: 'single_blocks'?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/marian/ComfyUI/execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/marian/ComfyUI/execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/marian/ComfyUI/execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)
  File "/home/marian/ComfyUI/execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/marian/ComfyUI/nodes.py", line 1430, in sample
    return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/marian/ComfyUI/nodes.py", line 1397, in common_ksampler
    samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/marian/ComfyUI/custom_nodes/ComfyUI-Advanced-ControlNet/adv_control/sampling.py", line 206, in acn_sample
    model.model.diffusion_model.forward = reference_injections.diffusion_model_orig_forward.__get__(model.model.diffusion_model, type(model.model.diffusion_model))
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute '__get__'. Did you mean: '__ge__'?

2024-09-22 12:39:03,664 - root - INFO - Prompt executed in 3.32 seconds

Attached Workflow

Please make sure that workflow does not contain any sensitive information such as API keys or passwords.

{"last_node_id":41,"last_link_id":70,"nodes":[{"id":38,"type":"LoadImage","pos":{"0":31,"1":706},"size":[315,314],"flags":{},"order":0,"mode":0,"inputs":[],"outputs":[{"name":"IMAGE","type":"IMAGE","links":[59],"shape":3,"slot_index":0},{"name":"MASK","type":"MASK","links":null,"shape":3}],"properties":{"Node name for S&R":"LoadImage"},"widgets_values":["flux_dev_checkpoint_example.png","image"]},{"id":41,"type":"ACN_ReferenceControlNet","pos":{"0":369,"1":853},"size":{"0":365.4000244140625,"1":106},"flags":{},"order":1,"mode":0,"inputs":[],"outputs":[{"name":"CONTROL_NET","type":"CONTROL_NET","links":[62],"shape":3,"slot_index":0}],"properties":{"Node name for S&R":"ACN_ReferenceControlNet"},"widgets_values":["reference_attn",0.5,1]},{"id":39,"type":"ACN_ReferencePreprocessor","pos":{"0":423,"1":693},"size":[268.79998779296875,68.25919688425745],"flags":{},"order":6,"mode":0,"inputs":[{"name":"image","type":"IMAGE","link":59},{"name":"vae","type":"VAE","link":60},{"name":"latent_size","type":"LATENT","link":61}],"outputs":[{"name":"proc_IMAGE","type":"IMAGE","links":[63],"shape":3,"slot_index":0}],"properties":{"Node name for S&R":"ACN_ReferencePreprocessor"}},{"id":6,"type":"CLIPTextEncode","pos":{"0":27,"1":198},"size":{"0":422.84503173828125,"1":164.31304931640625},"flags":{},"order":4,"mode":0,"inputs":[{"name":"clip","type":"CLIP","link":58}],"outputs":[{"name":"CONDITIONING","type":"CONDITIONING","links":[64],"slot_index":0}],"title":"CLIP Text Encode (Positive Prompt)","properties":{"Node name for S&R":"CLIPTextEncode"},"widgets_values":["cute anime girl with massive fluffy fennec ears and a big fluffy tail blonde messy long hair blue eyes wearing a maid outfit with a long black gold leaf pattern dress and a white apron mouth open placing a fancy black forest cake with candles on top of a dinner table of an old dark Victorian mansion lit by candlelight with a bright window to the foggy forest and very expensive stuff everywhere there are paintings on the walls"],"color":"#232","bgcolor":"#353"},{"id":33,"type":"CLIPTextEncode","pos":{"0":33,"1":408},"size":[399.6305144057293,76],"flags":{"collapsed":false},"order":5,"mode":0,"inputs":[{"name":"clip","type":"CLIP","link":70,"slot_index":0}],"outputs":[{"name":"CONDITIONING","type":"CONDITIONING","links":[66],"slot_index":0}],"title":"CLIP Text Encode (Negative Prompt)","properties":{"Node name for S&R":"CLIPTextEncode"},"widgets_values":[""],"color":"#322","bgcolor":"#533"},{"id":35,"type":"FluxGuidance","pos":{"0":458,"1":73},"size":{"0":211.60000610351562,"1":58},"flags":{},"order":8,"mode":0,"inputs":[{"name":"conditioning","type":"CONDITIONING","link":65}],"outputs":[{"name":"CONDITIONING","type":"CONDITIONING","links":[57],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"FluxGuidance"},"widgets_values":[3.5]},{"id":8,"type":"VAEDecode","pos":{"0":836,"1":126},"size":{"0":210,"1":46},"flags":{},"order":10,"mode":0,"inputs":[{"name":"samples","type":"LATENT","link":52},{"name":"vae","type":"VAE","link":68}],"outputs":[{"name":"IMAGE","type":"IMAGE","links":[9],"slot_index":0}],"properties":{"Node name for S&R":"VAEDecode"}},{"id":9,"type":"SaveImage","pos":{"0":1215,"1":246},"size":[329.7038846057296,428.70110468425764],"flags":{},"order":11,"mode":0,"inputs":[{"name":"images","type":"IMAGE","link":9}],"outputs":[],"properties":{},"widgets_values":["ComfyUI"]},{"id":31,"type":"KSampler","pos":{"0":626,"1":260},"size":{"0":315,"1":262},"flags":{},"order":9,"mode":0,"inputs":[{"name":"model","type":"MODEL","link":69},{"name":"positive","type":"CONDITIONING","link":57},{"name":"negative","type":"CONDITIONING","link":67},{"name":"latent_image","type":"LATENT","link":51}],"outputs":[{"name":"LATENT","type":"LATENT","links":[52],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"KSampler"},"widgets_values":[392704293521184,"randomize",20,1,"euler","simple",1]},{"id":40,"type":"ACN_AdvancedControlNetApply","pos":{"0":766,"1":646},"size":{"0":285.6000061035156,"1":286},"flags":{},"order":7,"mode":0,"inputs":[{"name":"positive","type":"CONDITIONING","link":64},{"name":"negative","type":"CONDITIONING","link":66},{"name":"control_net","type":"CONTROL_NET","link":62},{"name":"image","type":"IMAGE","link":63},{"name":"mask_optional","type":"MASK","link":null},{"name":"timestep_kf","type":"TIMESTEP_KEYFRAME","link":null},{"name":"latent_kf_override","type":"LATENT_KEYFRAME","link":null},{"name":"weights_override","type":"CONTROL_NET_WEIGHTS","link":null},{"name":"model_optional","type":"MODEL","link":null},{"name":"vae_optional","type":"VAE","link":null}],"outputs":[{"name":"positive","type":"CONDITIONING","links":[65],"shape":3,"slot_index":0},{"name":"negative","type":"CONDITIONING","links":[67],"shape":3,"slot_index":1},{"name":"model_opt","type":"MODEL","links":null,"shape":3}],"properties":{"Node name for S&R":"ACN_AdvancedControlNetApply"},"widgets_values":[1,0,1,""]},{"id":27,"type":"EmptySD3LatentImage","pos":{"0":36,"1":541},"size":{"0":315,"1":106},"flags":{},"order":2,"mode":0,"inputs":[],"outputs":[{"name":"LATENT","type":"LATENT","links":[51,61],"slot_index":0,"shape":3}],"properties":{"Node name for S&R":"EmptySD3LatentImage"},"widgets_values":[768,768,1],"color":"#323","bgcolor":"#535"},{"id":37,"type":"CheckpointLoaderNF4","pos":{"0":32,"1":54},"size":{"0":315,"1":98},"flags":{},"order":3,"mode":0,"inputs":[],"outputs":[{"name":"MODEL","type":"MODEL","links":[69],"shape":3,"slot_index":0},{"name":"CLIP","type":"CLIP","links":[58,70],"shape":3,"slot_index":1},{"name":"VAE","type":"VAE","links":[60,68],"shape":3,"slot_index":2}],"properties":{"Node name for S&R":"CheckpointLoaderNF4"},"widgets_values":["flux1-dev-bnb-nf4-v2.safetensors"]}],"links":[[9,8,0,9,0,"IMAGE"],[51,27,0,31,3,"LATENT"],[52,31,0,8,0,"LATENT"],[57,35,0,31,1,"CONDITIONING"],[58,37,1,6,0,"CLIP"],[59,38,0,39,0,"IMAGE"],[60,37,2,39,1,"VAE"],[61,27,0,39,2,"LATENT"],[62,41,0,40,2,"CONTROL_NET"],[63,39,0,40,3,"IMAGE"],[64,6,0,40,0,"CONDITIONING"],[65,40,0,35,0,"CONDITIONING"],[66,33,0,40,1,"CONDITIONING"],[67,40,1,31,2,"CONDITIONING"],[68,37,2,8,1,"VAE"],[69,37,0,31,0,"MODEL"],[70,37,1,33,0,"CLIP"]],"groups":[],"config":{},"extra":{"ds":{"scale":0.7513148009015781,"offset":[252.48083169426988,30.552703115742183]}},"version":0.4}

Additional Context

[qlewer]

Same problem..

[Kosinkadink]

ReferenceNet only works with unets, not transformer-based models. However, someone has worked on a different type of reference controlnet thing that is specifically designed for flux. I will be porting that over soon.

[hellloveyy]

Does Reference already support flux?