Issue when running run_generator.py (input depth must be evenly divisible by filter depth: 6 vs 4)

[vivekbharadhwajsa]

Constant issue about (input depth must be evenly divisible by filter depth: 6 vs 4)

`2021-10-13 12:57:27.644988: W tensorflow/core/framework/op_kernel.cc:1692] OP_REQUIRES failed at conv_ops.cc:654 : Invalid argument: input depth must be evenly divisible by filter depth: 6 vs 4
Traceback (most recent call last):
File "/home/sulugodu/.virtualenvs/ma_gan/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1375, in _do_call
return fn(*args)
File "/home/sulugodu/.virtualenvs/ma_gan/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1359, in _run_fn
return self._call_tf_sessionrun(options, feed_dict, fetch_list,
File "/home/sulugodu/.virtualenvs/ma_gan/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1451, in _call_tf_sessionrun
return tf_session.TF_SessionRun_wrapper(self._session, options, feed_dict,
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: input depth must be evenly divisible by filter depth: 6 vs 4
[[{{node Gs/_Run/Gs/G_synthesis/E_512x512/FromRGB/Conv2D}}]]
[[Gs/_Run/Gs/images_out/_1587]]
(1) Invalid argument: input depth must be evenly divisible by filter depth: 6 vs 4
[[{{node Gs/_Run/Gs/G_synthesis/E_512x512/FromRGB/Conv2D}}]]
0 successful operations.
0 derived errors ignored.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "run_generator.py", line 61, in
apply_predefined()
File "run_generator.py", line 55, in apply_predefined
create_from_images(pkl_path, img, mask)
File "run_generator.py", line 31, in create_from_images
fake = Gs.run(latent, None, real[np.newaxis], mask[np.newaxis], truncation_psi=truncation)[0]
File "/home/sulugodu/ma_gan/MA/Anonymization/co-mod-gan/dnnlib/tflib/network.py", line 445, in run
mb_out = tf.compat.v1.get_default_session().run(out_expr, dict(zip(in_expr, mb_in)))
File "/home/sulugodu/.virtualenvs/ma_gan/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 967, in run
result = self._run(None, fetches, feed_dict, options_ptr,
File "/home/sulugodu/.virtualenvs/ma_gan/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1190, in _run
results = self._do_run(handle, final_targets, final_fetches,
File "/home/sulugodu/.virtualenvs/ma_gan/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1368, in _do_run
return self._do_call(_run_fn, feeds, fetches, targets, options,
File "/home/sulugodu/.virtualenvs/ma_gan/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1394, in _do_call
raise type(e)(node_def, op, message) # pylint: disable=no-value-for-parameter
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: input depth must be evenly divisible by filter depth: 6 vs 4
[[node Gs/_Run/Gs/G_synthesis/E_512x512/FromRGB/Conv2D (defined at :60) ]]
[[Gs/_Run/Gs/images_out/_1587]]
(1) Invalid argument: input depth must be evenly divisible by filter depth: 6 vs 4
[[node Gs/_Run/Gs/G_synthesis/E_512x512/FromRGB/Conv2D (defined at :60) ]]
0 successful operations.
0 derived errors ignored.

Errors may have originated from an input operation.
Input Source operations connected to node Gs/_Run/Gs/G_synthesis/E_512x512/FromRGB/Conv2D:
Gs/_Run/Gs/G_synthesis/concat (defined at :387)
Gs/_Run/Gs/G_synthesis/E_512x512/FromRGB/mul (defined at :36)

Input Source operations connected to node Gs/_Run/Gs/G_synthesis/E_512x512/FromRGB/Conv2D:
Gs/_Run/Gs/G_synthesis/concat (defined at :387)
Gs/_Run/Gs/G_synthesis/E_512x512/FromRGB/mul (defined at :36)

Original stack trace for 'Gs/_Run/Gs/G_synthesis/E_512x512/FromRGB/Conv2D':
File "run_generator.py", line 61, in
apply_predefined()
File "run_generator.py", line 55, in apply_predefined
create_from_images(pkl_path, img, mask)
File "run_generator.py", line 31, in create_from_images
fake = Gs.run(latent, None, real[np.newaxis], mask[np.newaxis], truncation_psi=truncation)[0]
File "/home/sulugodu/ma_gan/MA/Anonymization/co-mod-gan/dnnlib/tflib/network.py", line 420, in run
out_gpu = net_gpu.get_output_for(*in_gpu, return_as_list=True, **dynamic_kwargs)
File "/home/sulugodu/ma_gan/MA/Anonymization/co-mod-gan/dnnlib/tflib/network.py", line 224, in get_output_for
out_expr = self._build_func(*final_inputs, **build_kwargs)
File "", line 239, in G_main
File "/home/sulugodu/ma_gan/MA/Anonymization/co-mod-gan/dnnlib/tflib/network.py", line 224, in get_output_for
out_expr = self._build_func(*final_inputs, **build_kwargs)
File "", line 391, in G_synthesis_RegionGAN
File "", line 350, in E_fromrgb
File "", line 60, in conv2d_layer
File "/home/sulugodu/.virtualenvs/ma_gan/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py", line 206, in wrapper
return target(*args, **kwargs)
File "/home/sulugodu/.virtualenvs/ma_gan/lib/python3.8/site-packages/tensorflow/python/ops/nn_ops.py", line 2281, in conv2d_v2
return conv2d(input, # pylint: disable=redefined-builtin
File "/home/sulugodu/.virtualenvs/ma_gan/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py", line 206, in wrapper
return target(*args, **kwargs)
File "/home/sulugodu/.virtualenvs/ma_gan/lib/python3.8/site-packages/tensorflow/python/ops/nn_ops.py", line 2388, in conv2d
return gen_nn_ops.conv2d(
File "/home/sulugodu/.virtualenvs/ma_gan/lib/python3.8/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 969, in conv2d
_, _, _op, _outputs = _op_def_library._apply_op_helper(
File "/home/sulugodu/.virtualenvs/ma_gan/lib/python3.8/site-packages/tensorflow/python/framework/op_def_library.py", line 748, in _apply_op_helper
op = g._create_op_internal(op_type_name, inputs, dtypes=None,
File "/home/sulugodu/.virtualenvs/ma_gan/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 3561, in _create_op_internal
ret = Operation(
File "/home/sulugodu/.virtualenvs/ma_gan/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 2045, in init
self._traceback = tf_stack.extract_stack_for_node(self._c_op) `

[vivekbharadhwajsa]

print((latent.shape, real[np.newaxis].shape, mask[np.newaxis].shape))
gives
((1, 512), (1, 3, 512, 512), (1, 3, 512, 512))
as output

[vivekbharadhwajsa]

I tried this with the ffhq-9(512512) and places-2(512512) pre-trained models from your repo. I get the same error all the time.

[zsyzzsoft]

Seems that your input image has 6 channels? How did you generate the custom dataset?

[vivekbharadhwajsa]

I did not train my model. I wanted to run inference with your pre-trained model and my image and mask.

The create_from_images in run_generator.py just takes the real image and mask right? It doesn't take the tfrecord as input.
When I use my custom image and mask (both 512x512) I get this issue of shapes.

[ghost]

Oh I misunderstood. Can you print out the shape of the mask image after line 11?

[vivekbharadhwajsa]

print((mask[np.newaxis].shape, mask.shape))
returns
((1, 3, 512, 512), (3, 512, 512))

[zsyzzsoft]

Can you provide your mask image? On my side the shape of the mask should always be (512, 512) before newaxis

[vivekbharadhwajsa]

Here are the image and mask samples I use.

[zsyzzsoft]

What is your PIL version? You can use 'pip show Pillow'

[vivekbharadhwajsa]

What is your PIL version? You can use 'pip show Pillow'

It is 8.3.2

[zsyzzsoft]

I still cannot reproduce your error... Did you modify the code?

[vivekbharadhwajsa]

I still cannot reproduce your error... Did you modify the code?

Here is my run_generator.py script

import argparse
import numpy as np
import PIL.Image
from numpy.testing._private.utils import import_nose
from dnnlib import tflib
from training import misc
import os
import glob
import cv2
import numpy as np
from dataset_tools.create_from_images import create_from_image
import tensorflow as tf
from tensorflow.keras.preprocessing.image import img_to_array

if not os.path.exists('co-mod-gan_imgs'):
os.mkdir('co-mod-gan_imgs')

def create_from_images(checkpoint, image, mask, truncation=None):
real_img = PIL.Image.open(image).convert('RGB')
real_img = real_img.resize((512, 512))
f_name = image.split('/')[-1]
real_img.save('co-mod-gan_imgs/'+f_name)
real = np.asarray(real_img).transpose([2, 0, 1])
real = misc.adjust_dynamic_range(real, [0, 255], [-1, 1])
mask = np.asarray(mask.transpose([2, 0, 1]), dtype=np.float32)[np.newaxis]
mask = np.squeeze(mask, axis=0)
#mask = np.squeeze(mask, axis=0)
#real = np.squeeze(real, axis=0)
tflib.init_tf()
_, , Gs = misc.load_pkl(checkpoint)
print('loaded network from pkl...')
latent = np.random.randn(1, *Gs.input_shape[1:])
#print((latent.shape, img_to_array(real).shape, img_to_array(mask).shape))
fake = Gs.run(latent, None, tf.convert_to_tensor(real[np.newaxis]), tf.convert_to_tensor(mask[np.newaxis]), truncation_psi=truncation)[0]
fake = misc.adjust_dynamic_range(fake, [-1, 1], [0, 255])
fake = fake.clip(0, 255).astype(np.uint8).transpose([1, 2, 0])
fake = PIL.Image.fromarray(fake)
fake.save('co-mod-gan_imgs/out'+f_name)

def create_from_image_test(checkpoint, image_dir, mask_dir, truncation=None):
real_img = PIL.Image.open(image_dir).convert('RGB')
real_img = real_img.resize((512, 512))
f_name = image_dir.split('/')[-1]
real_img.save('co-mod-gan_imgs/'+f_name)
tflib.init_tf()
_, , Gs = misc.load_pkl(checkpoint)
print('loaded network from pkl...')
latent = np.random.randn(1, *Gs.input_shape[1:])
#print((latent.shape, real[np.newaxis].shape, mask[np.newaxis].shape))
real = create_from_image('real', image_dir, 512, 3, 8, False, False)
mask = create_from_image('mask', mask_dir, 512, 3, 8, False, False)
fake = Gs.run(latent, None, real, mask, truncation_psi=truncation)[0]
fake = misc.adjust_dynamic_range(fake, [-1, 1], [0, 255])
fake = fake.clip(0, 255).astype(np.uint8).transpose([1, 2, 0])
fake = PIL.Image.fromarray(fake)
fake.save('co-mod-gan_imgs/out'+f_name)

def main():
parser = argparse.ArgumentParser()
parser.add_argument('-c', '--checkpoint', help='Network checkpoint path', required=True)
parser.add_argument('-i', '--image', help='Original image path', required=True)
parser.add_argument('-m', '--mask', help='Mask path', required=True)
parser.add_argument('-t', '--truncation', help='Truncation psi for the trade-off between quality and diversity. Defaults to 1.', default=None)
args = parser.parse_args()
create_from_images(**vars(args))

def apply_predefined():
pkl_path = '/home/sulugodu/ma_gan/MA/Test/co_mod_gan/co-mod-gan-ffhq-9-025000.pkl'
data_path = '/home/sulugodu/ma_gan/MA/Anonymization/Datasets/faces_bbox/images/test'
imgs = glob.glob(data_path+'/*')[:10]
for i, img in enumerate(imgs):
mask_path = img.replace('Datasets/faces_bbox/images/test', 'u2net_seg_imgs')
mask = PIL.Image.open(mask_path).resize((512, 512))
mask = cv2.bitwise_not(np.array(mask))
mask = cv2.threshold(mask, 0, 255, cv2.THRESH_BINARY)
create_from_images(pkl_path, img, mask[1])
if i%1000 == 0:
print('{} images processed with co-mod-gan...'.format(i))
#for i, img in enumerate(imgs):
# mask_path = img.replace('Datasets/faces_bbox/images/test', 'u2net_seg_imgs')
# create_from_image_test(pkl_path, img, mask_path)
# if i%1000 == 0:
# print('{} images processed with co-mod-gan...'.format(i))

if name == "main":
#main()
apply_predefined()

[zsyzzsoft]

I see. You should make the shape of the mask to (1, 512, 512) before Gs.run. In your case you can 'mask = mask[:, 0]' after line 11.

[vivekbharadhwajsa]

It worked.

Can you provide an example mask image and it's shape as well?

[zsyzzsoft]

There is an example mask image in the 'imgs' folder.

[vivekbharadhwajsa]

I tried and it worked. But the results looks very bad. Here are some examples:

Example 1:
Image:
Mask:
Output:

Example 2:
Image:
Mask:
Output:

Any suggestions on why the results are bad?

[zsyzzsoft]

You did not adjust dynamic range for input images like line 10 in run_generator.py