wide-resnet network structure is not standard or exits errors.

[gogo03]

I have tried pytorch version wide-resnet, N=4,ch=10,It has 28 Conv2Ds,
but yours only have 20 Conv2Ds.
and the accuarcy of cifar100 is only up to about 75%.

[koshian2]

@gogo03
The number of layers was different. Thank you. I was wondering that WRN should not be so poor without OctConv.
Here is the WRN 28-10 implemented by the auto augment official.
Because I used WRN 28-10 at another opportunity, I ported the WRN code in the auto augment repo of TensorFlow to Keras. If we replace Conv2D here with OctConv, we may get better accuracy.
I will try if there is time. Please let us know your result on PyTorch if possible.

[koshian2]

@gogo03
Hi, I fixed the bug that the number of layers of wide resnet is different. It has been committed in the repository. The learning curve and prediction time are all remeasured, and the new value is displayed in the readme.md.

The changes in models.py are as follows.

# Before
def _create_normal_residual_block(inputs, ch, N):
    # adujust channels
    x = layers.Conv2D(ch, 3, padding="same")(inputs)
    x = layers.BatchNormalization()(x)
    x = layers.Activation("relu")(x)
    # Conv with skip connections
    for i in range(N-1):
        skip = x
        x = layers.Conv2D(ch, 3, padding="same")(x)
        x = layers.BatchNormalization()(x)
        x = layers.Activation("relu")(x)
        x = layers.Conv2D(ch, 3, padding="same")(x)
        x = layers.BatchNormalization()(x)
        x = layers.Activation("relu")(x)
        x = layers.Add()([x, skip])
    return x

# After
def _create_normal_residual_block(inputs, ch, N):
    # Conv with skip connections
    x = inputs
    for i in range(N):
        # adjust channels
        if i == 0:
            skip = layers.Conv2D(ch, 1)(x)
            skip = layers.BatchNormalization()(skip)
            skip = layers.Activation("relu")(skip)
        else:
            skip = x
        x = layers.Conv2D(ch, 3, padding="same")(x)
        x = layers.BatchNormalization()(x)
        x = layers.Activation("relu")(x)
        x = layers.Conv2D(ch, 3, padding="same")(x)
        x = layers.BatchNormalization()(x)
        x = layers.Activation("relu")(x)
        x = layers.Add()([x, skip])

Before correction, the first Conv-BN-ReLU of res block was one less. This has been fixed. The overall conclusion has not changed.
Thank you for the report.

[gogo03]

excellent work! thank you, I have test Cutout using Pytorch and wide-resnet without OctConv,also don't use cutout method.the accuracy of Cifar100 is about 81.5%。 here is the website:
https://github.com/uoguelph-mlrg/Cutout/blob/master/train.py

Could you test OctConv in keras using octconv on the dataset of Cifar100?

[koshian2]

@gogo03
Hi. Thank you for sharing the result. Cutout is data augmentation, OctConv is network structure, and it can not be simply compared with accuracy.
We can improve the accuracy with data augmentation alone. If the accuracy of a model using only OctConv is lost to the accuracy of the model using a lot of data augmentation, it does not mean that OctConv is bad.
The effectiveness of OctConv may not be understood well unless we compare with whether or not use OctConv under the same conditions of data augmentation and regularization.