Part 2: GAN for image translation

[edyoshikun]

This PR has exercise 2 using the GAN approach. Ideally, this should be merged after #6 or merged into #6.

I put the previous PR here to see if someone else can reproduce my errors here.

Major Changes:

• TODO @Tonks684 please write a summary.

TODO:

• Review and merge part_1 before part_2
• Review that the setup.sh runs
• Test everything from scratch

[edyoshikun]

since you are using mixed between html and markdown these don't render well in VSCode.

[edyoshikun]

Take a look at all the markdown cells

[edyoshikun]

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[12], line 2
      1 # %%
----> 2 train_model(
      3     opt,
      4     phase2nuclei_model,
      5     visualizer,
      6     dataset_train,
      7     dataset_val,
      8     optimizer_G,
      9     optimizer_D,
     10     start_epoch,
     11     epoch_iter,
     12     writer,
     13 )

File ~/data/06_image_translation/part2/GAN_code/GANs_MI2I/pix2pixHD/train_dlmbl.py:209, in train(opt, model, visualizer, dataset_train, dataset_val, optimizer_G, optimizer_D, start_epoch, epoch_iter, writer)
    206 train_loss_D_fake, train_loss_D_real, train_loss_G_GAN, train_loss_G_Feat, train_loss_G_VGG, = train_epoch(opt, model, visualizer, dataset_train, optimizer_G, optimizer_D, total_steps, epoch, epoch_iter)
    208 [val_loss_D_fake, val_loss_D_real, val_loss_G_GAN, val_loss_G_Feat, val_loss_G_VGG], virtual_stain, fluorescence, brightfield = val_epoch(opt, model, dataset_val)
--> 209 visualizer.results_plot(brightfield,fluorescence,virtual_stain,['Phase Contrast', 'Fluorescence', 'Virtual Stain'],writer,epoch,rows=brightfield.shape[0])
    211 # Tensorboard Logging
    212 epoch_discriminator = {'fake_is_fake': train_loss_D_fake, 'real_is_real': train_loss_D_real}

File ~/data/06_image_translation/part2/GAN_code/GANs_MI2I/pix2pixHD/util/visualizer.py:190, in Visualizer.results_plot(self, input_x, target, predictions, titles, writer, epoch, rows)
    187 fig, axs = plt.subplots(input_x.shape[0], 3, figsize=(10, 30))
    189 # Set the titles for each column
--> 190 axs[0, 0].set_title(titles[0])
    191 axs[0, 1].set_title(titles[1])
    192 axs[0, 2].set_title(titles[2])

IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed

[edyoshikun]

I suggest you split this visualizer and show the stuff later.

[Tonks684]

Hi Ed I have commited the fix for this to the https://github.com/Tonks684/GANs_MI2I repo (Tonks684/GANs_MI2I@47c6344). It was due to the default batch being set to 1 (for speed of showing the training but the plot expected multiple images to display). The plot now works for batch size 1 and the default batch size has been set to 8 to show the students multiple images

[Tonks684]

Currently have the following issue

[Tonks684]

All packages are installed that are suggested. @edyoshikun are you aware of any other dependencies?

[edyoshikun]

Ah yes, good catch. I'll fix the coco-metrics dependency. I forgot in viscy we have the custom pip install "viscy[metrics]"
https://github.com/mehta-lab/VisCy/blob/9b6e3f257ded5a0d7d8d5e8d9388b5e485c1ec1e/pyproject.toml#L28C1-L34C2