Guidance on the training process/folder structure

[igoralvarezz]

Hi @OndrejTexler, I've read the paper and watched the video of your presentation on SIGGRAPH and want to congratulate you on the amazing work you made! As an artist/designer, I think that the possibilities your paper opens up for creativity are amazing, thanks for that!

I'm interested on the training process (to be able to create my own styles and apply them to my videos) and, although I understood the steps theoretically, I'm having a hard time running it on my side due to not being able to understand the training folder structure and what content I should have beforehand and what is generated by the _tools binaries before the training process itself. I have some python experience but I am really new on the torch/DL side of the things so looking at the code didn't helped me understand where each file should be.

Do you have any step-by-step guide/video on how to organize the files (e.g. frames from the original video and style frames)? I've used EbSynth before and love its results, but I am thinking on migrate to this python solution due to the increase in coherence in the end result and the automatic blending of the various references which makes not only the process more straightforward but also generate betters results overall from the results I was able to see.

Thanks in advance for the help and for the outstanding work

[nicolai256]

hey @igoralvarezz
in my fork of this project the folder structure is automatically created and u just need to run one script :)
https://github.com/nicolai256/Few-Shot-Patch-Based-Training

[aungkhant0911]

@nicolai256 Did you manage replicate the live training and inference on a live video feed as described by the author? The authors said the training shouldn't take more than a few mins (ideally 1 min) on training.

[nicolai256]

@nicolai256 Did you manage replicate the live training and inference on a live video feed as described by the author? The authors said the training shouldn't take more than a few mins (ideally 1 min) on training.

I've managed to enable the live video feed inference but not with training, you have to train it beforehand, there is probably a way to make it train while doing the live feed and refresh the model every 100 iterations or so