Optimal hyperparameters and optimal flow design

[dodreh]

Hello! Thank you for excellent framework!

For my bachelor thesis I have to adapt SAILOR (Link to project: https://github.com/malicd/sailor). I should implement NF instead of Gaussian Mixture Model.
Short about SAILOR: SAILOR is unsupervised algorithm for domain adoption. It uses GMM to compress big dataset and to train the model on target dataset.

How I used NF in SAILOR:
Architecture of transformation/flow:
Number of flows = 4

I am not sure if the number of hidden layers is optimally chosen or not?

number of epochs = 10 000
learning rate = 0.000002

Problem: I have tried to design flows in different ways (flows = 32, 16, 8, 4; changing lr and epochs many times, diff. number of hidden layers, different number of units in hidden layers and so on), but no version gave me good results. During the fitting the model in data I have most of the time case that my loss value is "inf" (or "Nan"). A value for loss starts from 365 and decreasing, and in one moment it starts to print "inf" or "nan". I have also printed loss in diagram (number of epochs = 15 000) and at around 2500 and 3000 epoch the diagram shows two very big spikes, after it continues to decrease.
Question: Is there some standard procedure how to choose hyperparameters (lr, number of epochs) and how to design flows? If no, can you give me advice how to create optimal flows with optimal parameters (lr, epochs)? Is 15 000 epochs too big or I can consider it as normal value for epochs?

Thank you in advance!

[dodreh]

P.S. Here are two screenshots of loss diagram: