Questions about the networks

[bluenights]

Thanks for the code! I'm wondering why using stochastic network in the feature extractor, and why adding noise in StatisticsNet.

[mohith-sakthivel]

Hi @bluenights,
I'm not aware of the intuition/rationale behind the use of a statistics network. I've used it only because the authors had used the same in the paper.

But if I remember right, I think you would get similar performance even if you replace the statistics network with an ordinary MLP. There are other similar MI bounds that use just an ordinary MLP network for discriminating the marginal/joint distributions. So it should be fine I guess.

Sorry that I couldn't be more helpful.