Toy data is already independent

[caph1993]

Hello Gilles, I wanted to verify the following claim about your toy example.

The data (X, y, z) in your example is generated from a source for which Y and Z are independent, i.e I(Y, Z)=0. If this is correct, then the adversarial network R will try to find the marginal distribution of Z regardless of the prediction f(X) produced by D because p(z|y) = p(z) for all y~Y. And as a consequence the optimal solution would be to train D and R separately.

It appears to me that the adversarial network R found a very small relationship between Z and Y that was introduced during the sampling process and we are trying to eliminate (or reduce) this fictitious relationship by tuning D. That explains the plots, and the large value of lambda that was needed, but I think it was not the original purpose. Do you think it would be better to introduce some relationship between Z and Y for future experiments?