distinction of cross entropy and KL divergence

[schwabPhysics]

I'm not entirely sure of the inner workings of the algorithm, but when reading this documentation and comparing with other sources I found that the expression for what was named 'cross entropy' did not seem correct. Instead, there are two separate terms describing two separate KL divergences (one for the change in entropy in the probability of the simplex existing, and one for not existing). It is not clear in the text (even with my suggestions) why one needs two divergences. I make no claims to the workings of the algorithm, but only suggest changes to the descriptions of the mathematics in the documentation.

I hope it makes sense, and thanks to everyone for putting this great resource together!

[jlmelville]

Seems like the current definition for the fuzzy cross-entropy is the same as Definition 3.1 in https://link.springer.com/article/10.1186/s40467-015-0029-5?

[schwabPhysics]

thanks for the resource jlmelville, I just found that as well. I guess the way to remedy the ambiguity in the doc is to use 'fuzzy set cross entropy' instead of cross entropy. Do you think that would solve it?

[jlmelville]

I agree that specifying the fuzzy set cross entropy (with a link to that publication via https://doi.org/10.1186/s40467-015-0029-5) might help avoid any potential confusion.