Prevent unnecessary recalculation of proxy values

[alexhernandezgarcia]

Context

While the Batch kept track of the availability of rewards and log-rewards, it did not store the proxy values, which are not needed for the computation of the loss. However, the proxy values were (re-)computed for logging purposes and for the Buffer. This was a waste of computation, especially if the proxy is expensive.

Main changes

• The call of proxy.rewards() now allows returning the proxy values, alongside the rewards, if return_proxy is True.
• After computing the rewards, the Batch now also stores the proxy values.
• The GFlowNet agent now retrieves the proxy values from the Batch, without recomputing them.
• The GFlowNet agent computes the (natural) rewards from the log-rewards and vice-versa.

Other changes

• The buffer does not store the proxy values, since they are unused.
• Some aspects of the Batch have been extended and refactored for better functionality and clarity.
• The logger now logs stats about the log-rewards too.
• Extended tests.
• Extended docstring.

[ftherrien]

Thanks so much that was quick! I will "review" it later today

[ftherrien]

Seems great to me. My understanding is that you mainly added a method to access self.proxy_values, made use of an existing method to get the rewards and added methods to check if the reward is log. Doc-strings are clear, internal variables renaming makes sense and tests seem relevant.