Trainable decay scheme issue

[CameronSCarlin]

GRU-D/GRUD.py

Line 120 in 9f877f8

x = mask * x + (1 - mask) * ( delta_x * x + (1 - delta_x) * x_mean)

Hi,
My understanding of Equation 11 in the GRU-D paper is that the two x values on the right side of this equation are supposed to be different values. I believe mask * x to be correct, as that is multiplying x of the current time step. However, I believe delta_x * x is incorrect, as this x should be the value of x most recently observed prior to the current time step t.

[Thartvigsen]

This is worth fixing! Otherwise it's decaying the current imputed value.

[2M-kotb]

@Thartvigsen I noticed this issue not fixed yet in the code!!!!

[Thartvigsen]

@DeepWolf90 For now, update the equations in your local version so they are accurate. I'm sure the author is aware of the bug by now. Hopefully it'll be sorted out soon!

[zhiyongc]

Hi @CameronSCarlin,
Thanks for posting the flaw in the code. You are definitely correct. The issue is fixed in the updated version.
Also, thanks for your reminding and suggestions! @Thartvigsen and @DeepWolf90.