Question about coding choice: why are beta and alpha clamped?

[landoskape]

This isn't a bug - just a scientific / implementation question.

In the Leaky and Synaptic class (and probably other places), alpha and beta are constrained to be between 0 and 1 by torch.clamp(beta, 0, 1). This means that they both could grow outside the range from learning (or by the user setting a higher value without realizing), but that value won't actually be used.

Any reason not to use torch.sigmoid() as a wrapper on beta so it can have values from $-\infty$ to $\infty$?

This would make reading out self.beta less informative, but of course you could just write:

def get_beta(self):
    return torch.sigmoid(self.beta)

[jeshraghian]

Using sigmoid could be totally legitimate - it's something I thought of when first putting the Leaky/Synaptic neurons together. I don't have a good answer for you - I don't know whether one approach works better or not.

I just let minimalism guide a lot of the choices I made... But if we were to discover adding sigmoid helped a lot, then I'd be open to enabling options for it.