Diminishing weights

[numersoz]

Does river support diminish weights?

As new data comes in and model updates, I want the newer samples added to have more weight compared to the oldest.

[gbolmier]

Hi @numersoz 👋

Online learning algorithms naturally "forget" about past observations over time as they learn from new ones. If you want to speed up that forgetting part you can use the sample_weight argument of the learn_one method of your model. Simple example that adds an increment of 0.01 to each new sample weight:

sample_weight, increment = 1, 0.01
for x, y in X_y:
    y_pred = model.predict_one(x)

    sample_weight += increment
    model.learn_one(x, y, sample_weight=sample_weight)

The drawback here is to persist the last value of sample_weight and the value of increment in order to resume training later on. It could be nice to do that with a scheduler. I think the learn_one methods could be made compatible with the optim.schedulers module interface in a simple way, though it would require changes in many places. Any thoughts @online-ml/devs?

Feel free to raise a PR @numersoz if you feel like you have the momentum.

Hope that helps, cheers!

[numersoz]

Thank you @gbolmier. Can you please detail how river ml naturally forgets?

To me it sounds like every iteration is like training with the latest income data plus the past data.

It's not clear how the forget mechanics work. I would assume there is something similar to LSTM forget gate, but haven't seen much documentation on the site.

[MaxHalford]

That's just the nature of online learning. If you keep doing stochastic gradient descent, then gradually you forget past data. Indeed, the influence of past data on the weights diminishes. That's no different to any deep learning algorithm that is trained through SGD. That's why it's recommended to shuffle your data when you train a deep learning model.

[nocluebutalotofit]

Is there a way to estimate how long the memory is of a model is?
In the engineering world "exponential" forgetting is often referred to as "tracking".
For an exponential moving average, I can calculate the half life and know that observations older than this only have have the weight as the newer observations. I guess in general, it is not possible to quantify a metric like this for a deep learning algorithm that is trained through SGD otherwise we would not have the problem of catastrophic forgetting, right?

[gbolmier]

Not that I'm aware of. I guess it's data and model dependent. The more different new data points are and the more sensitive the learning algorithm is, the quicker the model will "forget" previously learnt patterns.

One could monitor the performance of a given model on old data points to spot when the model is "forgetting".