How 'metric' is being updated

[cdeterman]

I would like to clarify how the 'metric' (regardless of the specific metric chosen) is aggregated in different situations. For example, in the 'nowcasting' example here there is a metric.update call. And a single MAE is reported at the top of the chart. Whereas in the bike sharing forecasting here the use of evaluate.progressive_val_score is used where incremental MAE is printed. Is it safe to say that the error in both cases should be the same at the end of evaluate.progressive_val_score whereby the metric.update loop is just the final aggregate?

Or is there a lag to define the MAE in these examples? I wasn't sure where the utils.Rolling comes in to play.

[MaxHalford]

For example, in the 'nowcasting' example here there is a metric.update call. And a single MAE is reported at the top of the chart.

That's the last value of the metric.

Is it safe to say that the error in both cases should be the same at the end of evaluate.progressive_val_score whereby the metric.update loop is just the final aggregate?

Correct!

Or is there a lag to define the MAE in these examples? I wasn't sure where the utils.Rolling comes in to play.

One could also calculate a metric in a rolling fashion. If you printed out utils.Rolling(metrics.MAE(), 2000), you would get the MAE measured over the past 2000 samples, relative to when you're printing. The final output would be the MAE over the last 2000 samples.

I hope that clarifies your doubts :)

[cdeterman]

Yes, this clarifies everything, thank you.