How to check which model is better when handling uncertinity?

[jomarca11]

Hi.
I'm checking two models, in fact they are the same but in one I'm using adaptation to drift, but not in the other.
But not sure if handling drift is better or not than not handling drift.
Which measures I should check?

Thanks in advance!

[kulbachcedric]

HI @jomarca11,

for checking which model performs better in adaptation to concept drifts, I think you could use the Rolling metric with a small window size and pass the metric to the wrapper you want to measure (e.g. Accuracy, MSE). Plotting the rolling metric over the instances of the data stream helps to see which model adapts better.

Best
Cedric

[jomarca11]

Hi @kulbachcedric , many thanks for your support here. That's what I did for me it's a bit bizzarre that handling concept drift wheh handling uncertinity, the model almost does not improve. I though it would be better.

Best Xavi