Using base classifiers (HT & NB) or Adaptive classifiers with different drift detectors

[TawabG]

I am working on a research project that is focused around benchmarking drift detection algorithms.
I am wondering if it is possible to use base classifiers such as HT (https://riverml.xyz/dev/api/tree/HoeffdingTreeClassifier/) or adaptive models such as ARF (https://riverml.xyz/dev/api/ensemble/AdaptiveRandomForestClassifier/) with different drift detectors.
As of yet, ARF uses ADWIN as the default drift detector. Would it be possible to add other drift detectors?

[smastelini]

Hi @TawabG, you can change the drift and warning detectors in ARF. The change detectors can be set explicitly when initializing the model. This is also valid for multiple models.

Keep in mind that some learners are stationary, though. For instance, HT and HB do not have explicitly concept drift adaptation capabilities.

[TawabG]

Wouldn't it be possible to transform these learners? E.g. creating a pipeline of HT as a base learner. Then using DDM to detect the warning level. This should trigger a function to store values from the start of the warning detection up until DDM truly detects a drift.
After which these values are used to re-train the base learner.

[smastelini]

Hi @TawabG, that's totally doable! ARF starts creating a new model to replace the affected one after issuing a drift warning. If no warning detector is passed, the current model is simply replaced by a new one when a drift is detected. Otherwise, the model trained in the background replaces the outdated one.

Perhaps a standalone utility like that would be helpful. Something like a wrapper that receives a model and warning (optional) and drift detectors.

Feel free to create a PR using this idea. You can take ARF, Streaming Random Patches, or Leverage Bagging as inspiration.

What do you think @MaxHalford, should we add something like that in the drift module?

[MaxHalford]

Sounds interesting! But not sure I understand 100%. Would you like to chain a regressor with a drift detector in a pipeline? Or would the idea be to reset a model when a drift is detected? All this is doable of course :)