fix: Re-instantiate stateful metrics in PerformanceTrigger (#630)

Since our metric objects are stateful, we had a form of averaging on the
metrics, instead of evaluating them on each window separately.

modyn/supervisor/internal/triggers/performance/performancetrigger_mixin.py

@@ -93,6 +93,9 @@ def _run_evaluation(
         assert self.model_downloader is not None
         assert self.context and self.context.pipeline_config is not None
 
+        # Since the metric objects are stateful, we need to re-instantiate them before each evaluation.
+        self._metrics = setup_metrics(self.config.evaluation.dataset.metrics)
+
         evaluation_dataloader = prepare_trigger_dataloader_fixed_keys(
             self.dataloader_info, [key for key, _ in interval_data]
         )