Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chronos: When the dataset is multivariate, Auto Model and Autotsest may have negative improvements. #35

Open
liangs6212 opened this issue Oct 19, 2021 · 1 comment

Comments

@liangs6212
Copy link

liangs6212 commented Oct 19, 2021

When I used forecaster\automodel\autotsest in the same file and tested it six times with the parameters unchanged, there were 3 negative improvements and 3 positive improvements.
P.S. The dataset is network_traffic, and there is no similar situation on the univariate dataset.
log path: cpx2-->/home/cpx/liangs/mul_single_log.txt

# TCNForecaster mse(AvgRate)  28026.504694819632
# TCNForecaster mse(total)  2.283820957717442e+22
# AutoTCN mse(AvgRate)  15804.34013031733 improve by  43.609307323868414 %
# AutoTCN mse(total)  1.2805747653238046e+22 improve by  43.92840817943666 %
# AutoTSEstimator mse(AvgRate)  11827.337451242214 improve by  57.799455979152626 %
# AutoTSEstimator mse(total)  9.407867757317523e+21 improve by  58.80645667285501 %

# TCNForecaster mse(AvgRate)  14584.228766034217
# TCNForecaster mse(total)  1.1828622532188679e+22
# AutoTCN mse(AvgRate)  15945.982666076641 improve by  -9.337167716498428 %
# AutoTCN mse(total)  1.30109186373673e+22 improve by  -9.995213744975754 %
# AutoTSEstimator mse(AvgRate)  18524.652257574562 improve by  -27.018387840413965 %
# AutoTSEstimator mse(total)  1.5047155456221437e+22 improve by  -27.20970185052668 %
@TheaperDeng
Copy link

The issue may caused by your param setting in AutoTSEstimator

future_seq_len=hp.randint(5, 10*2),

This may make the forecasting task to change (and it should not happen).

Still this give us an alarm that we need to give a runtime error when user give an hp sampling function to future_seq_len

related doc: https://analytics-zoo.readthedocs.io/en/latest/doc/PythonAPI/Chronos/autotsestimator.html#autotsestimator

@liangs6212 liangs6212 reopened this Oct 19, 2021
@liu-shaojun liu-shaojun transferred this issue from intel-analytics/BigDL-2.x Mar 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants