Reproducing results on MD-17 dataset (Training vs Validation MAE)

[Ashishrout1999]

Hi

I am trying to reprodcue results on MD-17 dataset molecules. However, from few of my runs, I observed that MAE in energy for training set is one order larger than that in validation and test set. Similarly, for forces also, MAE or RMSE of forces for training set is larger than that in validation and test set. This seems a bit counter-intuitive to me. However validation and test errors are pretty close to each other.

I am running the training with 950 samples, validation with 50 samples and test on the remaining samples. I am using all the default hyperparameters as mentioned in the paper except that I'm training the model for only 100 epochs. Even though, it is being run for small number of epochs (100), it's a bit difficult to understand why training error will be one order larger than validation/test error. Ex : For Aspirin, the energy MAE for training set was 1.733 kcal/mol(~74 meV), validation set was 0.1813 kcal/mol(~8 meV) and for test set was 0.1914 kcal/mol(~8meV).

Kindly let me know the plausible reason for this behaviour or if I am missing something.

[simonbatzner]

Can you share your inptu file? Also please train the model to end, it's difficult to judge a lot from 100 epochs. If you mind, please also share the force errors.