minimization step failing

[nisargjoshi10]

I have trained a water model, and looking at the training and validation loss it seems converged. When I try to run a minimization step with the trained potential, the system explodes after a few steps. I'm wondering if I need to train the model for longer epochs or change something?

I'm also attaching my config files and loss plots with the error.



water_train.txt

[simonbatzner]

Units?

[simonbatzner]

Your lenghts units are [A]? ASE's interal units are eV and A, but if your data are kcal/mol then NequIP will be in kcal/mol. I'm assuing that should be fine when interfacing with BFGS although it would change the step size I guess.

What is the force and energy validation mae (loss is not in proper units, hard to judge from that).

Are you training with or without stresses? If without and you're relying on good stress during deployment that won't work.

[nisargjoshi10]

the units are in [A]. here are my plots for energy and force validation mae: https://wandb.ai/njoshi/water250_3j_gmx/runs/1rv88yao?workspace=user-njoshi

Regarding training, I believe I am training without stresses

[simonbatzner]

That would be a force error of ~1meV/A which is essentially perfect, that's probably close to DFT noise, similar for energy. So it looks like what you're doing by just calling BFGS is simply relaxing the ionic positions and isn't changing the cell. My assumption of what's happening is that under these particular constraints, you get into a situation where you are out-of-distribution, i.e. some atoms are much closer together than what they should be and as a result you're seeing the blow up. What I would do is simply look at the structure that fails, visualize and plot the histogram of interatomic distances in the training set vs those in the frame that blows up. If it's indeed that, then add that frame to the training set.

I'd also make sure that this particular set up (not optimizing the cell) is really what you want to do.

[nisargjoshi10]

@simonbatzner I think i got a work around the energy minimization, however I'm getting a weird error while running the simulation in LAMMPS (I attached the output file below). I believe it could be the Tdamp which right now is 50 (but I'm trying to follow 100*{dt} based on the timestep.) I also tried changing the Tdamp for the same timestep and also used different timesteps but sort of observing the same fashion in the temperature fluctuations.

# bulk water
  
units           real
boundary        p p p
newton          off
atom_style      atomic

neighbor        2.0 bin
neigh_modify    delay 0 every 1 check yes

read_data       npt_data.lmp
mass            1 2
mass            2 16

variable        sysvol          equal vol
variable        sysmass         equal mass(all)/6.0221367e+23
variable        sysdensity      equal v_sysmass/v_sysvol/1.0e-24

pair_style      nequip
pair_coeff      * * water250_1k.pth H O


#nvt
reset_timestep  0
fix             3 all nvt temp 300.0 300.0 50.0
timestep        0.5
thermo_style    custom step pe ke etotal v_sysdensity temp press vol
thermo          1
dump            2 all custom 100 water_nvt.dump id type x y z

run             2000

slurm-478782.txt

[nisargjoshi10]

@Linux-cpp-lisp @simonbatzner wondering if you had a chance to look at this