Visibly different structure depending on OMP_NUM_THREADS

[TimoSommer]

Describe the bug
xtb structure optimization seems to give visibly different structures depending on the value of OMP_NUM_THREADS.

To Reproduce
Steps to reproduce the behaviour:

1. Input file:
   test.xyz.gz
2. Extract input file with gunzip test.xyz.gz
   Serial run of xTB
3. run export OMP_NUM_THREADS=1
4. Run xtb --gfn 2 --chrg 0 --uhf 4 --etemp 300 --iterations 250 --tblite --alpb water --spinpol --ohess tight test.xyz
5. Rename the output file xtbopt.xyz to xtbopt_1core.xyz
6. Delete output files g98.out, xtbrestart, vibspectrum, hessian, .xtboptok, xtbopt.log
   Parallel run of xTB
7. Run export OMP_NUM_THREADS=2
8. Run xtb --gfn 2 --chrg 0 --uhf 4 --etemp 300 --iterations 250 --tblite --alpb water --spinpol --ohess tight test.xyz (exactly as above)
9. Rename the output file xtbopt.xyz to xtbopt_2cores.xyz
   Results
10. The xyz files and final energies (as in the xyz files comments) between xtbopt_1core.xyz and xtbopt_2cores.xyz are slightly different. When looking at the structures in the viewer VESTA, the atoms visibly shift.
    xtbopt_1core.xyz.gz
    xtbopt_2cores.xyz.gz

Expected behaviour
Ideally, I would have expected that the output is exactly the same, independent of the number of cores used. I can understand that there might be some tiny numerical differences between the runs, but here it seems like the differences have become quite big. If we set OMP_NUM_THREADS to something higher than 2, the structures are different again.

Additional context
I have tried to find a minimal working example with less xtb options, but reducing the options seemed to also reduce the magnitude of the difference. The minimal example in which I could still visibly see atoms shift in VESTA was with the following command: xtb --gfn 2 --chrg 0 --uhf 4 --alpb water --tblite --opt tight test.xyz

Versions
xtb version 6.6.1 (8d0f1dd) compiled by '[email protected]' on 2023-08-01

[Albkat]

Hi,

Yes, there were some race conditions in the previous version. Have you tested xtb version 6.7.0?

[TimoSommer]

My laptop is a Mac, it seems like version 6.7.0 is out only for Linux and Windows so far. I will try it as soon as conda is updated to 6.7.0. Btw, I noticed in my tests that sometimes there can also be quite large differences in energy (up to 0.1 eV) for a structural optimization between different runs on different machines (e.g. my Mac laptop and a Linux cluster), even with OMP_NUM_THREADS=1. Would this also be fixed by the update?

[TimoSommer]

Hey I tried it out with xtb version 6.7.1, and while there are still slight numerical differences in the range of ~1e-5 Angstrom, the issue is much smaller now.

I notice these small numerical differences between xTB runs a lot though. Is that expected?