tests failing in docker container

[smiet]

I am unable to get the tests to run successfully. This is annoying on my local build, but very puzzling that it even fails in the simsopt docker container.

Reproduce:

docker run --rm -it  hiddensymmetries/simsopt
# install new pyoculus in container
yes | apt-get install gfortran && pip install git+https://github.com/zhisong/pyoculus.git
# install new py_spec as packaging error causes import (and spec) to fail in container
pip install -U py_spec
# clone the repo to get access to test files
git clone https://github.com/hiddenSymmetries/simsopt.git
# move into docker's simsopt container
cd simsopt
# 
./run_tests 2>&1 |tee testoutput.log

It consistently fails and kills the python kernel on the test mhd/test_integrated_vmec_mpi.py with the error

This script implements the "1DOF_circularCrossSection_varyR0_targetVolume" ...  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  VMEC OUTPUT FILES ALREADY EXIST: OVERWRITING THEM ...
  nthreed =            9  istat_threed =            2  nmac0   =           10  istat_mac0 =            0
 Error opening output file in VMEC open_output_files
STOP 10

also on my cluster and workstation installation.

This is quite frustrating as it complicates testing before pushing and the kernel killing prevents unittests from printing the error logs. It is also quite inconsistent, as calling that test directly does not error.

Can anyone try to reproduce? Know where to look? Do others run the tests sucessfully?

The failed tests fail along the line of:

======================================================================
ERROR: test_toroidal_flux_partial_derivatives_wrt_coils (geo.test_surface_objectives.ToroidalFluxTests) (surfacetype='SurfaceXYZFourier', stellsym=True)
Taylor test for partial derivative of toroidal flux with respect to surface coefficients
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/smiet/code/simsopt/tests/geo/test_surface_objectives.py", line 112, in test_toroidal_flux_partial_derivatives_wrt_coils
    self.subtest_toroidal_flux3(surfacetype, stellsym)
  File "/home/smiet/code/simsopt/tests/geo/test_surface_objectives.py", line 158, in subtest_toroidal_flux3
    curves, currents, ma = get_ncsx_data()
  File "/home/smiet/code/simsopt/src/simsopt/configs/zoo.py", line 24, in get_ncsx_data
    curves = CurveXYZFourier.load_curves_from_file(filename, order=Nt_coils, ppp=ppp)
  File "/home/smiet/code/simsopt/src/simsopt/geo/curvexyzfourier.py", line 85, in load_curves_from_file
    coil_data = np.loadtxt(filename, delimiter=delimiter)
  File "/home/smiet/miniconda3/envs/spec_simsopt/lib/python3.10/site-packages/numpy/lib/npyio.py", line 1373, in loadtxt
    arr = _read(fname, dtype=dtype, comment=comment, delimiter=delimiter,
  File "/home/smiet/miniconda3/envs/spec_simsopt/lib/python3.10/site-packages/numpy/lib/npyio.py", line 992, in _read
    fh = np.lib._datasource.open(fname, 'rt', encoding=encoding)
  File "/home/smiet/miniconda3/envs/spec_simsopt/lib/python3.10/site-packages/numpy/lib/_datasource.py", line 192, in open
    ds = DataSource(destpath)
  File "/home/smiet/miniconda3/envs/spec_simsopt/lib/python3.10/site-packages/numpy/lib/_datasource.py", line 249, in __init__
    self._destpath = os.path.abspath(destpath)
  File "/home/smiet/miniconda3/envs/spec_simsopt/lib/python3.10/posixpath.py", line 384, in abspath
    cwd = os.getcwd()
FileNotFoundError: [Errno 2] No such file or directory

which is also baffling to me. Both in docker container and in home system and on server.
how can os.getcwd() fail?!?!?!

Test output with the offending test removed is attached here:
test_output.log

[mbkumar]

The docker container is released only for tagged releases. Can you make sure your local copy of the simsopt is matching with the release version?

[smiet]

All latest version 0.19.0:

in container

(spec_simsopt) smiet@cl-cours@spcpc629:~> docker run --rm -it  hiddensymmetries/simsopt
root@3555c79e168c:/# python -c"import simsopt; print(simsopt.__version__)"
0.19.0.post90+g490282b3

on my own system:

(spec_simsopt) smiet@cl-cours@spcpc629:~> python -c"import simsopt; print(simsopt.__version__)"
0.19.0.post42+gddef238e.d20231026.dirty

[mbkumar]

Can you try this?

git clone https://github.com/hiddenSymmetries/simsopt.git
cd simsopt
docker run --name=mycontainer -it -v $PWD:/my_mount hiddensymmetries/simsopt
... intermediate steps to install additional packages inside the container...
<container ###> cd /my_mount
<container ###> ./run_tests 2>&1 |tee testoutput.log

[smiet]

Tried it, same happens when calling the tests from the simsopt repo on host system that is mounted into the docker:

testoutput.log

When I uncomment the crashing test, unittests show me the cause of the errors:
testoutput_failskip.log

do the above steps run sucessfully on your system?