Multiple CI jobs broken on master

[sloede]

It seems like multiple CI jobs are broken, e.g.,

• test-openmpi-jll (ubuntu-latest, 1.6, x86) fails with ERROR: LoadError: UndefVarError: libhsa_runtime64 not defined (upstream bug in AMDGPU.jl 0.2 with Julia v1.6, but AMDGPU v0.3 requires Julia v1.7). It doesn't seem to occur anymore.
• test-mpitrampoline-intel-linux (1.6, 2.10.3) fails with a segfault in some multithreaded part (MPI implementations intercepting Signals is incompatible with Julia GC safepoint #725). This job doesn't exist anymore with [CI] Use Spack OCI build cache for MPI packages #788
• test-spack-mvapich (1) fails with yet another segfault (upstream bug in mvapich: Tests segfaulting on Ookami #684, fixed by mvapich2: Add patch to avoid segmentation fault in MPIR_Attr_delete_list spack/spack#39058)
• test-mpitrampoline-jll (ubuntu-latest, mpitrampoline, 1.6, x64) has some problems with error testing
• Testing on Windows is failing in multiple places (Spurios test failures on Windows for get_position_shared #555). Test skipped in Skip threads tests which are known to fail #791

Any idea what's going on here?

[sloede]

Ah, sorry, I should've checked more carefully - it seems like these tests have been broken for a while now, so nothing to get alarmed about. Still, at least some of those errors seem legit (e.g., the first one complaining about a missing symbol)

[luraess]

Yeah, seems most of the AMDGPU CI fails but still tests status shows success.

[giordano]

test-openmpi-jll (ubuntu-latest, 1.6, x86) fails with ERROR: LoadError: UndefVarError: libhsa_runtime64 not defined

That's a crash in building AMDGPU.jl

test-mpitrampoline-intel-linux (1.6, 2.10.3) fails with a segfault in some multithreaded part

That's #725.

test-spack-mvapich (1) fails with yet another segfault

I think that's a bug in MVAPICH which may have been fixed in a later version.

Testing on Windows is failing in multiple places

#555

[giordano]

That's a crash in building AMDGPU.jl

More specifically, the problem is that an old version of AMDGPU.jl (0.2.17) is picked up, despite

MPI.jl/Project.toml

Line 22 in 4ba36fa

AMDGPU = "0.3, 0.4, 0.5"

This should have been fixed in newer versions.

[sloede]

Testing on Windows is failing in multiple places

#555

Are you sure the referenced issue is related to all these failures? Here, I also see premature aborts and errors in MPI_Waitall.

[luraess]

I'll check more carefully but to me it seems that none of the AMDGPU CI succeeds.

[giordano]

That's a crash in building AMDGPU.jl

More specifically, the problem is that an old version of AMDGPU.jl (0.2.17) is picked up, despite

MPI.jl/Project.toml

Line 22 in 4ba36fa

AMDGPU = "0.3, 0.4, 0.5"

This should have been fixed in newer versions.

AMDGPU.jl 0.3 isn't compatible with Julia v1.6. I don't see how we can possibly make it work. We should either drop support for Julia v1.6 or not use AMDGPU at all.

[sloede]

AMDGPU.jl 0.3 isn't compatible with Julia v1.6. I don't see how we can possibly make it work. We should either drop support for Julia v1.6 or not use AMDGPU at all.

Call me naive, but I don't see how anyone still uses MPI.jl + Julia v1.6 for productive workloads. For the sake of reducing maintenance efforts, I'd thus vote for dropping v1.6 support.

If this is not possible/desirable, then disable AMDGPU support for Julia v1.6 by adding a corresponding statement to the docs and removing the corresponding test matrix entries.

[luraess]

If this is not possible/desirable, then disable AMDGPU support for Julia v1.6 by adding a corresponding statement to the docs and removing the corresponding test matrix entries.

I'd say that AMDGPU Ext + MPI should be bound to Julia 1.9 & AMDGPU 0.5 as explained in #753

[giordano]

AMDGPU is always a test dependency:

MPI.jl/test/Project.toml

Line 2 in abebf65

AMDGPU = "21141c5a-9bdb-4563-92ae-f87d6854732e"

[giordano]

I think we have now cleared most of if not all the persistent failures by either addressing them or skipping consistently failing tests, which are already tracked by specific issues. If there are other relevant issues it'd be good to open dedicated issues, but I'm going to close this ticket, as CI should now be in a much better shape.