Add missing GPU synchronize

[marchdf]

Summary

Pull request type

Please check the type of change introduced:

• Bugfix
• Feature
• Code style update (formatting, renaming)
• Refactoring (no functional changes, no api changes)
• Build related changes
• Documentation content changes
• Other (please describe):

Checklist

The following is included:

• new unit-test(s)
• new regression test(s)
• documentation for new capability

This PR was tested by running:

• the unit tests
  • on GPU
  • on CPU
• the regression tests
  • on GPU
  • on CPU

Additional background

Issue Number:

[marchdf]

The main ones I am struggling with now are the ones in overset_ops_routines.cpp. For example, populate_psi is being used like this:

for (int lev = 0; lev < nlevels; ++lev) {
        // Thickness used here is user parameter, whatever works best
        const auto dx = (geom[lev]).CellSizeArray();
        const amrex::Real i_th =
            m_relative_length_scale * std::cbrt(dx[0] * dx[1] * dx[2]);

        // Populate approximate signed distance function
        overset_ops::populate_psi(levelset(lev), vof(lev), i_th, m_asdf_tiny);

        gp(lev).setVal(0.0);
        amrex::MultiFab::Copy(gp(lev), gp_noghost(lev), 0, 0, 3, 0);
        gp(lev).FillBoundary(geom[lev].periodicity());

        // Get pseudo-velocity scale, proportional to smallest dx in iblank
        const amrex::Real pvscale_lev =
            overset_ops::calculate_pseudo_velocity_scale(
                iblank_cell(lev), dx, max_pvscale);
        pvscale = std::min(pvscale, pvscale_lev);
    }
    amrex::Gpu::synchronize();

It has a synchronize but there is also a FillBoundary call right after it...

[marchdf]

Apparently the fill boundary call is safe like it is.