Particle container fixes

[debog]

• Made the particle advection functions virtual.
• Added the new particle files to BuildERFExe.cmake (this was missed earlier)
• Temporary bugfix in updating particle k-index

[asalmgren]

@debog -- as long as you're fixing things up maybe you can incorporate the rest of the changes we need for now. Specifically, add

template <typename P>
AMREX_GPU_HOST_DEVICE AMREX_FORCE_INLINE
void update_location_idata (P& p,
                            amrex::GpuArray<amrex::Real,AMREX_SPACEDIM> const& plo,
                            amrex::GpuArray<amrex::Real,AMREX_SPACEDIM> const& dxi,
                            const amrex::Array4<amrex::Real const>&  height_arr)
{
    amrex::IntVect iv( int(amrex::Math::floor((p.pos(0)-plo[0])*dxi[0])),
                       int(amrex::Math::floor((p.pos(1)-plo[1])*dxi[1])),
                       p.idata(0) );

    amrex::Real lx = (p.pos(0)-plo[0])*dxi[0] - static_cast<amrex::Real>(iv[0]);
    amrex::Real ly = (p.pos(1)-plo[1])*dxi[1] - static_cast<amrex::Real>(iv[1]);
    auto zlo = height_arr(iv[0]  ,iv[1]  ,iv[2]  ) * (1.0-lx) * (1.0-ly) +
               height_arr(iv[0]+1,iv[1]  ,iv[2]  ) *      lx  * (1.0-ly) +
               height_arr(iv[0]  ,iv[1]+1,iv[2]  ) * (1.0-lx) * ly +
               height_arr(iv[0]+1,iv[1]+1,iv[2]  ) *      lx  * ly;
    auto zhi = height_arr(iv[0]  ,iv[1]  ,iv[2]+1) * (1.0-lx) * (1.0-ly) +
               height_arr(iv[0]+1,iv[1]  ,iv[2]+1) *      lx  * (1.0-ly) +
               height_arr(iv[0]  ,iv[1]+1,iv[2]+1) * (1.0-lx) * ly +
               height_arr(iv[0]+1,iv[1]+1,iv[2]+1) *      lx  * ly;

    if (p.pos(2) > zhi) {
        p.idata(0) += 1;
    } else if (p.pos(2) <= zlo) {
        p.idata(0) -= 1;
    }
}

into the .H file and use the following snipped in the Evolve*cpp file

       ParallelFor(n, [=] AMREX_GPU_DEVICE (int i)
        {
            ParticleType& p = p_pbox[i];
            if (p.id() <= 0) { return; }
            ParticleReal v[AMREX_SPACEDIM];
            mac_interpolate_mapped_z(p, plo, dxi, umacarr, zheight, v);
            if (ipass == 0)
            {
                for (int dim=0; dim < AMREX_SPACEDIM; dim++)
                {
                    p.rdata(dim) = p.pos(dim);
                    p.pos(dim) += static_cast<ParticleReal>(ParticleReal(0.5_rt)*dt*v[dim]);
                }

                // Update z-coordinate stored in p.idata(0)
                update_location_idata(p,plo,dxi,zheight);
            }
            else
            {
                for (int dim=0; dim < AMREX_SPACEDIM; dim++)
                {
                    p.pos(dim) = p.rdata(dim) + static_cast<ParticleReal>(dt*v[dim]);
                    p.rdata(dim) = v[dim];
                }

                // Update z-coordinate stored in p.idata(0)
                update_location_idata(p,plo,dxi,zheight);
            }
        });

[asalmgren]

you'll also need to update the amrex submodule to be able to use the new interpolation routine from amrex

[asalmgren]

We will migrate the update_location_idata into amrex in the longer term but for right now this should make ERF particle evolution correct

[debog]

you'll also need to update the amrex submodule to be able to use the new interpolation routine from amrex

How do I do this?

[asalmgren]

cd Submodules/AMReX
git checkout development
git pull
cd ../..
git add Submodules/AMReX
git commit -m "update amrex"
git push

I believe that what ends up being stored is the new hash, not the actual code, so then when people git pull, they will have to "git submodule update" as well.

[debog]

@asalmgren Thanks! I did both and pushed the commits.