Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Numerical advection of shape matrix using Lie integrator #579

Open
wants to merge 78 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
4d7f971
temp
RuiApostolo Oct 24, 2023
e2e39ec
Changed size of B matrix variables
RuiApostolo Oct 30, 2023
913ace0
Removed unused volume parameters in function calls
RuiApostolo Oct 30, 2023
cce6b59
Changes to parcel_split
RuiApostolo Oct 30, 2023
3b27963
Changes to parcel_split
RuiApostolo Oct 30, 2023
ea2956f
Changes to parcel_split
RuiApostolo Oct 30, 2023
f4df232
Changes to parcel_diagnostics
RuiApostolo Oct 30, 2023
6f0bee5
Changes to parcel_init
RuiApostolo Oct 30, 2023
f7f5992
Change to parcel container
RuiApostolo Oct 30, 2023
4dceb95
Change to test_mpi_parcel_split
RuiApostolo Oct 30, 2023
0993089
Change to parcel_bc
RuiApostolo Oct 30, 2023
1b79fc5
Change to test_mpi_parcel_split
RuiApostolo Oct 30, 2023
fe71797
Added B33 indexes to parcel communication in parcel_container
RuiApostolo Nov 7, 2023
a503642
Changed delta_b size
RuiApostolo Nov 7, 2023
5f7521e
Docs
RuiApostolo Nov 7, 2023
d7a559a
Modified unit test
RuiApostolo Nov 8, 2023
ad987df
Rank mismatch
RuiApostolo Nov 8, 2023
9188c54
get-B33 calls MPI, can't be pure function
RuiApostolo Nov 8, 2023
70b76ed
1-indexed
RuiApostolo Nov 8, 2023
1d63ba7
1-indexed again
RuiApostolo Nov 8, 2023
e3d93c5
Initialised B33 in more tests
RuiApostolo Nov 13, 2023
49c9beb
Updated imports on test_mpi_parcel_init_3d.f90
RuiApostolo Nov 13, 2023
8536225
Updated variable declaration on test_mpi_trilinear.f90
RuiApostolo Nov 13, 2023
e231555
Updated imports on test_mpi_trilinear.f90
RuiApostolo Nov 13, 2023
0e17cb4
Updated import and variable declaration on test_mpi_grid2par.f90
RuiApostolo Nov 13, 2023
64bd0a9
Updated variable declaration on test_mpi_parcel_diagnostic.f90
RuiApostolo Nov 13, 2023
cbb290d
Updated test_mpi_parcel_correction_3d.f90 and test_mpi_parecl_pack.f90
RuiApostolo Nov 13, 2023
a8fc1b3
get_dBdt needs Bout(IDX_B33) fixed
RuiApostolo Nov 13, 2023
76739f9
Changed get_dBdt
RuiApostolo Nov 13, 2023
f9aeb90
Changed get_dBdt to remove volume
RuiApostolo Nov 13, 2023
4c8c029
test debug
RuiApostolo Nov 13, 2023
b7163f2
Revert "test debug"
RuiApostolo Nov 13, 2023
45e8f00
Fixed test_mpi_laplace_correction
RuiApostolo Nov 13, 2023
063803a
Formula for get_dBdt Bout(I_B33)
RuiApostolo Nov 27, 2023
3c454f8
Fixed Bm(6:n) in parcel_merge.f90
RuiApostolo Dec 6, 2023
f352f14
B matrix normalisation at parcel initialisation
RuiApostolo Jan 12, 2024
ef7f685
B33 init in parcel_init.f90
RuiApostolo Jan 15, 2024
dd57f70
Parcel merge and split bugfixes
sjboeing Jan 18, 2024
04004ca
RK module: small fixes to (mostly) OpenMP
sjboeing Jan 19, 2024
6c663e1
Correct delta_b to be consistent with actual B increment
sjboeing Jan 21, 2024
3b6d994
Small alignment correction
sjboeing Jan 22, 2024
36e6b0d
Use gridded fields for dbdt
sjboeing Feb 4, 2024
b9b0139
Work on matrix exponential branch
Feb 5, 2024
9c2309b
Fix bugs, remove delta_b
sjboeing Feb 5, 2024
f178974
Using one more iteration in the exponentiation
sjboeing Feb 6, 2024
a841d81
Using velgradg with 8 components
sjboeing Feb 6, 2024
523edd6
Revert "Using one more iteration in the exponentiation"
sjboeing Feb 6, 2024
dcae3a9
Work on 2d version
sjboeing Feb 14, 2024
3307c19
Merge remote-tracking branch 'sjboeing/gridded_strain_b33' into gridd…
sjboeing Feb 15, 2024
348675d
Merge branch 'gridded_strain_pr' into gridded_b33_pr
sjboeing Feb 15, 2024
606b70d
Merge remote-tracking branch 'sjboeing/matrix_exponential_2d' into ma…
sjboeing Feb 15, 2024
921363e
Optimise 3D matrix exponentiation
sjboeing Feb 16, 2024
50c498a
Fancy matrix exp in 2d and 3d
sjboeing Feb 16, 2024
b2ba10c
Merge branch 'gridded_strain_pr' into gridded_b33_pr
sjboeing Feb 18, 2024
3fcff4a
Merge branch 'gridded_b33_pr' into matrix_exp_pr
sjboeing Feb 18, 2024
2a89bbe
Merge branch 'gridded_strain_pr' into gridded_b33_pr
sjboeing Feb 19, 2024
8117956
Merge remote-tracking branch 'origin/dev' into gridded_b33_pr
sjboeing Feb 19, 2024
58b8e0c
Merge remote-tracking branch 'origin/dev' into gridded_b33_pr
sjboeing Feb 21, 2024
a727f96
Correct B only at end of time step
sjboeing Feb 21, 2024
231ebba
Fix parcel split
sjboeing Feb 22, 2024
cf0b47a
Temporary fixes for debug compilation
sjboeing Feb 18, 2024
b07df48
Fix debug compilation in 1-point p2g and g2p mode
sjboeing Feb 22, 2024
a81a17a
Merge branch 'debug_mode_pr' into matrix_exp_pr
sjboeing Feb 22, 2024
16b0567
Fix unit test issue with renaming parcel strain
sjboeing Feb 22, 2024
b0ca3e6
Merge branch 'main' into matrix_exp_pr
sjboeing Jul 25, 2024
3bc8221
Merge remote-tracking branch 'origin/fixed_dt' into matrix_exp_pr
sjboeing Jul 25, 2024
859afac
Remove unneeded calls
sjboeing Aug 5, 2024
a88ded1
Minor optimisations in rk utils
sjboeing Aug 5, 2024
ee03e03
Write B33 in netCDF
sjboeing Aug 5, 2024
33ecfdb
Merge remote-tracking branch 'origin/main' into matrix_exp_pr
sjboeing Aug 5, 2024
731c325
Merge remote-tracking branch 'origin/main' into matrix_exp_pr
sjboeing Aug 5, 2024
f31d81b
Merge remote-tracking branch 'origin/main' into matrix_exp_pr
sjboeing Aug 5, 2024
6cdbeb1
Rename matrix exponential constants
sjboeing Aug 7, 2024
8678f8b
Merge remote-tracking branch 'origin/main' into matrix_exp_pr
sjboeing Sep 25, 2024
81489cd
Fixes in debugging
sjboeing Sep 26, 2024
049ffe5
Comments on parcel container (3D)
sjboeing Sep 26, 2024
4ccf7f1
Changes to incorporate prognostic B22
sjboeing Sep 26, 2024
4245c24
Merge remote-tracking branch 'origin/main' into matrix_exp_pr
sjboeing Sep 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/2d/parcels/parcel_bc.f90
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ end subroutine apply_periodic_bc
! @param[inout] position vector of parcel
! @param[inout] B matrix of parcel
subroutine apply_reflective_bc(position, B)
double precision, intent(inout) :: position(2), B(2)
double precision, intent(inout) :: position(2), B(3)

if (position(2) > upper(2)) then
position(2) = two * upper(2) - position(2)
Expand Down
4 changes: 2 additions & 2 deletions src/2d/parcels/parcel_container.f90
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ module parcel_container
type parcel_container_type
double precision, allocatable, dimension(:, :) :: &
position, &
B ! B matrix entries; ordering B(:, 1) = B11, B(:, 2) = B12
B ! B matrix entries; ordering B(1, :) = B11, B(2, :) = B12, B(3, :) = B33

double precision, allocatable, dimension(:) :: &
volume, &
Expand Down Expand Up @@ -88,7 +88,7 @@ subroutine parcel_alloc(num)

allocate(parcels%position(2, num))
allocate(parcels%vorticity(num))
allocate(parcels%B(2, num))
allocate(parcels%B(3, num))
allocate(parcels%volume(num))
allocate(parcels%buoyancy(num))
#ifndef ENABLE_DRY_MODE
Expand Down
7 changes: 3 additions & 4 deletions src/2d/parcels/parcel_diagnostics.f90
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ subroutine calculate_parcel_diagnostics(velocity)
double precision :: velocity(:, :)
integer :: n
double precision :: b, z, vel(2), vol, zmin
double precision :: eval, lam, B22, lsum, l2sum, vsum, v2sum
double precision :: eval, lam, lsum, l2sum, vsum, v2sum

call start_timer(parcel_stats_timer)

Expand Down Expand Up @@ -121,7 +121,7 @@ subroutine calculate_parcel_diagnostics(velocity)
std_vol = zero

!$omp parallel default(shared)
!$omp do private(n, vel, vol, b, z, eval, lam, B22) &
!$omp do private(n, vel, vol, b, z, eval, lam) &
!$omp& reduction(+: ke, ape, lsum, l2sum, vsum, v2sum, n_small, rms_zeta) &
!$omp& reduction(-: pe)
do n = 1, n_parcels
Expand All @@ -141,8 +141,7 @@ subroutine calculate_parcel_diagnostics(velocity)
ape = ape + ape_den(b, z) * vol
endif

B22 = get_B22(parcels%B(1, n), parcels%B(2, n), vol)
eval = get_eigenvalue(parcels%B(1, n), parcels%B(2, n), B22)
eval = get_eigenvalue(parcels%B(1, n), parcels%B(2, n), parcels%B(3, n))
lam = get_aspect_ratio(eval, vol)

lsum = lsum + lam
Expand Down
21 changes: 7 additions & 14 deletions src/2d/parcels/parcel_ellipse.f90
Original file line number Diff line number Diff line change
Expand Up @@ -55,17 +55,14 @@ function get_eigenvector(a2, B11, B12, B22) result(evec)
end function get_eigenvector

! used in unit tests only
function get_angle(B11, B12, volume) result(angle)
function get_angle(B11, B12, B22) result(angle)
double precision, intent(in) :: B11
double precision, intent(in) :: B12
double precision, intent(in) :: volume
double precision :: B22
double precision, intent(in) :: B22
double precision :: a2
double precision :: evec(2)
double precision :: angle

B22 = get_B22(B11, B12, volume)

a2 = get_eigenvalue(B11, B12, B22)

evec = get_eigenvector(a2, B11, B12, B22)
Expand Down Expand Up @@ -119,24 +116,20 @@ elemental function get_aspect_ratio(a2, volume) result(lam)
! @param[in] volume of the parcel
! @param[in] B matrix elements of the parcel
! @returns the parcel support points
function get_ellipse_points(position, volume, B) result(points)
function get_ellipse_points(position, B) result(points)
double precision, intent(in) :: position(2)
double precision, intent(in) :: volume
double precision, intent(in) :: B(2) ! B11, B12
double precision :: B22
double precision, intent(in) :: B(3) ! B11, B12, B22
double precision :: c
double precision :: a2
double precision :: evec(2)
double precision :: h(2)
double precision :: points(2, 2)

B22 = get_B22(B(1), B(2), volume)

a2 = get_eigenvalue(B(1), B(2), B22)
a2 = get_eigenvalue(B(1), B(2), B(3))

c = dsqrt(dabs(two * a2 - B(1) - B22))
c = dsqrt(dabs(two * a2 - B(1) - B(3)))

evec = get_eigenvector(a2, B(1), B(2), B22)
evec = get_eigenvector(a2, B(1), B(2), B(3))

h = f12 * c * evec

Expand Down
10 changes: 6 additions & 4 deletions src/2d/parcels/parcel_init.f90
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ module parcel_init
use options, only : parcel, output, verbose, field_tol
use constants, only : zero, two, one, f12
use parcel_container, only : parcels, n_parcels
use parcel_ellipse, only : get_ab, get_B22, get_eigenvalue
use parcel_ellipse, only : get_ab, get_eigenvalue, get_B22
use parcel_split, only : split_ellipses
use parcel_interpl, only : bilinear, ngp
use parameters, only : dx, vcell, ncell, &
Expand Down Expand Up @@ -83,6 +83,9 @@ subroutine init_parcels(fname, tol)

! B12
parcels%B(2, n) = zero

! B22
parcels%B(3, n) = get_B22(parcels%B(1, n), zero, parcels%volume(1))
enddo
!$omp end do
!$omp end parallel
Expand Down Expand Up @@ -145,13 +148,12 @@ end subroutine init_regular_positions

subroutine init_refine(lam)
double precision, intent(inout) :: lam
double precision :: B22, a2
double precision :: a2

! do refining by splitting
do while (lam >= parcel%lambda_max)
call split_ellipses(parcels, parcel%lambda_max)
B22 = get_B22(parcels%B(1, 1), zero, parcels%volume(1))
a2 = get_eigenvalue(parcels%B(1, 1), zero, B22)
a2 = get_eigenvalue(parcels%B(1, 1), parcels%B(2, 1), parcels%B(3, 1))
lam = a2 / get_ab(parcels%volume(1))
end do
end subroutine init_refine
Expand Down
13 changes: 6 additions & 7 deletions src/2d/parcels/parcel_interpl.f90
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ subroutine vol2grid
pvol = parcels%volume(n)

points = get_ellipse_points(parcels%position(:, n), &
pvol, parcels%B(:, n))
parcels%B(:, n))


! we have 2 points per ellipse
Expand Down Expand Up @@ -85,7 +85,7 @@ end subroutine vol2grid
#ifndef NDEBUG
! Interpolate the parcel volume to the grid to check symmetry
subroutine vol2grid_symmetry_error
double precision :: points(2, 2), V, B(2), pos(2)
double precision :: points(2, 2), V, B(3), pos(2)
integer :: n, p, l, m
double precision :: pvol

Expand All @@ -107,7 +107,7 @@ subroutine vol2grid_symmetry_error

B(2) = dble(m) * B(2)

points = get_ellipse_points(pos, V, B)
points = get_ellipse_points(pos, B)

! we have 2 points per ellipse
do p = 1, 2
Expand Down Expand Up @@ -180,7 +180,7 @@ subroutine par2grid
btot = parcels%buoyancy(n)
#endif
points = get_ellipse_points(parcels%position(:, n), &
pvol, parcels%B(:, n))
parcels%B(:, n))

call get_index(parcels%position(:, n), i, j)
i = mod(i + nx, nx)
Expand Down Expand Up @@ -312,6 +312,7 @@ subroutine grid2par(vel, vor, vgrad, add)
do n = 1, n_parcels
vel(:, n) = zero
vor(n) = zero
vgrad(:, n) = zero
enddo
!$omp end do
!$omp end parallel
Expand All @@ -322,6 +323,7 @@ subroutine grid2par(vel, vor, vgrad, add)
do n = 1, n_parcels
vel(:, n) = zero
vor(n) = zero
vgrad(:, n) = zero
enddo
!$omp end do
!$omp end parallel
Expand All @@ -331,10 +333,7 @@ subroutine grid2par(vel, vor, vgrad, add)
!$omp do private(n, p, l, points, weight, is, js, weights)
do n = 1, n_parcels

vgrad(:, n) = zero

points = get_ellipse_points(parcels%position(:, n), &
parcels%volume(n), &
parcels%B(:, n))

! we have 2 points per ellipse
Expand Down
13 changes: 5 additions & 8 deletions src/2d/parcels/parcel_merge.f90
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ module parcel_merge
, n_parcels &
, parcel_replace &
, get_delx
use parcel_ellipse, only : get_B22, get_ab
use parcel_ellipse, only : get_ab
use options, only : parcel, verbose
use parcel_bc
use timer, only : start_timer, stop_timer
Expand Down Expand Up @@ -80,7 +80,7 @@ subroutine do_group_merge(parcels, isma, iclo, n_merge, B11m, B12m, B22m, vm)
integer :: m, ic, is, l, n
integer :: loca(n_parcels)
double precision :: x0(n_merge)
double precision :: posm(2, n_merge), delx, vmerge, dely, B22, mu
double precision :: posm(2, n_merge), delx, vmerge, dely, mu
double precision :: buoym(n_merge), vortm(n_merge)
#ifndef ENABLE_DRY_MODE
double precision :: hum(n_merge)
Expand Down Expand Up @@ -186,15 +186,13 @@ subroutine do_group_merge(parcels, isma, iclo, n_merge, B11m, B12m, B22m, vm)

vmerge = one / vm(l)

B22 = get_B22(parcels%B(1, ic), parcels%B(2, ic), parcels%volume(ic))

delx = get_delx(parcels%position(1, ic), posm(1, l))
dely = parcels%position(2, ic) - posm(2, l)

mu = parcels%volume(ic) * vmerge
B11m(l) = mu * (four * delx ** 2 + parcels%B(1, ic))
B12m(l) = mu * (four * delx * dely + parcels%B(2, ic))
B22m(l) = mu * (four * dely ** 2 + B22)
B22m(l) = mu * (four * dely ** 2 + parcels%B(3, ic))

parcels%volume(ic) = vm(l)
parcels%position(1, ic) = posm(1, l)
Expand All @@ -216,14 +214,12 @@ subroutine do_group_merge(parcels, isma, iclo, n_merge, B11m, B12m, B22m, vm)
delx = get_delx(parcels%position(1, is), posm(1, n))
dely = parcels%position(2, is) - posm(2, n)

B22 = get_B22(parcels%B(1, is), parcels%B(2, is), parcels%volume(is))

! volume fraction A_{is} / A
mu = vmerge * parcels%volume(is)

B11m(n) = B11m(n) + mu * (four * delx ** 2 + parcels%B(1, is))
B12m(n) = B12m(n) + mu * (four * delx * dely + parcels%B(2, is))
B22m(n) = B22m(n) + mu * (four * dely ** 2 + B22)
B22m(n) = B22m(n) + mu * (four * dely ** 2 + parcels%B(3, is))
enddo

end subroutine do_group_merge
Expand Down Expand Up @@ -266,6 +262,7 @@ subroutine geometric_merge(parcels, isma, iclo, n_merge)

parcels%B(1, ic) = B11(l) * factor
parcels%B(2, ic) = B12(l) * factor
parcels%B(3, ic) = B22(l) * factor

call apply_periodic_bc(parcels%position(:, ic))
endif
Expand Down
19 changes: 18 additions & 1 deletion src/2d/parcels/parcel_netcdf.f90
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ module parcel_netcdf
integer :: ncid
integer :: npar_dim_id, vol_id, buo_id, &
x_pos_id, z_pos_id, vor_id, &
b11_id, b12_id, &
b11_id, b12_id, b22_id, &
t_axis_id, t_dim_id
double precision :: restart_time

Expand Down Expand Up @@ -133,6 +133,15 @@ subroutine create_netcdf_parcel_file(basename, overwrite, l_restart)
dimids=dimids, &
varid=b12_id)

call define_netcdf_dataset(ncid=ncid, &
name='B22', &
long_name='B22 element of shape matrix', &
std_name='', &
unit='m^2', &
dtype=NF90_DOUBLE, &
dimids=dimids, &
varid=b22_id)

call define_netcdf_dataset(ncid=ncid, &
name='volume', &
long_name='parcel volume', &
Expand Down Expand Up @@ -205,6 +214,7 @@ subroutine write_netcdf_parcels(t)

call write_netcdf_dataset(ncid, b11_id, parcels%B(1, 1:n_parcels), start, cnt)
call write_netcdf_dataset(ncid, b12_id, parcels%B(2, 1:n_parcels), start, cnt)
call write_netcdf_dataset(ncid, b22_id, parcels%B(3, 1:n_parcels), start, cnt)

call write_netcdf_dataset(ncid, vol_id, parcels%volume(1:n_parcels), start, cnt)

Expand Down Expand Up @@ -263,6 +273,13 @@ subroutine read_netcdf_parcels(fname)
stop
endif

if (has_dataset(ncid, 'B22')) then
call read_netcdf_dataset(ncid, 'B22', parcels%B(3, 1:n_parcels), start, cnt)
else
print *, "The parcel shape component B22 must be present! Exiting."
stop
endif

if (has_dataset(ncid, 'x_position')) then
call read_netcdf_dataset(ncid, 'x_position', &
parcels%position(1, 1:n_parcels), start, cnt)
Expand Down
3 changes: 2 additions & 1 deletion src/2d/parcels/parcel_split.f90
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ subroutine split_ellipses(parcels, threshold)
B11 = parcels%B(1, n)
B12 = parcels%B(2, n)
V = parcels%volume(n)
B22 = get_B22(B11, B12, V)
B22 = parcels%B(3, n)

a2 = get_eigenvalue(B11, B12, B22)

Expand All @@ -64,6 +64,7 @@ subroutine split_ellipses(parcels, threshold)

parcels%B(1, n) = B11 - f34 * a2 * evec(1) ** 2
parcels%B(2, n) = B12 - f34 * a2 * (evec(1) * evec(2))
parcels%B(3, n) = B22 - f34 * a2 * evec(2) ** 2

h = f14 * dsqrt(three * a2)
parcels%volume(n) = f12 * V
Expand Down
Loading