Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Masking with conservative_normed #82

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 32 additions & 3 deletions xesmf/backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ def warn_lat_range(lat):
warnings.warn("Latitude is outside of [-90, 90]")


def esmf_grid(lon, lat, periodic=False):
def esmf_grid(lon, lat, periodic=False, mask=None):
'''
Create an ESMF.Grid object, for contrusting ESMF.Field and ESMF.Regrid
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Create an ESMF.Grid object, for contrusting ESMF.Field and ESMF.Regrid
Create an ESMF.Grid object, for constructing ESMF.Field and ESMF.Regrid.


Expand All @@ -70,6 +70,10 @@ def esmf_grid(lon, lat, periodic=False):
Periodic in longitude? Default to False.
Only useful for source grid.

mask : 2D numpy array, optional
Grid mask. Follows SCRIP convention where 1 is unmasked and 0 is
masked.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mention what dimensions should mask be. E.g. (Nlon, Nlat).

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that is the ESMF convention as well, I'd replace SCRIP by ESMF to avoid confusion.


Returns
-------
grid : ESMF.Grid object
Expand Down Expand Up @@ -111,6 +115,20 @@ def esmf_grid(lon, lat, periodic=False):
lon_pointer[...] = lon
lat_pointer[...] = lat

# Follows SCRIP convention where 1 is unmasked and 0 is masked.
# See https://github.com/NCPP/ocgis/blob/61d88c60e9070215f28c1317221c2e074f8fb145/src/ocgis/regrid/base.py#L391-L404
if mask is not None:
grid_mask = np.swapaxes(mask.astype(np.int32), 0, 1)
grid_mask = np.where(grid_mask == 0, 0, 1)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this to support unmasked values other than 1 ? If so, maybe worth mentioning in the docstring.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a conversion to boolean then to integer.

if not (grid_mask.shape == lon.shape):
raise ValueError(
"mask must have the same shape as the latitude/longitude"
"coordinates, got: mask.shape = %s, lon.shape = %s" %
(mask.shape, lon.shape))
grid.add_item(ESMF.GridItem.MASK, staggerloc=ESMF.StaggerLoc.CENTER,
from_file=False)
grid.mask[0][:] = grid_mask

return grid


Expand Down Expand Up @@ -175,6 +193,7 @@ def esmf_regrid_build(sourcegrid, destgrid, method,

- 'bilinear'
- 'conservative', **need grid corner information**
- 'conservative_normed', **need grid corner information**
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if mapping ESMFpy method options to new xESMF methods will scale well.

ESMFPy has these clear and well documented IntEnum classes in api.constants for options. To make it more pythonic, maybe xESMF could convert those to string options, e.g.

norm_type : {'dstarea', 'fracarea'}
  The type of normalization used when producing the weights.

and then

ESMF.Regrid(..., 
norm_type=getattr(NormType, norm_type.upper()),
...)

- 'patch'
- 'nearest_s2d'
- 'nearest_d2s'
Expand Down Expand Up @@ -204,6 +223,7 @@ def esmf_regrid_build(sourcegrid, destgrid, method,
# use shorter, clearer names for options in ESMF.RegridMethod
method_dict = {'bilinear': ESMF.RegridMethod.BILINEAR,
'conservative': ESMF.RegridMethod.CONSERVE,
'conservative_normed': ESMF.RegridMethod.CONSERVE,
'patch': ESMF.RegridMethod.PATCH,
'nearest_s2d': ESMF.RegridMethod.NEAREST_STOD,
'nearest_d2s': ESMF.RegridMethod.NEAREST_DTOS
Expand All @@ -215,7 +235,7 @@ def esmf_regrid_build(sourcegrid, destgrid, method,
'{}'.format(list(method_dict.keys())))

# conservative regridding needs cell corner information
if method == 'conservative':
if method in ['conservative', 'conservative_normed']:
if not sourcegrid.has_corners:
raise ValueError('source grid has no corner information. '
'cannot use conservative regridding.')
Expand All @@ -235,12 +255,21 @@ def esmf_regrid_build(sourcegrid, destgrid, method,
assert not os.path.exists(filename), (
'Weight file already exists! Please remove it or use a new name.')

# re-normalize conservative regridding results
# https://github.com/JiaweiZhuang/xESMF/issues/17
if method == 'conservative_normed':
norm_type = ESMF.NormType.FRACAREA
else:
norm_type = ESMF.NormType.DSTAREA

# Calculate regridding weights.
# Must set unmapped_action to IGNORE, otherwise the function will fail,
# if the destination grid is larger than the source grid.
regrid = ESMF.Regrid(sourcefield, destfield, filename=filename,
regrid_method=esmf_regrid_method,
unmapped_action=ESMF.UnmappedAction.IGNORE)
unmapped_action=ESMF.UnmappedAction.IGNORE,
norm_type=norm_type,
src_mask_values=[0], dst_mask_values=[0])

return regrid

Expand Down
16 changes: 13 additions & 3 deletions xesmf/frontend.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,14 @@ def ds_to_ESMFgrid(ds, need_bounds=False, periodic=None, append=None):
lat = np.asarray(ds['lat'])
lon, lat = as_2d_mesh(lon, lat)

if 'mask' in ds:
mask = np.asarray(ds['mask'])
print(mask.shape)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove print statement.

else:
mask = None

# tranpose the arrays so they become Fortran-ordered
grid = esmf_grid(lon.T, lat.T, periodic=periodic)
grid = esmf_grid(lon.T, lat.T, periodic=periodic, mask=mask)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be clearer to have mask.T as well insted of transposing it inside esgf_grid ?


if need_bounds:
lon_b = np.asarray(ds['lon_b'])
Expand All @@ -77,17 +83,21 @@ def __init__(self, ds_in, ds_out, method, periodic=False,
ds_in, ds_out : xarray DataSet, or dictionary
Contain input and output grid coordinates. Look for variables
``lon``, ``lat``, and optionally ``lon_b``, ``lat_b`` for
conservative method.
add method.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?


Shape can be 1D (Nlon,) and (Nlat,) for rectilinear grids,
or 2D (Ny, Nx) for general curvilinear grids.
Shape of bounds should be (N+1,) or (Ny+1, Nx+1).

If either dataset includes a 2d mask variable, that will also be
used to inform the regridding.

method : str
Regridding method. Options are

- 'bilinear'
- 'conservative', **need grid corner information**
- 'conservative_normed', **need grid corner information**
- 'patch'
- 'nearest_s2d'
- 'nearest_d2s'
Expand Down Expand Up @@ -115,7 +125,7 @@ def __init__(self, ds_in, ds_out, method, periodic=False,
"""

# record basic switches
if method == 'conservative':
if method in ['conservative', 'conservative_normed']:
self.need_bounds = True
periodic = False # bound shape will not be N+1 for periodic grid
else:
Expand Down
17 changes: 17 additions & 0 deletions xesmf/tests/test_frontend.py
Original file line number Diff line number Diff line change
Expand Up @@ -170,3 +170,20 @@ def test_regrid_with_1d_grid():

# clean-up
regridder.clean_weight_file()


def test_build_regridder_with_masks():
ds_in['mask'] = xr.DataArray(
np.random.randint(2, size=ds_in['data'].shape),
dims=('y', 'x'))
print(ds_in)
# 'patch' is too slow to test
for method in ['bilinear', 'conservative', 'nearest_s2d', 'nearest_d2s']:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't conservative_normed be exercised ?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep I agree. In fact, all conservative methods must be tested numerically, especially with masking.

regridder = xe.Regridder(ds_in, ds_out, method)

# check screen output
assert repr(regridder) == str(regridder)
assert 'xESMF Regridder' in str(regridder)
assert method in str(regridder)

regridder.clean_weight_file()