Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement ESMpy mask handling #23

Closed
wants to merge 5 commits into from
Closed

Implement ESMpy mask handling #23

wants to merge 5 commits into from

Conversation

jhamman
Copy link

@jhamman jhamman commented Jun 22, 2018

This comes from the discussion in #22. It is intended to help users specify masks for the valid cells in the source and/or destination grids. It does this in the following way:

  • looks for mask variable in ds_in and ds_out
  • if mask variable is present, it adds that mask to the ESMF grid object
  • specifies the mask values in the ESMF.Regrid() call.

I've been testing this and it seems to be headed in the right direction. There are certainly additional features/tests/docs that this will need but I'm putting up here to aid in our discussion.

@codecov-io
Copy link

codecov-io commented Jun 22, 2018

Codecov Report

Merging #23 into master will decrease coverage by 0.21%.
The diff coverage is 91.66%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #23      +/-   ##
==========================================
- Coverage   95.19%   94.97%   -0.22%     
==========================================
  Files           6        6              
  Lines         229      239      +10     
==========================================
+ Hits          218      227       +9     
- Misses         11       12       +1
Impacted Files Coverage Δ
xesmf/frontend.py 93.75% <100%> (+0.23%) ⬆️
xesmf/backend.py 94.8% <85.71%> (-0.97%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 97ec30a...bf712c0. Read the comment docs.

@jhamman
Copy link
Author

jhamman commented Jun 22, 2018

Here's the same example I showed in #22, but using a mask.

image

I haven't figured out yet how to mask the grid cells with nans where there were no valid weights. Perhaps @bekozi knows where to get that info from the ESMpy objects?

@bekozi
Copy link

bekozi commented Jun 22, 2018

I haven't figured out yet how to mask the grid cells with nans where there were no valid weights. Perhaps @bekozi knows where to get that info from the ESMpy objects?

We could potentially pull in the unmapped destination list to ESMPy, but I think it is easier to use fill values (or nans) during the sparse matrix multiplication. Basically, fill the destination matrix with nans before applying the sparse matrix. Afterwards, if the masks are set up correctly, only the mapped values will be non-nan. See filling and finding for mildly relevant examples.

@JiaweiZhuang
Copy link
Owner

Thanks for the PR (the first PR for xESMF)!

I still have some concerns: #22 (comment). Let's continue the discussion in the main issue.

JiaweiZhuang added a commit that referenced this pull request Aug 6, 2018
See #17.
This together with masking can address the normalization issue
in #23
@JiaweiZhuang JiaweiZhuang mentioned this pull request Aug 20, 2018
1 task
@rabernat
Copy link

@jhamman - this PR was discussed today at the xESMF dev meeting. There was a consensus that it's a very important feature.

Could you let us know whether you plan to keep working on the PR? Do you need any help from the dev team? If you don't have time to finish it, let us know so someone else can take it over.

@jhamman
Copy link
Author

jhamman commented Jun 16, 2020

Thanks for the ping. I'm not likely to have time to pick this up in the near future so would welcome someone else stepping in if there is time/interest.

@rabernat
Copy link

What is the difference between this and #82? Does that supersede this?

If we want to move forward with this PR, I recommend it be moved to https://github.com/pangeo-data/xESMF? Unfortunately this has to be done manually.

@raphaeldussin
Copy link
Contributor

Comparing across forks:

JiaweiZhuang:masking is up to date with all commits from jhamman:feature/masks.

so this PR can be closed

raphaeldussin pushed a commit to raphaeldussin/xESMF that referenced this pull request Jul 24, 2020
See JiaweiZhuang#17.
This together with masking can address the normalization issue
in JiaweiZhuang#23
raphaeldussin added a commit to raphaeldussin/xESMF that referenced this pull request Sep 10, 2020
* doc links pointing to pangeo-xesmf

let's have the links pointing to the updated doc

* update requirements versions

* Update doc/requirements.txt

Co-authored-by: Anderson Banihirwe <[email protected]>

* wrong syntax

* python3.8.5 not available

* remote python dep

* relax versions

Co-authored-by: Anderson Banihirwe <[email protected]>
@jhamman jhamman closed this by deleting the head repository Dec 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants