-
Notifications
You must be signed in to change notification settings - Fork 299
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Correct Unexpected floats when reading LI L2 LFL #2998
base: main
Are you sure you want to change the base?
Conversation
…he _FillValue is not into the attributes
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2998 +/- ##
==========================================
+ Coverage 96.08% 96.09% +0.01%
==========================================
Files 377 377
Lines 55150 55167 +17
==========================================
+ Hits 52992 53014 +22
+ Misses 2158 2153 -5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Pull Request Test Coverage Report for Build 12184523570Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the investigations! I left some comments in the code.
I am also wondering if we still also need the fix for the float64
upcasting when multiplying with the scale_factor
, as discussed in the issue...
…tests to check the dtype improve the mock datas
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the fix and the extra more comprehensive tests! Just two things inline.
satpy/readers/li_base_nc.py
Outdated
@@ -306,7 +306,7 @@ def get_projection_config(self): | |||
"""Retrieve the projection configuration details.""" | |||
# We retrieve the projection variable name directly from our swath settings: | |||
proj_var = self.swath_coordinates["projection"] | |||
|
|||
logger.error(proj_var) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess this one should be removed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes I have removed this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One suggestion inline.
Maybe we should start splitting the long test helpers to smaller chunks at some point?
I also restarted the one windows CI run.
@@ -158,11 +158,9 @@ def get_array_on_fci_grid(self, data_array: xr.DataArray): | |||
data_2d = da.map_blocks(_np_add_at_wrapper, data_2d, (rows, cols), data_array, | |||
dtype=data_array.dtype, | |||
chunks=(LI_GRID_SHAPE[0], LI_GRID_SHAPE[1])) | |||
data_2d = da.where(data_2d > 0, data_2d, np.nan) | |||
|
|||
data_2d = da.where(data_2d > 0, data_2d, np.nan).astype(np.float32) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would prevent one unnecessary upcasting:
data_2d = da.where(data_2d > 0, data_2d, np.nan).astype(np.float32) | |
data_2d = da.where(data_2d > 0, data_2d, np.float32(np.nan)) | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pnuu casting a nan
who is a float32 is not enough to convert all the arrays into a float32
. For example if data_2d
is an int32
the method where will convert it to a float64
. To prevent it I have used the method astype(np.float32)
.
Makes sure that the method apply_fill_value is not applied when
_FillaValue
is not into the attributes. It is fixing the bug mentioned there #2854 .