gdal version needs capping + hanging on scratch/masking_ancillary_darkland_removed.tif
#61
Comments
|
I can confirm capping the However, even so, I am not able to generate products as there is a step in the workflow that does not complete. The process hangs here: I have tried this on my local workstation and on the server aurora using the capped Thank you. |
cmarshak
changed the title
gdal.Open - possible gdal version issue?gdal version needs capping + hanging on scratch/masking_ancillary_darkland_removed.tif
|
Dear @oberonia78, Thank you for your help thus far with getting DSWx-S1 running; per your excellent observation (my output
For 1. and 2., I am still getting the issue I pointed out above, namely, hanging on Weirdly, when I use the VRT in the shared server you provided as in 3., it gets past this "hanging" portion. However, it still fails I will share the output below past that "hanging area": I compared the
If I see where precisely 0s occur I get this difference where the value of
Questions:
|
|
Hi @cmarshak, Thanks for posting the issues you have. I'll investigate your issues. But before that, I suggest you to process with the database together. I'll share the files with LFT. The problem seems to happen in the last step of making OPERA packages.
|
|
Following up on each:
The main difference I see between the environment you shared (i.e. the text file with Ideally, you could provide instructions to replicate your environment on a new machine. And just for posterity, the instructions here do not work on either my mac or aurora (the shared server): https://github.com/opera-adt/DSWX-SAR/blob/main/docker/lock_package.lock#L2
Would it just be
My hand tif is just slightly smaller. |
|
We can follow up next week. |
|
So to my surprise - 3. (using the VRT provided) does work! And it saves to So, the HAND and environment issues still need to be sorted out. Thanks @oberonia78 !! |
|
Dear @oberonia78 and dswx-sar team, I can confirm that the issue regarding the hanging step related to Not sure if this could be more elegantly solved providing a global vrt with ASF urls but don't know what the code is doing. You mentioned the relevant areas of the code are found in this file. Thank you, Junkgyo! |
|
This is assuming that |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
In the latest version the preprocessing routines, I was getting:
At first, I thought it had to do with auxiliary data not having enough buffer since it said that the nodata was being carried over into the reprojected reference water map. However, buffering by 1 degree yielded a similar issue.
Specifically, the function
validate_gtiffhere makes sure there are nonodatavalues as indicated here:DSWX-SAR/src/dswx_sar/pre_processing.py
Lines 688 to 691 in 5f4c320
dswx-s1workflow) looked like this:First, I removed the
no_datain the validation here:DSWX-SAR/src/dswx_sar/pre_processing.py
Lines 688 to 691 in 5f4c320
Still same error.
So I went into
dswx_utilsand modifying this line:to
Allowed the code to continue. Not sure if it's correct.
This seems like it could be gdal issue. I am using version
3.8.3for the above as indicated withconda list | grep gdal:My issues could be related to the minor release in November and making the runtime environment inconsistent with that which is being developed here. I have not checked this as I had a hard time pinpointing the bug. I am saying this because I had similar issues with DSWx-HLS recently as documented here. It turned out the expected runtime environment required
gdal<3.8and I was similarly using the latestgdal.Please let me know:
gdal.GA_Updatein this particular place is intended/acceptable/etc.The text was updated successfully, but these errors were encountered: