-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Job splitters should retain original geometries #153
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just one remark on the speed of reprojecting s2 grid.
|
||
# Dataset containing all the S2 tiles, find the nearest S2 tile for each point | ||
s2_grid = load_s2_grid() | ||
s2_grid = s2_grid.to_crs(epsg=3857) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this not a slow step for the entire s2 grid?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@VincentVerelst Maybe if this is a speed issue you could upload a second s2 grid in the artifactory with the CRS reprojected, and then change the load_s2_grid()
function in gfmap to accept a parameter web_mercator: bool = False
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
May be worth the effort indeed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the suggestion, I'll already update that.
Do have to note that the largest bottleneck is typically not the CRS conversion of the s2 grid, but rather the CRS conversion of the dataframe to split itself.
For example, for 2018_BEL_LPIS-Flanders_POLY_110
the old job_splitter takes 10s to run, while the new one takes 20s.
@kvantricht, updated both
split_job_hex
andsplit_job_s2grid
so that they now:Also added unit tests for both.