Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GeoTiff partial loading optimize for all roi #57

Open
HowcanoeWang opened this issue Dec 2, 2022 · 0 comments
Open

GeoTiff partial loading optimize for all roi #57

HowcanoeWang opened this issue Dec 2, 2022 · 0 comments
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@HowcanoeWang
Copy link
Member

HowcanoeWang commented Dec 2, 2022

if RAM >> GeoTiff.filesize and len(roi) > 100:
    full load GeoTiff, and delete the variable after finish.

Have test fulling loading into memory, DOM.dtype(uint8) == DSM.dtype(float32), 20w x 20w dom requires 91GB RAM. Not applicable.


The optimal choice:

read one line tail for muitiple roi if applicable.

No need duplicately read geotiff for each roi.

> ROI.get_z_from_dsm():

1. calculate each roi and its bbox.
2. bbox to geotiff tail line & colume
3. revert previous index, to each geotiff tail, which columns are which roi and roi corresponding parts
4. For loop, for each geotiff tail, read to numpy, and crop & save to correspoing roi parts
    1. If one roi is completed loaded, return cropped results for calculation.
    2. delete the completed roi in memory.
    3. continue for the next tail.
@HowcanoeWang HowcanoeWang added the enhancement New feature or request label Dec 2, 2022
@HowcanoeWang HowcanoeWang added this to the 2.0.1 milestone Dec 2, 2022
@HowcanoeWang HowcanoeWang self-assigned this Dec 2, 2022
@HowcanoeWang HowcanoeWang modified the milestones: 2.0.1, 2.0.0.dev4 Dec 5, 2022
@HowcanoeWang HowcanoeWang moved this to Todo in EasyIDP v2.0 Dec 5, 2022
@HowcanoeWang HowcanoeWang changed the title Load full GeoTiff if enough RAM Load GeoTiff partial loading optimize for all roi Dec 16, 2022
@HowcanoeWang HowcanoeWang changed the title Load GeoTiff partial loading optimize for all roi GeoTiff partial loading optimize for all roi Dec 16, 2022
@HowcanoeWang HowcanoeWang modified the milestones: 2.0.0.dev4, 2.0.1 Jan 10, 2023
@HowcanoeWang HowcanoeWang modified the milestones: 2.0.0, 2.1.0, 2.0.1 May 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Todo
Development

No branches or pull requests

1 participant