Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generation stops at almost last download #22

Open
UltraDanHR opened this issue Dec 25, 2024 · 5 comments
Open

Generation stops at almost last download #22

UltraDanHR opened this issue Dec 25, 2024 · 5 comments
Assignees
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@UltraDanHR
Copy link

For big downloads (~90k), generation stops at almost last download, both for albedo and heightmap

@manticorp
Copy link
Owner

manticorp commented Dec 25, 2024 via email

@manticorp manticorp self-assigned this Dec 31, 2024
@manticorp
Copy link
Owner

Hmm - I will look into this in more detail when I have some time in the new year.

I've just sat down at my computer and tried the generation myself, and the URL you provided is trying to create a 81920 x 71680 pixel image.

That's a 5,872 megapixel image!!

So, it's first got to download each of the 90,201 tiles which are ~120kb each (which is 10.8GB of download, for a start):

image

It will then need to load all those images into memory, and create an output image in memory as a HTML canvas.

Uncompressed, 5,872 MP in 8 bit RBG is going to be ~ 17GB.

So at least 34GB of memory is required for an albedo image, and I think a bit more for the heightmap as it loads it into a UINT16 array.

Overall, the memory requirements of creating such a large image might be the stumbling block, and beyond the scale of this application.

My suggestions would be either:

I will have a think as to how we can modify the application for extremely large generations, but it's unlikely that there will be an automatic solution because of browser limitations. It would probably require being able to stream directly to files, which is note widely supported.

@manticorp
Copy link
Owner

I'll leave this issue open, because it would be awesome to do super large exports like this!

@manticorp manticorp added enhancement New feature or request help wanted Extra attention is needed labels Dec 31, 2024
@UltraDanHR
Copy link
Author

So I gave a thought about this, this needs some steps:

  • Option to freeze cursor in place
  • Sometimes you want to check certain terrain in the area you've selected, but this moves the cursor
  • Divide area by cells, select one area as focus
  • This will allow to export big maps, one area at a time, without spending a whole day selecting squares and trying to match these
  • Remember which area you've downloaded
  • I'm not sure if this will even be worth the effort to be done, basically store in cookies or cache what has been downloaded
  • Maybe manual user input?
  • User can pretty much just use a Google sheets, hence previous comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants