[WIP] Elevation

[scd31]

This is heavily WIP. I'm opening this PR at request of @louis-e - it still needs a ton of work. I'm going to do my best to summarize below, but it's been a few months, so I likely won't remember everything and may make some mistakes.

There are two pieces to this:

1. Take elevation data and turn it into a y-map. I have not started on this. For debugging I'm just generating sinewaves. I'm doing this in a modular way though, so it should be pretty easy to swap out.

2. Take the y-map and use it to generate the minecraft map. This is the part I've been focusing on. It works reasonably well but there are a ton of edge cases to deal with still. Off the top of my head:

• Rivers! The elevation map should have rivers as the low point, but depending on their resolution and how accurate they actually are, we may need to manually "push" rivers down into the ground so that they don't overflow onto the terrain.

• River banks - we'll want to slope the terrain down towards rivers, probably depending on how wide the river is.

• Buildings/parking lots on hills - ideally we want to figure out which road they attach to and "anchor" them at the road's height. Right now I'm just anchoring the buildings to the lowest or highest point on the y-map under their foundation (I forget which)

I think it's these edge-cases that caused me to lose steam. There were a bunch of them and it started to make me realize how much effort it would be, and then I got busy with other projects.

Probably there is opportunity here to have multiple people work on this. I'm wondering if we merge this in (possibly with some cleanup) and have an experimental flag to turn it on. Then people can merge in improvements and only when everything is ready do we drop the flag.

I'm definitely open to continuing working on this!

[scd31]

This is where the sinewave comes from. When we start using actual elevation data this is where it would go

[scd31]

You'll see these in a few spots. They're only used by elements that I haven't updated to use the Ground struct yet.

[amir16yp]

you could download SRTM data from the european space agency (only mirror i found that doesnt require some kind of registration)
PoC for ground.rs, to get an idea:

use crate::cartesian::XZPoint;
use std::collections::HashMap;
use std::fs::File;
use std::io::{BufReader, Read, Seek, SeekFrom, Write};
use std::path::Path;
use reqwest::blocking::get;
use zip::ZipArchive;

pub struct Ground {
    srtm: SRTMData,
}

impl Ground {
    pub fn new(bbox: (f64, f64, f64, f64), _scale_factor_x: f64, _scale_factor_z: f64) -> Self {
        Self {
            srtm: SRTMData::new(bbox)
        }
    }

    #[inline(always)]
    pub fn level(&self, coord: XZPoint) -> i32 {
        let elevation = self.srtm.get_elevation(coord);
        self.srtm.normalize_to_minecraft(elevation)
    }

    #[inline(always)]
    pub fn min_level<I: Iterator<Item = XZPoint>>(&self, coords: I) -> Option<i32> {
        coords.map(|c| self.level(c)).min()
    }

    #[inline(always)]
    pub fn max_level<I: Iterator<Item = XZPoint>>(&self, coords: I) -> Option<i32> {
        coords.map(|c| self.level(c)).max()
    }
}

struct SRTMData {
    elevations: HashMap<(i32, i32), i32>,
    min_elevation: i32,
    max_elevation: i32,
}

impl SRTMData {
    fn new(bbox: (f64, f64, f64, f64)) -> Self {
        let mut srtm = Self {
            elevations: HashMap::new(),
            min_elevation: i32::MAX,
            max_elevation: i32::MIN,
        };

        // Calculate required SRTM tiles
        let min_lat = bbox.1.floor() as i32;
        let min_lon = bbox.0.floor() as i32;
        let max_lat = bbox.3.ceil() as i32;
        let max_lon = bbox.2.ceil() as i32;

        println!("Needed SRTM tiles:");
        for lat in min_lat..=max_lat {
            for lon in min_lon..=max_lon {
                let file_path = ensure_srtm_file(lat, lon);
                if let Some(path) = file_path {
                    srtm.process_srtm_file(&path, lat, lon, bbox);
                }
            }
        }

        if !srtm.elevations.is_empty() {
            srtm.min_elevation = *srtm.elevations.values().min().unwrap();
            srtm.max_elevation = *srtm.elevations.values().max().unwrap();
            println!("Elevation data loaded: {}m to {}m", srtm.min_elevation, srtm.max_elevation);
        } else {
            println!("Warning: No elevation data found, using default values");
            srtm.min_elevation = 0;
            srtm.max_elevation = 255;
        }

        srtm
    }

    fn process_srtm_file(&mut self, path: &str, base_lat: i32, base_lon: i32, bbox: (f64, f64, f64, f64)) {
        println!("Processing SRTM file: {}", path);
        let file = File::open(path).expect("Failed to open SRTM file");
        let mut reader = BufReader::new(file);
        let mut buffer = vec![0u8; 2];

        // SRTM files are 1-degree squares with 1201x1201 points
        const SIZE: usize = 1201;
        const POINTS_PER_DEGREE: f64 = (SIZE - 1) as f64;

        // Convert bbox coordinates to file indices
        let min_y = ((1.0 - (bbox.3 - base_lat as f64)) * POINTS_PER_DEGREE) as usize;
        let max_y = ((1.0 - (bbox.1 - base_lat as f64)) * POINTS_PER_DEGREE) as usize;
        let min_x = ((bbox.0 - base_lon as f64) * POINTS_PER_DEGREE) as usize;
        let max_x = ((bbox.2 - base_lon as f64) * POINTS_PER_DEGREE) as usize;

        println!("Reading coordinates from y={}-{}, x={}-{}", min_y, max_y, min_x, max_x);

        for y in min_y.min(SIZE-1)..=max_y.min(SIZE-1) {
            for x in min_x.min(SIZE-1)..=max_x.min(SIZE-1) {
                // Get coordinates for this point
                let lat = base_lat as f64 + 1.0 - (y as f64 / POINTS_PER_DEGREE);
                let lon = base_lon as f64 + (x as f64 / POINTS_PER_DEGREE);

                // Read elevation from file (big-endian 16-bit signed integer)
                let offset = ((y * SIZE + x) * 2) as u64;
                if reader.seek(SeekFrom::Start(offset)).is_ok() {
                    if reader.read_exact(&mut buffer).is_ok() {
                        let height = ((buffer[0] as i16) << 8 | buffer[1] as i16) as i32;
                        
                        // Skip SRTM voids (-32768)
                        if height != -32768 {
                            let (mc_x, mc_z) = self.to_minecraft_coords(lat, lon, bbox);
                            self.elevations.insert((mc_x, mc_z), height);
                        }
                    }
                }
            }
        }
    }

    fn to_minecraft_coords(&self, lat: f64, lon: f64, bbox: (f64, f64, f64, f64)) -> (i32, i32) {
       // todo: complete this
    }

    fn get_elevation(&self, point: XZPoint) -> i32 {
        self.elevations.get(&(point.x, point.z)).copied().unwrap_or(0)
    }

    fn normalize_to_minecraft(&self, elevation: i32) -> i32 {
        if self.max_elevation == self.min_elevation {
            return 60;
        }

        // Scale elevation to reasonable Minecraft height
        let range = (self.max_elevation - self.min_elevation) as f64;
        let normalized = ((elevation - self.min_elevation) as f64 / range * 128.0) as i32 + 32;
        normalized.clamp(1, 255)
    }
}

fn ensure_srtm_file(lat: i32, lon: i32) -> Option<String> {
    let filename = format!("{}{}{}{}",
        if lat >= 0 { "N" } else { "S" },
        lat.abs().to_string().pad_left('0', 2),
        if lon >= 0 { "E" } else { "W" },
        lon.abs().to_string().pad_left('0', 3)
    );

    // Create cache directory if needed
    let cache_dir = Path::new("srtm_cache");
    if !cache_dir.exists() {
        std::fs::create_dir(cache_dir).expect("Failed to create SRTM cache directory");
    }

    let cache_path = cache_dir.join(format!("{}.hgt", filename));
    println!("Looking for SRTM file: {}", filename);

    if !cache_path.exists() {
        // EU SPACE AGENCY SRTM URL
        let url = format!(
            "https://step.esa.int/auxdata/dem/SRTMGL1/{}.SRTMGL1.hgt.zip",
            filename
        );

        println!("Downloading from EU Space Agency: {}", url);
        match get(&url) {
            Ok(response) => {
                if response.status().is_success() {
                    // Create a temporary file to store the zip
                    let temp_path = cache_dir.join(format!("{}.zip", filename));
                    match File::create(&temp_path) {
                        Ok(mut temp_file) => {
                            match response.bytes() {
                                Ok(data) => {
                                    if temp_file.write_all(&data).is_ok() {
                                        println!("Downloaded zip file successfully");
                                        
                                        // Extract the HGT file from the zip
                                        match File::open(&temp_path) {
                                            Ok(zip_file) => {
                                                match ZipArchive::new(zip_file) {
                                                    Ok(mut archive) => {
                                                        match archive.by_name(&format!("{}.hgt", filename)) {
                                                            Ok(mut hgt_file) => {
                                                                match File::create(&cache_path) {
                                                                    Ok(mut output) => {
                                                                        match std::io::copy(&mut hgt_file, &mut output) {
                                                                            Ok(_) => {
                                                                                println!("Successfully extracted HGT file");
                                                                                // Clean up zip file
                                                                                let _ = std::fs::remove_file(&temp_path);
                                                                                return Some(cache_path.to_str().unwrap().to_string());
                                                                            }
                                                                            Err(e) => println!("Failed to copy HGT data: {}", e)
                                                                        }
                                                                    }
                                                                    Err(e) => println!("Failed to create output file: {}", e)
                                                                }
                                                            }
                                                            Err(e) => println!("Failed to find HGT file in zip: {}", e)
                                                        }
                                                    }
                                                    Err(e) => println!("Failed to read zip file: {}", e)
                                                }
                                            }
                                            Err(e) => println!("Failed to open zip file: {}", e)
                                        }
                                    }
                                }
                                Err(e) => println!("Failed to get response bytes: {}", e)
                            }
                        }
                        Err(e) => println!("Failed to create temp file: {}", e)
                    }
                } else {
                    println!("Download failed with status: {}", response.status());
                }
            }
            Err(e) => println!("Download error: {}", e)
        }
        None
    } else {
        Some(cache_path.to_str().unwrap().to_string())
    }
}

[louis-e]

This is so great, thanks for sharing it with us! Already working on resolving the merge conflicts for the current code base. Maybe we could work on the elevation feature in a seperate branch?

[louis-e]

I’ve merged your work into a new branch called elevation-wip, where I’ve resolved conflicts and where we can continue developing the feature. Your commits are preserved, and you’ll be credited in the final merge to main. Thanks a lot again, this is such a good foundation to work on!