GeoHub Component design guide

[iferencik]

Main assumptions and facts

GeoHub leverages Maplibre GL to handle geospatial data visualization and svelte/sveltekit to create slick and reactive UI. Because Malibre Map has it's own lifecycle it is necessary to synchronize the lifecycle of svelte components with the lifecycle of the Map object.

There are a range of options that could be used to achieve this but the practicalities of developing components has shown not all to be reliable. These are the main assumptions and facts that underline this process:

1. svelte-kit drives the synchronization process
2. the Maplibre loaded() event is used to notify the svelte components the map is loaded/ready
3. the first stage where this is happening is the component initialization. Note that in this case the classical onMount() is not sufficient because, the Geohub is trying to facilitate the user experience by taking small actions (eg. choosing what legend to apply a given layer by default) and this means imperative initialization. onMount() is executed after the DOM has been rendered and this prevents such actions
4. the second stage is UI interaction (change color map, etc)
5. reactivity plays an important role because it can slim down the code base provided it is used correctly
6. the bulk of inter-component communications should be done through
   a). simple props sharing
   b). events -> child-parent and parent-child
   c). svelte stores
7. the Map object needs to be shared through a store so various independent components can use it
8. whenever the map object is changed/written to the store it needs to be ready or loaded. Otherwise the other components that read various of it's props are not going to receive notifications and work correctly
9. the UI should always be wrapped inside a svelte await block resolving an async init function

Svelte component lifecycle

There are some specific aspects of how svelte components life-cycle happens and this has implication on how GeoHub components have to be designed. There are three main parts of a svelte component,

<script>
</script>

<div>I am a component</div

<style></style>

The most important part is the script and it features a specific execution order as follows:

1. static declarative initialization
2. reactive statements
3. special lifecycle functions

GeoHub lifecycle

1. declarative initialization
2. imperative initialization
3. reactive statements
4. UI rendering

It is important to note the rendering happens between somewhere inside step three, theoretically before the beforeUpdate() and onMount().Geohub componets use a custom approach with a custom async method called init() wharred inside an await block. All of this is illustrated in the code below

<script lang="ts">
  /*IMPORTS*/
  import {
    getValueFromRasterTileUrl,
    sleep,
    getLayerSourceUrl,
    fetchUrl,
    loadMap,
    updateParamsInURL,
    getValueFromRasterTileUrlAsync,
    getRandomColormap,
    updateLayerURL,
  } from '$lib/helper'
  import { Loader, Tabs } from '@undp-data/svelte-undp-design'
  import { map, layerList } from '$stores'
  import DefaultRasterLegend from '$components/raster/DefaultRasterLegend.svelte'
  import type { Layer, RasterTileMetadata } from '$lib/types'

  console.clear()

  /*EXPORTS*/
  export let layerId: string

  /*DECLARATIVE INIT*/
  console.log('init')

  /*REACTIVE DECLARATIVE STATE*/
  $: colormapName = getValueFromRasterTileUrl($map, layerId, 'colormap_name') as string
  $: layer = $layerList.find((l) => l.id == layerId) as Layer
  $: info = layer?.info as RasterTileMetadata

  /*IMPERATIVE INIT*/
  const init = async () => {
    colormapName = (await getValueFromRasterTileUrlAsync($map, layerId, 'colormap_name')) as string

  }
  /*FUCTIONS*/

  
  /*EVENTS*/
  const changeColorMap = async (e: Event) => {
    const newColormapName: string = getRandomColormap()
    //await updateColorMapName(newColormapName)
    const layerUrl = getLayerSourceUrl($map, layerId) as string
    const layerURL = new URL(layerUrl)
    await updateLayerURL(layerId, layerURL, { colormap_name: newColormapName })

    info.test = 'RasterLayer'
    layer = { ...layer, info: info }
    const layers = $layerList.map((lyr) => {
      return layer.id !== lyr.id ? lyr : layer
    })
    layerList.set([...layers])
  }

  console.log('regular')

  $: console.log('in RasterLayer',  info?.test)



</script>

{#await init()}
  <Loader size="small" />
{:then loaded}
  {colormapName}
  <DefaultRasterLegend {layerId} />
  <hr />
  <button
    class="button is-info"
    on:click={changeColorMap}>
    Change me
  </button>
{/await}

<style lang="scss">
</style>

This approach ensures the geoHub components are being initialized consistently.

Map <=> UI intercommunication

The interaction between the Maplibre and the UI is implemented through a series of utility functions. In general the communication goes

UI -> Map -> UI

There are directions modes of communications:

1. UI->Map

This functionality is implemented as an async utility function.

export const updateLayerURL = async (layerID: string, layerURL: URL, params: Record<string, string>) => {
  const map = get(mapStore)
  //console.log(JSON.stringify(Object.fromEntries(layerURL.searchParams), null, '\t'))
  Object.keys(params).forEach((key) => {
    layerURL.searchParams.set(key, params[key])
  })
  //console.log(JSON.stringify(Object.fromEntries(layerURL.searchParams), null, '\t'))

  if ('getStyle' in map) {
    const style = map.getStyle()

    if (style?.sources) {
      // eslint-disable-next-line @typescript-eslint/ban-ts-comment
      // @ts-ignore
      style.sources[layerID].tiles = [decodeURI(layerURL.toString())]
      // delete all props which have undefined value
      // probably it is a bug of maplibre to add undefined property (like url, bounds) to the style,
      // and maplibre complains it has error which some of properties are not defined.
      Object.keys(style.sources).forEach((key) => {
        const src = style.sources[key]
        Object.keys(src).forEach((prop) => {
          if (!src[prop]) {
            delete src[prop]
          }
        })
      })

      map.setStyle(style)
      await loadMap(map)
      mapStore.set(map)
    }
  }
}

The most important section of the function are the last three lines. We can see the map is updated, and saved only after it is ready. This ensures that all reactive subscribers are notified with valid values.

2. Map -UI
This direction is covered by two specific functions targeting two different initialization modes: declarative/synchronous

export const getValueFromRasterTileUrl = (
  map: Map,
  layerId: string,
  paramName: 'colormap_name' | 'rescale' | 'expression' | 'colormap' | 'url',
): string | number[] | number[][][] | { [key: string]: number[] } => {
  const source: RasterTileSource = map.getSource(map.getLayer(layerId).source) as RasterTileSource
  if (source.type !== 'raster') {
    throw new Error(`Invalid source type: ${source.type}`)
  }
  if (!(source && source.tiles && source.tiles.length > 0)) return
  const tileUrl = new URL(source.tiles[0])
  let value: string | number[] | number[][][] | { [key: string]: number[] } = tileUrl.searchParams.get(paramName)
  if (value && paramName === 'rescale') {
    const values = value.split(',')
    value = values.map((v) => Number(v))
  } else if (value && paramName === 'colormap') {
    if (Array.isArray(value)) {
      // interval
      value = JSON.parse(value) as number[][][]
    } else {
      // unique
      value = JSON.parse(value) as unknown as { [key: string]: number[] }
    }
  }
  return value
}

and imperative/async

export const getValueFromRasterTileUrlAsync = async (
  map: Map,
  layerId: string,
  paramName: 'colormap_name' | 'rescale' | 'expression' | 'colormap' | 'url',
): Promise<string | number[] | number[][][] | { [key: string]: number[] }> => {
  await loadMap(map)
  const source: RasterTileSource = map.getSource(map.getLayer(layerId).source) as RasterTileSource
  if (source.type !== 'raster') {
    throw new Error(`Invalid source type: ${source.type}`)
  }
  //if (!(source && source.tiles && source.tiles.length > 0)) await loadMap(map)
  const tileUrl = new URL(source.tiles[0])
  let value: string | number[] | number[][][] | { [key: string]: number[] } = tileUrl.searchParams.get(paramName)
  if (value && paramName === 'rescale') {
    const values = value.split(',')
    value = values.map((v) => Number(v))
  } else if (value && paramName === 'colormap') {
    if (Array.isArray(value)) {
      // interval
      value = JSON.parse(value) as number[][][]
    } else {
      // unique
      value = JSON.parse(value) as unknown as { [key: string]: number[] }
    }
  }
  return value
}

The functions performs exactly the same thing, the only difference being in the way they behave. The async function is designed to be fetch state in an imperative context or on demand and because of this it makes sure the map is loaded before it extracts the state. The declarative version assumes the map is loaded/ready and fetches the state from the map.

The last piece of the setup is the loadMap function

import { sleep } from '$lib/helper'
import type { Map } from 'maplibre-gl'

export const loadMap = async (map: Map, ms = 100, wait_ms = 5000) => {
  const startTime: number = Date.now()
  if (map) {
    if (!map.loaded()) {
      while (!map.loaded()) {
        await sleep(ms)
        if (Date.now() - startTime > wait_ms) return false
      }
    }
    return true
  } else {
    return false
  }
}

This is a small workhorse designed to be robust and and features some safety measures like a maximum amount of time to wait before it returns as not to get blocked.

Using reactivity and synchronization correctly will ensure the components are consistent and GeoHub behaves correctly and is predicatable, two very important features

[JinIgarashi]

In addition to the above state management of No.6 on svelte/sveltekit, GeoHub also uses URL query params and page load data as state managements.

These two state management methods are used in many places in GeoHub. For instance,

• data tab in main Geohub to search datasets by using query params and load data
• /data page to get table data of published and ingesting datasets from server side.
• /data/publish page to initialize a dataset specified by query param of url
• /map page to search saved maps from server side.
• etc...

load data

For page data, please refer the following URL.
https://kit.svelte.dev/docs/load

If +page.ts, +page.server.ts, +layout.ts or +layout.server.ts returns an object, the data will be accessible through $page.data in +page.svelte or any other components used from +page.svelte`.

Some of data can be directly accessed from components via page data.

An example of +page.sever.ts to return data to frontend.

geohub/sites/geohub/src/routes/data/+page.server.ts

Lines 49 to 55 in a7bd62b

	return {
	promises: {
	datasets: getDatasets(event.fetch, apiUrl),
	ingestingDatasets: getIngestingDatasets(event.fetch),
	tags: getTags(event.fetch, new URL(`${url.origin}/api/datasets${apiUrl.search}`)),
	},
	}

An example of +page.svelte

geohub/sites/geohub/src/routes/data/+page.svelte

Lines 1 to 12 in a7bd62b

	<script lang="ts">
	import type { PageData } from './$types'
	import { invalidateAll } from '$app/navigation'
	import PublishedDatasets from '$components/data-upload/PublishedDatasets.svelte'
	import IngestingDatasets from '$components/data-upload/IngestingDatasets.svelte'
	import type { DatasetFeatureCollection, IngestingDataset } from '$lib/types'
	import DataUploadButton from '$components/data-upload/DataUploadButton.svelte'
	export let data: PageData
	let datasets: Promise<DatasetFeatureCollection> = data.promises.datasets
	let ingestingDatasets: Promise<IngestingDataset[]> = data.promises.ingestingDatasets

query params between +page.svelte and +page.server.ts

In +page.server.ts, it can access URL query params on the page as following code.

geohub/sites/geohub/src/routes/data/+page.server.ts

Lines 7 to 10 in a7bd62b

	export const load: PageServerLoad = async (event) => {
	const { locals, url, parent } = event
	const session = await locals.getSession()
	if (!session) return {}

This enables robust communication between frontend and backend.

In frontend side, a component can access current URL by the below code

import { page } from '$app/stores';

let url = $page.url

You can simply update query params of current URL directly. But you need to use goto to set new URL. Otherwise URL will not be reflected in the browser. After calling goto, then have to call invalidateAllmethod to reload all load functions in server side. You can access updated loading data after invalidating.

import { page } from '$app/stores';

import {
    goto,
    invalidateAll
} from '$app/navigation';

let url = $page.url

url.params.set('key', 'value')

// set new URL to browser. In this stage, `load` function in server.ts will not be called again.
await goto(url)

// `load` function will be called again once this method is executed. Then $page.data will be updated by the latest URL query params.
await invalidateAll()

[JinIgarashi]

For maplibre's style state management, maplibre has new option of transformStyle in setStyle function. This new option maybe can improve our code of style updating process

https://maplibre.org/maplibre-gl-js-docs/api/map/#map#setstyle