Accessing Dask Cluster Dashboard

[caitlinadams]

When creating a Dask cluster and client using existing EASI code, I don't get the behaviour I would typically expect based on my experience in DE Africa. The code I am using is

from dask.distributed import Client, LocalCluster

# Connect to an existing LocalCluster if available
# The default port is 8786
try:
    client = Client(f'localhost:8786', timeout='2s')
    cluster = client.cluster  # None
except:
    cluster = LocalCluster(scheduler_port=8786)
    client = Client(cluster)

display(cluster if cluster else client)

This returns the dashboard url as http://127.0.0.1:8787/status but when I follow this link, I find that the site can't be reached, and the dashboard does not display.

If I use the following, I get the behavoiur I expect, with the dashboard url being listed as https://hub.asia.easi-eo.solutions/user/<username>/proxy/<port>/status which I can view.

from datacube.utils.dask import start_local_dask

client = start_local_dask()

display(client)

What's the correct practice for starting a Dask Cluster on EASI and getting access to the Dashboard?

[woodcockr]

When starting a local dask cluster the url needs to be rewritten to point at the jupyter hub proxy. The datacube.utils.dask function does that for you as I recall. Matt's got a similar function stashed somewhere in the examples.