Dask Update #1577
Dask Update #1577
Conversation
| @@ -2076,16 +2076,16 @@ def connect_dask( | |||
| ) | |||
|
|
|||
| # Note: We need to do this on the head node too, because this creates all the worker processes | |||
| for node in self.ips: | |||
| for idx, node in enumerate(self.ips): | |||
There was a problem hiding this comment.
Probably cleaner to just iterate over self.internal_ips - otherwise there could potentially be an index out of range error thrown
There was a problem hiding this comment.
We no longer have a config field which contains both, so we have to iterate over one of the two since we use both. self.ips gives us node which is used in self.run()
There was a problem hiding this comment.
Ah I see, are the array lengths always the same? Instead, it might be good to add a check before calling self.internal_ips[idx] like:
if len(self.internal_ips) > idx:
self.run...
There was a problem hiding this comment.
I think they are guaranteed to be the same length, no? Since each represents each node of a cluster.
Something much more wrong has happened with cluster launch if the cluster config dropped one of internal or external IPs. I'm not religiously opposed to adding a check, but this is the wrong place to do it if so.
No description provided.