Q: on merge_stations_lines_by_station_id_and_voltage

[pz-max]

Hi @davide-f,
The merge_stations_lines_by_station_id_and_voltage function takes quite a while. I added an option to turn it off but realised that it is an essential function? Like bus0 and bus1 won't be set if the the functions is not run.

So next to removing the option to run (my bad), are there ideas to increase the performance?

https://github.com/pypsa-meets-africa/pypsa-africa/blob/757c418aad5e17d361fa6e2ee02619ba30a874a2/scripts/osm_built_network.py#L554-L558

[davide-f]

Unfortunately, I am aware of that problem and it is not an easy answer.
First, don't get scared about the numbers, which often are like X/69000 points, because after 30-40%, it speeds up incredibly.
Anyway that function is slow and the ways to improve the performance relate to different ways to cluster the nodes.
That functions cluster with a tolerance in meters.

The best would be to use a function like groupby on geometries with tolerances. I have searched for it but that's not a trivial think to do.
I thought that we could use a groupby using rounded GPS coordinates, which would be way faster, yet sometimes the accuracy is not equivalent and dependent on the specific GIS location.

If tolerances of 5-10km are acceptable, than the solution above could work. Note, the tol=4000 is just a temporary fix because of the issue #124 ; once the latter is solved, the tolerance could be reduced

[pz-max]

Yeah, just realized that it runs for whole Africa below 40min so not too bad. 👍
Debugging should also happen not on an Africa scale - something I still need to learn ;)
Guess speeding the process up is maybe not yet a priority, though, we could add an issue if desired