Hydrogen pipeline efficiency

[Eric-Nitschke]

Checklist

• I am using the current main branch or the latest release. Please indicate.
• I am running on an up-to-date pypsa-earth environment. Update via conda env update -f envs/environment.yaml.

Describe the Bug

Summary

The H2 pipeline efficiency behavior is inconsistent and unexpected depending on enable: retrieve_cost_data.

Description

When setting retrieve_cost_data: false in the config, PyPSA-Earth uses the cost.csv provided in the data folder to determine the cost, efficiency, lifetime, etc. of the network components. When using retrieve_cost_data: true, PyPSA-Earth downloads a similar dataset from the PyPSA technology database, saves it in resources and uses it to determine the aforementioned values. Importantly, not all values listed in the data/costs.csv appear in the technology database and vice versa. Specifically, the technology database does not include efficiency values for pipelines, as PyPSA-Eur uses a different, more complex way to calculate them than PyPSA-Earth. To handle the missing values, PyPSA-Earth uses the default values specified in the costs section of the config, setting the pipeline efficiency to 1. When using the data/costs.csv however, the efficiency would be set to 0.98 as it still contains a value for the pipeline efficiency.
I think this behavior is problematic, as it is difficult to spot and an non intuitive result of changing the retrieve_cost_data setting. It might affect other pipelines implemented in the sector coupled network as well.

Reproduction

The bug can be reproduced by running setting electricity: extendable_carriers: Store: ["H2"] and electricity: extendable_carriers: Link: ["H2 pipeline"] in the config and varying the enable: retrieve_cost_data setting between true and false.

[ekatef]

Hello @Eric-Nitschke and thanks a lot for the detailed description! Sounds indeed like an issue potentially very relevant for modeling workflows. Do you have any ideas on how to fix that? [We are aware of your PR #1192 on the bidirectional links and working on reviewing it, sorry for being slow!]

@Eddy-JV I know that you have been working on H2 pipelines, too. May you have any insights regarding the issue and possible fixes?

[Eric-Nitschke]

Hey @ekatef!
I can think of two ways of fixing this:

• implementing the efficiency similar to PyPSA-Eur, which calculates the efficiency based on the pipeline length and compression required. I think this would't be that complicated, but would require adding compression rates, etc. to data/costs.csv.
• alternitively, the efficiency could be added back into the PyPSA technology database. I don't think this would affect PyPSA-Eur, as they simply do not use that value.

Technically, the efficiency could also be hardcoded somewhere, but I think this should be avoided.

[Eric-Nitschke]

The pull request is my idea how to implement the PyPSA-Eur version, though I did't have the time yet to test it, so it might be still super buggy.

[Eric-Nitschke]

Generally I've found a couple issues with this approach:

• PyPSA-Eur doesn't load the efficiency values from either datafile, but from the config. I've done the same here, but I find this suboptimal, as it clutters the config and probably should be in a datafile instead.
• This fix requires multi-input links, which requires changing the networks component_attrs. I don't think this is problematic.
• The PyPSA-Eur version relies on the bus attribute "location", which I don't think is fully implemented yet in PyPSA-Earth (though it might be part of the sector optimization). As a result I had to write a workaround, which works fine for the hydrogen pipelines, but easily breaks when applied to components with inconsistent naming conventions.
• Though this fix works without the changes to lossy bidirectional links (PR Lossy Bidirectional Links #1192), it only really makes sense when using both changes together. This requires some changes, which I've added as comments and highlighted as #TODO.

[ekatef]

Hello @Eric-Nitschke, thank you so much for looking into it!

Completely support the approach and love the spirit of your implementation 😄 Generally, it's a good idea to stay consistent with PyPSA-Eur approach and improve it in parts where we see the need for improvements.

Thank you for the detailed consideration of a possible implementation plan. I think, it makes sense to go step-by-step.

• PyPSA-Eur doesn't load the efficiency values from either datafile, but from the config. I've done the same here, but I find this suboptimal, as it clutters the config and probably should be in a datafile instead.

Personally, I'm fully aligned with your idea to change config only if it's really needed. However, our default.config.yaml is still quite messy which must be improved, in any case. [We have a draft PR #1174 on that] On physical side of things, hydraulic losses in a pipeline are determined not only by the pipe parameters (surface roughness and diameter + geometry which determines local hydraulic resistances), but also by a square of the flow speed according to Darcy-Weissbach equation.

That means that our efficiency parameters depend not only on the pipeline design but also on it's operation regime and essentially provide a simplistic way to account also for this regime. So, I'd agree with PyPSA-Eur approach here as regime-dependent parameters belong rather to the config than to technology data. Does it make sense for you?

• This fix requires multi-input links, which requires changing the networks component_attrs. I don't think this is problematic.

Agree 😄

• The PyPSA-Eur version relies on the bus attribute "location", which I don't think is fully implemented yet in PyPSA-Earth (though it might be part of the sector optimization). As a result I had to write a workaround, which works fine for the hydrogen pipelines, but easily breaks when applied to components with inconsistent naming conventions.

Yeah, one of PyPSA versions introduced breaking changes which we have not been so eager to accommodate straight away as it means interruptions for all the works which relay to our model. But we have on a list to give it a go in the next couple of weeks, so probably the implementation can still account for that. If you wish, you can test these updates using pypsa_linopy_update branch

• Though this fix works without the changes to lossy bidirectional links (PR Lossy Bidirectional Links #1192), it only really makes sense when using both changes together. This requires some changes, which I've added as comments and highlighted as #TODO.

Makes perfect sense! May you be willing to introduce these updates directly into #1192? [Sharing a review for it shortly, sorry for the delay!]