Finalize duplication of retrieve_data

[davide-f]

Closes #1073 and #1074

Changes proposed in this Pull Request

This PR builds on top of #1120 to avoid duplication of retrieve_data function, bump technology data version and drop cost_dir option.

Note: dropping cost_dir is in line with pypsa-eur and we also had a TODO on that, but before merging this PR, there is the need for a check by @hazemakhalek , @Eddy-JV or @energyLS given its use in the -sec model.
Moreover, bumping the latest version of the default technology data to the latest will change values of the CI

Checklist

• I consent to the release of this PR's code under the AGPLv3 license and non-code contributions under CC0-1.0 and CC-BY-4.0.
• I tested my contribution locally and it seems to work fine.
• Code and workflow changes are sufficiently documented.
• Newly introduced dependencies are added to envs/environment.yaml and doc/requirements.txt.
• Changes in configuration options are added in all of config.default.yaml and config.tutorial.yaml.
• Add a test config or line additions to test/ (note tests are changing the config.tutorial.yaml)
• Changes in configuration options are also documented in doc/configtables/*.csv and line references are adjusted in doc/configuration.rst and doc/tutorial.rst.
• A note for the release notes doc/release_notes.rst is amended in the format of previous release notes, including reference to the requested PR.

[ekatef]

Hello @davide-f, looks like a very nice improvement!

My major concern is that the PR also contains some improvements which relate to improvements for alternative clustering. It feels like there may be some connections with other issues and fixes, but the context is not quite clear from the PR. Given the fact that alternative clustering seems to be one of the major points to be tackled, I'd suggest to keep in this PR only cost-related improvements and open another dedicated PR for the improvements on alternative clustering. What do you think?

The costs-related part looks great and basically ready to merge, unless there are some comments from @hazemakhalek @Eddy-JV and @energyLS

[ekatef]

@davide-f thanks a lot for introducing the changes! Totally agree that tackling the hydro improvements deserves a dedicated issue. Thanks for taking care about that 🙂

I think this PR is ready to merge if there is no objections from the experts on the sector-coupled part 🚀 Great work! 😄