Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test win shadow #1299

Closed
wants to merge 81 commits into from
Closed

Conversation

davide-f
Copy link
Member

Closes # (if applicable).

Changes proposed in this Pull Request

Checklist

  • I consent to the release of this PR's code under the AGPLv3 license and non-code contributions under CC0-1.0 and CC-BY-4.0.
  • I tested my contribution locally and it seems to work fine.
  • Code and workflow changes are sufficiently documented.
  • Newly introduced dependencies are added to envs/environment.yaml and doc/requirements.txt.
  • Changes in configuration options are added in all of config.default.yaml and config.tutorial.yaml.
  • Add a test config or line additions to test/ (note tests are changing the config.tutorial.yaml)
  • Changes in configuration options are also documented in doc/configtables/*.csv and line references are adjusted in doc/configuration.rst and doc/tutorial.rst.
  • A note for the release notes doc/release_notes.rst is amended in the format of previous release notes, including reference to the requested PR.

github-actions bot and others added 30 commits October 30, 2024 14:47
* Add a zenodo link to natura.tiff

* Update environment

* Revise structure definition for lines

* Remove get_aggregation_strategies

* Fix typo aggregation_strategies

* Replace aggregategenerators with aggregateoneport

* Add aggregation strategies as a parameter

* Re-define aggregation strategies

* Update aggregation strategies

* Update aggregation strategies for lines

* Update aggregation strategies for buses

* Fix typo

* Put aggregation strategies into a variable

* Parametrize the aggregation strategies

* Refactor update of the aggregation strategies

* Clean-up the code

* Revert "Add a zenodo link to natura.tiff"

This reverts commit 7700759.

* Define an explicit clustering strategy for v_nom

* Add a release note

* Get glpk back

* Specify v_nom for buses explicitly

* Revert "Specify v_nom for buses explicitly"

This reverts commit 20192e6.

* Add a version restriction to the environment specification

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Adjust naming

* Move the variable definition

* Move the variable

* Upgrade PyPSA version

---------

Co-authored-by: Davide Fioriti <[email protected]>
Co-authored-by: Davide Fioriti <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add a zenodo link to natura.tiff

* Update environment

* Revise structure definition for lines

* Remove get_aggregation_strategies

* Fix typo aggregation_strategies

* Replace aggregategenerators with aggregateoneport

* Add aggregation strategies as a parameter

* Re-define aggregation strategies

* Update aggregation strategies

* Update aggregation strategies for lines

* Update aggregation strategies for buses

* Fix typo

* Put aggregation strategies into a variable

* Parametrize the aggregation strategies

* Refactor update of the aggregation strategies

* Clean-up the code

* Revert "Add a zenodo link to natura.tiff"

This reverts commit 7700759.

* Define an explicit clustering strategy for v_nom

* Add a release note

* Get glpk back

* Specify v_nom for buses explicitly

* Revert "Specify v_nom for buses explicitly"

This reverts commit 20192e6.

* Add a version restriction to the environment specification

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Adjust naming

* Move the variable definition

* Move the variable

* Upgrade PyPSA version

* Update docstring

* Fix imports duplication

* Update imports

* Update the carrier-capacity constraint

* Add docstring

* Update the equity constraint

* Add docstring

* Update BAU constraint

* Update SAFE constraint

* Add docstring

* Update operational reserve margin constraint

* Add docstring

* Add an new argument to the RM constraint

* Update the update of capacity constraints

* Update adding an operational reserve margin constraint

* Update docstring

* Update battery constraint

* Add docstring

* Update a constraint related to a RES share

* Fix usage of add_ERS_constraints

* Update solving script

* Update a solving run

* Fix typos

---------

Co-authored-by: Davide Fioriti <[email protected]>
Co-authored-by: Davide Fioriti <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…ets-earth#1162)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Add a zenodo link to natura.tiff

* Update environment

* Revise structure definition for lines

* Remove get_aggregation_strategies

* Fix typo aggregation_strategies

* Replace aggregategenerators with aggregateoneport

* Add aggregation strategies as a parameter

* Re-define aggregation strategies

* Update aggregation strategies

* Update aggregation strategies for lines

* Update aggregation strategies for buses

* Fix typo

* Put aggregation strategies into a variable

* Parametrize the aggregation strategies

* Refactor update of the aggregation strategies

* Clean-up the code

* Revert "Add a zenodo link to natura.tiff"

This reverts commit 7700759.

* Define an explicit clustering strategy for v_nom

* Add a release note

* Get glpk back

* Specify v_nom for buses explicitly

* Revert "Specify v_nom for buses explicitly"

This reverts commit 20192e6.

* Add a version restriction to the environment specification

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Adjust naming

* Move the variable definition

* Move the variable

* Upgrade PyPSA version

---------

Co-authored-by: Davide Fioriti <[email protected]>
Co-authored-by: Davide Fioriti <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add a zenodo link to natura.tiff

* Update environment

* Revise structure definition for lines

* Remove get_aggregation_strategies

* Fix typo aggregation_strategies

* Replace aggregategenerators with aggregateoneport

* Add aggregation strategies as a parameter

* Re-define aggregation strategies

* Update aggregation strategies

* Update aggregation strategies for lines

* Update aggregation strategies for buses

* Fix typo

* Put aggregation strategies into a variable

* Parametrize the aggregation strategies

* Refactor update of the aggregation strategies

* Clean-up the code

* Revert "Add a zenodo link to natura.tiff"

This reverts commit 7700759.

* Define an explicit clustering strategy for v_nom

* Add a release note

* Get glpk back

* Specify v_nom for buses explicitly

* Revert "Specify v_nom for buses explicitly"

This reverts commit 20192e6.

* Add a version restriction to the environment specification

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Adjust naming

* Move the variable definition

* Move the variable

* Upgrade PyPSA version

* Update docstring

* Fix imports duplication

* Update imports

* Update the carrier-capacity constraint

* Add docstring

* Update the equity constraint

* Add docstring

* Update BAU constraint

* Update SAFE constraint

* Add docstring

* Update operational reserve margin constraint

* Add docstring

* Add an new argument to the RM constraint

* Update the update of capacity constraints

* Update adding an operational reserve margin constraint

* Update docstring

* Update battery constraint

* Add docstring

* Update a constraint related to a RES share

* Fix usage of add_ERS_constraints

* Update solving script

* Update a solving run

* Fix typos

---------

Co-authored-by: Davide Fioriti <[email protected]>
Co-authored-by: Davide Fioriti <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@davide-f
Copy link
Member Author

davide-f commented Jan 16, 2025

Hello @Eric-Nitschke ,
In #1276 the error on windows is visible.
The issue on windows is extremely tricky and linked to a combination of snakemake shutil dependency, see snakemake/snakemake#2383

This is a tentative PR to check if the problem still persists but it is highly tricky; I've also seen issues with the use of absolute paths in snakemake. I'm not sure the use of the abs path for overrides may work in general. Have you tested it locally?

FYI @Eddy-JV

Update: it may be worth testing if the update of snakemake fixes this

@Eric-Nitschke
Copy link
Contributor

Eric-Nitschke commented Jan 16, 2025

Hey @davide-f,
when working on PR #1192 I first encountered the bug on my local machine on Windows 11. When testing PyPSA-Earth today with the newest merges I again encountered it:

Traceback (most recent call last):
  File "[...]\snakemake\__init__.py", line 792, in snakemake
    success = workflow.execute(
  File "[...]\snakemake\workflow.py", line 1246, in execute
    raise e
  File "[...]\snakemake\workflow.py", line 1242, in execute
    success = self.scheduler.schedule()
  File "[...]\snakemake\scheduler.py", line 636, in schedule
    self.run(runjobs)
  File "[...]\snakemake\scheduler.py", line 685, in run
    executor.run_jobs(
  File "[...]\snakemake\executors\__init__.py", line 158, in run_jobs
    self.run(
  File "[...]\snakemake\executors\__init__.py", line 582, in run
    future = self.run_single_job(job)
  File "[...]\snakemake\executors\__init__.py", line 635, in run_single_job
    self.cached_or_run, job, run_wrapper, *self.job_args_and_prepare(job)
  File "[...]\snakemake\executors\__init__.py", line 587, in job_args_and_prepare
    job.prepare()
  File "[...]\snakemake\jobs.py", line 924, in prepare
    shutil.copy(rel_path, copy)
  File "[...]\envs\pypsa-earth\lib\shutil.py", line 417, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "[...]\envs\pypsa-earth\lib\shutil.py", line 254, in copyfile
    with open(src, 'rb') as fsrc:
PermissionError: [Errno 13] Permission denied: 'data\\override_component_attrs'

When working on PR #1192 changing the input to os.getcwd() + "/data/override_component_attrs" fixed the issue for me. However, if I understood your previous comments properly, this approach can lead to other issues.

@davide-f
Copy link
Member Author

Yes, the os.getcwd cannot be used.
The use of that is not compatible with the use of submodules.
#1301 should fix that. @Eric-Nitschke could you test it?

@Eric-Nitschke Eric-Nitschke mentioned this pull request Jan 16, 2025
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants