Q on rule 'build_natura_raster'

[jarry7]

Hi,

In the interest of testing "from the outside" - beyond just the tutorial, I started playing around utilising the config.default.yaml suitably renamed to config.yaml and bringing geographical scope down to just ZA (as opposed to africa).

The workflow runs quite well until the build_natura_raster rule.

Here, logging runs until Stage 4/5: Mask geometry. Following this, a RuleException is raised as below:

Error in rule build_natura_raster:
    jobid: 17
    output: resources/natura.tiff
    log: logs/build_natura_raster.log (check log file(s) for error message)
RuleException:
CalledProcessError in line 196 of /home/jarrad/pypsa-africa/Snakefile:
Command 'set -euo pipefail;  /home/jarrad/anaconda3/envs/pypsa-africa/bin/python3.9 /home/jarrad/pypsa-africa/.snakemake/scripts/tmpvydim272.build_natura_raster.py' returned non-zero exit status 137.
  File "/home/jarrad/pypsa-a
[build_natura_raster.log](https://github.com/pypsa-meets-africa/pypsa-africa/files/7837480/build_natura_raster.log)
frica/Snakefile", line 196, in __rule_build_natura_raster
  File "/home/jarrad/anaconda3/envs/pypsa-africa/lib/python3.9/concurrent/futures/thread.py", line 58, in run

The build_natura_raster.log file is also attached for reference.

I must be missing something or doing something wrong with the workflow and/or data dependencies.

Could anyone assist me here?

Thanks.

[pz-max]

Hi @jarry7 , Do you have the natura.tiff in your resources folder?
If not, you might want to download first the data by setting retrieve_databundle: true (something you have to do the first time you try to run pypsa-africa). Just a note if the tutorial is true the retrieve data bundle function will download different data. So make sure in case you download the data for ZA that tutorial setting is set to false.
https://github.com/pypsa-meets-africa/pypsa-africa/blob/2cd796463004d8aeb8751cf21d8fe466a85936cb/config.default.yaml#L28-L32

[jarry7]

Hi Max,

Thanks for the quick response.

The tutorial flag is tutorial: False (as I am using the config.default.yaml file renamed to config.yaml).

I also did update the enable: section of the config.yaml to make sure the download happens. Thus, the enable section looks like this:

  retrieve_databundle: true  #  Recommended 'true', for the first run. Otherwise data might be missing.
  download_osm_data: true  # If 'true', OpenStreetMap data will be downloaded for the above given countries
  build_natura_raster: true  # If True, than an exclusion raster will be build
  build_cutout: true

[davide-f]

Hi @jarry7 , I notice that build_cutout is enabled; was it enabled also in the first attempt you made?
Be aware that to successfully run the build_cutout rule, you need the cdsapi installed to access the copernicus data dynamically, see build_cutout rule
This may lead to the issue that you are experiencing; I suggest you to rerun the script with the build_cutout disabled. build_cutout: false

[jarry7]

Hi Davide,

Yes, it was enabled and I do have cdsapi installed (as guided in the installation) to get the data from copernicus.

Nonetheless, I seem to have moved on to a further issue in add electricity rule (seems to be linked to the hydro inflows in resources/profile_hydro.nc):

Traceback (most recent call last):
  File "/home/jarrad/pypsa-africa/.snakemake/scripts/tmpoctxlch_.add_electricity.py", line 722, in <module>
    attach_hydro(n, costs, ppl)
  File "/home/jarrad/pypsa-africa/.snakemake/scripts/tmpoctxlch_.add_electricity.py", line 450, in attach_hydro
    assert missing_c.empty, (
AssertionError: 'resources/profile_hydro.nc' is missing inflow time-series for at least one bus: 1084, 1164, 1438, 1637, 177, 1849, 186, 1865, 1911, 1927, 2182, 2358, 2371, 2462, 2464, 268, 731, 777, 887

[pz-max]

Hi Jarad,
We comment attach_hydro out now which solves this issue.
While run of river is stable we need to fix some hydro reservoir issues.

Does it work now with the the updated repo?

[jarry7]

Hi,

Yes, I commented that out earlier today to see if all would run and it seems to have worked.

Thanks for the follow-up @pz-max!