Optional snakemake workflow

[pz-max]

Hi @FabianHofmann, while adding a new optional snakemake rule I had a feeling the workflow could be better integrated. But I am not sure if there is a better way (imported python function vs rule?). Since we might face similar situations in future, maybe you have an immediate idea of a better solution. We practically have the case that:

option == True executes
rule A
rule B
rule C

option == False executes
rule A
rule C

We created rule B to have a separate workflow which creates in our case a more "meshed network". We anticipate that this rule might contain alternative methods in future. What are your thoughts on it? :)

https://github.com/pypsa-meets-africa/pypsa-africa/blob/37afb4333092b8ddbda6c4cafd61e2c9edc64dd2/Snakefile#L257-L316

[FabianHofmann]

mmh, hard to say what is the best option. An alternative approach could be to carry the augmentation flag within the clusters wildcard and add the augmentation code to cluster_network.py. So for example the clusters wildcard could be something like "120aug" and the cluster code follow the idea of

n_clusters, augmented = re.match(r"([0-9]+)($|[a-zA-Z]+)", wildcards.clusters).groups()
n_clusters = int(n_clusters)

if augmented.endswith('m')
    aggregate_carriers = pd.Index(n.generators.carrier.unique()).difference(renewable_carriers)
    augmented = augmented[:-1]
else:
    aggregate_carriers = None # All

....normal main code...

if augmented == "aug":
    ... run augmentation code...
elif augmented:
    logger.warning(f"Wildcard 'clusters' contains non-supported string {augmented}. Ignoring it.")

EDIT: changed the code a bit to catch the m letter case.

Would that be an option?

[pz-max]

Good to hear that the current implementation is not completely wrong. Your suggestion sounds also great. Especially since you can then model different combinations with one config. We probably wait until this wildcard is required and will implement it on demand. Thanks Fabian 🙂

…

________________________________ From: Fabian Hofmann ***@***.***> Sent: 17 December 2021 15:30 To: pypsa-meets-africa/pypsa-africa ***@***.***> Cc: PARZEN Maximilian ***@***.***>; Author ***@***.***> Subject: Re: [pypsa-meets-africa/pypsa-africa] Optional snakemake workflow (Discussion #184) This email was sent to you by someone outside the University. You should only click on links or attachments if you are certain that the email is genuine and the content is safe. mmh, hard to say what is the best option. An alternative approach could be to carry the augmentation flag within the clusters wildcard and add the augmentation code to cluster_network.py. So for example the clusters wildcard could be something like "120aug" and the cluster code follow the idea of n_clusters, augmented = re.match(r"([0-9]+)($|[a-zA-Z]+)", wildcards.clusters).groups() n_clusters = int(n_clusters) ....normal main code... if augmented == "aug": ... run augmentation code... elif augmented: logger.warning(f"Wildcard 'clusters' contains non-supported string {augmented}. Ignoring it.") Would that be an option? — Reply to this email directly, view it on GitHub<#184 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AOYZENKXSO3VKULVACXMR5TURNJTDANCNFSM5KHPVGMA>. You are receiving this because you authored the thread.Message ID: ***@***.***> The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.