Making snakemake workflows modular

[SilasK]

Hey @alienzj and @franciscozorrilla

I contact you because you are both developers of outstanding metagenome pipelines implemented in snakemake.

I read somewhere a quote that went like this:

A bioinformatics tool is not helpful until it's integrated into another pipeline
I think there is something true about it, and I fear that my pipeline is too large to integrate into other pipelines reasonably. I don't know how you feel about metaEM and metaAPI.

I have been thinking for some time about making metagenome-atlas more modulable. Snakemake provides now all the necessary framework for that, however, as I see there are still some points to be clarified e.g. how to pass config arguments and file specification between workflows.

I propose a collaboration

1. to create a github organisation for metagenome-snakemake-workflows
2. Define the interface between workflows

• how we would specify the input/output files of a workflow (Best-paractice of cross-workflow specification of files snakemake-workflows/docs#20)
• How to configure different pathways
• ...

3. Adapt/create modules for (metagenomics) analysis that can then be re-combined in our existing pipline or a new one.

What do you tink? Would you have time to contribute to this?

[SilasK]

Other developers interested are happy to join.

[alienzj]

Hi, @SilasK
Sure, I am very excited to contribute to the modular metagenomics pipeline.

[franciscozorrilla]

Hey Silas & Jie,
I think this is a great initiative that would benefit the both the community of end-users and developers in the metagenomics space 🧬 💻 Would be happy to contribute! Probably would be good to also get suggestions/feeback from Johannes Köster.