processes in setup block are not guaranteed to execute before processes in when block

[tnetx]

The nf-test test attempts to run a subworkflow with a setup block that deletes s3 buckets using process DELETE_S3.

nextflow_workflow {

    name "Test Workflow CHECK_MD5_DOWNLOAD_UNICHEM_WORKFLOW"
    script "subworkflows/download_unichem_workflow.nf"
    workflow "CHECK_MD5_DOWNLOAD_UNICHEM_WORKFLOW"

    test("Should run without failures and should download all files") {


        setup {
            // Delete the test files in the s3 input files dir
            // so that we can test the download and upload
            run("DELETE_S3") {
                script "../../modules/util.nf"
                process {
                    """
                    input[0] =  Channel.value('s3://mybucket')
                    """
                }
            }

        }

        when {
            params {
                unichem {
                    s3_input_files_dir = 's3://mybucket'
                }
            }

            workflow {
                
                """
                input[0] = Channel.value(params.unichem.s3_input_files_dir)
                ....
                """
            }
...
        }
...

We see from the logs that the setup process DELETE_S3 is executed after other processes in the workflow block.

> Nextflow 24.10.0 is available - Please consider updating your version to it
   > N E X T F L O W  ~  version 24.04.4
   > Launching `[...]/.nf-test-3fa44d841839dc0c22a4f838a5652433.nf` [focused_mercator] DSL2 - revision: dc6044fdc0
   > WARN: Task runtime metrics are not reported when using macOS without a container engine
   > [a0/d1aefb] Submitted process > DOWNLOAD_UNICHEM_WORKFLOW:DOWNLOAD_UNICHEM
   > [fe/a5a33a] Submitted process > DELETE_S3
   > [86/8e38fa] Submitted process > DOWNLOAD_UNICHEM_WORKFLOW:WRITE_UNICHEM_ONTOLOGY_READER_CONFIG
   > [fc/122115] Submitted process > DOWNLOAD_UNICHEM_WORKFLOW:WRITE_MD5_FILE
   > Waiting for file transfers to complete (1 files)
...

[lukfor]

Thanks, you are right. Since the test process is not using the output of the setup process, there is no dependency between them, and Nextflow could execute the setup process afterward. I will think about finding a way to fix this and guarantee the correct order 👍