From 7333c7c32c2ba42068dda98f3a5ac85ceabdf56a Mon Sep 17 00:00:00 2001 From: aawdeh Date: Thu, 30 Nov 2023 10:22:21 -0500 Subject: [PATCH 1/2] Updated description for Fastqprocessing as SplitFastq (#1134) --- website/docs/Pipelines/ATAC/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/Pipelines/ATAC/README.md b/website/docs/Pipelines/ATAC/README.md index e740188e73..10b8c8e4dc 100644 --- a/website/docs/Pipelines/ATAC/README.md +++ b/website/docs/Pipelines/ATAC/README.md @@ -71,7 +71,7 @@ To see specific tool parameters, select the task WDL link in the table; then vie | Task name and WDL link | Tool | Software | Description | | --- | --- | --- | ------------------------------------ | -| [FastqProcessing as SplitFastq](https://github.com/broadinstitute/warp/blob/master/tasks/skylab/FastqProcessing.wdl) | fastqprocess | custom | Dynamically selects the correct barcode orientation, corrects cell barcodes and splits FASTQs into 30 GB FASTQs that are grouped by cell barcode with each read having the corrected (CB) and raw barcode (CR) in the read name. | +| [FastqProcessing as SplitFastq](https://github.com/broadinstitute/warp/blob/master/tasks/skylab/FastqProcessing.wdl) | fastqprocess | custom | Dynamically selects the correct barcode orientation, corrects cell barcodes and splits FASTQs into smaller FASTQs. The number of files output depends on either the bam_size parameter, which determines the size of the output FASTQs produced, or the num_output_files parameter, which determines the number of FASTQS that should be output. The smaller FASTQs are grouped by cell barcode with each read having the corrected (CB) and raw barcode (CR) in the read name. | | [TrimAdapters](https://github.com/broadinstitute/warp/blob/master/pipelines/skylab/multiome/atac.wdl) | Cutadapt v4.4 | cutadapt | Trims adaptor sequences. | | [BWAPairedEndAlignment](https://github.com/broadinstitute/warp/blob/master/pipelines/skylab/multiome/atac.wdl) | bwa-mem2 | mem | Aligns reads from each set of partitioned FASTQ files to the genome and outputs a BAM with ATAC barcodes in the CB:Z tag. | | [Merge.MergeSortBamFiles as MergeBam](https://github.com/broadinstitute/warp/blob/master/tasks/skylab/MergeSortBam.wdl) | MergeSamFiles | Picard | Merges each BAM into a final aligned BAM with corrected cell barcodes in the CB tag. | From 41bb4b306c49c1fcd13dcd85d3a2fb162e849ac8 Mon Sep 17 00:00:00 2001 From: Nikelle Petrillo <38223776+nikellepetrillo@users.noreply.github.com> Date: Thu, 30 Nov 2023 11:49:58 -0500 Subject: [PATCH 2/2] Np revert both vbid dockers (#1136) * add summary task * revert vbid dockers * changelogs * changelogs * changelogs * Update pipelines/broad/dna_seq/germline/single_sample/ugwgs/UltimaGenomicsWholeGenomeGermline.changelog.md * Update pipelines/broad/dna_seq/germline/single_sample/ugwgs/UltimaGenomicsWholeGenomeGermline.wdl * Update pipelines/broad/internal/dna_seq/germline/single_sample/UltimaGenomics/BroadInternalUltimaGenomics.changelog.md Co-authored-by: ekiernan <55763654+ekiernan@users.noreply.github.com> --------- Co-authored-by: ekiernan <55763654+ekiernan@users.noreply.github.com> --- .../exome/ExomeGermlineSingleSample.changelog.md | 3 ++- .../ugwgs/UltimaGenomicsWholeGenomeGermline.changelog.md | 1 + .../wgs/WholeGenomeGermlineSingleSample.changelog.md | 3 ++- .../germline/variant_calling/VariantCalling.changelog.md | 3 ++- .../ugwgs/UltimaGenomicsWholeGenomeCramOnly.changelog.md | 3 ++- .../UltimaGenomics/BroadInternalUltimaGenomics.changelog.md | 3 ++- .../broad/reprocessing/exome/ExomeReprocessing.changelog.md | 3 ++- .../external/exome/ExternalExomeReprocessing.changelog.md | 3 ++- .../wgs/ExternalWholeGenomeReprocessing.changelog.md | 3 ++- .../reprocessing/wgs/WholeGenomeReprocessing.changelog.md | 3 ++- tasks/broad/BamProcessing.wdl | 6 +----- tasks/broad/UltimaGenomicsWholeGenomeGermlineTasks.wdl | 5 ++--- 12 files changed, 22 insertions(+), 17 deletions(-) diff --git a/pipelines/broad/dna_seq/germline/single_sample/exome/ExomeGermlineSingleSample.changelog.md b/pipelines/broad/dna_seq/germline/single_sample/exome/ExomeGermlineSingleSample.changelog.md index a2bf1daf8d..e9e01f966a 100644 --- a/pipelines/broad/dna_seq/germline/single_sample/exome/ExomeGermlineSingleSample.changelog.md +++ b/pipelines/broad/dna_seq/germline/single_sample/exome/ExomeGermlineSingleSample.changelog.md @@ -1,7 +1,8 @@ # 3.1.14 -2023-11-21 (Date of Last Commit) +2023-11-29 (Date of Last Commit) * Fixed bug in ReblockGVCFs; this does not affect this pipeline. +* Reverted the VerifyBamID docker image back to the 3.1.10 ExomeGermlineSingleSample pipeline version # 3.1.13 2023-10-10 (Date of Last Commit) diff --git a/pipelines/broad/dna_seq/germline/single_sample/ugwgs/UltimaGenomicsWholeGenomeGermline.changelog.md b/pipelines/broad/dna_seq/germline/single_sample/ugwgs/UltimaGenomicsWholeGenomeGermline.changelog.md index 800760cba3..c6e9d375b3 100644 --- a/pipelines/broad/dna_seq/germline/single_sample/ugwgs/UltimaGenomicsWholeGenomeGermline.changelog.md +++ b/pipelines/broad/dna_seq/germline/single_sample/ugwgs/UltimaGenomicsWholeGenomeGermline.changelog.md @@ -2,6 +2,7 @@ 2023-11-21 (Date of Last Commit) * Fixed bug in ReblockGVCFs; this does not affect this pipeline. +* Reverted the VerifyBamID docker image back to the 1.0.9 UltimaGenomicsWholeGenomeGermline pipeline version # 1.0.10 2023-09-18 (Date of Last Commit) diff --git a/pipelines/broad/dna_seq/germline/single_sample/wgs/WholeGenomeGermlineSingleSample.changelog.md b/pipelines/broad/dna_seq/germline/single_sample/wgs/WholeGenomeGermlineSingleSample.changelog.md index 3ad8276d1d..767f71ed06 100644 --- a/pipelines/broad/dna_seq/germline/single_sample/wgs/WholeGenomeGermlineSingleSample.changelog.md +++ b/pipelines/broad/dna_seq/germline/single_sample/wgs/WholeGenomeGermlineSingleSample.changelog.md @@ -1,7 +1,8 @@ # 3.1.15 -2023-11-21 (Date of Last Commit) +2023-11-29 (Date of Last Commit) * Fixed bug in ReblockGVCFs; this does not affect this pipeline. +* Reverted the VerifyBamID docker image back to the 3.1.11 WholeGenomeGermlineSingleSample pipeline version # 3.1.14 2023-10-10 (Date of Last Commit) diff --git a/pipelines/broad/dna_seq/germline/variant_calling/VariantCalling.changelog.md b/pipelines/broad/dna_seq/germline/variant_calling/VariantCalling.changelog.md index 8371336f38..f798a6f474 100644 --- a/pipelines/broad/dna_seq/germline/variant_calling/VariantCalling.changelog.md +++ b/pipelines/broad/dna_seq/germline/variant_calling/VariantCalling.changelog.md @@ -1,7 +1,8 @@ # 2.1.13 -2023-11-21 (Date of Last Commit) +2023-11-29 (Date of Last Commit) * Fixed bug in ReblockGVCFs; this does not affect this pipeline. +* Reverted the VerifyBamID docker image back to the 2.1.10 VariantCalling pipeline version # 2.1.12 2023-09-18 (Date of Last Commit) diff --git a/pipelines/broad/dna_seq/somatic/single_sample/ugwgs/UltimaGenomicsWholeGenomeCramOnly.changelog.md b/pipelines/broad/dna_seq/somatic/single_sample/ugwgs/UltimaGenomicsWholeGenomeCramOnly.changelog.md index c05d9daac9..b6e8fbfaf2 100644 --- a/pipelines/broad/dna_seq/somatic/single_sample/ugwgs/UltimaGenomicsWholeGenomeCramOnly.changelog.md +++ b/pipelines/broad/dna_seq/somatic/single_sample/ugwgs/UltimaGenomicsWholeGenomeCramOnly.changelog.md @@ -1,7 +1,8 @@ # 1.0.11 -2023-11-21 (Date of Last Commit) +2023-11-29 (Date of Last Commit) * Fixed bug in ReblockGVCFs; this does not affect this pipeline. +* Reverted the VerifyBamID docker image back to the 1.0.8 UltimaGenomicsWholeGenomeCramOnly pipeline version # 1.0.10 2023-09-18 (Date of Last Commit) diff --git a/pipelines/broad/internal/dna_seq/germline/single_sample/UltimaGenomics/BroadInternalUltimaGenomics.changelog.md b/pipelines/broad/internal/dna_seq/germline/single_sample/UltimaGenomics/BroadInternalUltimaGenomics.changelog.md index c9da7399bf..513b8a6bf0 100644 --- a/pipelines/broad/internal/dna_seq/germline/single_sample/UltimaGenomics/BroadInternalUltimaGenomics.changelog.md +++ b/pipelines/broad/internal/dna_seq/germline/single_sample/UltimaGenomics/BroadInternalUltimaGenomics.changelog.md @@ -1,7 +1,8 @@ # 1.0.12 -2023-11-21 (Date of Last Commit) +2023-11-29 (Date of Last Commit) * Fixed bug in ReblockGVCFs; this does not affect this pipeline. +* Reverted the VerifyBamID docker image back to the 1.0.9 BroadInternalUltimaGenomics pipeline version # 1.0.11 2023-09-18 (Date of Last Commit) diff --git a/pipelines/broad/reprocessing/exome/ExomeReprocessing.changelog.md b/pipelines/broad/reprocessing/exome/ExomeReprocessing.changelog.md index 8868f5b5f0..4990ee2230 100644 --- a/pipelines/broad/reprocessing/exome/ExomeReprocessing.changelog.md +++ b/pipelines/broad/reprocessing/exome/ExomeReprocessing.changelog.md @@ -1,7 +1,8 @@ # 3.1.14 -2023-11-21 (Date of Last Commit) +2023-11-29 (Date of Last Commit) * Fixed bug in ReblockGVCFs; this does not affect this pipeline. +* Reverted the VerifyBamID docker image back to the 3.1.10 ExomeReprocessing pipeline version # 3.1.13 2023-10-10 (Date of Last Commit) diff --git a/pipelines/broad/reprocessing/external/exome/ExternalExomeReprocessing.changelog.md b/pipelines/broad/reprocessing/external/exome/ExternalExomeReprocessing.changelog.md index 1b36962f35..65fedc52df 100644 --- a/pipelines/broad/reprocessing/external/exome/ExternalExomeReprocessing.changelog.md +++ b/pipelines/broad/reprocessing/external/exome/ExternalExomeReprocessing.changelog.md @@ -1,7 +1,8 @@ # 3.1.16 -2023-11-21 (Date of Last Commit) +2023-11-29 (Date of Last Commit) * Fixed bug in ReblockGVCFs; this does not affect this pipeline. +* Reverted the VerifyBamID docker image back to the 3.1.12 ExternalExomeReprocessing pipeline version # 3.1.15 2023-10-10 (Date of Last Commit) diff --git a/pipelines/broad/reprocessing/external/wgs/ExternalWholeGenomeReprocessing.changelog.md b/pipelines/broad/reprocessing/external/wgs/ExternalWholeGenomeReprocessing.changelog.md index bb34b614de..bd3a7a02b1 100644 --- a/pipelines/broad/reprocessing/external/wgs/ExternalWholeGenomeReprocessing.changelog.md +++ b/pipelines/broad/reprocessing/external/wgs/ExternalWholeGenomeReprocessing.changelog.md @@ -1,7 +1,8 @@ # 2.1.16 -2023-11-21 (Date of Last Commit) +2023-11-29 (Date of Last Commit) * Fixed bug in ReblockGVCFs; this does not affect this pipeline. +* Reverted the VerifyBamID docker image back to the 2.1.12 ExternalWholeGenomeReprocessing pipeline version # 2.1.15 2023-10-10 (Date of Last Commit) diff --git a/pipelines/broad/reprocessing/wgs/WholeGenomeReprocessing.changelog.md b/pipelines/broad/reprocessing/wgs/WholeGenomeReprocessing.changelog.md index 3f96fea061..9cc58d254d 100644 --- a/pipelines/broad/reprocessing/wgs/WholeGenomeReprocessing.changelog.md +++ b/pipelines/broad/reprocessing/wgs/WholeGenomeReprocessing.changelog.md @@ -1,7 +1,8 @@ # 3.1.15 -2023-11-21 (Date of Last Commit) +2023-11-29 (Date of Last Commit) * Fixed bug in ReblockGVCFs; this does not affect this pipeline. +* Reverted the VerifyBamID docker image back to the 3.1.11 WholeGenomeReprocessing pipeline version # 3.1.14 2023-10-10 (Date of Last Commit) diff --git a/tasks/broad/BamProcessing.wdl b/tasks/broad/BamProcessing.wdl index 3c18e97920..e5ae21039a 100644 --- a/tasks/broad/BamProcessing.wdl +++ b/tasks/broad/BamProcessing.wdl @@ -417,9 +417,7 @@ task CheckContamination { Float contamination_underestimation_factor Boolean disable_sanity_check = false - # this docker image was built using the commit hash c1cba76e979904eb69c31520a0d7f5be63c72253 (v1.0.6)from the VerifyBamID repo - # https://github.com/Griffan/VerifyBamID/commit/c1cba76e979904eb69c31520a0d7f5be63c72253 - String docker = "us.gcr.io/broad-gotc-prod/verify-bam-id:1.0.1-c1cba76e979904eb69c31520a0d7f5be63c72253-1692099985" + String docker = "us.gcr.io/broad-gotc-prod/verify-bam-id:1.0.1-c1cba76e979904eb69c31520a0d7f5be63c72253-1639071840" } Int disk_size = ceil(size(input_bam, "GiB") + size(ref_fasta, "GiB")) + 30 @@ -468,8 +466,6 @@ task CheckContamination { preemptible: preemptible_tries memory: "7.5 GiB" disks: "local-disk " + disk_size + " HDD" - # this docker image was built using the commit hash c1cba76e979904eb69c31520a0d7f5be63c72253 from the VerifyBamID repo - # https://github.com/Griffan/VerifyBamID/commit/c1cba76e979904eb69c31520a0d7f5be63c72253 docker: docker cpu: 2 } diff --git a/tasks/broad/UltimaGenomicsWholeGenomeGermlineTasks.wdl b/tasks/broad/UltimaGenomicsWholeGenomeGermlineTasks.wdl index 90625a7d4e..d1761ba041 100644 --- a/tasks/broad/UltimaGenomicsWholeGenomeGermlineTasks.wdl +++ b/tasks/broad/UltimaGenomicsWholeGenomeGermlineTasks.wdl @@ -390,9 +390,8 @@ task CheckContamination { References references String output_prefix - # this docker image was built using the commit hash f6cb51761861e57c43879aa262df5cf8e670cf7c (v2.0.1) from the VerifyBamID repo - # https://github.com/Griffan/VerifyBamID/commit/f6cb51761861e57c43879aa262df5cf8e670cf7c - String docker = "us.gcr.io/broad-gotc-prod/verify-bam-id:1.0.1-f6cb51761861e57c43879aa262df5cf8e670cf7c-1692216226" + + String docker = "us.gcr.io/broad-gotc-prod/verify-bam-id:f6cb51761861e57c43879aa262df5cf8e670cf7c-1606775309" Int disk_size_gb = ceil(if ceil((size(input_bam, "GB")) + (size(references.ref_fasta, "GB") + size(references.ref_fasta_index, "GB") +