diff --git a/pipelines/LATISS/DRP.yaml b/pipelines/LATISS/DRP.yaml index 4c1dedb5..a9da7c12 100644 --- a/pipelines/LATISS/DRP.yaml +++ b/pipelines/LATISS/DRP.yaml @@ -164,6 +164,8 @@ subsets: - forcedPhotCoadd - writeObjectTable - transformObjectTable + - selectGoodSeeingVisits + - templateGen description: | Tract-level coaddition tasks. Allowed data query constraints: instrument, skymap, tract. @@ -212,9 +214,15 @@ subsets: step4: subset: + - forcedPhotCcd + - forcedPhotDiffim + - getTemplate + - subtractImages + - detectAndMeasureDiaSources + - transformDiaSourceCat + - writeForcedSourceTable - writeRecalibratedSourceTable - transformSourceTable - - consolidateSourceTable description: | Tasks that be run together, but only after the 'step1', 'step2', and 'step3' subsets. @@ -223,3 +231,42 @@ subsets: These detector-level tasks should not be run with 'tract' or 'patch' as part of the data ID expression if all reference catalogs are desired, and to ensure that consolidateSourceTable contains complete visits. + + step5: + subset: + - drpAssociation + - drpDiaCalculation + - forcedPhotCcdOnDiaObjects + - forcedPhotDiffOnDiaObjects + - transformForcedSourceTable + - consolidateForcedSourceTable + - consolidateAssocDiaSourceTable + - consolidateFullDiaObjectTable + - writeForcedSourceOnDiaObjectTable + - transformForcedSourceOnDiaObjectTable + - consolidateForcedSourceOnDiaObjectTable + description: | + Tasks that can be run together, but only after the 'step1', 'step2', + 'step3', and 'step4' subsets. + + This step includes patch-level aggregation Tasks. + 'consolidateForcedSourceTable' is a tract-level task that aggregates + patches and should be rerun if any of the patches fail. + + step6: + subset: + - consolidateDiaSourceTable + - consolidateSourceTable + description: | + Tasks that can be run together, but only after the 'step1', 'step2', + 'step3', 'step4', and 'step5' subsets. + + This step includes visit-level aggregation tasks. Running without tract + or patch in the data query is recommended, otherwise the outputs of + consolidateDiaSourceTable will not contain complete visits. + + This subset is separate from step4 to signal to operators to pause to + assess unexpected image differencing failures before these aggregation + steps. Otherwise, if run in the same quantum graph, aggregated data + products (e.g. diaObjects) would not be created if one or more of the + expected inputs is missing.