Merge pull request #98 from lsst/tickets/DM-39038

Add difference imaging steps to LATISS DRP.yaml

pipelines/LATISS/DRP.yaml

@@ -164,6 +164,8 @@ subsets:
       - forcedPhotCoadd
       - writeObjectTable
       - transformObjectTable
+      - selectGoodSeeingVisits
+      - templateGen
     description: |
       Tract-level coaddition tasks.
       Allowed data query constraints: instrument, skymap, tract.
@@ -212,9 +214,15 @@ subsets:
 
   step4:
     subset:
+      - forcedPhotCcd
+      - forcedPhotDiffim
+      - getTemplate
+      - subtractImages
+      - detectAndMeasureDiaSources
+      - transformDiaSourceCat
+      - writeForcedSourceTable
       - writeRecalibratedSourceTable
       - transformSourceTable
-      - consolidateSourceTable
     description: |
       Tasks that be run together, but only after the 'step1', 'step2', and
       'step3' subsets.
@@ -223,3 +231,42 @@ subsets:
       These detector-level tasks should not be run with 'tract' or 'patch' as
       part of the data ID expression if all reference catalogs are desired,
       and to ensure that consolidateSourceTable contains complete visits.
+
+  step5:
+    subset:
+      - drpAssociation
+      - drpDiaCalculation
+      - forcedPhotCcdOnDiaObjects
+      - forcedPhotDiffOnDiaObjects
+      - transformForcedSourceTable
+      - consolidateForcedSourceTable
+      - consolidateAssocDiaSourceTable
+      - consolidateFullDiaObjectTable
+      - writeForcedSourceOnDiaObjectTable
+      - transformForcedSourceOnDiaObjectTable
+      - consolidateForcedSourceOnDiaObjectTable
+    description: |
+      Tasks that can be run together, but only after the 'step1', 'step2',
+      'step3', and 'step4' subsets.
+
+      This step includes patch-level aggregation Tasks.
+      'consolidateForcedSourceTable' is a tract-level task that aggregates
+      patches and should be rerun if any of the patches fail.
+
+  step6:
+    subset:
+      - consolidateDiaSourceTable
+      - consolidateSourceTable
+    description: |
+      Tasks that can be run together, but only after the 'step1', 'step2',
+      'step3', 'step4', and 'step5' subsets.
+
+      This step includes visit-level aggregation tasks. Running without tract
+      or patch in the data query is recommended, otherwise the outputs of
+      consolidateDiaSourceTable will not contain complete visits.
+
+      This subset is separate from step4 to signal to operators to pause to
+      assess unexpected image differencing failures before these aggregation
+      steps. Otherwise, if run in the same quantum graph, aggregated data
+      products (e.g. diaObjects) would not be created if one or more of the
+      expected inputs is missing.