Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-47210: Add analysis tasks to the injected RC2 pipelines. #174

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

DM-47210: Add analysis tasks to the injected RC2 pipelines. #174

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 44 additions & 4 deletions SConstruct
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# -*- python -*-
import os

from lsst.sconsUtils import scripts
from lsst.sconsUtils.state import env
from lsst.sconsUtils.utils import libraryLoaderEnvironment
Expand All @@ -18,7 +19,25 @@ scripts.BasicSConstruct(
)
PKG_ROOT = env.ProductDir("drp_pipe")

# make deepCoadd injection pipelines for rc2_subset and RC2
additional_pipeline_RC2 = os.path.join(
PKG_ROOT,
"pipelines",
"HSC",
"DRP-RC2-post-injected.yaml",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we append -brightstars to this? Or bright-stars? Whatever we want to call this (@leeskelvin ?).

)

additional_pipeline_LSSTComCam = os.path.join(
PKG_ROOT,
"pipelines",
"LSSTComCam",
"DRP-post-injected.yaml",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, add -brightstars.

)

subset_name = "injected_coadd_analysis"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe injected_bright_stars_coadd_analysis?

subset_description = "Analysis tasks for object_table level injected catalogs."


# Make deepCoadd injection pipelines for rc2_subset, RC2, and LSSTComCam.
rc2_subset_injected_deepCoadd = env.Command(
target=os.path.join(
PKG_ROOT, "pipelines", "HSC", "DRP-RC2_subset+injected_deepCoadd.yaml"
Expand All @@ -27,7 +46,8 @@ rc2_subset_injected_deepCoadd = env.Command(
action=" ".join(
[
libraryLoaderEnvironment(),
"make_injection_pipeline -t deepCoadd -r $SOURCE -f $TARGET --overwrite",
f"make_injection_pipeline -t deepCoadd -r $SOURCE -f $TARGET -a {additional_pipeline_RC2} "
f"-s {subset_name} -d '{subset_description}' --overwrite",
]
),
)
Expand All @@ -39,8 +59,28 @@ RC2_injected_deepCoadd = env.Command(
action=" ".join(
[
libraryLoaderEnvironment(),
"make_injection_pipeline -t deepCoadd -r $SOURCE -f $TARGET --overwrite",
f"make_injection_pipeline -t deepCoadd -r $SOURCE -f $TARGET -a {additional_pipeline_RC2} "
f"-s {subset_name} -d '{subset_description}' --overwrite",
]
),
)
LSSTComCam_injected_deepCoadd = env.Command(
target=os.path.join(
PKG_ROOT, "pipelines", "LSSTComCam", "DRP+injected_deepCoadd.yaml"
),
source=os.path.join(PKG_ROOT, "pipelines", "LSSTComCam", "DRP.yaml"),
action=" ".join(
[
libraryLoaderEnvironment(),
f"make_injection_pipeline -t deepCoadd -r $SOURCE -f $TARGET -a {additional_pipeline_LSSTComCam} "
f"-s {subset_name} -d '{subset_description}' --overwrite",
]
),
)
Default([rc2_subset_injected_deepCoadd, RC2_injected_deepCoadd])
Default(
[
rc2_subset_injected_deepCoadd,
RC2_injected_deepCoadd,
LSSTComCam_injected_deepCoadd,
]
)
46 changes: 46 additions & 0 deletions pipelines/HSC/DRP-RC2-post-injected.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
description: Match tract-level injected_deepCoadd_catalog datasets to injected_objectTable_tract datasets.
instrument: lsst.obs.subaru.HyperSuprimeCam
imports:
- location: $SOURCE_INJECTION_DIR/pipelines/match_injected_tract_catalog.yaml
- location: $ANALYSIS_TOOLS_DIR/pipelines/coaddDiffMatchedInjectedCore.yaml
tasks:
compare_object_to_injected:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compare_object_to_injected_bright_stars?

class: lsst.pipe.tasks.diff_matched_tract_catalog.DiffMatchedTractCatalogTask
config:
connections.name_input_cat_ref: injected_deepCoadd_catalog_tract
connections.name_input_cat_target: injected_objectTable_tract
column_matched_prefix_ref: ref_

# TODO: Remove as part of DM-44139
columns_ref_mag_to_nJy: {g_mag: "g_flux", r_mag: "r_flux", i_mag: "i_flux", z_mag: "z_flux", y_mag: "y_flux"}
columns_ref_copy: [
"injected_id", "source_type",
"g_mag", "r_mag", "i_mag", "z_mag", "y_mag",
"injection_flag", "injected_isPatchInner",
]
columns_target_coord_err: ["coord_raErr", "coord_decErr"]
columns_target_copy: [
"g_psfFlux", "r_psfFlux", "i_psfFlux", "z_psfFlux", "y_psfFlux",
"g_psfFluxErr", "r_psfFluxErr", "i_psfFluxErr", "z_psfFluxErr", "y_psfFluxErr",
"g_psfFlux_flag", "r_psfFlux_flag", "i_psfFlux_flag", "z_psfFlux_flag", "y_psfFlux_flag",
"patch", "detect_isDeblendedSource", "detect_isPatchInner", "detect_isPrimary", "merge_peak_sky",
"refExtendedness", "refSizeExtendedness",
]
diff_matched_analysis:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's time to give this a better name too - analyze_matched_injected_bright_stars?

class: lsst.analysis.tools.tasks.DiffMatchedAnalysisTask
config:
connections.inputName: matched_injected_deepCoadd_catalog_tract_injected_objectTable_tract
connections.outputName: matched_injected_deepCoadd_catalog_tract_injected_objectTable_tract

bands: ["g", "r", "i", "z", "y"]
python: |
from lsst.analysis.tools.atools.diffMatched import reconfigure_diff_matched_defaults

reconfigure_diff_matched_defaults(
config=config,
context="injection",
key_flux_meas="psf",
bands_color = {"g", "r", "i", "z", "y"},
use_any=False,
use_galaxies=False,
)
46 changes: 46 additions & 0 deletions pipelines/LSSTComCam/DRP-post-injected.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
description: Match tract-level injected_deepCoadd_catalog datasets to injected_objectTable_tract datasets.
instrument: lsst.obs.lsst.LsstComCam
imports:
- location: $SOURCE_INJECTION_DIR/pipelines/match_injected_tract_catalog.yaml
- location: $ANALYSIS_TOOLS_DIR/pipelines/coaddDiffMatchedInjectedCore.yaml
tasks:
compare_object_to_injected:
class: lsst.pipe.tasks.diff_matched_tract_catalog.DiffMatchedTractCatalogTask
config:
connections.name_input_cat_ref: injected_deepCoadd_catalog_tract
connections.name_input_cat_target: injected_objectTable_tract
column_matched_prefix_ref: ref_

# TODO: Remove as part of DM-44139
columns_ref_mag_to_nJy: {u_mag: "u_flux", g_mag: "g_flux", r_mag: "r_flux", i_mag: "i_flux", z_mag: "z_flux", y_mag: "y_flux"}
columns_ref_copy: [
"injected_id", "source_type",
"u_mag", "g_mag", "r_mag", "i_mag", "z_mag", "y_mag",
"injection_flag", "injected_isPatchInner",
]
columns_target_coord_err: ["coord_raErr", "coord_decErr"]
columns_target_copy: [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I forgot objectId here.

"u_psfFlux", "g_psfFlux", "r_psfFlux", "i_psfFlux", "z_psfFlux", "y_psfFlux",
"u_psfFluxErr", "g_psfFluxErr", "r_psfFluxErr", "i_psfFluxErr", "z_psfFluxErr", "y_psfFluxErr",
"u_psfFlux_flag", "g_psfFlux_flag", "r_psfFlux_flag", "i_psfFlux_flag", "z_psfFlux_flag", "y_psfFlux_flag",
"patch", "detect_isDeblendedSource", "detect_isPatchInner", "detect_isPrimary", "merge_peak_sky",
"refExtendedness", "refSizeExtendedness",
]
diff_matched_analysis:
class: lsst.analysis.tools.tasks.DiffMatchedAnalysisTask
config:
connections.inputName: matched_injected_deepCoadd_catalog_tract_injected_objectTable_tract
connections.outputName: matched_injected_deepCoadd_catalog_tract_injected_objectTable_tract

bands: ["u", "g", "r", "i", "z", "y"]
python: |
from lsst.analysis.tools.atools.diffMatched import reconfigure_diff_matched_defaults

reconfigure_diff_matched_defaults(
config=config,
context="injection",
key_flux_meas="psf",
bands_color = {"u", "g", "r", "i", "z", "y"},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be okay to omit since ugrizy is the default.

use_any=False,
use_galaxies=False,
)

Check failure on line 46 in pipelines/LSSTComCam/DRP-post-injected.yaml

View workflow job for this annotation

GitHub Actions / call-workflow / yamllint 

46:10 [new-line-at-end-of-file] no new line character at the end of file
Loading