Releases: sdss/idlspec2d
Releases · sdss/idlspec2d
Kaalut
Taranushi
- Fixed coadded metadata
- fixed Healpix path (healpixgrp) columns to use integer healpix group values
- Added fits columns descriptions to fits headers
- Fixed SDSS_id bug in SDSSV plates
- added targeting flags
- fixed SDSSV plate (field) epoch coadds bug
- only coadd a single plugging
- streamlined running of full pipeline catchup runs
- added model column back to specfull and speclite files
- added observatory specific custom coadds
- added code to update targeting flags within the spAll files
Olkoth
- Added SimpleDust2023 preliminary dustmap
- Added dither flag to spplan files
- updates to spcalib to save additional information and help with calibration library tests
- Added check of hartmann door status
- Migrated parts of pipeline to python
- spplan
- readfibermap - as an initial step to the pipeline (updated readplugmap.pro for updated format)
- fieldmerge - builds on previous spAll file (does not start from scratch daily)
- fieldlist
- spSpec_reformat - converts spSpec files to spec files (and produces pngs)
- Added basic fits datamodel
- par files with columns
- python/idl codes to handle models
- updated sos to add a second SN2 calculation
- updated for sdsscore migration
- added codes for FPS epoch and custom (target level) coadds
- fix bug with combining resolution arrays
- Add additional qa plots and files (with debug flag)
- Updated LCO configuration for updated Flat lamps
- inclusion of SDSS_ID
Daoloth
- updated SOS for LCO
- Updated SOS controller to the systemd (systemctl) standard
- optimize SOS calibration matching
- clean old SOS warnings when rerunning an exposure
- clarification of Arc Lamp Status messages - removed photolog dependency
- Update idlutils dependency to v6.X.X (SDSSV version)
- updates to add LCO location
- Add an option to override flat/arc rejection - update main pipeline for LCO
- add observatory to final products
- fixed double counting of Exposure time
- add an option to run pipeline without SDSS-V database access
- add an option to use arc from another field
- update Arc lamp line file, opFibers, and gains for LCO
- migrate from bundled fiber setup to monolithic slit
- trim early LCO blue data where there is no flat lamp coverage - fix bad line labels in PNGS
- migrated uurundaily and uubatchpbs to python
- setup to run fieldmerge separately from daily run
- update sphdrfix for SDSS-V (in SDSSCORE)
- add sdR_hdrfix.py - Fix bug in spall fils where models were incorrect
- remove offset fibers and "bhm_gua*" targets from SN2
- Correct v0.5 magnitudes with RevisedMagnitude Table
- fix bugs in healpix columns
- fix bug in spField headers
- Add BOSS_log function for night logs
- Add GAIA_DR2 ids
Abhoth
- Updates to SOS and idlspec2d for LCO operation
- r1/b1 vs r2/b2
- pass LCO lat/long/alt option to pass to heliocentric/barycentric velocity correction
- determine LCO vs APO at plan level and pass that to the pipeline - Updated readplugmap & prerun_readplugmap
- Update to fibermap fits formats (added a summary extension with fibermap metadata)
- Merged FPS and plate fibermap datamodels
- modify to use parallax and propermotion from fps configuration files, otherwise get from database
- fixed bugs to properly read fibermap from fits file if available rather then recreating it
- clean plugmap
- add code to get survey name for plates - Automate selection of parameters/options for pipeline for plate vs fps data
- added wrapper for yanny_par (yanny_par_fc) to add fold_case for string matching
- added v_rad fits header keyword and modified helio_rv description to conform with the definition of heliocentric vs barycentric RVs.
- New keyword (v_rad) was chosen to be consistent between APOGEE, ASTRA, and BOSS - Exclude Nan mags (mag < -99) for SOS SN2 fits
- Changed sdss_spec_image error bar shading code to address shading errors
- Add columns to spall for
- catalogIDs from various catalog crossmatch versions
- delta_RA_LIST and delta_DEC_LIST
- SPEC2* (LCO) - Updated fieldlist
- fixed bugs to fill out fieldlist dat file
- add columns associated with b2/r2 (LCO) - Corrected condition meta data for spall for single exposures
- Updated spplan2d and spplan1d
- Automatically determine plate/legacy/fps data
- merged plate and fps spplan2d/1d models
- Add observatory header keyword
- Updated code to filter out engineering data and reference baddata.par to flag other bad data
- add options to exclude dithers - updated daily run scripts to enable LCO master and give option to have a 2nd daily run (if necessary)
- update run_plugmap_supplements
- corrected bug in get_RJCE_ext
- get field cadence information from designID rather then fieldid - updated run_PyXCSAO to use splog.py (python version of idl splog) to enable live logging
Ghroth
We implement a number of changes to fix bugs and small changes to both the main IDLspec2d pipeline and the SOS mountain pipeline
- Update spplan2d
- to utilize twilight flats (if no field-mjd matched flats are available)
- allow skip of commission fields
- allow test exposures with field = -999 to run with fieldid=0 - Addition of a prototype spAll-lite file
- Addition of spectrophotometric QA plots on a field-mjd basis
- Fixed a bug in Extinction values from dust3d model
- The addition of columns identifying utilized extinction source (EBV_TYPE)
- Updated SFD_EBV column name in spall (and other steps) to EBV - Corrected bug in loadSN2Value to allow updates of SN2 values
- Updated APO red CCD gain values to SDSS-V updated CCD
- modified sdssproc to read these from a new opGain.par file to clean up and allow for easier future updates - Added LCO arc lamp line file (lampHeNe.dat) and included an option to use in fitarcimage.pro
- modified SN2 and fieldquality color/values to be consistent
- both in fieldlist and spall
- scales SN2 of fields with < 3600s exposure to 3600s before comparing with reference values of SN2_b <10 and SN2_r<22 - Updated Fieldlist
- added fieldcadence (after retrieving from targetdb via run_plugmap_supplements) - Updated spAll
- RACAT, DECCAT, MJD_FINAL are reported as double
- additional decimal point included in RA/DEC_LIST
- sfd_ebv → ebv
- added ebv_type
- added cadence
- Implemented an option to skip the step to build specprimary and nspecobs columns (but still allow as post step)
- optimized the code to produce specprimary, nspecobs, etc.
- added FIBER_RA, FIBER_DEC, and specobjid (required for sas_mwm_healpix)
- specobjid is actually reproduced in this step (to ensure consistency
- copying them from the specreduce1d step outputs was producing inconsistent values due to data types) - Utilizing astrometry values from target_db instead of crossmatching to Gaia for retrieval
- modified read_sos.py to get parent confSummary files (for dithered fields) if it is not included in the fibermap fits file
- Cleaned up fibermap/spall columns
- replacing -999. with NANs - fixed bug with utilizing fibermask for calibrations, when more than 1 calibration frame of type is included in the plan
Bast
We implement a modification to the daily target level coadd algorithm. This modification splits the coadd into a two-stage process, with the first stage combining the blue and red camera (including outlier rejection), and the second stage combining the (already red+blue combined) exposures (without outlier rejection). This strategy was implemented to improve the coadds of exposures with significant flux differences seen in the SDSS-V FPS era. EXP_DISP_MED was added to SpAll to characterize the flux dispersion between the exposures. All exposures are included as extensions in the full versions of the spec files for custom post-pipeline coadds. Additional columns of lists of exposure level metadata are included in the SpAll file for reference. The reading of the confSummary files was replaced with the confSummaryF files to incorporate the more accurate fiber positions (as measured by the FVC loop). Additionally, several other bug fixes were incorporated into the main IDLspec2d and SOS pipelines.
Hypnos
We implement several bug fixes to the flux calibration and flat fielding of the reduced spectra. We also implement improvements to the final spectral metadata included in the spec-field-mjd-catalogid.fits file headers and the spAll extensions and file. These improvements included propagating some information on a per-target rather than per-field basis since we coadd on a target level. We switch from coadding via an RA-DEC match to a coadd based on catalogid. We also switched off the interpolated flux calibration vectors for all fields with less than 60 standards (in practice only leaving this on for BHM-RM fields).
Amaterasu
The initial version of the idlspec2d and SOS for the FPS.
- Implement the new confSummary files for FPS in place of the plate plugmaps.
- Optimize the readplugmap for multiple calls
- Implement single MJD coadd of all observations of a field
- Correct specobjid data types to produce correct output
- Implement gaia dust extinction maps for galactic latitudes <15
- Replaced reference to plate with fields
fieldID < 16000 are plates
16000 < fieldID < 100000 are FPS commissioning Fields
fieldID > 100000 are FPS Survey Fields
fieldID strings are zero-padded 6-character strings - Clarify output magnitudes are fiber2mags
