Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add corrections #33

Merged
merged 34 commits into from
Sep 7, 2023
Merged
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
81e1683
Merge pull request #10 from askap-vast/add-crop
ddobie Jul 6, 2023
3e6dc1d
Pulled in Andrew's code for corrections and re-wrote the entire corre…
AkashA98 Jul 12, 2023
ce61320
Cleaned up minor naming issues with variables
AkashA98 Jul 12, 2023
270ec3e
Removed redundant code
AkashA98 Jul 12, 2023
66919e1
Fixed quantities with units; component files matching made easy
AkashA98 Jul 14, 2023
842267d
Start of reorg
hansenjiang Aug 6, 2023
fcd07cd
created scripts directory
hansenjiang Aug 6, 2023
b417e34
created docker directory
hansenjiang Aug 6, 2023
f696681
scripts directory
hansenjiang Aug 6, 2023
a906540
added READMEs to explain directories
hansenjiang Aug 6, 2023
852ca0e
moved documentation instructions to docs
hansenjiang Aug 6, 2023
d0fb958
separating logic - cleanup module
hansenjiang Aug 9, 2023
66d50f9
moved logic from cli calls into relevant modules
hansenjiang Aug 9, 2023
07ae31c
Corrected the wrong path for the catalog files
AkashA98 Aug 10, 2023
660245f
Merge pull request #34 from askap-vast/dev-reorg
mubdi Aug 10, 2023
b2fccaf
Re-organized code so that this can be passed to cropping, added docst…
AkashA98 Aug 14, 2023
c3c020d
Fixed typos
AkashA98 Aug 14, 2023
cd14713
New log message
AkashA98 Aug 14, 2023
a4d316d
Pulled in Andrew's code for corrections and re-wrote the entire corre…
AkashA98 Jul 12, 2023
77e49b1
Cleaned up minor naming issues with variables
AkashA98 Jul 12, 2023
efaf369
Fixed quantities with units; component files matching made easy
AkashA98 Jul 14, 2023
7e1be37
Corrected the wrong path for the catalog files
AkashA98 Aug 10, 2023
1d3bd39
Re-organized code so that this can be passed to cropping, added docst…
AkashA98 Aug 14, 2023
d4477a8
Fixed typos
AkashA98 Aug 14, 2023
513d2c3
New log message
AkashA98 Aug 14, 2023
bb00cc5
Merge branch 'add_corrections' of github.com:askap-vast/vast-post-pro…
AkashA98 Aug 14, 2023
005f45f
Deleted older corrections cli file
AkashA98 Aug 14, 2023
27395f2
Make new directories only when write_output=True
AkashA98 Aug 15, 2023
ef8698e
Updated the filtering function for catalogs
AkashA98 Aug 16, 2023
b495dba
Deal with all epochs or single epoch the same way
AkashA98 Aug 16, 2023
3450e47
User decides whether to skip entire epoch or a single file
AkashA98 Aug 17, 2023
903b04b
changed variables to function arguments for filtering sources
AkashA98 Aug 21, 2023
ffeae45
Tested it on a couple of epoch, changed code to use f-strings
AkashA98 Aug 21, 2023
f3a5ae8
Added catalog filtering parameters as CLI arguments.
AkashA98 Aug 28, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
changed variables to function arguments for filtering sources
AkashA98 committed Aug 21, 2023
commit 903b04b4c6bc2f14497e56597324e142a624132b
40 changes: 34 additions & 6 deletions vast_post_processing/catalogs.py
Original file line number Diff line number Diff line change
@@ -182,10 +182,32 @@ def __init__(
path: Path,
psf: Optional[Tuple[float, float]] = None,
input_format: str = "selavy",
condon: bool = False,
apply_flux_limit: bool = True,
condon: bool = True,
flux_limit: float = 0,
snr_limit: float = 20,
nneighbor: float = 1,
apply_flux_limit: bool = True,
select_point_sources: bool = True,
):
"""Defines a catalog class to read the component files

Args:
path (Path): path to the component file (selavy/aegean supported right now)
psf (Optional[Tuple[float, float]], optional): The major and minor axis dimensions
in arcsec. Defaults to None. Used to calculate condon errors
input_format (str, optional): are the component files selavy or aegean generated?.
Defaults to "selavy".
condon (bool, optional): Apply condon corrections. Defaults to True.
flux_limit (float, optional): Flux limit to select sources (sources with peak flux
> this will be selected). Defaults to 0.
snr_limit (float, optional): SNR limit to select sources (sources with SNR > this
will be selected). Defaults to 20.
nneighbor (float, optional): Distance to nearest neighbor (in arcmin). Sources with
neighbors < this will be removed. Defaults to 1.
apply_flux_limit (bool, optional): Flag to decide to apply flux limit. Defaults to True.
select_point_sources (bool, optional): Flag to decide to select point sources.
Defaults to True
"""
self.path: Path
self.table: QTable
self.input_format: Optional[str]
@@ -202,6 +224,9 @@ def __init__(
self.input_format = input_format
self.flux_flag = apply_flux_limit
self.flux_lim = flux_limit
self.snr_lim = snr_limit
self.sep_lim = nneighbor # In arcmin
self.point_sources = select_point_sources

# Read the catalog
self._read_catalog()
@@ -284,14 +309,17 @@ def _filter_sources(self):
psf_mask = (self.table["maj_axis"] > 0) & (self.table["min_axis"] > 0)

# point source flag
ps_metric = self.table["flux_peak"] / self.table["flux_int"]
ps_mask = ps_metric < 1.5
if self.point_sources:
ps_metric = self.table["flux_peak"] / self.table["flux_int"]
ps_mask = ps_metric < 1.5
else:
ps_mask = np.ones(len(self.table)).astype(bool)

# Add snr flag
snr_mask = self.table["flux_peak"] / self.table["rms_image"] > 20
snr_mask = self.table["flux_peak"] / self.table["rms_image"] > self.snr_lim

# Select distant sources
dist_mask = self.table["nn_separation"].to(u.arcsec).value > 60
dist_mask = self.table["nn_separation"].to(u.arcsec).value > 60 * self.sep_lim

mask = (flux_mask) & (psf_mask) & (ps_mask) & (snr_mask) & (dist_mask)
self.table = self.table[mask]
4 changes: 2 additions & 2 deletions vast_post_processing/corrections.py
Original file line number Diff line number Diff line change
@@ -493,11 +493,11 @@ def correct_field(

# construct output path to store corrections for each epoch
corr_dir = outdir / "corr_db"
if not corr_dir.isdir():
if not corr_dir.is_dir():
corr_dir.mkdir()
epoch_corr_dir = corr_dir / epoch_dir

if not epoch_corr_dir.isdir():
if not epoch_corr_dir.is_dir():
epoch_corr_dir.mkdir()

# check for auxiliary files