Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add corrections #33

Merged
merged 34 commits into from
Sep 7, 2023
Merged
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
81e1683
Merge pull request #10 from askap-vast/add-crop
ddobie Jul 6, 2023
3e6dc1d
Pulled in Andrew's code for corrections and re-wrote the entire corre…
AkashA98 Jul 12, 2023
ce61320
Cleaned up minor naming issues with variables
AkashA98 Jul 12, 2023
270ec3e
Removed redundant code
AkashA98 Jul 12, 2023
66919e1
Fixed quantities with units; component files matching made easy
AkashA98 Jul 14, 2023
842267d
Start of reorg
hansenjiang Aug 6, 2023
fcd07cd
created scripts directory
hansenjiang Aug 6, 2023
b417e34
created docker directory
hansenjiang Aug 6, 2023
f696681
scripts directory
hansenjiang Aug 6, 2023
a906540
added READMEs to explain directories
hansenjiang Aug 6, 2023
852ca0e
moved documentation instructions to docs
hansenjiang Aug 6, 2023
d0fb958
separating logic - cleanup module
hansenjiang Aug 9, 2023
66d50f9
moved logic from cli calls into relevant modules
hansenjiang Aug 9, 2023
07ae31c
Corrected the wrong path for the catalog files
AkashA98 Aug 10, 2023
660245f
Merge pull request #34 from askap-vast/dev-reorg
mubdi Aug 10, 2023
b2fccaf
Re-organized code so that this can be passed to cropping, added docst…
AkashA98 Aug 14, 2023
c3c020d
Fixed typos
AkashA98 Aug 14, 2023
cd14713
New log message
AkashA98 Aug 14, 2023
a4d316d
Pulled in Andrew's code for corrections and re-wrote the entire corre…
AkashA98 Jul 12, 2023
77e49b1
Cleaned up minor naming issues with variables
AkashA98 Jul 12, 2023
efaf369
Fixed quantities with units; component files matching made easy
AkashA98 Jul 14, 2023
7e1be37
Corrected the wrong path for the catalog files
AkashA98 Aug 10, 2023
1d3bd39
Re-organized code so that this can be passed to cropping, added docst…
AkashA98 Aug 14, 2023
d4477a8
Fixed typos
AkashA98 Aug 14, 2023
513d2c3
New log message
AkashA98 Aug 14, 2023
bb00cc5
Merge branch 'add_corrections' of github.com:askap-vast/vast-post-pro…
AkashA98 Aug 14, 2023
005f45f
Deleted older corrections cli file
AkashA98 Aug 14, 2023
27395f2
Make new directories only when write_output=True
AkashA98 Aug 15, 2023
ef8698e
Updated the filtering function for catalogs
AkashA98 Aug 16, 2023
b495dba
Deal with all epochs or single epoch the same way
AkashA98 Aug 16, 2023
3450e47
User decides whether to skip entire epoch or a single file
AkashA98 Aug 17, 2023
903b04b
changed variables to function arguments for filtering sources
AkashA98 Aug 21, 2023
ffeae45
Tested it on a couple of epoch, changed code to use f-strings
AkashA98 Aug 21, 2023
f3a5ae8
Added catalog filtering parameters as CLI arguments.
AkashA98 Aug 28, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Deal with all epochs or single epoch the same way
  • Loading branch information
AkashA98 committed Aug 16, 2023
commit b495dba62a523c245eb4297cf6dacfded8ab6f3a
66 changes: 34 additions & 32 deletions vast_post_processing/corrections.py
Original file line number Diff line number Diff line change
@@ -570,39 +570,41 @@ def correct_files(
logger.remove()
logger.add(sys.stderr, level="INFO")

# read corrections
image_path_glob_list: list[Generator[Path, None, None]] = []
components_path_glob_list: list[Generator[Path, None, None]] = []

# Read all the epochs
if epoch is None or len(epoch) == 0:
image_path_glob_list.append(
vast_tile_data_root.glob("STOKESI_IMAGES/epoch_*/*.fits")
)
components_path_glob_list.append(
vast_tile_data_root.glob("STOKESI_SELAVY/epoch_*/*.xml")
)
epoch_dirs = list(vast_tile_data_root.glob("STOKESI_IMAGES/epoch_*"))
else:
for n in epoch:
image_path_glob_list.append(
vast_tile_data_root.glob(f"STOKESI_IMAGES/epoch_{n}/*.fits")
epoch_dirs = []
epoch_dirs = [
vast_tile_data_root / "STOKESI_IMAGES" / f"epoch_{e}" for e in epoch
]

logger.info(
f"Corrections requested of these epochs: {[i.name for i in epoch_dirs]}"
)

# Work on individual epochs
for e in epoch_dirs:
# read fits/xml files
image_path_glob_list: list[Generator[Path, None, None]] = []
components_path_glob_list: list[Generator[Path, None, None]] = []

image_path_glob_list.append(e.glob("*.fits"))
components_path_glob_list.append(e.glob("*.xml"))

# get corrections for every image and the correct it
for image_path in chain.from_iterable(image_path_glob_list):
correct_field(
image_path=image_path,
vast_corrections_root=vast_corrections_root,
radius=radius,
condon=condon,
psf_ref=psf_ref,
psf=psf,
write_output=write_output,
outdir=outdir,
overwrite=overwrite,
)
components_path_glob_list.append(
vast_tile_data_root.glob(f"STOKESI_SELAVY/epoch_{n}/*.xml")
logger.info(
f"Successfully corrected the images and catalogs for {image_path.as_posix()}"
)

# get corrections for an image and the correct it
for image_path in chain.from_iterable(image_path_glob_list):
correct_field(
image_path=image_path,
vast_corrections_root=vast_corrections_root,
radius=radius,
condon=condon,
psf_ref=psf_ref,
psf=psf,
write_output=write_output,
outdir=outdir,
overwrite=overwrite,
)
logger.info(
f"Successfully corrected the images and catalogs for {image_path.as_posix()}"
)