Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-running CosmoDC2 #359

Closed
wants to merge 64 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
2f35072
fix Neff calculation in calib for parallel systems
joezuntz Nov 6, 2023
a379da3
path and config updates for cosmodc2
joezuntz Nov 7, 2023
d17b25a
generate quantiles in a new stage
joezuntz Nov 16, 2023
9450e31
changes to make CosmoDC2 run work
joezuntz Nov 17, 2023
4f84d77
trying to switch selector
joezuntz Nov 17, 2023
79e5311
fix TXRandomForestLensSelector options
joezuntz Nov 17, 2023
3380ec2
Merge master into branch
joezuntz Jul 23, 2024
368aa20
update 20 deg^2 pipeline file
joezuntz Jul 23, 2024
d092e12
Merge branch 'master' into cosmodc2-rerun
joezuntz Sep 3, 2024
1865ff5
update yaml files for cosmodc2
joezuntz Sep 3, 2024
a70b7f1
image updates
joezuntz Sep 3, 2024
fa33ca6
Merge branch 'cosmodc2-rerun' of https://github.com/LSSTDESC/TXPipe i…
joezuntz Sep 3, 2024
aaf3059
run ceci update here too
joezuntz Sep 4, 2024
5611869
Merge master
joezuntz Sep 4, 2024
28aceb3
Update TXSourceTrueNumberDensity
joezuntz Sep 4, 2024
b70abfa
WIP
joezuntz Sep 4, 2024
3bfca09
Fix zero sized patches
joezuntz Sep 4, 2024
ecc5593
Merge branch 'cosmodc2-rerun' of https://github.com/LSSTDESC/TXPipe i…
joezuntz Sep 4, 2024
6a7d44c
Make min size 2
joezuntz Sep 4, 2024
cd16874
config and code fixes
joezuntz Sep 4, 2024
81703c4
add theory to plots
joezuntz Sep 4, 2024
e45e4ea
a few more tweaks
joezuntz Sep 4, 2024
d0ad811
diagnostic quantile update
joezuntz Sep 4, 2024
eac9a82
make quantiles of every column
joezuntz Sep 4, 2024
b1b9d21
add nersc installer
joezuntz Sep 5, 2024
e69947c
wtf
joezuntz Sep 5, 2024
7b0fde8
oh that
joezuntz Sep 5, 2024
29eeae2
fix = ==
joezuntz Sep 5, 2024
ac54843
new installer might even work
joezuntz Sep 5, 2024
c12c2e1
make installer executable
joezuntz Sep 5, 2024
2a748bd
add include flag maybe
joezuntz Sep 5, 2024
876c63c
fix path
joezuntz Sep 5, 2024
7eae204
new version based on working one
joezuntz Sep 6, 2024
399266c
move to scratch install
joezuntz Sep 6, 2024
ce41924
working installer
joezuntz Sep 6, 2024
7b34aeb
working installer
joezuntz Sep 6, 2024
9b3f5df
20deg2 update
joezuntz Sep 6, 2024
2ed6c8c
more error handling in line fit
joezuntz Sep 6, 2024
9a1719c
work in progress
joezuntz Sep 10, 2024
f3c3a93
go back to older diagnostics file
joezuntz Sep 10, 2024
a92ea88
fix source-only
joezuntz Sep 10, 2024
aa26c65
fix lensfit
joezuntz Sep 10, 2024
d43958b
fix dry runs
joezuntz Sep 10, 2024
000e434
fix return from rank>0 metadetectcalculator
joezuntz Sep 10, 2024
4174931
Fix parallelization bug in plot_psf_shear
joezuntz Sep 11, 2024
f3aeb27
restore options in 20 deg^2 config
joezuntz Sep 11, 2024
8455a01
print lens counts in selector
joezuntz Sep 13, 2024
597e26b
print out more progress
joezuntz Sep 13, 2024
fa2dfa2
update cosmodc2 config files
joezuntz Sep 13, 2024
9c6bec8
add option to smooth n(z) in theory
joezuntz Sep 13, 2024
1a069da
fix error message in theory_model
joezuntz Sep 13, 2024
cc5b97f
WIP covariance with MPI
joezuntz Sep 16, 2024
4840214
Switch to MPI for basic covariance
joezuntz Sep 16, 2024
517855a
Switch to dask-based TXAuxiliarySourceMaps
joezuntz Sep 16, 2024
05b465f
fix parallel type
joezuntz Sep 16, 2024
ee1fab2
Fix weight mistake in new aux mapper
joezuntz Sep 18, 2024
03a5188
Make it easier to debug dask in future using TX_DASK_DEBUG
joezuntz Sep 18, 2024
1a1da93
replace lens mapper with much faster dask version
joezuntz Sep 18, 2024
6f6fdf3
save metadfata
joezuntz Sep 18, 2024
753feed
add dask_parallel flag
joezuntz Sep 18, 2024
54745ae
force all dask block sizes to be the same even if dtypes are different
joezuntz Sep 19, 2024
c31879d
fix typo
joezuntz Sep 19, 2024
3f16bcf
Merge branch 'master' into cosmodc2-rerun
joezuntz Sep 19, 2024
abee95c
remove wigner_transform new file
joezuntz Sep 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge master
joezuntz committed Sep 4, 2024
commit 56118698a5326e9d5a37ef597cb17f9fc3fc6e48
3 changes: 2 additions & 1 deletion examples/metacal/config.yml
Original file line number Diff line number Diff line change
@@ -303,7 +303,6 @@ PZRailSummarizeLens:
catalog_group: lens
tomography_name: lens
bands: ugrizy
model: None


PZRailSummarizeSource:
@@ -315,6 +314,8 @@ PZRailSummarizeSource:
tomography_name: source
catalog_group: shear
bands: riz
name: PZRailSummarizeSource

TXPhotozPlotLens:
name: TXPhotozPlotLens
TXPhotozPlotSource:
2 changes: 2 additions & 0 deletions examples/metacal/pipeline.yml
Original file line number Diff line number Diff line change
@@ -37,13 +37,15 @@ stages:
binned_catalog: binned_lens_catalog
model: lens_direct_calibration_model
photoz_stack: lens_photoz_stack
photoz_realizations: lens_photoz_realizations
- name: PZRailSummarizeSource # Run the DIR method on the lens sample to find n(z)
classname: PZRailSummarize
aliases:
tomography_catalog: shear_tomography_catalog
binned_catalog: binned_shear_catalog
model: source_direct_calibration_model
photoz_stack: shear_photoz_stack
photoz_realizations: source_photoz_realizations
- name: TXSourceSelectorMetacal # select and split objects into source bins
- name: NZDirInformerSource # Prepare the DIR method inputs for the source sample
classname: NZDirInformer
1 change: 0 additions & 1 deletion examples/metadetect/config.yml
Original file line number Diff line number Diff line change
@@ -312,7 +312,6 @@ PZRailSummarizeLens:
catalog_group: lens
tomography_name: lens
bands: ugrizy
model: None


PZRailSummarizeSource:
2 changes: 1 addition & 1 deletion examples/metadetect_source_only/config.yml
Original file line number Diff line number Diff line change
@@ -185,12 +185,12 @@ PZRailSummarizeSource:
zmin: 0.0
zmax: 3.0
nzbins: 50
nsamples: 100
mag_prefix: /shear/00/mag_
catalog_group: shear
tomography_name: source
bands: riz
name: PZRailSummarizeSource

FlowCreator:
n_samples: 1000000
seed: 5763248
You are viewing a condensed version of this merge commit. You can view the full changes here.