Skip to content

Pipeline elements for 3x2pt analysis (shear-shear, shear-density, density-density) for DC2

License

Notifications You must be signed in to change notification settings

LSSTDESC/TXPipe

Folders and files

NameName
Last commit message
Last commit date

Latest commit

a03c421 · Aug 5, 2024
Jun 21, 2024
Feb 22, 2024
Jun 16, 2022
Aug 24, 2023
Jul 5, 2024
Oct 31, 2023
Dec 30, 2022
Aug 5, 2024
Mar 4, 2022
Nov 2, 2022
Jul 6, 2023
Jun 28, 2023
Oct 19, 2023
Jun 18, 2020
Feb 3, 2023
Apr 21, 2023
Feb 22, 2024
Mar 11, 2024
Nov 3, 2023

Repository files navigation

TXPipe

TXPipe is the DESC 3x2pt Pipeline implementation. It measures lensing and clustering 2pt quantities in real and Fourier space and performs associated tests.

It is written in Python, and requires version 3.6 or above.

Installing

Download TXPipe like this:

git clone --recurse-submodules https://github.com/LSSTDESC/TXPipe

and then install dependencies for it with conda like this on a laptop/desktop (M1 macs are not yet supported, sorry!):

cd TXPipe
./bin/install.sh

You can now set up your environment using:

source ./conda/bin/activate

which you should run each time you start a new terminal, from the TXPipe directory.

For using TXPipe on NERSC, you can use the Shifter system, which creates a container with all the requirements pre-built. See the ReadtheDocs page (https://txpipe.readthedocs.io/en/latest) under 'Running TXPipe at NERSC.'

Running

Download some test data:

curl -O https://portal.nersc.gov/cfs/lsst/txpipe/data/example.tar.gz
tar -zxvf example.tar.gz

Then run a pipeline on that data:

ceci examples/metadetect/pipeline.yml

You can find the outputs in the directory data/example/outputs_metadetect

Running the example notebook on jupyter at NERSC

Follow these steps to run the jupyter notebook at nersc.

  1. In a terminal on NERSC, run these commands to prepare and download code and data
# Set up the TXPipe kernel
/global/cfs/cdirs/lsst/groups/WL/users/zuntz/setup-txpipe-jupyter.sh

# Get TXPipe
cd $SCRATCH
git clone --recurse-submodules https://github.com/LSSTDESC/TXPipe
cd TXPipe
# While this is still in a branch, you also have to do this:
git checkout example-notebook

# Get data
curl -O https://portal.nersc.gov/cfs/lsst/txpipe/data/example.tar.gz
tar -zxvf example.tar.gz
  1. Go to https://jupyter.nersc.gov, log in, and select "Perlmutter shared node"

  2. Navigate to this notebook in your cloned directory's notebooks subdirectory (via the $PSCRATCH link) and open it.

  3. Click the button on the top right of the notebook that usually says something "NERSC Python" and select the "TXPipe" kernel in the pop-up.

  4. You can now run the notebook

Learning more

See the ReadTheDocs page for much more documentation.

Permissions

Email or Slack Joe Zuntz to be added to the development team.

Continuous Integration

Github actions is set up to run unit tests several pipelines whenever commits are pushed to an open pull request. We need to keep these pipelines up to date: https://github.com/LSSTDESC/TXPipe/actions