Skip to content

Commit

Permalink
Generate README from docs and code via GH actions
Browse files Browse the repository at this point in the history
  • Loading branch information
thomasmfish authored and Thomas Fish committed Sep 9, 2024
1 parent 394c6f0 commit 615aae3
Show file tree
Hide file tree
Showing 8 changed files with 237 additions and 131 deletions.
53 changes: 53 additions & 0 deletions .github/workflows/README from docs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: Generate README

on:
pull_request:
paths:
- "docs/**/*"
workflow_run:
workflows:
- "generate_help_docs"

jobs:
make-README:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v4
- name: Fill template
working-directory: "docs/"
timeout-minutes: 0.1
run: |
sub_doc_parts_from() {
template="$1"
directory="$2"
for file in "$directory"/*; do
if [ -f "$file" ]; then
placeholder="{{ $(echo "${file^^}" | tr ./ _) }}"
echo "Saving $file to $placeholder with expanded environment variables" >&2
# Export file contents to text following a pattern in template.md, e.g.:
# section/test_file.md would replace "{{ SECTION_TEST_FILE_MD }}""
template="${template//"$placeholder"/"$(<"$file")"}"
fi
done
echo "$template"
}
# Fills from docs/sections sections, then docs/help (note "working-directory")
sub_doc_parts_from "$(sub_doc_parts_from "$(<template.md)" "sections")" "help">../README.md
- name: Setup git
run: |
git config --local user.name "github-actions[bot]"
git config --local user.email "github-actions[bot]@users.noreply.github.com"
git remote set-url origin https://${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}
- name: Push changes (if any)
run: |
git pull
if git update-index --refresh && git diff-index --quiet HEAD --; then
git add README.md
git commit -m "Automated update of README.md from docs"
branch_name="${GITHUB_HEAD_REF:-${GITHUB_REF#refs/heads/}}"
git push origin HEAD:$branch_name
fi
47 changes: 47 additions & 0 deletions .github/workflows/generate_help_docs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: Generate README

on:
pull_request:
paths:
- "src/sim_recon/cli/parsing/*"
- "src/sim_recon/settings/formatting.py"

jobs:
generate-help-docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
- name: Display Python version
run: python -c "import sys; print(sys.version)"
- name: Copy template
run: cp -f docs/template.md README.md
- name: Install (without dependencies)
run: |
python -m pip install --upgrade pip
python -m pip install . --no-deps
- name: Ensure README exists
run: |
if [ ! -f "README.md" ]; then
touch README.md
fi
- name: Get help strings from package
run: |
python -m sim_recon.cli.parsing.otf '--help' >docs/help/sim_otf.txt
python -m sim_recon.cli.parsing.recon '--help' >docs/help/sim_recon.txt
python -m sim_recon.cli.parsing.otf_view '--help' >docs/help/otf_view.txt
- name: Setup git
run: |
git config --local user.name "github-actions[bot]"
git config --local user.email "github-actions[bot]@users.noreply.github.com"
git remote set-url origin https://${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}
- name: Push changes (if any)
run: |
if git update-index --refresh && git diff-index --quiet HEAD --; then
git add docs/help/*
git commit -m "Automated update of help docs"
branch_name="${GITHUB_HEAD_REF:-${GITHUB_REF#refs/heads/}}"
git push origin HEAD:$branch_name
fi
20 changes: 20 additions & 0 deletions docs/help/otf_view.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
usage: otf-view [-h] [--show] [--show-only] [-o OUTPUT_DIRECTORY] [-v]
[--no-progress]
otf_paths [otf_paths ...]

Create OTF views

positional arguments:
otf_paths OTF file paths

options:
-h, --help show this help message and exit
--show Display the plots while running
--show-only Show plots without saving
-o OUTPUT_DIRECTORY, --output-directory OUTPUT_DIRECTORY
Save to this directory if saving plots, otherwise each
plot will be saved with its input file
-v, --verbose Show more logging
--no-progress turn off progress bars (only has an effect if tqdm is
installed)
sdfsdf
55 changes: 55 additions & 0 deletions docs/help/sim_otf.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
usage: sim-otf [-c CONFIG_PATH] [-p PSF_PATHS [PSF_PATHS ...]]
[-o OUTPUT_DIRECTORY] [--overwrite] [--no-cleanup]
[--shape XY_SHAPE XY_SHAPE] [-v] [--no-progress]
[--nphases NPHASES] [--ls LS] [--na NA] [--nimm NIMM]
[--background BACKGROUND] [--beaddiam BEADDIAM] [--angle ANGLE]
[--nocompen NOCOMPEN] [--fixorigin FIXORIGIN FIXORIGIN]
[--leavekz LEAVEKZ LEAVEKZ LEAVEKZ] [-h]

SIM PSFs to OTFs

options:
-c CONFIG_PATH, --config-path CONFIG_PATH
Path to the root config that specifies the paths to
the OTFs and the other configs
-p PSF_PATHS [PSF_PATHS ...], --psf PSF_PATHS [PSF_PATHS ...]
Paths to PSF files to be reconstructed (multiple paths
can be given)
-o OUTPUT_DIRECTORY, --output-directory OUTPUT_DIRECTORY
If specified, the output directory that the OTFs will
be saved in, otherwise each OTF will be saved in the
same directory as its PSF
--overwrite If specified, files will be overwritten if they
already exist (unique filenames will be used
otherwise)
--no-cleanup If specified, files created during the OTF creation
process will not be cleaned up
--shape XY_SHAPE XY_SHAPE
Takes 2 integers (X Y), specifying the shape to crop
PSFs to before converting (powers of 2 are fastest)
-v, --verbose Show more logging
--no-progress turn off progress bars (only has an effect if tqdm is
installed)
-h, --help show this help message and exit

Overrides:
Arguments that override configured values. Defaults stated are only used
if no value is given or configured.

--nphases NPHASES Number of pattern phases per SIM direction
--ls LS Line spacing of SIM pattern in microns
--na NA Detection objective's numerical aperture
--nimm NIMM Refractive index of immersion medium
--background BACKGROUND
Camera readout background
--beaddiam BEADDIAM The diameter of the bead in microns, by default 0.12
--angle ANGLE The k0 vector angle with which the PSF is taken, by
default 0
--nocompen NOCOMPEN Do not perform bead size compensation, default False
(do perform)
--fixorigin FIXORIGIN FIXORIGIN
The starting and end pixel for interpolation along kr
axis, by default (2, 9)
--leavekz LEAVEKZ LEAVEKZ LEAVEKZ
Pixels to be retained on kz axis, by default (0, 0, 0)
sad
134 changes: 3 additions & 131 deletions README.md → docs/help/sim_recon.txt
Original file line number Diff line number Diff line change
@@ -1,105 +1,3 @@
# PySIMRecon

Easy to use wrapper for pyCUDAsirecon, allowing the use of DV (DeltaVision) files without IVE/Priism (UCSF library with MRC/DV support).

This is built for easy use from the command line or as part of an auto-processing pipeline, allowing parameters to be pre-configured or defined at runtime.

### Goal

To reconstruct the SIM data and create OTFs from PSFs from the cryoSIM at B24, Diamond from the .dv files created by Cockpit. Requirements are:

- Easy to use
- Open source
- Reproducable results
- Can be used as part of an automatic processing pipeline

### Current state

Under development

### Configuration

Calls to `sim-otf` and `sim-recon` can both take a `-c`/`--config` argument. This should be similar to the `config.ini` file in the configs directory, which specifies the defaults config, any per-channel configs (some values should be set on a per-channel basis), and the locations of OTFs for each channel.
The channels are specified based on the emission wavelength in nanometres (must be an integer).

##### Example:

For a channel with an emission wavelength of 525nm:

- In the `[configs]` section, the channel config can be specified as `525=/path/to/configs/525.cfg`
- In the `[otfs]` section, the channel OTF file can be specified as `525=/path/to/otfs/525_otf.tiff`

A directory can be specified within each of the `[configs]` and `[otfs]` sections. If given, files can also be defined relative to that directory, e.g. if `directory=/path/to`, then `525=/path/to/otfs/525_otf.tiff` could simply be `525=otfs/525_otf.tiff`.
Config settings will be overriden by higher-priority sources.

##### Order of setting priority:

1. Command line arguments (override all)
2. Per-channel configs (override defaults for the appropriate channel only)
3. Defaults (override any cudasirecon defaults)

### CLI

##### sim-otf

```
usage: sim-otf [-c CONFIG_PATH] [-p PSF_PATHS [PSF_PATHS ...]]
[-o OUTPUT_DIRECTORY] [--overwrite] [--no-cleanup]
[--shape XY_SHAPE XY_SHAPE] [-v] [--no-progress]
[--nphases NPHASES] [--ls LS] [--na NA] [--nimm NIMM]
[--background BACKGROUND] [--beaddiam BEADDIAM] [--angle ANGLE]
[--nocompen NOCOMPEN] [--fixorigin FIXORIGIN FIXORIGIN]
[--leavekz LEAVEKZ LEAVEKZ LEAVEKZ] [-h]
options:
-c CONFIG_PATH, --config-path CONFIG_PATH
Path to the root config that specifies the paths to
the OTFs and the other configs
-p PSF_PATHS [PSF_PATHS ...], --psf PSF_PATHS [PSF_PATHS ...]
Paths to PSF files to be reconstructed (multiple paths
can be given)
-o OUTPUT_DIRECTORY, --output-directory OUTPUT_DIRECTORY
If specified, the output directory that the OTFs will
be saved in, otherwise each OTF will be saved in the
same directory as its PSF
--overwrite If specified, files will be overwritten if they
already exist (unique filenames will be used
otherwise)
--no-cleanup If specified, files created during the OTF creation
process will not be cleaned up
--shape XY_SHAPE XY_SHAPE
Takes 2 integers (X Y), specifying the shape to crop
PSFs to before converting (powers of 2 are fastest)
-v, --verbose Show more logging
--no-progress turn off progress bars (only has an effect if tqdm is
installed)
-h, --help show this help message and exit
Overrides:
Arguments that override configured values. Defaults stated are only used
if no value is given or configured.
--nphases NPHASES Number of pattern phases per SIM direction
--ls LS Line spacing of SIM pattern in microns
--na NA Detection objective's numerical aperture
--nimm NIMM Refractive index of immersion medium
--background BACKGROUND
Camera readout background
--beaddiam BEADDIAM The diameter of the bead in microns, by default 0.12
--angle ANGLE The k0 vector angle with which the PSF is taken, by
default 0
--nocompen NOCOMPEN Do not perform bead size compensation, default False
(do perform)
--fixorigin FIXORIGIN FIXORIGIN
The starting and end pixel for interpolation along kr
axis, by default (2, 9)
--leavekz LEAVEKZ LEAVEKZ LEAVEKZ
Pixels to be retained on kz axis, by default (0, 0, 0)
```

##### sim-recon

```
usage: sim-recon [-c CONFIG_PATH] [-d SIM_DATA_PATHS [SIM_DATA_PATHS ...]]
[-o OUTPUT_DIRECTORY] [--overwrite] [--no-cleanup]
[--keep-split] [--parallel] [-v] [--no-progress]
Expand All @@ -119,6 +17,8 @@ usage: sim-recon [-c CONFIG_PATH] [-d SIM_DATA_PATHS [SIM_DATA_PATHS ...]]
[--zres ZRES] [--zresPSF ZRESPSF] [--wavelength WAVELENGTH]
[-h]

Reconstruct SIM data

options:
-c CONFIG_PATH, --config-path CONFIG_PATH
Path to the root config that specifies the paths to
Expand Down Expand Up @@ -211,32 +111,4 @@ Overrides:
--zresPSF ZRESPSF Z pixel size (used in PSF TIFF files)
--wavelength WAVELENGTH
Emission wavelength in nm (only used for TIFF files)
```

##### otf-view

```
usage: otf-view [-h] [--show] [--show-only] [-o OUTPUT_DIRECTORY] [-v]
[--no-progress]
otf_paths [otf_paths ...]
positional arguments:
otf_paths OTF file paths
options:
-h, --help show this help message and exit
--show Display the plots while running
--show-only Show plots without saving
-o OUTPUT_DIRECTORY, --output-directory OUTPUT_DIRECTORY
Save to this directory if saving plots, otherwise each
plot will be saved with its input file
-v, --verbose Show more logging
--no-progress turn off progress bars (only has an effect if tqdm is
installed)
```


##### dv2tiff

Accepts a list of DV files to be converted to TIFFs (with some OME metadata).
safas
23 changes: 23 additions & 0 deletions docs/sections/command_line.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
### CLI

##### sim-otf

```
{{ HELP_SIM_OTF_TXT }}
```

##### sim-recon

```
{{ HELP_SIM_RECON_TXT }}
```

##### otf-view

```
{{ HELP_OTF_VIEW_TXT }}
```

##### dv2tiff

Accepts a list of DV files to be converted to TIFFs (with some OME metadata).
14 changes: 14 additions & 0 deletions docs/sections/configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
### Configuration
Calls to `sim-otf` and `sim-recon` can both take a `-c`/`--config` argument. This should be similar to the `config.ini` file in the configs directory, which specifies the defaults config, any per-channel configs (some values should be set on a per-channel basis), and the locations of OTFs for each channel.
The channels are specified based on the emission wavelength in nanometres (must be an integer).
##### Example:
For a channel with an emission wavelength of 525nm:
- In the `[configs]` section, the channel config can be specified as `525=/path/to/configs/525.cfg`
- In the `[otfs]` section, the channel OTF file can be specified as `525=/path/to/otfs/525_otf.tiff`
A directory can be specified within each of the `[configs]` and `[otfs]` sections. If given, files can also be defined relative to that directory, e.g. if `directory=/path/to`, then `525=/path/to/otfs/525_otf.tiff` could simply be `525=otfs/525_otf.tiff`.
Config settings will be overriden by higher-priority sources.

##### Order of setting priority:
1. Command line arguments (override all)
2. Per-channel configs (override defaults for the appropriate channel only)
3. Defaults (override any cudasirecon defaults)
22 changes: 22 additions & 0 deletions docs/template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# PySIMRecon

Easy to use wrapper for pyCUDAsirecon, allowing the use of DV (DeltaVision) files without IVE/Priism (UCSF library with MRC/DV support).

This is built for easy use from the command line or as part of an auto-processing pipeline, allowing parameters to be pre-configured or defined at runtime.

### Goal

To reconstruct the SIM data and create OTFs from PSFs from the cryoSIM at B24, Diamond from the .dv files created by Cockpit. Requirements are:

- Easy to use
- Open source
- Reproducable results
- Can be used as part of an automatic processing pipeline

### Current state

Under development

{{ SECTIONCONFIGURATION_MD }}

{{ SECTIONS_COMMAND_LINE_MD }}

0 comments on commit 615aae3

Please sign in to comment.