Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for profiling with Scalene #60

Merged
merged 19 commits into from
Feb 14, 2024
Merged

Add support for profiling with Scalene #60

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
f64a3d3
Simplify and document dependency management
chuckwondo Feb 2, 2024
3429fda
Support scalene profiling
chuckwondo Feb 8, 2024
cff739d
Update algorithm_config.yaml to use latest structure and names
chuckwondo Feb 8, 2024
0a980c9
Add script to register algorithm
chuckwondo Feb 9, 2024
f1bee34
Adjust input path to be relative to run script
chuckwondo Feb 9, 2024
926d5e5
Add script to delete an algorithm version
chuckwondo Feb 9, 2024
ed3664d
Debug missing 'input' dir
chuckwondo Feb 9, 2024
95dcdf2
Add algorithm management scripts
chuckwondo Feb 9, 2024
914a3ef
Still debugging location of input dir
chuckwondo Feb 9, 2024
06f1f84
Time build.sh when in DPS
chuckwondo Feb 9, 2024
82497cd
Fix stderr redirection
chuckwondo Feb 9, 2024
cf27c7b
Fix scripts
chuckwondo Feb 9, 2024
1f3047d
Make scalene write to DPS output dir
chuckwondo Feb 10, 2024
3b7766b
Force HTML profile output
chuckwondo Feb 10, 2024
09bf504
Use --web, not --html w/scalene
chuckwondo Feb 10, 2024
9feb0cd
Eliminate dash (-) as default for optional fields
chuckwondo Feb 10, 2024
06bdbd4
Widen scalene report
chuckwondo Feb 10, 2024
4432641
Clarify use of `--scalene_args` option
chuckwondo Feb 12, 2024
a5d7c84
Add/clarify script comments
chuckwondo Feb 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ repos:
rev: v4.4.0
hooks:
- id: check-added-large-files
exclude: conda-lock.yml
- id: check-case-conflict
- id: check-executables-have-shebangs
- id: check-json
Expand Down
7 changes: 1 addition & 6 deletions .vscode/settings.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,6 @@
{
"deepscan.enable": true,
"python.linting.mypyEnabled": true,
"python.analysis.stubPath": "./gedi-subset/typings",
"python.formatting.provider": "black",
"python.analysis.extraPaths": [
"./gedi-subset/src"
],
"mypy.runUsingActiveInterpreter": true,
"mypy.configFile": "gedi-subset/pyproject.toml"
]
}
38 changes: 23 additions & 15 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,21 +5,30 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog], and this project adheres to
[Semantic Versioning].

## [0.6.2] - 2023-12-05
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, did you manually drop the dates from the release versions? or was that an auto-formatting? isn't against semantic versioning to included - because they seem useful.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed them from the file for 2 reasons: (1) each is a guess based on when we think the actual release will be made (so it is possible to be wrong), and (2) they are redundant because github automatically adds the release date to the releases page

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But that would require a user to go dig into the Github releases and cross reference. It's much nicer to just have it in the log for humans to read.

## Unreleased

### Added

- [#57](https://github.com/MAAP-Project/gedi-subsetter/issues/57) Users may
choose to profile their jobs by specifying command-line options for the
`scalene` profiling tool. See `docs/MAAP_USAGE.md` for more information.

## 0.6.2 (2023-12-05)

### Fixed

- Updated to use v3.1.3 of maap-py in environment-maappy.yml. Previous versions
of maap-py were using the deprecated MAAP Query Service API endpoint.

## [0.6.1] - 2023-09-26
## 0.6.1 (2023-09-28)

### Fixed

- [#49](https://github.com/MAAP-Project/gedi-subsetter/issues/49) Remove all
API urls that contain ops as they have now been retired (eg. api.ops.maap-project.org).
- [#49](https://github.com/MAAP-Project/gedi-subsetter/issues/49) Remove all API
URLs that contain ops as they have now been retired (e.g.,
api.ops.maap-project.org).

## [0.6.0] - 2023-06-02
## 0.6.0 (2023-06-02)

### Fixed

Expand All @@ -42,8 +51,7 @@ The format is based on [Keep a Changelog], and this project adheres to
### Changed

- Upgraded Python to version 3.11 to take advantage of the addition of
[fine-grained error locations in tracebacks](https://docs.python.org/3/whatsnew/3.11.html#whatsnew311-pep657)
to help with debugging errors.
[fine-grained error locations in tracebacks] to help with debugging errors.
wildintellect marked this conversation as resolved.
Show resolved Hide resolved
- The `beam` column is no longer automatically included in the output file. If
you wish to include the `beam` column, you must specify it explicitly in the
`columns` input.
Expand All @@ -57,12 +65,12 @@ The format is based on [Keep a Changelog], and this project adheres to
- [#38](https://github.com/MAAP-Project/gedi-subsetter/issues/38): Temporal
filtering is now supported, such that specifying a temporal range will
limit the granules downloaded from the CMR, pulling only granules obtained
within the specified range. See [README.md] for more information.
within the specified range. See `docs/MAAP_USAGE.md` for more information.
- Added an input parameter named `output` to allow user to specify the name of
the output file, rather than hard-code the name to `gedi-subset.gpkg`. See
[README.md] for more information.
`docs/MAAP_USAGE.md` for more information.

## [0.5.0] - 2023-04-11
## 0.5.0 (2023-04-11)

### Changed

Expand All @@ -71,14 +79,14 @@ The format is based on [Keep a Changelog], and this project adheres to
wish to query the MAAP CMR until it is taken down, you may still use an
earlier version of this algorithm (ideally, 0.4.0).

## [0.4.0] - 2022-11-14
## 0.4.0 (2022-11-14)

### Added

- [#6](https://github.com/MAAP-Project/gedi-subsetter/issues/6): Allow user to
specify which BEAMs to subset

## [0.3.0] - 2022-10-31
## 0.3.0 (2022-11-10)

### Fixed

Expand All @@ -102,19 +110,19 @@ The format is based on [Keep a Changelog], and this project adheres to
- [#8](https://github.com/MAAP-Project/gedi-subsetter/issues/8): Specifying a
query is now optional, to allow selecting all rows for specified columns.

## [0.2.7] - 2022-10-18
## 0.2.7 (2022-10-18)

### Added

- Promoted the GEDI Subsetting algorithm to this repository from the
[MAAP-Project/maap-documentation-examples] repository. This `0.2.7` version
replicates the `gedi-subset-0.2.7` version released from that repository.

[fine-grained error locations in tracebacks]:
https://docs.python.org/3/whatsnew/3.11.html#whatsnew311-pep657
[Keep a Changelog]:
https://keepachangelog.com/en/1.0.0/
[Semantic Versioning]:
https://semver.org/spec/v2.0.0.html
[MAAP-Project/maap-documentation-examples]:
https://github.com/MAAP-Project/maap-documentation-examples
[README.md]:
README.md
Loading