Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy docs and add performance benchmark #42

Open
wants to merge 18 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 0 additions & 37 deletions .github/workflows/benchmark.yml

This file was deleted.

2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ jobs:
- name: Install uv
uses: astral-sh/setup-uv@v3
with:
version: "0.4.22"
version: "0.5.*"
enable-cache: true

- name: Set up Python ${{ matrix.python-version }}
Expand Down
219 changes: 219 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,219 @@
name: Documentation
on:
push:
branches:
- develop
tags:
- 'v*'
pull_request:
types: [opened, synchronize, reopened, closed]

permissions:
contents: write
pull-requests: write

jobs:
# Job for versioned deployments (runs on develop branch and tags)
deploy-versions:
if: github.event_name == 'push' && (github.ref == 'refs/heads/develop' || startsWith(github.ref, 'refs/tags/v'))
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Important for mike to work with tags

- uses: astral-sh/setup-uv@v3
with:
version: "0.5.*"
enable-cache: true
- name: Sync
run: |
uv sync --locked
git restore uv.lock

- name: Configure Git
run: |
git config --local user.name "GitHub Actions"
git config --local user.email "[email protected]"

- name: Deploy docs
env:
EARTHDATA_USERNAME: ${{ secrets.earthdata_username }}
EARTHDATA_PASSWORD: ${{ secrets.earthdata_password }}
run: |
echo "machine urs.earthdata.nasa.gov\nlogin ${EARTHDATA_USERNAME}\npassword ${EARTHDATA_PASSWORD}" > ~/.netrc
if [[ $GITHUB_REF == refs/tags/* ]]; then
# For tags, deploy the tagged version
VERSION=${GITHUB_REF#refs/tags/v}
uv run mike deploy --push $VERSION

# If this is the latest version, update the 'latest' alias
LATEST_TAG=$(git tag --sort=-creatordate | head -n 1)
if [[ "v$VERSION" == "$LATEST_TAG" ]]; then
uv run mike alias --push $VERSION latest
uv run mike set-default --push latest
fi
else
# For develop branch, deploy to 'dev'
uv run mike deploy --push --update-aliases dev
fi

# Job for PR previews
preview:
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: astral-sh/setup-uv@v3
with:
version: "0.5.*"
enable-cache: true
- name: Sync
run: |
uv sync
git diff

# Only build and deploy if PR is opened/synchronized/reopened
- name: Build docs
if: github.event.action != 'closed'
env:
EARTHDATA_USERNAME: ${{ secrets.earthdata_username }}
EARTHDATA_PASSWORD: ${{ secrets.earthdata_password }}
run: |
echo "machine urs.earthdata.nasa.gov\nlogin ${EARTHDATA_USERNAME}\npassword ${EARTHDATA_PASSWORD}" > ~/.netrc
uv run mkdocs build --site-dir site/pr-${{ github.event.pull_request.number }}

- name: Deploy preview
id: deploy
if: github.event.action != 'closed'
run: |
git config --global user.name "GitHub Actions"
git config --global user.email "[email protected]"

# Checkout gh-pages branch
git restore .
git fetch origin gh-pages --depth=1
git checkout gh-pages

# Create preview directory if it doesn't exist
mkdir -p pr-previews

# Copy new preview
rm -rf pr-previews/pr-${{ github.event.pull_request.number }}
cp -r site/pr-${{ github.event.pull_request.number }} pr-previews/

# Check if there are actual changes in git
git add pr-previews
if git diff --staged --quiet; then
echo "No changes in documentation. Skipping deployment."
echo "has_changes=false" >> $GITHUB_OUTPUT
exit 0
fi

# If we get here, there are changes
git commit -m "Deploy preview for PR #${{ github.event.pull_request.number }}"
git push origin gh-pages
echo "has_changes=true" >> $GITHUB_OUTPUT

- name: Post/Update PR Comment
if: github.event.action != 'closed'
uses: actions/github-script@v6
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
console.log('Starting PR comment update...');
const has_changes = '${{ steps.deploy.outputs.has_changes }}' === 'true';
const preview_url = `https://${context.repo.owner}.github.io/${context.repo.repo}/pr-previews/pr-${context.payload.pull_request.number}/`;
const message = `📚 Documentation preview will be available at: ${preview_url}

Status: ${has_changes ? '✅ Preview is ready!' : '🔄 No changes in documentation since last update'}`;

const comments = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.payload.pull_request.number,
});

const docComment = comments.data.find(comment =>
comment.user.login === 'github-actions[bot]' &&
comment.body.includes('Documentation preview will be available at:')
);

if (docComment) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: docComment.id,
body: message
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.payload.pull_request.number,
body: message
});
}

- name: Cleanup preview
if: github.event.action == 'closed'
run: |
git config --global user.name "GitHub Actions"
git config --global user.email "[email protected]"

# Checkout gh-pages branch
git restore uv.lock
git fetch origin gh-pages --depth=1
git checkout gh-pages

# Remove the preview directory for this PR
rm -rf pr-previews/pr-${{ github.event.pull_request.number }}

# Commit and push if there are changes
git add pr-previews
git diff --staged --quiet || (git commit -m "Remove preview for PR #${{ github.event.pull_request.number }}" && git push origin gh-pages)

# Optional job to cleanup old PR previews
cleanup-old-previews:
if: github.event_name == 'push' && github.ref == 'refs/heads/develop'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: gh-pages

- name: Get closed PRs
id: closed-prs
uses: actions/github-script@v7
with:
script: |
const prs = await github.rest.pulls.list({
owner: context.repo.owner,
repo: context.repo.repo,
state: 'closed'
});
return prs.data.map(pr => pr.number);
result-encoding: string

- name: Cleanup old previews
run: |
git config --global user.name "GitHub Actions"
git config --global user.email "[email protected]"

# Get list of preview directories
cd pr-previews
for preview in pr-*; do
if [ -d "$preview" ]; then
PR_NUM=$(echo $preview | sed 's/pr-//')
if ! echo "${{ steps.closed-prs.outputs.result }}" | grep -q "$PR_NUM"; then
rm -rf "$preview"
echo "Removed old preview: $preview"
fi
fi
done

# Commit and push if there are changes
git add .
git diff --staged --quiet || (git commit -m "Cleanup old PR previews" && git push origin gh-pages)

1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -107,4 +107,3 @@ node_modules
cdk.context.json

notebooks/

2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
repos:
- repo: https://github.com/abravalheri/validate-pyproject
rev: v0.12.1
rev: v0.23
hooks:
- id: validate-pyproject

Expand Down
10 changes: 10 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,13 @@ Record the new responses and commit them to the repository.
```bash
uv run pytest -v -s --record-mode new_episodes
```

## Documentation

The documentation is generated using `mkdocs` and gets built and deployed to Github Pages when new tags are released and on pushes to the `develop` branch.

To preview the documentation in your browser you can run:

```bash
uv run mkdocs serve -o
```
9 changes: 4 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,20 +28,19 @@ An API for creating image tiles from CMR queries.

## Installation

To install from sources and run for development:
To install from sources and run for development, [install `uv`](https://docs.astral.sh/uv/getting-started/installation/) then:

```bash
git clone https://github.com/developmentseed/titiler-cmr.git
cd titiler-cmr

python -m pip install -U pip
python -m pip install uvicorn -e .[dev,test]
uv sync --all-extras
```

## Authentication for data read access

`titiler-cmr` can read data either over `HTTP` (external) or directly from `AWS S3` (direct) depending on the app configuration.
The behavior of the application is controlled by the S3 authentication settings in [`settings.py`](./titiler/cmr/settings.py), which you can set either with environment variables (`TITILER_CMR_S3_AUTH_ACCESS`, `TITILER_CMR_S3_AUTH_STRATEGY`) or in an environment file (`.env`).
The behavior of the application is controlled by the S3 authentication settings in `settings.py`, which you can set either with environment variables (`TITILER_CMR_S3_AUTH_ACCESS`, `TITILER_CMR_S3_AUTH_STRATEGY`) or in an environment file (`.env`).

### Direct from S3

Expand Down Expand Up @@ -77,7 +76,7 @@ echo "machine urs.earthdata.nasa.gov login ${EARTHDATA_USERNAME} password ${EART

## Docker deployment

You can run the application in a docker container using the [docker-compose.yml](./docker-compose.yml).
You can run the application in a docker container using the docker-compose.yml file.
The docker container is configured to read the `EARTHDATA_USERNAME` and `EARTHDATA_PASSWORD` environment variables so make sure set those before starting the docker network.

```bash
Expand Down
Binary file added docs/assets/logo_no_text.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/contributing.md
Loading
Loading