Skip to content

Commit

Permalink
Merge pull request #4278 from GSA-TTS/main
Browse files Browse the repository at this point in the history
  • Loading branch information
jadudm authored Sep 12, 2024
2 parents 8e4bf31 + e84a236 commit fc502d0
Show file tree
Hide file tree
Showing 33 changed files with 38,874 additions and 10,332 deletions.
8 changes: 3 additions & 5 deletions .github/workflows/terraform-apply-env.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,9 @@ jobs:
apt-get install -y zip python
# Install CF CLI
mkdir -p /etc/apt/keyrings/
wget -q -O - https://packages.cloudfoundry.org/debian/cli.cloudfoundry.org.key | gpg --dearmor | tee /etc/apt/trusted.gpg.d/cf.gpg > /dev/null
echo "deb https://packages.cloudfoundry.org/debian stable main" | tee /etc/apt/sources.list.d/cloudfoundry-cli.list
apt-get update && DEBIAN_FRONTEND=noninteractive
apt-get install --assume-yes cf8-cli
curl -k -O -L https://github.com/cloudfoundry/cli/releases/download/v8.8.0/cf8-cli-installer_8.8.0_x86-64.deb
apt-get install --assume-yes ./cf8-cli-installer_8.8.0_x86-64.deb
cf api api.fr.cloud.gov
cf auth ${{ secrets.CF_USERNAME }} ${{ secrets.CF_PASSWORD }}
TF_VAR_postgrest_image: 'ghcr.io/gsa-tts/fac/postgrest:latest'
Expand Down
8 changes: 3 additions & 5 deletions .github/workflows/terraform-plan-env.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,9 @@ jobs:
apt-get install -y zip python
# Install CF CLI
mkdir -p /etc/apt/keyrings/
wget -q -O - https://packages.cloudfoundry.org/debian/cli.cloudfoundry.org.key | gpg --dearmor | tee /etc/apt/trusted.gpg.d/cf.gpg > /dev/null
echo "deb https://packages.cloudfoundry.org/debian stable main" | tee /etc/apt/sources.list.d/cloudfoundry-cli.list
apt-get update && DEBIAN_FRONTEND=noninteractive
apt-get install --assume-yes cf8-cli
curl -k -O -L https://github.com/cloudfoundry/cli/releases/download/v8.8.0/cf8-cli-installer_8.8.0_x86-64.deb
apt-get install --assume-yes ./cf8-cli-installer_8.8.0_x86-64.deb
cf api api.fr.cloud.gov
cf auth ${{ secrets.CF_USERNAME }} ${{ secrets.CF_PASSWORD }}
TF_VAR_postgrest_image: 'ghcr.io/gsa-tts/fac/postgrest:latest'
Expand Down
1 change: 1 addition & 0 deletions backend/audit/intakelib/checks/check_version_number.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
"1.1.0",
"1.1.1",
"1.1.2",
"1.1.3",
}


Expand Down
6 changes: 3 additions & 3 deletions backend/schemas/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ xlsx = $(wildcard output/excel/xlsx/*-workbook*.xlsx)
json = $(wildcard output/excel/json/*.json)

source_data:
python scripts/generate_lookup_schemas.py source/data/cfda-lookup-20230626.csv source/base/FederalProgramNames.json
python scripts/generate_lookup_schemas.py source/data/cluster-names-20230626.csv source/base/ClusterNames.json
python scripts/generate_lookup_schemas.py cfda-lookup source/base/FederalProgramNames.json
python scripts/generate_lookup_schemas.py cluster-names source/base/ClusterNames.json

clean:
for f in $(xlsx); do \
Expand All @@ -25,7 +25,7 @@ clean:
rm $$f; \
done

build_audit_json:
build_audit_json:
for jsonnet_file in $(audit_specs); do \
base_name=$$(basename "$$jsonnet_file" .jsonnet); \
jsonnet -o output/audit/"$$base_name.json" "$$jsonnet_file"; \
Expand Down
15 changes: 15 additions & 0 deletions backend/schemas/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Bumping workbook template version

Follow these steps to version bump the workbook templates:
- `backend/schemas/source/excel/libs/Sheets.libsonnet`: Update the `WORKBOOKS_VERSION` variable
- `backend/audit/intakelib/checks/check_version_number.py`: Update the `AUTHORIZED_VERSIONS` variable
- Run `make all` to generate new schemas and tempaltes
- Once your PR is merged, don't forget to copy the new templates, found in `backend/schemas/output/excel/xlsx/`, into `assets/workbooks/` of the [static site repo](https://github.com/GSA-TTS/FAC-transition-site).

# Updating the CFDA listings

The current CFDA assistance listings are in the CSV found [here](https://sam.gov/data-services/Assistance%20Listings/datagov?privacy=Public). When downloading the file, save it in the format `cfda-lookup-YYYYMMDD.csv` in the `/schemas/source/data` directory. Running `make all` should be sufficent to regenerate the lookup schemas and the Excel templates.

More specifically, `make all` executes `make source_data`, which, calls `generate_lookup_schemas.py`. This script can generate either cluster names or CFDA listings or agencies, depending on the args given (see docstring in the script.) The script will automatically used the latest-dated CSV file for processing. This way, the Makefile doesn't have to be repeatedly changed and we can retain the historic files. The format of the CSVs can change (and have), so changes to `generate_lookup_schemas.py` may be necessary may be necessary in the future and non-current files may no longer be processable.

If you get a `UnicodeDecodeError`, you may have to manually save it with UTF-8 encoding (in VSCode, click UTF-8 in the bottom right and select "Save with encoding".)
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
"validation": {
"type": "NOVALIDATION"
},
"value": "1.1.2",
"value": "1.1.3",
"width": 48
},
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
"validation": {
"type": "NOVALIDATION"
},
"value": "1.1.2",
"value": "1.1.3",
"width": 48
},
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
"validation": {
"type": "NOVALIDATION"
},
"value": "1.1.2",
"value": "1.1.3",
"width": 48
},
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
"validation": {
"type": "NOVALIDATION"
},
"value": "1.1.2",
"value": "1.1.3",
"width": 48
},
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
"validation": {
"type": "NOVALIDATION"
},
"value": "1.1.2",
"value": "1.1.3",
"width": 48
},
{
Expand Down
Loading

0 comments on commit fc502d0

Please sign in to comment.