Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetch extra reporting_org fields #35

Merged
merged 27 commits into from
Jan 22, 2025
Merged

Fetch extra reporting_org fields #35

merged 27 commits into from
Jan 22, 2025

Conversation

simon-20
Copy link
Contributor

The Bulk Data Service now:

  • Fetches reporting_org fields: short_name, human_readable_name, IATI Identifier, id
  • These fields are made available in a separate index

This release also includes a fix to re-generate the website on deploy, and not just on initial provision.

Simplifying refactor ahead of database changes:
* remove unneeded factory methods
* fetch org metadata each run
Tests to check list of IATI organisations are added, updated and deleted from new DB table.
* Refactored some tests and test data to facilitate testing given that org data will now be drawn from the full org metadata, and not the limited org metadata on the datasets
* Add 'dataset' to names now there are also entries for organisations.
Previously, if the zipper was run standalone from the command line, it would crash at the end when updating prom metrics. This would only ever occur during local development, but this fixes it.
* toml package installed to allow programmatic access to version
* Python upgraded because latest black doesn't run with 3.12.5
Save and make available additional reporting_org fields
fix: update website on redeploy
Create STORAGE_ACCOUNT_NAME directly because intermediate
env variable not available in same context as created
fix: create STORAGE_ACCOUNT_NAME directly
@simon-20 simon-20 requested a review from a team January 22, 2025 10:07
@simon-20
Copy link
Contributor Author

This has been running on dev since early this morning and has proceeded through several cycles without any problems.

@simon-20 simon-20 merged commit 7c31043 into main Jan 22, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants