Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Background worker setup (CKAN container) | ckanext-harvest & ckanext-xloader #100

Merged
merged 6 commits into from
Mar 8, 2024

Conversation

mjanez
Copy link
Owner

@mjanez mjanez commented Feb 21, 2024

Use supervisor to allow fetch and gather consumers in the background: https://github.com/ckan/ckanext-harvest#setting-up-the-harvesters-on-a-production-server

@mjanez mjanez added documentation Improvements or additions to documentation enhancement New feature or request ckan-ogc https://github.com/mjanez/ckan-ogc ckanext-schemingdcat labels Feb 21, 2024
@mjanez mjanez self-assigned this Feb 21, 2024
@mjanez mjanez changed the title Setting up the harvesters (ckan container) Background worker setup (CKAN container) | ckanext-harvest & ckanext-xloader Feb 21, 2024
mjanez added 5 commits March 5, 2024 07:40
- Update Dockerfiles
- Update start_ckan.sh with custom workers
- Update start_ckan_development.sh.override to remove automatic workers
- Add supervisor harvester.conf
- Add supervisor xloader.conf
- Update Scheming DCAT version to 2.1.0
@mjanez mjanez linked an issue Mar 8, 2024 that may be closed by this pull request
2 tasks
@mjanez mjanez merged commit 904530e into ckan-2.9.9 Mar 8, 2024
1 check passed
@mjanez mjanez mentioned this pull request Mar 8, 2024
3 tasks
@mjanez mjanez deleted the feature/harvester-worker branch November 22, 2024 20:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ckan-ogc https://github.com/mjanez/ckan-ogc ckanext-schemingdcat documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature - Harvest/Xloader workers in background Load data into datastore
1 participant