Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Solr schema and minor updates #168

Merged
merged 4 commits into from
Dec 2, 2024
Merged

Improve Solr schema and minor updates #168

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e464047
Add acces_services multiValued fields
mjanez Nov 13, 2024
6a4c1ae
Update ckanext-geoview to version 0.2.2 in Dockerfile
mjanez Nov 13, 2024
47b7011
Update OpenAPI endpoints in .env.example for consistency
mjanez Nov 19, 2024
fba627b
Add CKAN core settings to update the displayed timezone when configur…
mjanez Nov 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,7 @@ CKANEXT__SCHEMINGDCAT__OPEN_DATA_STATISTICS=True
CKANEXT__SCHEMINGDCAT__OPEN_DATA_STATISTICS_THEMES=True
# Private fields to be hidden in the API responses
CKANEXT__SCHEMINGDCAT__API__PRIVATE_FIELDS="private_name private_email"
CKANEXT__OPENAPI__ENDPOINTS='[{"url":"https://raw.githubusercontent.com/mjanez/ckanext-openapi/refs/heads/develop/ckanext/openapi/public/static/openapi/datastore.yaml","name":"datastore","title":{"en":"CKAN - Datastore API","es":"Portal de datos abiertos de CKAN - API Datastore"},"description":{"en":"This API provides live access to the Datastore portion of the CKAN Open Data Portal.","es":"Esta API proporciona acceso en tiempo real a la parte de Datastore del Portal de Datos Abiertos CKAN."}},{"url":"https://raw.githubusercontent.com/mjanez/ckanext-openapi/refs/heads/develop/ckanext/openapi/public/static/openapi/ckan.yaml","name":"ckan","title":{"en":"CKAN Open Data Portal - CKAN API (ES)","es":"Portal de datos abiertos de CKAN - API CKAN"},"description":{"en":"This API provides live access to the CKAN portion of the CKAN Open Data Portal.","es":"Esta API proporciona acceso en tiempo real a la parte de CKAN del Portal de Datos Abiertos CKAN."}}]'
CKANEXT__OPENAPI__ENDPOINTS='[{"url":"https://raw.githubusercontent.com/mjanez/ckanext-openapi/refs/heads/develop/ckanext/openapi/public/static/openapi/datastore.yaml","name":"datastore","title":{"en":"CKAN - Datastore API","es":"Portal de datos abiertos de CKAN - API Datastore"},"description":{"en":"This API provides live access to the Datastore portion of the CKAN Open Data Portal.","es":"Esta API proporciona acceso en tiempo real a la parte de Datastore del Portal de Datos Abiertos CKAN."}},{"url":"https://raw.githubusercontent.com/mjanez/ckanext-openapi/refs/heads/develop/ckanext/openapi/public/static/openapi/ckan.yaml","name":"ckan","title":{"en":"CKAN Open Data Portal - CKAN API","es":"Portal de datos abiertos de CKAN - API CKAN"},"description":{"en":"This API provides live access to the CKAN portion of the CKAN Open Data Portal.","es":"Esta API proporciona acceso en tiempo real a la parte de CKAN del Portal de Datos Abiertos CKAN."}}]'

# ckanext-pages
CKANEXT__PAGES__ALOW_HTML=False
Expand Down
9 changes: 8 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,11 @@ doc/scripts/ckanapi/*_ckanapi.py
doc/scripts/ckanapi/sites.yml
sites.yml
venv
env
env

# Solr $ CKAN custom Dockerfiles
!solr/Dockerfile.spatial
solr/Dockerfile.*
!ckan/Dockerfile.dev
!ckan/Dockerfile.ghcr
ckan/Dockerfile.*
4 changes: 2 additions & 2 deletions ckan/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
# CKAN configuration & extensions
## XLoader - 1.0.1-stable (mjanez/Forked fixed version) ##
## Harvest - v1.5.6 (mjanez Enhanced Privacy and Security for Remote Harvesting) ##
## Geoview - v0.2.0 ##
## Geoview - v0.2.2 ##
## Spatial - v2.1.1 ##
## DCAT - v1.8.0 (Latest stable version of ckanext-dcat with minor fixes) ##
## Scheming - release-3.0.0 ##
Expand All @@ -24,7 +24,7 @@
## Fluent - v1.0.1 (mjanez/Forked stable version) ##
## OpenAPI - v1.0.0 (mjanez stable version) ##
## Scheming DCAT - v4.2.3 (mjanez/GeoDCAT-AP/NTI-RISP extended version) ##
RUN echo ${TZ} > /etc/timezone && \

Check warning on line 27 in ckan/Dockerfile

View workflow job for this annotation

GitHub Actions / runner/test-docker-pr:ckan-2.10.5 

Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`

Check notice on line 27 in ckan/Dockerfile

View workflow job for this annotation

GitHub Actions / runner/test-docker-pr:ckan-2.10.5 

Avoid additional packages by specifying `--no-install-recommends`

Check warning on line 27 in ckan/Dockerfile

View workflow job for this annotation

GitHub Actions / runner/test-docker-pr:ckan-2.10.5 

Pin versions in pip. Instead of `pip install <package>` use `pip install <package>==<version>` or `pip install --requirement <requirements file>`

Check failure on line 27 in ckan/Dockerfile

View workflow job for this annotation

GitHub Actions / runner/build-docker-push:ckan-2.10.5 

SC3013 warning: In POSIX sh, -ef is undefined.

Check failure on line 27 in ckan/Dockerfile

View workflow job for this annotation

GitHub Actions / runner/build-docker-push:ckan-2.10.5 

DL3008 warning: Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`

Check failure on line 27 in ckan/Dockerfile

View workflow job for this annotation

GitHub Actions / runner/build-docker-push:ckan-2.10.5 

DL3015 info: Avoid additional packages by specifying `--no-install-recommends`

Check failure on line 27 in ckan/Dockerfile

View workflow job for this annotation

GitHub Actions / runner/build-docker-push:ckan-2.10.5 

DL3013 warning: Pin versions in pip. Instead of `pip install <package>` use `pip install <package>==<version>` or `pip install --requirement <requirements file>`
if ! [ /usr/share/zoneinfo/${TZ} -ef /etc/localtime ]; then cp /usr/share/zoneinfo/${TZ} /etc/localtime; fi && \
# Install patch utility
apt-get update && \
Expand All @@ -40,7 +40,7 @@
pip3 install --no-cache-dir -e git+https://github.com/mjanez/[email protected]#egg=ckanext-harvest && \
pip3 install --no-cache-dir -r ${APP_DIR}/src/ckanext-harvest/requirements.txt && \
echo "ckan/ckanext-geoview" && \
pip3 install --no-cache-dir -e git+https://github.com/ckan/[email protected].0#egg=ckanext-geoview && \
pip3 install --no-cache-dir -e git+https://github.com/ckan/[email protected].2#egg=ckanext-geoview && \
echo "ckan/ckanext-spatial" && \
pip3 install --no-cache-dir -e git+https://github.com/ckan/[email protected]#egg=ckanext-spatial && \
pip3 install --no-cache-dir -r ${APP_DIR}/req_fixes/ckanext-spatial/requirements.txt && \
Expand Down Expand Up @@ -72,7 +72,7 @@
# Apply any patches needed to CKAN core
COPY patches patches

RUN for d in $APP_DIR/patches/*; do \

Check warning on line 75 in ckan/Dockerfile

View workflow job for this annotation

GitHub Actions / runner/test-docker-pr:ckan-2.10.5 

Use WORKDIR to switch to a directory

Check warning on line 75 in ckan/Dockerfile

View workflow job for this annotation

GitHub Actions / runner/test-docker-pr:ckan-2.10.5 

Set the SHELL option -o pipefail before RUN with a pipe in it. If you are using /bin/sh in an alpine image or if your shell is symlinked to busybox then consider explicitly setting your SHELL to /bin/ash, or disable this check

Check failure on line 75 in ckan/Dockerfile

View workflow job for this annotation

GitHub Actions / runner/build-docker-push:ckan-2.10.5 

SC2086 info: Double quote to prevent globbing and word splitting.

Check failure on line 75 in ckan/Dockerfile

View workflow job for this annotation

GitHub Actions / runner/build-docker-push:ckan-2.10.5 

DL3003 warning: Use WORKDIR to switch to a directory

Check failure on line 75 in ckan/Dockerfile

View workflow job for this annotation

GitHub Actions / runner/build-docker-push:ckan-2.10.5 

DL4006 warning: Set the SHELL option -o pipefail before RUN with a pipe in it. If you are using /bin/sh in an alpine image or if your shell is symlinked to busybox then consider explicitly setting your SHELL to /bin/ash, or disable this check
if [ -d $d ]; then \
for f in `ls $d/*.patch | sort -g`; do \
cd $SRC_DIR/`basename "$d"` && \
Expand Down
5 changes: 5 additions & 0 deletions ckan/docker-entrypoint.d/01_setup_ckanext_config.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@
echo "[docker-entrypoint.01_setup_ckanext_config] Clear index"
ckan -c $CKAN_INI search-index clear

# ckan: Update core settings
echo "[docker-entrypoint.01_setup_ckanext_config] Loading CKAN core settings in the CKAN config file"
ckan config-tool $CKAN_INI \
"ckan.display_timezone=$TZ"

# Rebuild index
echo "[docker-entrypoint.01_setup_ckanext_config] Rebuild index"
ckan -c $CKAN_INI search-index rebuild
2 changes: 2 additions & 0 deletions solr/Dockerfile.spatial
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ ENV SOLR_BBOX_FIELDS '<field name="bbox_area" type="float" indexed="true" stored
<field name="tag_uri" type="string" uninvertible="false" docValues="true" indexed="true" stored="true" multiValued="true"/> \
<field name="conforms_to" type="string" uninvertible="false" docValues="true" indexed="true" stored="true" multiValued="true"/> \
<field name="applicable_legislation" type="string" uninvertible="false" docValues="true" indexed="true" stored="true" multiValued="true"/> \
<field name="endpoint_url" type="string" uninvertible="false" docValues="true" indexed="true" stored="true" multiValued="true"/> \
<field name="serves_dataset" type="string" uninvertible="false" docValues="true" indexed="true" stored="true" multiValued="true"/> \
<field name="lineage_source" type="string" uninvertible="false" docValues="true" indexed="true" stored="true" multiValued="true"/> \
<field name="lineage_process_steps" type="string" uninvertible="false" docValues="true" indexed="true" stored="true" multiValued="true"/> \
<field name="reference" type="string" uninvertible="false" docValues="true" indexed="true" stored="true" multiValued="true"/> \
Expand Down
Loading