There is still something wrong with read_from_db when running CKAN 2.11.1

[hcvdwerf]

The changed code

https://github.com/ckan/ckanext-dcat/commit/6bcf6931158ce995487a0828accf926ecfb375b1

The error

  File "/srv/app/src/ckanext-harvest/ckanext/harvest/cli.py", line 276, in run_test
    utils.run_test_harvester(id, force_import)
  File "/srv/app/src/ckanext-harvest/ckanext/harvest/utils.py", line 405, in run_test_harvester
    lib.run_harvest_job(job_obj, harvester)
  File "/srv/app/src/ckanext-harvest/ckanext/harvest/tests/lib.py", line 59, in run_harvest_job
    queue.fetch_and_import_stages(harvester, harvest_object)
  File "/srv/app/src/ckanext-harvest/ckanext/harvest/queue.py", line 515, in fetch_and_import_stages
    success_import = harvester.import_stage(obj)
  File "/srv/app/src/ckanext-dcat/ckanext/dcat/harvesters/rdf.py", line 322, in import_stage
    existing_dataset = self._get_existing_dataset(harvest_object.guid)
  File "/srv/app/src/ckanext-dcat/ckanext/dcat/harvesters/base.py", line 183, in _get_existing_dataset
    datasets = self._read_datasets_from_db(guid)
  File "/srv/app/src/ckanext-dcat/ckanext/dcat/harvesters/base.py", line 171, in _read_datasets_from_db
    .filter(model.Package.extras['guid'] == f'"{guid}"') \
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/operators.py", line 461, in __getitem__
    return self.operate(getitem, index)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/ext/associationproxy.py", line 951, in operate
    self.remote_attr.operate(op, *other, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/orm/attributes.py", line 322, in operate
    return op(self.comparator, *other, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/operators.py", line 461, in __getitem__
    return self.operate(getitem, index)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/orm/properties.py", line 426, in operate
    return op(self.__clause_element__(), *other, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/operators.py", line 461, in __getitem__
    return self.operate(getitem, index)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 873, in operate
    return op(self.comparator, *other, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/operators.py", line 461, in __getitem__
    return self.operate(getitem, index)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/type_api.py", line 77, in operate
    return o[0](self.expr, op, *(other + o[1:]), **kwargs)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/default_comparator.py", line 181, in _getitem_impl
    _unsupported_impl(expr, op, other, **kw)
  File "/usr/local/lib/python3.10/site-packages/sqlalchemy/sql/default_comparator.py", line 185, in _unsupported_impl
    raise NotImplementedError(
NotImplementedError: Operator 'getitem' is not supported on this expression`

The local environment file

# CKAN databases
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=postgres
POSTGRES_HOST=db
CKAN_DB_USER=ckandbuser
CKAN_DB_PASSWORD=ckandbpassword
CKAN_DB=ckandb
CKAN_SQLALCHEMY_URL=postgresql://ckandbuser:ckandbpassword@db/ckandb

# Dev settings
USE_HTTPS_FOR_DEV=false

# Host Ports
CKAN_PORT_HOST=5500

# CKAN core
CKAN_VERSION=2.11.1
CKAN_SITE_ID=default
CKAN_SITE_URL=http://localhost:5500

CKAN___BEAKER__SESSION__SECRET=CHANGE_ME
# See https://docs.ckan.org/en/latest/maintaining/configuration.html#api-token-settings
CKAN___API_TOKEN__JWT__ENCODE__SECRET=string:CHANGE_ME
CKAN___API_TOKEN__JWT__DECODE__SECRET=string:CHANGE_ME
CKAN_SYSADMIN_NAME=ckan_admin
CKAN_SYSADMIN_PASSWORD=test1234
[email protected]
CKAN_STORAGE_PATH=/var/lib/ckan
CKAN_SMTP_SERVER=smtp.corporateict.domain:25
CKAN_SMTP_STARTTLS=True
CKAN_SMTP_USER=user
CKAN_SMTP_PASSWORD=pass
CKAN_SMTP_MAIL_FROM=ckan@localhost
CKAN_MAX_UPLOAD_SIZE_MB=10
TZ=Europe/Amsterdam

# Redis
REDIS_VERSION=7.4.2
CKAN_REDIS_URL=redis://redis:6379/1

# Solr 
CKAN_SOLR_URL=http://solr:8983/solr/ckan

# Extensions
CKAN__PLUGINS="envvars scheming_datasets scheming_organizations gdi_userportal dcat dcat_json_interface harvest ckan_harvester dcat_rdf_harvester dcat_json_harvester fairdatapointharvester activity"
CKAN__HARVEST__MQ__HOSTNAME=redis
CKAN__HARVEST__MQ__PORT=6379
CKAN__HARVEST__MQ__REDIS_DB=1
CKAN__HARVEST__MQ__PASSWORD=
CKAN__HARVEST__MQ__TYPE=redis

#Logger
CKAN___DEBUG=true
CKAN__HARVEST__LOG_LEVEL=DEBUG
CKAN___LOGGER_CKANEXT__LEVEL=DEBUG

#OTEL
OTEL_SDK_DISABLED=true

[amercader]

The fix (6bcf693) is in the master branch but not in the latest published tag (v2.1.0).

It's time to do a new release anyway,I'll do that tomorrow, thanks for pointing it out.

[amercader]

v2.2.0 is out, and it includes the fix for this issue: https://github.com/ckan/ckanext-dcat/releases/tag/v2.2.0