diff --git a/doc/scripts/ckanapi/.gitignore b/doc/scripts/ckanapi/.gitignore deleted file mode 100644 index 7273b114..00000000 --- a/doc/scripts/ckanapi/.gitignore +++ /dev/null @@ -1,4 +0,0 @@ -venv -env -sites.yml -output/* \ No newline at end of file diff --git a/doc/scripts/ckanapi/.old/.gitignore b/doc/scripts/ckanapi/.old/.gitignore new file mode 100644 index 00000000..8a6f585c --- /dev/null +++ b/doc/scripts/ckanapi/.old/.gitignore @@ -0,0 +1,6 @@ +venv +env +sites.yml +sites.*.yml +output/* +!sites.example.yml \ No newline at end of file diff --git a/doc/scripts/ckanapi/.old/README.md b/doc/scripts/ckanapi/.old/README.md new file mode 100644 index 00000000..0184a20f --- /dev/null +++ b/doc/scripts/ckanapi/.old/README.md @@ -0,0 +1,175 @@ +# CKAN API Script + +This script allows you to interact with a CKAN instance to perform various actions such as updating packages, exporting groups, and exporting organizations. The configuration for different CKAN instances and actions is specified in a YAML file. + +## Prerequisites + +- Python 3.x + +## Setup + +### Creating a Virtual Environment + +1. Create a virtual environment: + +```sh +python3 -m venv venv +``` + +2. Activate the virtual environment: + +- On Linux and macOS: + +```sh +source venv/bin/activate +``` + +- On Windows: + +```sh +.\venv\Scripts\activate +``` + +### Installing Dependencies + +Install the required libraries using pip: + +```sh +pip install -r requirements.txt +``` + +## Configuration + +The configuration for the CKAN instances and actions is specified in a YAML file located at `./input/sites.yml`. If this file does not exist, you can use the provided `sites.example.yml` as a template. + +### Creating `sites.yml` + +1. Copy the `sites.example.yml` file to `sites.yml`: + +```sh +cp ./input/sites.example.yml ./input/sites.yml +``` + +2. Edit the `sites.yml` file to include your CKAN instance details and the actions you want to perform. Below is an example configuration: + +```yaml +# ./input/sites.yml + +default: + ckan_site_url: 'https://demo.ckan.dcat-ap-3.es' + api_token: 'your_api_token' + actions: + - update_packages + - export_groups + - export_organizations + override: + theme_es: "http://datos.gob.es/kos/sector-publico/sector/medio-ambiente" + target_values: + publisher_name: + condition: "Sample Company" + override: + publisher_type: "http://purl.org/adms/publishertype/Company" + +site_1: + ckan_site_url: 'https://site1.ckan.instance' + api_token: 'site1_api_token' + actions: + - update_packages + - export_groups + - export_organizations + override: + theme_es: "http://site1.theme.url" + target_values: + publisher_name: + condition: "Site 1 Company" + override: + publisher_type: "http://site1.publisher.type" + +site_2: + ckan_site_url: 'https://site2.ckan.instance' + api_token: null + actions: + - export_groups + - export_organizations + +site_3: + ckan_site_url: 'https://site2.ckan.instance' + api_token: 'site2_api_token' + actions: + - export_groups + - export_organizations + - create_organizations + - create_groups + - create_users + organizations: './input/site_3/organizations.json' + groups: './input/site_3/groups.json' + users: './input/site_3/users.json' + +``` + +## Usage + +To run the script, use the following command: + +```sh +python -m src -s +``` + +Replace `` with the site configuration you want to use (e.g., `default`, `site_1`, `site_2`). + +### Example + +To run the script using the `site_2` configuration: + +```sh +python -m src -s site_2 +``` + +### Command-line Arguments + +- `-s` or `--site`: The site configuration to load (default: `default`). +- `-c` or `--config`: The path to the YAML configuration file (default: `./input/sites.yml`). + +### Notes + +- Actions that modify data (e.g., `update_packages`, `create`, `delete`) require a non-null `api_token`. +- If only `GET` actions are specified (e.g., `export_groups`, `export_organizations`), the `api_token`, `override`, and `target_values` are not required. + +## Script Details + +The script performs the following actions based on the configuration: + +1. **Update Packages**: Updates the packages in the CKAN instance based on the specified overrides and target values. +2. **Export Groups**: Exports all groups from the CKAN instance to a JSON file (`groups.json`). +3. **Export Organizations**: Exports all organizations from the CKAN instance to a JSON file (`organizations.json`). + +### Functions + +#### Connect + +- `connect_to_ckan(ckan_site_url, api_token)`: Connects to the CKAN instance. + +#### Create + +- `create_organizations(rc, json_path)`: Creates new organizations in CKAN from a JSON file. +- `create_groups(rc, json_path)`: Creates new groups in CKAN from a JSON file. +- `create_users(rc, json_path)`: Creates new users in CKAN from a JSON file. + +#### Update + +- `update_package(rc, package_id, theme_es, publisher_name, publisher_type)`: Updates a package given its ID. + +#### Get + +- `export_groups_to_json(rc, file_path)`: Exports all groups to a JSON file. +- `export_organizations_to_json(rc, file_path)`: Exports all organizations to a JSON file. + + +### Example Output + +- `groups.json`: Contains detailed information about all groups. +- `organizations.json`: Contains detailed information about all organizations. + +## License + +This project is licensed under the MIT License. diff --git a/doc/scripts/ckanapi/input/sites.example.yml b/doc/scripts/ckanapi/.old/input/sites.example.yml similarity index 100% rename from doc/scripts/ckanapi/input/sites.example.yml rename to doc/scripts/ckanapi/.old/input/sites.example.yml diff --git a/doc/scripts/ckanapi/requirements.txt b/doc/scripts/ckanapi/.old/requirements.txt similarity index 100% rename from doc/scripts/ckanapi/requirements.txt rename to doc/scripts/ckanapi/.old/requirements.txt diff --git a/doc/scripts/ckanapi/src/__init__.py b/doc/scripts/ckanapi/.old/src/__init__.py similarity index 100% rename from doc/scripts/ckanapi/src/__init__.py rename to doc/scripts/ckanapi/.old/src/__init__.py diff --git a/doc/scripts/ckanapi/src/__main__.py b/doc/scripts/ckanapi/.old/src/__main__.py similarity index 100% rename from doc/scripts/ckanapi/src/__main__.py rename to doc/scripts/ckanapi/.old/src/__main__.py diff --git a/doc/scripts/ckanapi/src/config.py b/doc/scripts/ckanapi/.old/src/config.py similarity index 83% rename from doc/scripts/ckanapi/src/config.py rename to doc/scripts/ckanapi/.old/src/config.py index 8da44396..641a1b06 100644 --- a/doc/scripts/ckanapi/src/config.py +++ b/doc/scripts/ckanapi/.old/src/config.py @@ -17,6 +17,7 @@ class SampleConfig: groups (str): Path to the JSON file containing group data. users (str): Path to the JSON file containing user data. override (dict): Dictionary of override values. + delete (list): List of keys to delete. target_values (dict): Dictionary of target values. """ def __init__(self, config): @@ -27,7 +28,14 @@ def __init__(self, config): self.groups = config.get('groups') self.users = config.get('users') self.override = config.get('override', {}) + self.delete = config.get('delete', []) self.target_values = config.get('target_values', {}) + target_site_config = config.get('target_site', {}) + self.target_ckan_site_url = target_site_config.get('ckan_site_url') + self.target_api_token = target_site_config.get('api_token') + self.target_organization_id = target_site_config.get('target_organization_id') + self.delete_all_before_copy = target_site_config.get('delete_all_before_copy', False) # Nueva opción + def load_config(config_file, site='default'): """ @@ -72,6 +80,9 @@ def parse_args(): print(f"Actions: {config.actions}") if config.override: print(f"Override: {config.override}") + #FIXME: Not working, duplicate datasets. + # if config.delete: + # print(f"Delete fields: {config.delete}") if config.target_values: print(f"Target Values: {config.target_values}") if config.organizations: diff --git a/doc/scripts/ckanapi/src/logic/__init__.py b/doc/scripts/ckanapi/.old/src/logic/__init__.py similarity index 92% rename from doc/scripts/ckanapi/src/logic/__init__.py rename to doc/scripts/ckanapi/.old/src/logic/__init__.py index 4430246f..a8459b22 100644 --- a/doc/scripts/ckanapi/src/logic/__init__.py +++ b/doc/scripts/ckanapi/.old/src/logic/__init__.py @@ -39,12 +39,10 @@ def execute_action(action_name, rc, config, output_dir): # Iterate over each package and update it for package_id in package_list: - actions[action_name](rc, package_id, config.override, config.target_values) + actions[action_name](rc, package_id, config.override, config.delete, config.target_values) elif action_name == 'create_organizations': actions[action_name](rc, config.organizations) elif action_name == 'create_groups': actions[action_name](rc, config.groups) elif action_name == 'create_users': - actions[action_name](rc, config.users) - else: - actions[action_name](rc, output_dir) \ No newline at end of file + actions[action_name](rc, config.users) \ No newline at end of file diff --git a/doc/scripts/ckanapi/src/logic/create.py b/doc/scripts/ckanapi/.old/src/logic/create.py similarity index 96% rename from doc/scripts/ckanapi/src/logic/create.py rename to doc/scripts/ckanapi/.old/src/logic/create.py index 017e2250..cc760163 100644 --- a/doc/scripts/ckanapi/src/logic/create.py +++ b/doc/scripts/ckanapi/.old/src/logic/create.py @@ -1,5 +1,7 @@ import json import os +from ckanapi import RemoteCKAN, NotFound, ValidationError +import uuid def load_json(file_path): """ diff --git a/doc/scripts/ckanapi/src/logic/delete.py b/doc/scripts/ckanapi/.old/src/logic/delete.py similarity index 100% rename from doc/scripts/ckanapi/src/logic/delete.py rename to doc/scripts/ckanapi/.old/src/logic/delete.py diff --git a/doc/scripts/ckanapi/src/logic/get.py b/doc/scripts/ckanapi/.old/src/logic/get.py similarity index 100% rename from doc/scripts/ckanapi/src/logic/get.py rename to doc/scripts/ckanapi/.old/src/logic/get.py diff --git a/doc/scripts/ckanapi/src/logic/update.py b/doc/scripts/ckanapi/.old/src/logic/update.py similarity index 82% rename from doc/scripts/ckanapi/src/logic/update.py rename to doc/scripts/ckanapi/.old/src/logic/update.py index 0b3ceaed..2f263bbd 100644 --- a/doc/scripts/ckanapi/src/logic/update.py +++ b/doc/scripts/ckanapi/.old/src/logic/update.py @@ -31,7 +31,7 @@ def apply_target_values(package, target_values): package = apply_overrides(package, overrides) return package -def update_package(rc, package_id, overrides, target_values): +def update_package(rc, package_id, overrides, deletes, target_values=None): """ Update a package given its ID. @@ -39,8 +39,15 @@ def update_package(rc, package_id, overrides, target_values): rc (RemoteCKAN): The CKAN instance connection. package_id (str): The ID of the package. overrides (dict): The dictionary of fields to override. - target_values (dict): The dictionary of conditions and overrides. + deletes (list): The list of keys to delete. + target_values (dict, optional): The dictionary of conditions and overrides. Defaults to {}. + + Returns: + None """ + if target_values is None: + target_values = {} + package = rc.action.package_show(id=package_id) # Apply overrides diff --git a/doc/scripts/ckanapi/src/quick_ckanapi.py b/doc/scripts/ckanapi/.old/src/quick_ckanapi.py similarity index 70% rename from doc/scripts/ckanapi/src/quick_ckanapi.py rename to doc/scripts/ckanapi/.old/src/quick_ckanapi.py index 63e8c751..81cefe2a 100644 --- a/doc/scripts/ckanapi/src/quick_ckanapi.py +++ b/doc/scripts/ckanapi/.old/src/quick_ckanapi.py @@ -18,15 +18,15 @@ def connect_to_ckan(ckan_site_url, api_token): def is_modifying_action(action_name): """ - Determine if an action modifies data based on its name. + Determina si una acción modifica datos basándose en su nombre. Args: - action_name (str): The name of the action. + action_name (str): El nombre de la acción. Returns: - bool: True if the action modifies data, False otherwise. + bool: True si la acción modifica datos, False de lo contrario. """ - modifying_prefixes = ('update_', 'create_', 'delete_') + modifying_prefixes = ('update_', 'create_', 'delete_', 'copy_') return action_name.startswith(modifying_prefixes) def main(config): @@ -36,9 +36,6 @@ def main(config): Args: config (SampleConfig): The configuration object. """ - # Connect to the CKAN instance - rc = connect_to_ckan(config.ckan_site_url, config.api_token) - # Create output directories if they don't exist output_dir = os.path.join('output', config.site) os.makedirs(output_dir, exist_ok=True) @@ -51,7 +48,13 @@ def main(config): # Execute actions based on the configuration for action in config.actions: - execute_action(action, rc, config, output_dir) + if action == "copy_datasets": + # For copy_datasets, connections are handled within the function + execute_action(action, None, config, output_dir) + else: + # Connect to the CKAN instance + rc = connect_to_ckan(config.ckan_site_url, config.api_token) + execute_action(action, rc, config, output_dir) print("Actions completed.") diff --git a/doc/scripts/ckanapi/README.md b/doc/scripts/ckanapi/README.md index 0184a20f..281bb088 100644 --- a/doc/scripts/ckanapi/README.md +++ b/doc/scripts/ckanapi/README.md @@ -1,175 +1,342 @@ -# CKAN API Script +# ckanapi +A command line interface and Python module for accessing the [CKAN Action API](http://docs.ckan.org/en/latest/api/index.html#action-api-reference) -This script allows you to interact with a CKAN instance to perform various actions such as updating packages, exporting groups, and exporting organizations. The configuration for different CKAN instances and actions is specified in a YAML file. +>[!NOTE] +> `ckanapi`: https://github.com/ckan/ckanapi -## Prerequisites +## Installation -- Python 3.x +Installation with pip: +``` +pip install ckanapi +``` + +Installation with conda: +``` +conda install -c conda-forge ckanapi +``` + + +## ckanapi CLI + +The ckanapi command line interface lets you access local and +remote CKAN instances for bulk operations and simple API actions. + + +### Actions -## Setup +Simple actions with string parameters may be called directly. The +response is pretty-printed to STDOUT. -### Creating a Virtual Environment +used to adjust these limits. `CKANAPI_MY_SITES` (comma-delimited list of CKAN urls) +will not have the `PARALLEL_LIMIT` applied. -1. Create a virtual environment: +`dump` and `load` jobs may be resumed from the last completed +record or split across multiple servers by specifying record +start and max values. -```sh -python3 -m venv venv +#### 🔧 Dump datasets from CKAN into a local file with 4 processes + +``` +$ ckanapi dump datasets --all -O datasets.jsonl.gz -z -p 4 -r http://localhost ``` -2. Activate the virtual environment: +#### 🔧 Export datasets including private ones using search -- On Linux and macOS: +``` +$ ckanapi search datasets include_private=true -O datasets.jsonl.gz -z \ + -c /etc/ckan/production.ini -```sh -source venv/bin/activate +# Remote URL +$ ckanapi search datasets include_private=true -O datasets.jsonl.gz -z \ + -r http://myckan.org/catalog --apikey my_api_key ``` -- On Windows: +`search` is faster than `dump` because it calls `package_search` to retrieve +many records per call, paginating automatically. + +You may add parameters supported by `package_search` to filter the +records returned. + -```sh -.\venv\Scripts\activate +#### 🔧 Load/update datasets from a dataset JSON lines file with 3 processes + +``` +$ ckanapi load datasets -I datasets.jsonl.gz -z -p 3 -c /etc/ckan/production.ini + +# Remote URL +$ ckanapi load datasets -I datasets.jsonl.gz -z -p 3 -r http://myckan.org/catalog --apikey my_api_key ``` -### Installing Dependencies -Install the required libraries using pip: +### Bulk Delete + +Datasets, groups, organizations, users and related items may be deleted in +bulk with the delete command. This command accepts ids or names on the +command line or a number of different formats piped on standard input. -```sh -pip install -r requirements.txt + +```bash + ckanapi delete (datasets | groups | organizations | users | related) + (ID_OR_NAME ... | [-I JSONL_INPUT] [-s START] [-m MAX]) + [-p PROCESSES] [-l LOG_FILE] [-qwz] + [[-c CONFIG] [-u USER] | -r SITE_URL [-a APIKEY] [--insecure]] ``` -## Configuration +#### 🔧 All datasets (JSON list of "id" or "name" values) +``` +$ ckanapi action package_list -j | ckanapi delete datasets -The configuration for the CKAN instances and actions is specified in a YAML file located at `./input/sites.yml`. If this file does not exist, you can use the provided `sites.example.yml` as a template. +$ ckanapi action package_list -j -r http://ckan.source.org/catalog --apikey my_api_key | ckanapi delete datasets -r https://ckan.target.org -u ckan_admin -a target_api_key -### Creating `sites.yml` +``` -1. Copy the `sites.example.yml` file to `sites.yml`: +#### 🔧 Selective delete (JSON object with "results" list containing "id" values) +``` +$ ckanapi action package_search q=ponies | ckanapi delete datasets +``` -```sh -cp ./input/sites.example.yml ./input/sites.yml +#### 🔧 Processed JSON Lines (JSON objects with "id" or "name" value, one per line) ``` +$ ckanapi dump groups --all > groups.jsonl +$ grep ponies groups.jsonl | ckanapi delete groups -2. Edit the `sites.yml` file to include your CKAN instance details and the actions you want to perform. Below is an example configuration: -```yaml -# ./input/sites.yml +# Remote delete datasets +$ ckanapi action package_list -j -r http://ousrce.ckan.org/catalog --apikey my_api_key > default.jsonl +$ ckanapi delete datasets -I default.jsonl -p 3 -l log.log -u ckan_admin -r http://target.ckan.org/catalog --apikey my_api_key -default: - ckan_site_url: 'https://demo.ckan.dcat-ap-3.es' - api_token: 'your_api_token' - actions: - - update_packages - - export_groups - - export_organizations - override: - theme_es: "http://datos.gob.es/kos/sector-publico/sector/medio-ambiente" - target_values: - publisher_name: - condition: "Sample Company" - override: - publisher_type: "http://purl.org/adms/publishertype/Company" +``` -site_1: - ckan_site_url: 'https://site1.ckan.instance' - api_token: 'site1_api_token' - actions: - - update_packages - - export_groups - - export_organizations - override: - theme_es: "http://site1.theme.url" - target_values: - publisher_name: - condition: "Site 1 Company" - override: - publisher_type: "http://site1.publisher.type" +#### 🔧 Text list of "id" or "name" values (one per line) +``` +$ cat users_to_remove.txt +fred +bill +larry +$ ckanapi delete users < users_to_remove.txt -site_2: - ckan_site_url: 'https://site2.ckan.instance' - api_token: null - actions: - - export_groups - - export_organizations -site_3: - ckan_site_url: 'https://site2.ckan.instance' - api_token: 'site2_api_token' - actions: - - export_groups - - export_organizations - - create_organizations - - create_groups - - create_users - organizations: './input/site_3/organizations.json' - groups: './input/site_3/groups.json' - users: './input/site_3/users.json' +## Datasets +ckanapi delete users -r http://myckan.org/catalog --apikey my_api_key < output_pre.txt ``` -## Usage -To run the script, use the following command: +### Bulk Dataset and Resource Export - datapackage.json format + +Datasets may be exported to a simplified +[datapackage.json format](http://dataprotocols.org/data-packages/) +(which includes the actual resources, where available). -```sh -python -m src -s +If the resource url is not available, the resource will be included +in the datapackage.json file but the actual resource data will not be downloaded. + +``` +$ ckanapi dump datasets --all --datapackages=./output_directory/ -r http://sourceckan.example.com ``` -Replace `` with the site configuration you want to use (e.g., `default`, `site_1`, `site_2`). +### Batch Actions -### Example +Run a set of actions from a JSON lines file. For local actions this is much faster than running +`ckanapi action ...` in a shell loop because the local start-up time only happens once. -To run the script using the `site_2` configuration: +Batch actions can also be run in parallel with multiple processes and errors logged, just like the +dump and load commands. -```sh -python -m src -s site_2 +#### 🔧 Update a dataset field across a number of datasets +``` +$ cat update-emails.jsonl +{"action":"package_patch","data":{"id":"dataset-1","maintainer_email":"new@example.com"}} +{"action":"package_patch","data":{"id":"dataset-2","maintainer_email":"new@example.com"}} +{"action":"package_patch","data":{"id":"dataset-3","maintainer_email":"new@example.com"}} +$ ckanapi batch -I update-emails.jsonl ``` -### Command-line Arguments +#### 🔧 Replace a set of uploaded files +``` +$ cat upload-files.jsonl +{"action":"resource_patch","data":{"id":"408e1b1d-d0ca-50ca-9ae6-aedcee37aaa9"},"files":{"upload":"data1.csv"}} +{"action":"resource_patch","data":{"id":"c1eab17f-c2d0-536d-a3f6-41a3dfe6a2c3"},"files":{"upload":"data2.csv"}} +{"action":"resource_patch","data":{"id":"8ed068c2-4d4c-5f20-90db-39d2d596ce1a"},"files":{"upload":"data3.csv"}} +$ ckanapi batch -I upload-files.jsonl --local-files +``` -- `-s` or `--site`: The site configuration to load (default: `default`). -- `-c` or `--config`: The path to the YAML configuration file (default: `./input/sites.yml`). +The `"files"` values in the JSON lines file is ignored unless the `--local-files` parameter is passed. +Paths in the JSON lines file reference files on the local filesystems relative to the current working +directory. -### Notes +### Shell pipelines -- Actions that modify data (e.g., `update_packages`, `create`, `delete`) require a non-null `api_token`. -- If only `GET` actions are specified (e.g., `export_groups`, `export_organizations`), the `api_token`, `override`, and `target_values` are not required. +Simple shell pipelines are possible with the CLI. -## Script Details +#### 🔧 Copy the name of a dataset to its title with 'jq' +``` +$ ckanapi action package_show id=my-dataset \ + | jq '.+{"title":.name}' \ + | ckanapi action package_update -i +``` -The script performs the following actions based on the configuration: +#### 🔧 Mirror all datasets from one CKAN instance to another +``` +$ ckanapi dump datasets --all -q -r http://sourceckan.example.com \ + | ckanapi load datasets +``` -1. **Update Packages**: Updates the packages in the CKAN instance based on the specified overrides and target values. -2. **Export Groups**: Exports all groups from the CKAN instance to a JSON file (`groups.json`). -3. **Export Organizations**: Exports all organizations from the CKAN instance to a JSON file (`organizations.json`). +* `ValidationError` - field errors listed in `.error_dict` +* `SearchQueryError` - error reported from SOLR index +* `SearchError` +* `CKANAPIError` - incorrect use of ckanapi or unable to parse response +* `ServerIncompatibleError` - the remote API is not a CKAN API + +When using an action shortcut or the `call_action` method +failures are raised as exceptions just like when calling `get_action` from a +CKAN plugin: + +```python +from ckanapi import RemoteCKAN, NotAuthorized +ua = 'ckanapiexample/1.0 (+http://example.com/my/website)' + +demo = RemoteCKAN('https://demo.ckan.org', apikey='phony-key', user_agent=ua) +try: + pkg = demo.action.package_create(name='my-dataset', title='not going to work') +except NotAuthorized: + print('denied') +``` -### Functions +When it is possible to `import ckan` all the ckanapi exception classes are +replaced with the CKAN exceptions with the same names. -#### Connect -- `connect_to_ckan(ckan_site_url, api_token)`: Connects to the CKAN instance. +### File Uploads -#### Create +File uploads for CKAN 2.2+ are supported by passing file-like objects to action +shortcut methods: -- `create_organizations(rc, json_path)`: Creates new organizations in CKAN from a JSON file. -- `create_groups(rc, json_path)`: Creates new groups in CKAN from a JSON file. -- `create_users(rc, json_path)`: Creates new users in CKAN from a JSON file. +```python +from ckanapi import RemoteCKAN +ua = 'ckanapiexample/1.0 (+http://example.com/my/website)' -#### Update +mysite = RemoteCKAN('http://myckan.example.com', apikey='real-key', user_agent=ua) +mysite.action.resource_create( + package_id='my-dataset-with-files', + url='dummy-value', # ignored but required by CKAN<2.6 + upload=open('/path/to/file/to/upload.csv', 'rb')) +``` -- `update_package(rc, package_id, theme_es, publisher_name, publisher_type)`: Updates a package given its ID. +When using `call_action` you must pass file objects separately: -#### Get +```python +mysite.call_action('resource_create', + {'package_id': 'my-dataset-with-files'}, + files={'upload': open('/path/to/file/to/upload.csv', 'rb')}) +``` + + +### List all private datasets +```py +from ckanapi import RemoteCKAN, ValidationError, SearchQueryError, SearchError, CKANAPIError, ServerIncompatibleError +import requests + +ua = 'ckanapiexample/1.0 (+http://example.com/my/website)' + +# Configura tu instancia de CKAN y tu API key +ckan_instance = RemoteCKAN('http://myckan.example.com', apikey='api_key', user_agent=ua) + +try: + # Realiza la búsqueda de paquetes + result = ckan_instance.action.package_search( + q='*:*', + include_private=True, + include_drafts=True + ) + print(f"Datasets: {result['count']}") +except (ValidationError, SearchQueryError, SearchError, CKANAPIError, ServerIncompatibleError) as e: + print(f"Error: {e}") +``` + +### Session Control + +As of ckanapi 4.0 RemoteCKAN will keep your HTTP connection open using a +[requests session](http://docs.python-requests.org/en/master/user/advanced/). + +For long-running scripts make sure to close your connections by using +RemoteCKAN as a context manager: -- `export_groups_to_json(rc, file_path)`: Exports all groups to a JSON file. -- `export_organizations_to_json(rc, file_path)`: Exports all organizations to a JSON file. +```python +from ckanapi import RemoteCKAN +ua = 'ckanapiexample/1.0 (+http://example.com/my/website)' + +with RemoteCKAN('https://demo.ckan.org', user_agent=ua) as demo: + groups = demo.action.group_list(id='data-explorer') +print(groups) +``` + +Or by explicitly calling `RemoteCKAN.close()`. + +### LocalCKAN + +A similar class is provided for accessing local CKAN instances from a plugin in +the same way as remote CKAN instances. +Unlike [CKAN's get_action](http://docs.ckan.org/en/latest/extensions/plugins-toolkit.html?highlight=get_action#ckan.plugins.toolkit.get_action) +LocalCKAN prevents data from one action +call leaking into the next which can cause issues that are very hard do debug. + +This class defaults to using the site user with full access. + +```python +from ckanapi import LocalCKAN, ValidationError + +registry = LocalCKAN() +try: + registry.action.package_create(name='my-dataset', title='this will work fine') +except ValidationError: + print('unless my-dataset already exists') +``` + +For extra caution pass a blank username to LocalCKAN and only actions allowed +by anonymous users will be permitted. + +```python +from ckanapi import LocalCKAN + +anon = LocalCKAN(username='') +print(anon.action.status_show()) +``` + +#### Extra Loggging + +To enable extra info logging for the execution of LocalCKAN ckanapi commands, you can enable the config option in your CKAN INI file. + +``` +ckanapi.log_local = True +``` + +The output of the log will look like: + +``` +INFO [ckan.ckanapi] OS User executed LocalCKAN: ckanapi +``` + +### TestAppCKAN + +A class is provided for making action requests to a +[webtest.TestApp](http://webtest.readthedocs.org/en/latest/testapp.html) +instance for use in CKAN tests: + +```python +from ckanapi import TestAppCKAN +from webtest import TestApp + +test_app = TestApp(...) +demo = TestAppCKAN(test_app, apikey='my-test-key') +groups = demo.action.group_list(id='data-explorer') +``` -### Example Output +## Tests -- `groups.json`: Contains detailed information about all groups. -- `organizations.json`: Contains detailed information about all organizations. +To run the tests: -## License + python setup.py test -This project is licensed under the MIT License.