From 2cf0732a9bc0c2a51887469e7098bbcb09ee73dc Mon Sep 17 00:00:00 2001 From: Alexei Karikov Date: Sun, 9 Apr 2023 00:52:34 +0600 Subject: [PATCH 1/5] Create index lifecycle guide Signed-off-by: Alexei Karikov --- guides/index_lifecycle.md | 157 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 157 insertions(+) create mode 100644 guides/index_lifecycle.md diff --git a/guides/index_lifecycle.md b/guides/index_lifecycle.md new file mode 100644 index 00000000..7569caba --- /dev/null +++ b/guides/index_lifecycle.md @@ -0,0 +1,157 @@ +# Index Lifecycle + +This guide covers OpenSearch Python Client API actions for Index Lifecycle. You'll learn how to create, read, update, and delete indices in your OpenSearch cluster. We will also leverage index templates to create default settings and mappings for indices of certain patterns. + +## Setup + +In this guide, we will need an OpenSearch cluster with more than one node. Let's use the sample [docker-compose.yml](https://opensearch.org/samples/docker-compose.yml) to start a cluster with two nodes. The cluster's API will be available at `localhost:9200` with basic authentication enabled with default username and password of `admin:admin`. + +To start the cluster, run the following command: + +```bash +cd /path/to/docker-compose.yml +docker-compose up -d +``` + +Let's create a client instance to access this cluster: + +```python +from opensearchpy import OpenSearch + +client = OpenSearch( + hosts=['https://admin:admin@localhost:9200'], + use_ssl=True, + verify_certs=False +) + +print(client.info()) # Check server info and make sure the client is connected +``` + +## Index API Actions + +### Create a new index + +You can quickly create an index with default settings and mappings by using the `indices.create` API action. The following example creates an index named `paintings` with default settings and mappings: + +```python +client.indices.create(index='paintings') +``` + +To specify settings and mappings, you can pass them as the `body` of the request. The following example creates an index named `movies` with custom settings and mappings: + +```python +client.indices.create( + index='movies', + body={ + 'settings': { + 'index': { + 'number_of_shards': 2, + 'number_of_replicas': 1 + } + }, + 'mappings': { + 'properties': { + 'title': {'type': 'text'}, + 'year': {'type': 'integer'} + } + } + } +) +``` + +When you create a new document for an index, OpenSearch will automatically create the index if it doesn't exist: + +```python +print(client.indices.exists(index='burner')) # => False +client.create(index='burner', body={'lorem': 'ipsum'}) +print(client.indices.exists(index='burner')) # => True +``` + +### Update an Index + +You can update an index's settings and mappings by using the `indices.put_settings` and `indices.put_mapping` API actions. + +The following example updates the `movies` index's number of replicas to `0`: + +```python +client.indices.put_settings( + index='movies', + body={ + 'index': { + 'number_of_replicas': 0 + } + } +) +``` + +The following example updates the `movies` index's mappings to add a new field named `director`: + +```python +client.indices.put_mapping( + index='movies', + body={ + 'properties': { + 'director': {'type': 'text'} + } + } +) +``` + +### Get Metadata for an Index + +Let's check if the index's settings and mappings have been updated by using the `indices.get` API action: + +```python +print(client.indices.get(index='movies')) +``` + +The response body contains the index's settings and mappings: + +```python +{ + "movies": { + "aliases": {}, + "mappings": { + "properties": { + "title": {"type": "text"}, + "year": {"type": "integer"}, + "director": {"type": "text"} + } + }, + "settings": { + "index": { + "creation_date": "1680297372024", + "number_of_shards": "2", + "number_of_replicas": "0", + "uuid": "FEDWXgmhSLyrCqWa8F_aiA", + "version": {"created": "136277827"}, + "provided_name": "movies" + } + } + } +} +``` + +### Delete an Index + +Let's delete the `movies` index by using the `indices.delete` API action: + +```python +client.indices.delete(index='movies') +``` + +We can also delete multiple indices at once: + +```python +client.indices.delete(index=['movies', 'paintings', 'burner'], ignore=[404]) +``` + +Notice that we are passing `ignore: 404` to the request. This tells the client to ignore the `404` error if the index doesn't exist for deletion. Without it, the above `delete` request will throw an error because the `movies` index has already been deleted in the previous example. + +## Cleanup + +All resources created in this guide are automatically deleted when the cluster is stopped. You can stop the cluster by running the following command: + +```bash +docker-compose down +``` From 7c4d0e212b1a818632daff77a3fb1e22daa67eeb Mon Sep 17 00:00:00 2001 From: Alexei Karikov Date: Thu, 20 Apr 2023 00:34:37 +0600 Subject: [PATCH 2/5] Add changelog entry Signed-off-by: Alexei Karikov --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index b38fd86b..c2855020 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) ## [Unreleased] ### Added - Added async support for helpers that are merged from opensearch-dsl-py ([#329](https://github.com/opensearch-project/opensearch-py/pull/329)) +- Added index lifecycle guide ([#362](https://github.com/opensearch-project/opensearch-py/pull/362)) ### Changed - Upgrading pytest-asyncio to latest version - 0.21.0 ([#339](https://github.com/opensearch-project/opensearch-py/pull/339)) ### Deprecated From 66290287a2f3a43139bc11fb14f549c16a8eda47 Mon Sep 17 00:00:00 2001 From: Alexei Karikov Date: Fri, 21 Apr 2023 00:57:33 +0600 Subject: [PATCH 3/5] Replace ignore param and add short corrections Signed-off-by: Alexei Karikov --- guides/index_lifecycle.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/guides/index_lifecycle.md b/guides/index_lifecycle.md index 7569caba..5b3bed18 100644 --- a/guides/index_lifecycle.md +++ b/guides/index_lifecycle.md @@ -63,7 +63,7 @@ When you create a new document for an index, OpenSearch will automatically creat ```python print(client.indices.exists(index='burner')) # => False -client.create(index='burner', body={'lorem': 'ipsum'}) +client.index(id='1', index='burner', body={'lorem': 'ipsum'}) print(client.indices.exists(index='burner')) # => True ``` @@ -143,10 +143,10 @@ client.indices.delete(index='movies') We can also delete multiple indices at once: ```python -client.indices.delete(index=['movies', 'paintings', 'burner'], ignore=[404]) +client.indices.delete(index=['movies', 'paintings', 'burner'], ignore_unavailable=True) ``` -Notice that we are passing `ignore: 404` to the request. This tells the client to ignore the `404` error if the index doesn't exist for deletion. Without it, the above `delete` request will throw an error because the `movies` index has already been deleted in the previous example. +Notice that we are passing `ignore_unavailable=True` to the request. This tells the client to ignore the `404` error if the index doesn't exist for deletion. Without it, the above `delete` request will throw an error because the `movies` index has already been deleted in the previous example. ## Cleanup From a7123051bb59979c394e503fac8a743a73e75ca6 Mon Sep 17 00:00:00 2001 From: Alexei Karikov Date: Sat, 22 Apr 2023 01:00:15 +0600 Subject: [PATCH 4/5] reset spaces Signed-off-by: Alexei Karikov --- CHANGELOG.md | 44 +------------------------------------------- 1 file changed, 1 insertion(+), 43 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ea87450b..882bcc77 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,92 +1,56 @@ # CHANGELOG - Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) ## [Unreleased] - ### Added - - Added async support for helpers that are merged from opensearch-dsl-py ([#329](https://github.com/opensearch-project/opensearch-py/pull/329)) - Added search.md to guides ([#356](https://github.com/opensearch-project/opensearch-py/pull/356)) - Added index lifecycle guide ([#362](https://github.com/opensearch-project/opensearch-py/pull/362)) - ### Changed - - Upgrading pytest-asyncio to latest version - 0.21.0 ([#339](https://github.com/opensearch-project/opensearch-py/pull/339)) - ### Deprecated - ### Removed - ### Fixed - - Fixed import cycle when importing async helpers ([#311](https://github.com/opensearch-project/opensearch-py/pull/311)) - Fixed userguide for async client ([#340](https://github.com/opensearch-project/opensearch-py/pull/340)) - ### Security - - Fixed CVE-2022-23491 reported in opensearch-dsl-py ([#295](https://github.com/opensearch-project/opensearch-py/pull/295)) - Update ci workflows ([#318](https://github.com/opensearch-project/opensearch-py/pull/318)) ## [2.2.0] - ### Added - - Merging opensearch-dsl-py into opensearch-py ([#287](https://github.com/opensearch-project/opensearch-py/pull/287)) - Added upgrading.md file and updated it for opensearch-py 2.2.0 release ([#293](https://github.com/opensearch-project/opensearch-py/pull/293)) - ### Changed - ### Deprecated - ### Removed - - Removed 'out/opensearchpy' folder which was produced while generating pyi files for plugins ([#288](https://github.com/opensearch-project/opensearch-py/pull/288)) - Removed low-level and high-level client terminology from guides ([#298](https://github.com/opensearch-project/opensearch-py/pull/298)) - ### Fixed - - Fixed CVE - issue 86 mentioned in opensearch-dsl-py repo ([#295](https://github.com/opensearch-project/opensearch-py/pull/295)) - ### Security ## [2.1.1] - ### Added - ### Changed - ### Deprecated - ### Removed - ### Fixed - - Fixed SigV4 Signing for Managed Service ([#279](https://github.com/opensearch-project/opensearch-py/pull/279)) - Fixed SigV4 Signing for Async Requests with QueryStrings ([#272](https://github.com/opensearch-project/opensearch-py/pull/279)) - ### Security ## [2.1.0] - ### Added - - Added Support for AOSS ([#268](https://github.com/opensearch-project/opensearch-py/pull/268)) - ### Changed - ### Deprecated - ### Removed - ### Fixed - ### Security ## [2.0.1] - ### Added - - Added Point in time API rest API([#191](https://github.com/opensearch-project/opensearch-py/pull/191)) - Added pool_maxsize for RequestsHttpConnection ([#216](https://github.com/opensearch-project/opensearch-py/pull/216)) - Github workflow for changelog verification ([#218](https://github.com/opensearch-project/opensearch-py/pull/218)) @@ -96,27 +60,21 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) - Added SigV4 support for Async Opensearch Client ([#254](https://github.com/opensearch-project/opensearch-py/pull/254)) - Compatibility with OpenSearch 2.1.0 - 2.4.1 ([#257](https://github.com/opensearch-project/opensearch-py/pull/257)) - Adding explicit parameters for AIOHttpConnection and AsyncTransport ([#276](https://github.com/opensearch-project/opensearch-py/pull/276)) - ### Changed - - Updated getting started to user guide ([#233](https://github.com/opensearch-project/opensearch-py/pull/233)) - Updated CA certificate handling to check OpenSSL environment variables before defaulting to certifi ([#196](https://github.com/opensearch-project/opensearch-py/pull/196)) - Updates `master` to `cluster_manager` to be inclusive ([#242](https://github.com/opensearch-project/opensearch-py/pull/242)) - Support a custom signing service name for AWS SigV4 ([#268](https://github.com/opensearch-project/opensearch-py/pull/268)) - Updated CI tests to make them work locally ([#275](https://github.com/opensearch-project/opensearch-py/pull/275)) - ### Deprecated ### Removed - - Removed patch versions in integration tests for OpenSearch 1.0.0 - 2.3.0 to reduce Github Action jobs ([#262](https://github.com/opensearch-project/opensearch-py/pull/262)) - ### Fixed - - Fixed DeprecationWarning emitted from urllib3 1.26.13+ ([#246](https://github.com/opensearch-project/opensearch-py/pull/246)) - ### Security + [Unreleased]: https://github.com/opensearch-project/opensearch-py/compare/v2.2.0...HEAD [2.0.1]: https://github.com/opensearch-project/opensearch-py/compare/v2.0.0...v2.0.1 [2.1.0]: https://github.com/opensearch-project/opensearch-py/compare/v2.0.1...v2.1.0 From b852a88b2a898b468ffa58aeb52cf80b37bd94ce Mon Sep 17 00:00:00 2001 From: Alexei Karikov Date: Mon, 24 Apr 2023 23:22:59 +0600 Subject: [PATCH 5/5] Correct indents Signed-off-by: Alexei Karikov --- guides/index_lifecycle.md | 60 +++++++++++++++++++-------------------- 1 file changed, 30 insertions(+), 30 deletions(-) diff --git a/guides/index_lifecycle.md b/guides/index_lifecycle.md index 5b3bed18..3a9bf86d 100644 --- a/guides/index_lifecycle.md +++ b/guides/index_lifecycle.md @@ -19,9 +19,9 @@ Let's create a client instance to access this cluster: from opensearchpy import OpenSearch client = OpenSearch( - hosts=['https://admin:admin@localhost:9200'], - use_ssl=True, - verify_certs=False + hosts=['https://admin:admin@localhost:9200'], + use_ssl=True, + verify_certs=False ) print(client.info()) # Check server info and make sure the client is connected @@ -41,21 +41,21 @@ To specify settings and mappings, you can pass them as the `body` of the request ```python client.indices.create( - index='movies', - body={ - 'settings': { - 'index': { - 'number_of_shards': 2, - 'number_of_replicas': 1 - } - }, - 'mappings': { - 'properties': { - 'title': {'type': 'text'}, - 'year': {'type': 'integer'} - } - } - } + index='movies', + body={ + 'settings': { + 'index': { + 'number_of_shards': 2, + 'number_of_replicas': 1 + } + }, + 'mappings': { + 'properties': { + 'title': {'type': 'text'}, + 'year': {'type': 'integer'} + } + } + } ) ``` @@ -75,12 +75,12 @@ The following example updates the `movies` index's number of replicas to `0`: ```python client.indices.put_settings( - index='movies', - body={ - 'index': { - 'number_of_replicas': 0 - } - } + index='movies', + body={ + 'index': { + 'number_of_replicas': 0 + } + } ) ``` @@ -88,12 +88,12 @@ The following example updates the `movies` index's mappings to add a new field n ```python client.indices.put_mapping( - index='movies', - body={ - 'properties': { - 'director': {'type': 'text'} - } - } + index='movies', + body={ + 'properties': { + 'director': {'type': 'text'} + } + } ) ```