Merge branch '1.x' into backport-readonly-fix

DEVELOPER_GUIDE.md

@@ -9,16 +9,16 @@ So you want to contribute code to this project? Excellent! We're glad you're her
 ## Prerequisites
 This project runs as a plugin of OpenSearch. You could [download a minimal release of OpenSearch](https://opensearch.org/downloads.html#minimal) and then install the plugin there, but at this time there are no MacOS distributions (although you can run it as a docker container, but those images come with the security plugin already preinstalled). In addition, it might come in handy to be able to always run against the latest commit of the version under development, so we'll be compiling it from source instead.
 
-To get started, follow the [getting started section](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#getting-started) of OpenSearch's developer guide. This will get you up and running with OpenSearch built from source code. Get to the point where you can run a successful `curl localhost:9200` call (you can skip the `./gradlew check` step to save some time. Great! now kill the server with `Ctrl+c` and copy the built code to a new folder somewhere else where you'll be running your service (run this from the base directory of the OpenSearch fork you cloned above):
+To get started, follow the [getting started section](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#getting-started) of OpenSearch's developer guide. This will get OpenSearch up and running built from source code. You can skip the `./gradlew check` step to save some time. You should follow the steps until you reach the point where you can run a successful `curl localhost:9200` call. Great! now kill the server with `Ctrl+C`.
+
+Next, inside `OpenSearch` folder run the following commands to copy the built code (snapshot) to a new folder in a different location (this where you'll be running the OpenSearch service). Here **`darwin-tar`** is an example running on MacOS, adjust `$OPENSEARCH_BUILD` path based on your version and Operating System.
 
 ```bash
-export OPENSEARCH_HOME=~/Test/opensearch-1.3.0-SNAPSHOT
-export OPENSEARCH_BUILD=distribution/archives/darwin-tar/build/install/opensearch-1.3.0-SNAPSHOT
-cp -Rf $OPENSEARCH_BUILD $OPENSEARCH_HOME
+export OPENSEARCH_HOME=`pwd`/opensearch-$(./gradlew properties -q | grep -E '^version:' | awk '{print $2}' | sed 's/-SNAPSHOT//g')
+export OPENSEARCH_BUILD=distribution/archives/darwin-tar/build/install/opensearch-$(./gradlew properties -q | grep -E '^version:' | awk '{print $2}')
+cp -Rf $OPENSEARCH_BUILD/* $OPENSEARCH_HOME
 ```
 
-Choose `$OPENSEARCH_HOME` as the base folder where your server will live, and adjust `$OPENSEARCH_BUILD` based on your version and OS (this is an example running on MacOS.)
-
 Let's test it!
 
 ```bash
@@ -42,15 +42,101 @@ Install the built plugin into the OpenSearch server:
 
 ```bash
 export OPENSEARCH_SECURITY_HOME=$OPENSEARCH_HOME/plugins/opensearch-security
-cp build/distributions/opensearch-security-1.3.0.0-SNAPSHOT.zip $OPENSEARCH_SECURITY_HOME
+mkdir -p $OPENSEARCH_SECURITY_HOME
+cp build/distributions/opensearch-security-*.zip $OPENSEARCH_SECURITY_HOME
 cd $OPENSEARCH_SECURITY_HOME
-unzip opensearch-security-1.3.0.0-SNAPSHOT.zip
-rm opensearch-security-1.3.0.0-SNAPSHOT.zip
+unzip opensearch-security-*.zip
+rm opensearch-security-*.zip
+mkdir -p $OPENSEARCH_HOME/config/opensearch-security
+mv config/* $OPENSEARCH_HOME/config/opensearch-security/
+rm -rf config/
 ```
 
-You will have to adjust it to your specific version. In this example I'm using `1.3.0.0-SNAPSHOT`.
+### Refreshing demo certificates
+
+1. Use the following commands to generate new demo certificates:
+
+```zsh
+## ROOT
+
+openssl genrsa -out root-ca-key.pem 2048
+openssl req -new -x509 -sha256 -key root-ca-key.pem -subj "/DC=com/DC=example/O=Example Com Inc./OU=Example Com Inc. Root CA/CN=Example Com Inc. Root CA" -addext "basicConstraints = critical,CA:TRUE" -addext "keyUsage = critical, digitalSignature, keyCertSign, cRLSign" -addext "subjectKeyIdentifier = hash" -addext "authorityKeyIdentifier = keyid:always,issuer:always" -out root-ca.pem
+
+
+## NODE
+
+openssl genrsa -out esnode-key-temp.pem 2048
+openssl pkcs8 -inform PEM -outform PEM -in esnode-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out esnode-key.pem
+openssl req -new -key esnode-key.pem -subj "/C=de/L=test/O=node/OU=node/CN=node-0.example.com" -out esnode.csr
+openssl x509 -req -in esnode.csr -out esnode.pem -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -days 3650 -extfile <(printf "subjectAltName = RID:1.2.3.4.5.5, DNS:node-0.example.com, DNS:localhost, IP:::1, IP:127.0.0.1\nkeyUsage = digitalSignature, nonRepudiation, keyEncipherment\nextendedKeyUsage = serverAuth, clientAuth\nbasicConstraints = critical,CA:FALSE")
+
+
+## ADMIN
+
+openssl req -new -newkey rsa:2048 -keyout kirk-key.pem -out kirk.csr -nodes -subj "/C=de/L=test/O=client/OU=client/CN=kirk"
+openssl x509 -req -in kirk.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -out kirk.pem -days 3650 -extfile <(printf "basicConstraints = critical,CA:FALSE\nkeyUsage = critical,digitalSignature,nonRepudiation,keyEncipherment\nextendedKeyUsage = critical,clientAuth\nauthorityKeyIdentifier = keyid,issuer:always\nsubjectKeyIdentifier = hash")
+
+## Remove root-ca-key.pem and other temp keys
+
+## Generate new jks for sanity-tests which use demo certs
+#### kirk-root-chain.pem is chain certificate of kirk.pem followed by root-ca.pem
+openssl pkcs12 -export -in kirk-root-chain.pem -inkey kirk-key.pem -out kirk.p12 -name kirk
+keytool -importkeystore -srckeystore kirk.p12 -srcstoretype PKCS12 -destkeystore kirk.jks -deststoretype JKS
+```
+
+2. Update `install_demo_configuration.sh` and `install_demo_configuration.bat` with these new certificates.
+3. Add the SHA256 hashes for newly generated certs in OpenSearchSecurityPlugin.java
+```zsh
+cd <cert-folder>
+cat <cert>.pem | sha256sum
+```
+
+### Installing demo extension users and roles
+
+If you are working with an extension and want to set up demo users for the Hello-World extension, append following items to files inside `$OPENSEARCH_HOME/config/opensearch-security/`:
+1. In **internal_users.yml**
+```yaml
+hw-user:
+  hash: "$2a$12$VcCDgh2NDk07JGN0rjGbM.Ad41qVR/YFJcgHp0UGns5JDymv..TOG"
+  reserved: true
+  description: "Demo user for ext-test"
+```
+
+2. In **roles.yml**
+```yaml
+extension_hw_greet:
+  reserved: true
+  cluster_permissions:
+    - 'hw:greet'
+
+extension_hw_full:
+  reserved: true
+  cluster_permissions:
+    - 'hw:goodbye'
+    - 'hw:greet'
+    - 'hw:greet_with_adjective'
+    - 'hw:greet_with_name'
+
+legacy_hw_greet_with_name:
+  reserved: true
+  cluster_permissions:
+    - 'cluster:admin/opensearch/hw/greet_with_name'
+```
+
+3. In **roles_mapping.yml**
+```yaml
+legacy_hw_greet_with_name:
+  reserved: true
+  users:
+    - "hw-user"
+
+extension_hw_greet:
+  reserved: true
+  users:
+    - "hw-user"
+```
 
-Install the demo certificates and default configuration, answer `y` to the first two questions and `n` to the last one:
+To install the demo certificates and default configuration, answer `y` to the first two questions and `n` to the last one. The log should look like below:
 
 ```bash
 ./tools/install_demo_configuration.sh