Merge pull request #197 from PasanT9/4.2.0.1-release

Add APIM Distributed pattern

patterns/distributed/README.md

@@ -0,0 +1,160 @@
+# WSO2 API Management Ansible scripts
+
+This repository contains the Ansible scripts for installing and configuring WSO2 API Management.
+
+## Supported Operating Systems
+
+- Ubuntu 16.04 or higher
+- CentOS 7
+
+## Supported Ansible Versions
+
+- Ansible 2.5 or higher
+
+## Directory Structure
+```
+.
+├── dev
+│   ├── group_vars
+│   │   └──apim.yml
+│   ├── host_vars
+│   │   ├── apim-control-plane_1.yml
+│   │   ├── apim-gateway_1.yml
+│   │   └── apim-tm_1.yml
+│   └── inventory
+├── files
+│   ├── lib
+│   │   ├── amazon-corretto-17.0.6.10.1-linux-x64.tar.gz
+│   │   └── mysql-connector-j-8.0.31.jar
+│   └── packs
+│   │   └── wso2am-4.2.0.zip
+│   ├── system
+│   │   └── etc
+│   │       ├── security
+│   │       │   └── limits.conf
+│   │       └── sysctl.conf
+│   └── misc
+├── README.md
+├── roles
+│   ├── apim
+│   │   ├── tasks
+│   │   └── templates
+│   ├── apim-control-plane
+│   │   ├── tasks
+│   │   └── templates
+│   ├── apim-tm
+│   │   ├── tasks
+│   │   └── templates
+│   └── common
+│       └── tasks
+├── scripts
+│   ├── update.sh
+│   └── update_README.md
+└── site.yml
+
+```
+
+Following instructions can be followed to deploy a distributed APIM deployment pattern.
+
+
+## Copying packs locally
+Packs could be either copied to a local directory, or downloaded from a remote location.
+
+Copy the following files to `files/packs` directory.
+
+1. [WSO2 API Manager 4.2.0 package](https://wso2.com/api-management/install/)
+
+Copy the following files to `files/lib` directory.
+
+2. [Amazon Corretto for Linux x64 JDK](https://docs.aws.amazon.com/corretto/latest/corretto-17-ug/downloads-list.html)
+
+Copy the miscellaneous files to `files/misc` directory. To enable file copying,  uncomment the `misc_file_list` in the yaml files under `group_vars` and add the miscellaneous files to the list.
+
+## Downloading from remote location
+
+In **group_vars**, change the values of the following variables in all groups:
+1. The value of `pack_location` should be changed from "local" to "remote"
+2. The value of `remote_jdk` should be changed to the URL in which the JDK should be downloaded from, and remove it as a comment.
+3. The value of `remote_pack` should be changed to the URL in which the package should be downloaded from, and remove it as a comment.
+
+## Running Ansible scripts
+1. Configure an Ansible setup with three hosts.
+2. Replace `ansible_host` and `ansible_user` configurations given in `dev/inventory` file according to your Ansible hosts. An example is given below.
+
+```
+[apim]
+apim-tm_1               ansible_host=tm.apim.com     ansible_user=ubuntu
+apim-gateway_1          ansible_host=gw.apim.com     ansible_user=ubuntu
+apim-control-plane_1    ansible_host=cp.apim.com     ansible_user=ubuntu
+```
+
+3. Download the relevant JDBC driver into `files/lib` directory.
+4. Open `dev/host_vars/apim.yaml` and update the following configurations.
+  - Change the hostnames according to you Ansible hosts.
+  - Change the DB configurations.
+  - Change the JDBC driver name.
+
+5. Run the following command to execute playbook.
+
+`ansible-playbook -i dev site.yml`
+
+If you need to alter the configurations given, please change the parameterized values in the yaml files under `group_vars` and `host_vars`.
+
+**NOTE:**
+> If you have mounted the 'persistent artifacts' as guided [below](##configuration-guide), make sure to unmount the artifacts, prior to running the Ansible playbook as the playbook running process has a step to remove the existing setup. After completing the Ansible playbook running process, make sure to remount the artifacts.
+
+> If the `client-truststore.jks` is monuted among the Gateway nodes, then make sure to copy the `client-truststore.jks` from the mount source to the `<ANSIBLE_HOME>/files/security/wso2am/` directory of the Ansible resources, prior to re-running the playbook.
+
+### 2. Customize the WSO2 Ansible scripts
+
+The templates that are used by the Ansible scripts are in j2 format in-order to enable parameterization.
+
+The `deployment.toml.j2` file is added under `roles/apim-<profile>/templates/carbon-home/repository/conf/`, in order to enable customizations. You can add any other customizations to `custom.yml` under tasks of each role as well.
+
+#### Step 1
+Uncomment the following line in `main.yml` under the role you want to customize.
+```
+- import_tasks: custom.yml
+```
+
+#### Step 2
+Add the configurations to the `custom.yml`. A sample is given below.
+
+```
+- name: "Copy custom file"
+  template:
+    src: path/to/example/file/example.xml.j2
+    dest: destination/example.xml.j2
+  when: "(inventory_hostname in groups['am'])"
+```
+
+Follow the steps mentioned under `docs` directory to customize/create new Ansible scripts and deploy the recommended patterns.
+
+#### Including custom Keystore and Truststore
+If custom keystores and truststores are needed to be added, uncomment the below list in the yml file
+```
+# security_file_list:
+#   - { src: '{{ security_file_location }}/wso2am-analytics/client-truststore.jks',
+#       dest: '{{ carbon_home }}/resources/security/client-truststore.jks' }
+#   - { src: '{{ security_file_location }}/wso2am-analytics/wso2carbon.jks',
+#       dest: '{{ carbon_home }}/resources/security/wso2carbon.jks' }
+```
+Then save the changed file and add the required files under `files/security/<product-home>/<path-to-file>`
+
+## Performance Tuning
+
+System configurations can be changed through Ansible to optimize OS level performance. Performance tuning can be enabled by changing `enable_performance_tuning` in `dev/group_vars/apim.yml` to `true`.
+
+System files that will be updated when performance tuning are enabled is available in `files/system`. Update the configuration values according to the requirements of your deployment.
+
+## Configuration Guide
+
+Refer the below documentation on configuring key-stores for APIM and APIM-Analytics
+1. [WSO2 API Manager key-stores configuration guide](https://apim.docs.wso2.com/en/latest/install-and-setup/setup/security/configuring-keystores/configuring-keystores-in-wso2-api-manager/)
+
+Refer the below documentation on configuring persistent artifacts of the servers.
+1. [Persistent artifacts of the servers](https://apim.docs.wso2.com/en/latest/install-and-setup/setup/reference/common-runtime-and-configuration-artifacts/)
+
+Refer the below documentation on configuring Load-Balancers for your deoloyment.
+1. [Load balancer configurations](https://apim.docs.wso2.com/en/latest/install-and-setup/setup/setting-up-proxy-server-and-the-load-balancer/configuring-the-proxy-server-and-the-load-balancer/)
+

patterns/distributed/common-tasks/mi-secure-vault.yml

@@ -0,0 +1,7 @@
+- name: Run Secure Vault to Encrypt passwords
+  command: "{{ carbon_home }}/bin/ciphertool.sh -Dconfigure"
+  environment:
+    JAVA_HOME: "{{ java_home }}"
+  args:
+    chdir: "{{ carbon_home }}"
+  when: mi_secure_vault_enabled|bool

patterns/distributed/common-tasks/secure-vault.yml

@@ -0,0 +1,21 @@
+
+- name: Run Secure Vault to Encrypt passwords
+  command: "{{ carbon_home }}/bin/ciphertool.sh -Dconfigure -Dpassword={{ internal_key_store_password }}"
+  environment:
+    JAVA_HOME: "{{ java_home }}"
+  args: 
+    chdir: "{{ carbon_home }}" 
+  when: secure_vault_enabled|bool
+
+- name: Creating a file with content
+  copy:
+    dest: "{{ carbon_home }}/password-tmp"
+    content: |
+      {{ internal_key_store_password }}
+
+- name: Change the owner of WSO2 directory
+  file:
+    path: "{{ carbon_home }}/password-tmp"
+    owner: "{{ wso2_user }}"
+    group: "{{ wso2_group }}"
+    mode: 0755

patterns/distributed/dev/group_vars/apim.yml

@@ -0,0 +1,173 @@
+# ----------------------------------------------------------------------------
+#  Copyright (c) 2023 WSO2, LLC. http://www.wso2.org
+#
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+# ----------------------------------------------------------------------------
+
+cp_hostname: cp.apim.com
+gw_hostname: gw.apim.com
+tm_hostname: tm.apim.com
+
+wso2_group: wso2 # OS group to be created
+wso2_user: wso2carbon # OS user for WSO2 services
+product_name: wso2am
+product_version: 4.2.0
+target: /mnt # Product installation directory
+product_package_location: files 
+backup_dir: /tmp # Artifact backup directory in the instance
+misc_file_location: "{{ product_package_location }}/misc"
+misc_file_copy_location: "{{ target }}/misc"
+security_file_location: "{{ product_package_location }}/security"
+#If you uncomment the below line to set the jdbc driver, change the driver name accordingly
+jdbc_driver: mysql-connector-j-8.0.31.jar
+
+# Set backup_enabled to true to backup <carbon-home>/repository/deployment/server directory
+backup_enabled: false
+
+# Performance tuning configurations for the operating system
+enable_performance_tuning: false
+performance_tuning_file_list:
+  - { src: '{{ product_package_location }}/system/etc/sysctl.conf',
+      dest: '/etc/sysctl.conf' }
+  - { src: '{{ product_package_location }}/system/etc/security/limits.conf',
+      dest: '/etc/security/limits.conf' }
+
+# If custom primary keystores and trsustrores are need to be added, uncomment the below list
+# Add the required files under files/security/<product-home>/<path-to-file>
+# A sample is given below
+# security_file_list:
+#   - { src: '{{ security_file_location }}/wso2am/client-truststore.jks',
+#       dest: '{{ carbon_home }}/repository/resources/security/client-truststore.jks' }
+#   - { src: '{{ security_file_location }}/wso2am/wso2carbon.jks',
+#       dest: '{{ carbon_home }}/repository/resources/security/wso2carbon.jks' }
+
+# If custom tls keystore needs to be added, uncomment the below list
+# Add the required files under files/security/<product-home>/<path-to-file>
+# A sample is given below
+# security_file_list:
+#   - { src: '{{ security_file_location }}/wso2am/tls.jks',
+#       dest: '{{ carbon_home }}/repository/resources/security/tls.jks' }
+
+# If custom internal keystore needs to be added, uncomment the below list
+# Add the required files under files/security/<product-home>/<path-to-file>
+# A sample is given below
+# security_file_list:
+#   - { src: '{{ security_file_location }}/wso2am/internal.jks',
+#       dest: '{{ carbon_home }}/repository/resources/security/internal.jks' }
+
+# Uncomment and add any miscellaneous files you need to copy
+# A sample is given below
+# misc_file_list:
+#   - { src: '{{ misc_file_location }}/apictl-4.2.0-linux-amd64.tar.gz',
+#       dest: '{{ misc_file_copy_location }}/apictl-4.2.0-linux-amd64.tar.gz' }
+
+# Set the location the product packages should reside in (eg: "local" in the /files directory, "remote" in a remote location)
+pack_location: local
+#pack_location: remote
+#remote_jdk: "<URL_TO_JDK_FILE>"
+#remote_pack: "<URL_TO_APIM_PACK>"
+
+# JDK Distributions
+setup_java_enabled: true #If this is set to false java_home variable below needs to be specified.
+jdk_name: amazon-corretto-17.0.6.10.1-linux-x64 # Update this as per jdk version you use
+java_dir: /opt
+java_symlink: "{{ java_dir }}/java"
+java_home: "{{ java_dir }}/{{ jdk_name }}"
+
+# Server URL of the Authentication service. Make sure to import the Key Manager's public certificate to WSO2 API-M's
+# client-truststore.jks. For more information, see https://docs.wso2.com/display/ADMIN44x/Creating+New+Keystores
+key_manager_server_url: https://{{ cp_hostname }}:9443/services/
+
+# API Store related configurations
+api_devportal_url: https://{{ cp_hostname }}:9443/devportal
+api_devportal_server_url: https://{{ cp_hostname }}:9443/services/
+
+# Gateway configurations
+gateway_environments:
+  - { type: 'hybrid',
+      name: 'Default',
+      description: 'This is a hybrid gateway that handles both production and sandbox token traffic.',
+      service_url: 'https://{{ gw_hostname }}:9443/services/',
+      ws_endpoint: 'ws://{{ gw_hostname }}:9099',
+      wss_endpoint: 'wss://{{ gw_hostname }}:8099',
+      http_endpoint: 'http://{{ gw_hostname }}:8280',
+      https_endpoint: 'https://{{ gw_hostname }}:8243',
+      websub_event_receiver_http_endpoint: "http://{{ gw_hostname }}:9021",
+      websub_event_receiver_https_endpoint: "https://{{ gw_hostname }}:8021"}
+
+sync_runtime_gateway_labels: ["Default"]
+
+# Traffic Manager configurations
+traffic_manager_host_url: https://{{ tm_hostname }}:9443
+traffic_manager_receiver_url: tcp://{{ tm_hostname }}:${receiver.url.port}
+traffic_manager_auth_url: ssl://{{ tm_hostname }}:${auth.url.port}
+throttle_config_policy_deployer_url: https://{{ tm_hostname }}:9443/services/
+notification_endpoint_url: https://{{ tm_hostname }}:9443
+throttle_decision_endpoints:
+  - tcp://{{ tm_hostname }}:5672
+throttling_url_group:
+  - traffic_manager_urls:
+      - tcp://{{ tm_hostname }}:9611
+    traffic_manager_auth_urls:
+      - ssl://{{ tm_hostname }}:9711
+
+# Workflow configurations
+workflow_enable: "false"
+workflow_service_url: https://{{ cp_hostname }}:9445/bpmn
+workflow_callback_endpoint: https://{{ cp_hostname }}:9443/api/am/publisher/v0.17/workflows/update-workflow-status
+workflow_token_endpoint: https://{{ cp_hostname }}:9443/oauth2/token
+workflow_client_registration_endpoint: https://{{ cp_hostname }}:9443/client-registration/v0.17/register
+workflow_server_url: https://apis.apim.com:9443/services/  #Developer Portal Service URL
+
+# Datasource configurations
+wso2shared_db_type: mysql
+wso2shared_db_url: jdbc:mysql://localhost:3306/shared_db?ssl-mode=REQUIRED
+wso2shared_db_username: root
+wso2shared_db_password: root
+wso2shared_db_driver: com.mysql.cj.jdbc.Driver
+
+wso2am_db_type: mysql
+wso2am_db_url: jdbc:mysql://localhost:3306/apim_db?ssl-mode=REQUIRED
+wso2am_db_username: root
+wso2am_db_password: root
+wso2am_db_driver: com.mysql.cj.jdbc.Driver
+
+# Key Store and Trust Store configuration
+primary_key_store_name: wso2carbon.jks
+primary_key_store_password: wso2carbon
+primary_key_store_key_password: wso2carbon
+primary_key_store_key_alias: wso2carbon
+
+# Client trustore configurations
+trust_store_name: client-truststore.jks
+trust_store_password: wso2carbon
+
+# The KeyStore which is used for encrypting/decrypting internal data. This block is read by Carbon Crypto Service
+internal_key_store_name: wso2carbon.jks
+internal_key_store_key_password: wso2carbon
+internal_key_store_password: wso2carbon
+internal_key_store_key_alias: wso2carbon
+
+# The KeyStore which is used for encrypting/decrypting internal data. This block is read by Carbon Crypto Service
+tls_key_store_name: wso2carbon.jks
+tls_key_store_key_password: wso2carbon
+tls_key_store_password: wso2carbon
+tls_key_store_key_alias: wso2carbon
+
+# If following is enabled all the sensitive information in server configurations will be encrypted. 
+secure_vault_enabled: false
+
+# Add any new changes you want to add for the group/profile below.
+# If you add a new file under templates and parameterized the file, the values for those parameters should be added
+# below. An example is provided below.
+# clustering_enable: false