- Platform Specific Architecture for AWS (Amazon Web Services)
Following diagram is illustrating complete version of Reference Architecture for SAP HANA tailored for AWS (Amazon Web Services).
For detailed explanation of individual modules please see individual sections in Generic SAP HANA Architecture.
You can also review official AWS Reference Architecture and other documentation:
- AWS: SAP on AWS Technical Documentation: SAP HANA
- AWS: SAP HANA on the AWS Cloud: Quick Start Reference Deployment
- AWS: Multi-AZ (HA), Single-Node Architecture
Link to generic content: Module: Basic Architecture
Not every instance type is supported for productive SAP HANA usage.
Official list of SAP certified instance types is available at SAP: The SAP HANA Hardware Directory. This should always be used to decide whether the particular instance type is supported for SAP HANA or not.
AWS specific list of certified instances with additional details can be found in AWS: SAP HANA on AWS Quick Start Guide: AWS Instance Types for SAP HANA
SAP HANA Storage Configuration is coming in two flavors:
- General Purpose SSD (
gp2) storage - cheaper storage that meets SAP KPI requirements for most of the SAP HANA workloads - Provisioned IOPS SSD (
io1) storage - high-performance storage intended for most demanding SAP HANA workloads
Following disk setup is recommended:
| Filesystem | Name | Device type | Comment |
|---|---|---|---|
| / | Root volume | gp2 | |
| /hana/data | SAP HANA data | gp2 / io1 | |
| /hana/log | SAP HANA logs | gp2 / io1 | |
| /hana/shared | SAP HANA shared | gp2 | Provisioned to the master, NFS-mounted on other nodes |
| /usr/sap | SAP binaries | gp2 | |
| /backups | SAP HANA backup | st1 | Provisioned to the master, NFS-mounted on other nodes |
| /backups | SAP HANA backup | Amazon EFS | Alternative option for SAP HANA backup filesystem |
Instance specific sizing recommendations are available at AWS: SAP HANA on AWS Quick Start Guide: Storage Configuration for SAP HANA.
As visualized on the Overall Architecture diagram - in AWS each Availability Zone is having its own subnet. It is not possible to stretch one subnet across multiple Availability Zones. This needs to be taken into consideration during deployment planning.
Impact on implementation of Cluster IP in AWS is described in AWS: High Availability.
Link to generic content: Module: Virtual Hostname/IP
This chapter describes recommended implementation of Virtual Hostname and Virtual IP for an SAP HANA installation on the AWS cloud.
The implementation is based on assigning a Secondary private IP address to an existing network interface of the AWS instance. A Secondary private IP address can be easily reassigned to another AWS instance and so it can follow SAP HANA instance relocation. For more details see AWS: Multiple IP Addresses. This Secondary private IP address is associated with the Virtual Hostname which is used during SAP HANA instance installation.
- define Virtual IP (in the same subnet as the network interface) and Virtual Hostname for SAP HANA Instance
- assign Virtual IP as Secondary private IP address to existing network interface (see AWS: assign secondary private IP)
- configure OS to use Virtual IP Address (e.g. SUSE: Administration Guide - Configuring Multiple Addresses)
- make Virtual Hostname resolvable (e.g. updating
/etc/hosts) - install SAP HANA instance with the Virtual Hostname (see SAP: Administration Guide - Default Host Names and Virtual Host Names)
Note: Virtual IP can be be reassigned to another AWS instance thanks to option "Allow reassignment" of the network interface (see AWS: assign a secondary private IPv4 address)
A network communication initiated from a remote system to the Virtual IP of SAP HANA instance is established and takes place between remote system IP and the Virtual IP (Secondary private IP address). The Primary private IP address on the same interface is not involved in this communication.
In contrast to an inbound connection, when SAP HANA instance initiates a network connection to a remote system the Primary private IP address is used as source IP instead of Virtual IP (Secondary private IP address).
If there is requirement to use Virtual IP as the source IP, it could be achieved by means of linux routing. The core of the idea is to add route specifying source address like ip route add <network/mask> via <gateway> src <virtual_ip> (see Routing for multiple uplinks/providers).
Link to generic content: Module: High Availability
Best practice for deploying SAP HANA is to stretch High Availability cluster across Availability Zones. Each AWS Availability Zone is physically separate infrastructure, therefore deploying High Availability across Availability Zones ensures that there is no shared single-point-of-failure (SPOF) between primary and secondary SAP HANA system. This approach is significantly increasing overall resiliency of the High Availability of the solution.
List of existing Availability Zones for individual AWS Regions is available here: AWS: Regions and Availability Zones.
Most critical factor for selecting Availability Zones is network latency. Latency between individual Availability Zones can significantly differ and therefore it is important to measure network latency using SAP niping tool (see SAP Note 500235: Network Diagnosis with NIPING for additional information) and select Availability Zones with minimal latency.
Furthermore, it is important to note that internal numbering of Availability Zones is specific for each individual AWS account. Therefore, the network latency test must be performed in given account. For additional information please see AWS: Regions, Availability Zones, and Local Zones - Availability Zones.
Last thing to consider is whether desired instance types are available inside selected Availability Zones as not all Availability Zones are offering all instance types.
AWS is using IPMI-like Fencing (see Module: High Availability - IPMI-like Fencing for additional details). SBD (Storage Based Death) Fencing is not available in AWS.
Fencing agent source code is available here: external/ec2. Behind the scenes it is using StopInstances API call with force option. This will hard stop EC2 instance without gracefully stopping the Operating System.
Following are most important prerequisites for external/ec2 fencing mechanism to work properly:
- EC2 instances need to be properly tagged - tags are used by fencing agent to find correct instance to stop (see SUSE: Tagging the EC2 Instances)
- Configure AWS CLI - fencing agent is dependent on AWS CLI which needs to be configured properly (see SUSE: Creating an AWS CLI Profile on Both EC2 Instances)
- Configure HTTP Proxies (or use NAT gateway) to access AWS APIs over internet (see SUSE: Configure HTTP Proxies)
- Create STONITH policy and attach it as instance role (see SUSE: STONITH Policy)
Please see SUSE: SAP HANA High Availability Cluster for the AWS Cloud - Setup Guide - Prerequisites for the AWS-Specific HA Installation for complete list of prerequisites.
Additional Information:
- SUSE: SAP HANA High Availability Cluster for the AWS Cloud - Setup Guide
- Red Hat: Configure SAP HANA System Replication in Pacemaker on Amazon Web Services
Traditional implementation of Cluster IP (not applicable to AWS) is covered in section Module: High Availability - Typical Cluster IP Implementation.
In AWS each Availability Zone is having its own separate subnet and it is not possible to stretch one subnet across multiple Availability Zones. Therefore, different mechanism is required. AWS implementation of Cluster IP address is based on "Overlay IP" address concept. It is defined as additional entry in VPC routing table entry that is managed directly by Pacemaker Cluster. This entry is forwarding all packets sent to Overlay IP (third IP address in its own separate subnet) to the IP address of either primary or secondary server. During cluster takeover this VPC routing table entry is updated by Pacemaker cluster via AWS Command Line Interface (CLI) utility to point to new primary server. This concept is more in detail explained here:
- AWS: SAP on AWS High Availability with Overlay IP Address Routing
- IP Failover with Overlay IP Addresses
In order to ensure that Pacemaker cluster is able to properly manage the VPC route table adjustments, it needs proper IAM access to be assigned to given VM. Technical details are explained in following documentation:
- SUSE: SAP HANA High Availability Cluster for the AWS Cloud - Setup Guide
- Red Hat: Configure SAP HANA System Replication in Pacemaker on Amazon Web Services
Third requirement for this concept to work is to disable "Source/Destination Check" for given Network Interface as explained in above mentioned guides. This is to ensure that packets are not discarded based on using Cluster IP address. For additional information please see AWS: Disabling source/destination checks).
Link to generic content: Module: Disaster Recovery
Disaster Recovery for SAP HANA via SAP HANA System Replication is not infrastructure dependent.
Link to generic content: Module: Data Tiering Options
Following data tiering options are supposed on AWS:
| Data Tiering Option | Supported |
|---|---|
| Persistent Memory (NVRAM) | No |
| SAP HANA Native Storage Extensions (NSE) | Yes |
| SAP HANA Extension Nodes | Yes |
| SAP HANA Dynamic Tiering (DT) | Yes |
Additional Information:
Amazon Web Services platform does not offer any instance types having NVRAM that are supported for productive SAP HANA usage.
SAP HANA Native Storage Extensions only need additional disk space compared to regular SAP HANA deployments. Amazon Web Services platform does allow to provision additional disks to SAP HANA VM and add capacity into existing filesystems. There is no change to the design of the storage layout.
SAP HANA Extension Node implementation is based on provisioning additional SAP HANA node (with increased storage capacity) to existing SAP HANA system. Result is SAP HANA Scale-Out system where one of the nodes is designated as SAP HANA Extension Node. Storage layout is same as for regular SAP HANA nodes and it is visualized above in section AWS: Storage Setup for SAP HANA Implementation.
Additional Information:
SAP HANA Dynamic Tiering (DT) implementation in Amazon Web Services platform is based on provisioning additional VM for Dynamic Tiering component and connecting it to VM hosting SAP HANA instance, thus forming two-node distributed setup. Storage layout is identical to SAP HANA Scale-out setup as illustrated above in section AWS: Storage Setup for SAP HANA Implementation.
Additional Information:
Link to generic content: Module: SAP XSA
SAP HANA extended application services, advanced model (XSA) component is not infrastructure dependent.