Skip to content

Latest commit

 

History

History
150 lines (95 loc) · 15.3 KB

module_data_tiering.md

File metadata and controls

150 lines (95 loc) · 15.3 KB
Raw

Module: Data Tiering Options

SAP is offering range of capabilities how to optimize costs by segregating data into different storage and processing tiers. This module is discussing how individual Data Tiering options can be implemented as part of this Reference Architecture.

Overview of Data Tiering Options for SAP HANA

SAP is dividing the data based on the aging characteristics of the data and frequency of usage. Following data temperature tiers and tiering options are available:

Data Tiering Option Native HANA BW on HANA
BW/4HANA		 Suite on HANA
S/4HANA
Hot Data
- Dynamic Random Access Memory (DRAM) Yes Yes Yes
- Persistent Memory (NVRAM) Yes Yes Yes
Warm Data
- SAP HANA Native Storage Extensions (NSE) Yes Yes Yes
- SAP HANA Extension Nodes Yes Yes -
- SAP HANA Dynamic Tiering (DT) Yes - -
Cold Data
- SAP Near Line Storage
- SAP Data Hub / SAP Data Intelligence
- SAP HANA Spark Controller (Hadoop)

Selected Data Tiering Options (those impacting SAP HANA Reference Architecture) are discussed in sections below.

Additional Information:

Persistent Memory (NVRAM)

SAP HANA in-memory data can be divided into following usage types:

  • Main Data fragments (the in-memory copy of tables; infrequent changes)
  • Delta Data fragments (update information; frequent changes)
  • Temporary Data fragments (computational data; very frequent changes)

Server memory must be combination of Traditional RAM (DRAM) and Persistent Memory (Non-Volatile Random Access Memory - NVRAM). Traditional RAM (DRAM) is required during Operating System start and is also offering better performance for write operations. On the other hand, Persistent RAM (NVRAM) is cheaper and bigger and almost as fast as DRAM for read operations.

Therefore, Persistent Memory is intended only for Main Data fragments of Column Store tables that are changed very infrequently (only during Delta Merge operation).

Persistent Memory is supported since SAP HANA 2.0 SP03 (revision 35 and higher) and SAP HANA 2.0 SP04.

Usage of Persistent Memory can be activated on the level of whole SAP HANA Database, selected Tables, selected Table Partitions or only selected Table Columns.

Since Persistent Memory DIMMs are having much bigger capacity compared to Traditional DRAM DIMMs, SAP HANA System with Persistent Memory will have increased overall RAM capacity able to host higher amount of data.

Although replication between SAP HANA System having Persistent Memory and SAP HANA System without Persistent Memory is supported, it is not recommended for High Availability purpose. In any case proper memory sizing must be ensured to avoid out-of-memory situations after takeover.

Additional Information:

SAP HANA Native Storage Extensions (NSE)

SAP HANA is offering native option how to manage less frequently accessed data using built-in warm data store capability called Native Storage Extensions (NSE).

Data management of hot data is well described in Administration Guide: Memory Management in the Column Store.

Hot data is normally stored in "In-Memory Column Store". SAP HANA is automatically loading complete data structures (Table Columns or Table Column Partitions) into the memory based on first usage and will keep all data in memory as long as possible. These data structures are unloaded from memory only in case that allocated memory has reached the maximum limit and memory is required for processing another workload. In such case least recently used data structures are unloaded first.

SAP HANA Native Storage Extensions functionality is offering different approach based on "Disk Based Column Store". It can be activated for selected database objects (tables, partitions or columns). The data structures (Table Columns or Table Column Partitions) are kept on disk and only selected data pages are loaded into memory into "Buffer Cache". This concept is well-known from all other classical databases.

By using "Buffer Cache" that is significantly smaller than size of data in "Disk Based Column Store", Native Storage Extensions capability is increasing maximum amount of data that can be stored in SAP HANA database. Therefore, total storage requirements are also increased which needs to be reflected by infrastructure.

SAP HANA Native Storage Extension feature is supported since SAP HANA 2.0 SP04 and is limited only to Single-node SAP HANA Systems. Please note other functional restrictions as mentioned in SAP Note 2771956: SAP HANA Native Storage Extension Functional Restrictions.

Additional Information:

SAP HANA Extension Nodes

SAP HANA Extension Nodes

SAP HANA Scale-Out Systems can leverage SAP HANA Extension Nodes capability - new type of SAP HANA node type used exclusively for warm data.

SAP HANA Extension Node (configured as a slave node, worker group value worker_dt) is storing the warm data in "In-Memory Column Store" like regular SAP HANA node used for hot data. Since the warm data is less frequently used, the performance for SELECT statements against this data is not seen as important. Therefore, we can overload this node with amount of data to be doubled or in some cases even quadrupled compared to regular SAP HANA nodes.

Because the data is stored in "In-Memory Column Store" the internal mechanics is same as described in Administration Guide: Memory Management in the Column Store and in previous section. Due to a high volume of data in given node, the data structures are loaded and unloaded much more often than hot data in other nodes. Since this is associated with performance degradation, the Extension Node must be dedicated only for warm data.

Warm data must be placed in separate Tables or in separate Table Partitions. Subsequently those Tables and Table Partitions are relocated to SAP HANA Extension Node(s). Note that as described in Module: Basic Architecture each SAP HANA node is having its own subset of data in its own data files.

SAP HANA Extension Nodes are supported since SAP HANA 1.0 SP12 (for BW scenario) and since SAP HANA 2.0 SP03 (native scenario) and are limited only to Scale-Out SAP HANA Systems.

For SAP BW scenarios the hardware used for SAP HANA Extension Nodes can differ compared to other worker nodes starting from SAP HANA 2.0 SP03. For native scenarios this is supported from SAP HANA 2.0 SP4.

Additional Information:

SAP HANA Dynamic Tiering (DT)

SAP HANA Dynamic Tiering (DT) is optional add-on for SAP HANA database to manage warm data. Behind the scenes it is SAP IQ database that was modified and integrated into SAP HANA to act as new type of SAP HANA database process esserver. Since SAP IQ is based on "Disk Based Column Store", SAP HANA Dynamic Tiering database process (esserver) is offering very similar capabilities like SAP HANA in-memory database process (indexserver) with reduced memory requirements and reduced performance.

Although SAP HANA Dynamic Tiering is based on different database, the integration effort is ensuring that Dynamic Tiering node is embedded into SAP HANA operational processes like Start/Stop, Backup/Recovery and System Replication making it sub-component of SAP HANA database. Therefore, SAP HANA Dynamic Tiering cannot be operated independently from SAP HANA.

SAP HANA Dynamic Tiering database process (esserver) is typically deployed on dedicated host as part of SAP HANA Scale-Out scenario, however, there is option to co-deploy it on same host as SAP HANA in-memory database process (indexserver) for Single-Node scenario.

However, there are still several limitations that need to be taken into consideration when deploying SAP HANA Dynamic Tiering. These limitations include:

Basic deployment options are described in Installation and Update Guide: SAP HANA Dynamic Tiering Architecture.

Until SAP HANA Dynamic Tiering 2.0 SP04 there was no support for Cluster Manager - therefore High Availability using Pacemaker Cluster was not supported. Because this is recently released feature, this version of SAP HANA Reference Architecture is not yet supporting SAP HANA Dynamic Tiering. The support will be included in future versions based on detailed examination of functional restrictions mentioned above.

In order to move the data to Dynamic Tiering node SAP introduced concept of Extended Store Tables and Multistore Tables:

These new database table types are well explained and visualized in following blog: A Closer Look at SAP HANA Dynamic Tiering for Warm Data Management.

SAP HANA Dynamic Tiering is available only for use cases documented in Master Guide: Dynamic Tiering Use Cases. SAP HANA Dynamic Tiering is NOT supported for following applications:

Additional Information: