diff --git a/API.md b/API.md index fa10af18..a4a02f66 100644 --- a/API.md +++ b/API.md @@ -8,6 +8,7 @@ Name|Description [WatchDynamoTable](#cdk-watchful-watchdynamotable)|*No description* [WatchEcsService](#cdk-watchful-watchecsservice)|*No description* [WatchLambdaFunction](#cdk-watchful-watchlambdafunction)|*No description* +[WatchOpenSearchDomain](#cdk-watchful-watchopensearchdomain)|*No description* [WatchRdsAurora](#cdk-watchful-watchrdsaurora)|*No description* [WatchStateMachine](#cdk-watchful-watchstatemachine)|*No description* [Watchful](#cdk-watchful-watchful)|*No description* @@ -28,6 +29,8 @@ Name|Description [WatchEcsServiceProps](#cdk-watchful-watchecsserviceprops)|*No description* [WatchLambdaFunctionOptions](#cdk-watchful-watchlambdafunctionoptions)|*No description* [WatchLambdaFunctionProps](#cdk-watchful-watchlambdafunctionprops)|*No description* +[WatchOpenSearchOptions](#cdk-watchful-watchopensearchoptions)|*No description* +[WatchOpenSearchProps](#cdk-watchful-watchopensearchprops)|*No description* [WatchRdsAuroraOptions](#cdk-watchful-watchrdsauroraoptions)|*No description* [WatchRdsAuroraProps](#cdk-watchful-watchrdsauroraprops)|*No description* [WatchStateMachineOptions](#cdk-watchful-watchstatemachineoptions)|*No description* @@ -164,6 +167,49 @@ new WatchLambdaFunction(scope: Construct, id: string, props: WatchLambdaFunction +## class WatchOpenSearchDomain + + + +__Implements__: [IConstruct](#constructs-iconstruct), [IDependable](#constructs-idependable) +__Extends__: [Construct](#constructs-construct) + +### Initializer + + + + +```ts +new WatchOpenSearchDomain(scope: Construct, id: string, props: WatchOpenSearchProps) +``` + +* **scope** ([Construct](#constructs-construct)) *No description* +* **id** (string) *No description* +* **props** ([WatchOpenSearchProps](#cdk-watchful-watchopensearchprops)) *No description* + * **automatedSnapshotFailureThresholdMax** (number) Threshold for AutomatedSnapshotFailure metric. __*Default*__: 1 + * **clusterIndexWritesBlockedThresholdMax** (number) Threshold for ClusterIndexWritesBlocked metric. __*Default*__: 1 + * **clusterStatusRedThresholdMax** (number) Threshold for ClusterStatus.red metric. __*Default*__: 1 + * **clusterStatusYellowThresholdMax** (number) Threshold for ClusterStatus.yellow metric. __*Default*__: 1 + * **cpuMaximumThresholdPercent** (number) Threshold for the Cpu Maximum utilization. __*Default*__: 80 + * **freeStorageSpaceThresholdPercent** (number) Threshold for FreeStorageSpace metric. __*Default*__: 25% of storage + * **http5XXResponsesThresholdPercent** (number) Threshold for 5XX Errors metric. __*Default*__: 10 + * **jvmMemoryPressureThresholdMax** (number) Threshold for the JVMMMemoryPressure metric. __*Default*__: 80 + * **kmsKeyErrorThresholdMax** (number) Threshold for KMSKeyError. __*Default*__: 1 + * **kmsKeyInaccessibleThresholdMax** (number) Threshold for KMSKeyInaccessible. __*Default*__: 1 + * **masterCpuMaximumThresholdPercent** (number) Threshold for the Master Cpu Maximum utilization. __*Default*__: 50 + * **masterJVMMemoryPressureThresholdMax** (number) Threshold for the MasterJVMMMemoryPressure metric. __*Default*__: 80 + * **masterReachableFromNodeThresholdMin** (number) Threshold for MasterReachableFromNode metric. __*Default*__: 1 + * **nodesThresholdMin** (number) Threshold for Nodes metric. __*Default*__: instanceCount + * **shardsActiveThresholdSum** (number) Threshold for ShardsActive metric. __*Default*__: 30000 + * **threadpoolSearchQueueThresholdAvg** (number) Threshold for ThreadpoolSearchQueue metric. __*Default*__: 500 + * **threadpoolWriteQueueThresholdAvg** (number) Threshold for ThreadpoolWriteQueue metric. __*Default*__: 100 + * **domain** ([aws_opensearchservice.Domain](#aws-cdk-lib-aws-opensearchservice-domain) | [aws_opensearchservice.CfnDomain](#aws-cdk-lib-aws-opensearchservice-cfndomain)) *No description* + * **title** (string) *No description* + * **watchful** ([IWatchful](#cdk-watchful-iwatchful)) *No description* + + + + ## class WatchRdsAurora @@ -389,6 +435,38 @@ watchLambdaFunction(title: string, fn: Function, options?: WatchLambdaFunctionOp __Returns__: * [WatchLambdaFunction](#cdk-watchful-watchlambdafunction) +#### watchOpenSearch(title, domain, options?) + + + +```ts +watchOpenSearch(title: string, domain: Domain | CfnDomain, options?: WatchOpenSearchOptions): WatchOpenSearchDomain +``` + +* **title** (string) *No description* +* **domain** ([aws_opensearchservice.Domain](#aws-cdk-lib-aws-opensearchservice-domain) | [aws_opensearchservice.CfnDomain](#aws-cdk-lib-aws-opensearchservice-cfndomain)) *No description* +* **options** ([WatchOpenSearchOptions](#cdk-watchful-watchopensearchoptions)) *No description* + * **automatedSnapshotFailureThresholdMax** (number) Threshold for AutomatedSnapshotFailure metric. __*Default*__: 1 + * **clusterIndexWritesBlockedThresholdMax** (number) Threshold for ClusterIndexWritesBlocked metric. __*Default*__: 1 + * **clusterStatusRedThresholdMax** (number) Threshold for ClusterStatus.red metric. __*Default*__: 1 + * **clusterStatusYellowThresholdMax** (number) Threshold for ClusterStatus.yellow metric. __*Default*__: 1 + * **cpuMaximumThresholdPercent** (number) Threshold for the Cpu Maximum utilization. __*Default*__: 80 + * **freeStorageSpaceThresholdPercent** (number) Threshold for FreeStorageSpace metric. __*Default*__: 25% of storage + * **http5XXResponsesThresholdPercent** (number) Threshold for 5XX Errors metric. __*Default*__: 10 + * **jvmMemoryPressureThresholdMax** (number) Threshold for the JVMMMemoryPressure metric. __*Default*__: 80 + * **kmsKeyErrorThresholdMax** (number) Threshold for KMSKeyError. __*Default*__: 1 + * **kmsKeyInaccessibleThresholdMax** (number) Threshold for KMSKeyInaccessible. __*Default*__: 1 + * **masterCpuMaximumThresholdPercent** (number) Threshold for the Master Cpu Maximum utilization. __*Default*__: 50 + * **masterJVMMemoryPressureThresholdMax** (number) Threshold for the MasterJVMMMemoryPressure metric. __*Default*__: 80 + * **masterReachableFromNodeThresholdMin** (number) Threshold for MasterReachableFromNode metric. __*Default*__: 1 + * **nodesThresholdMin** (number) Threshold for Nodes metric. __*Default*__: instanceCount + * **shardsActiveThresholdSum** (number) Threshold for ShardsActive metric. __*Default*__: 30000 + * **threadpoolSearchQueueThresholdAvg** (number) Threshold for ThreadpoolSearchQueue metric. __*Default*__: 500 + * **threadpoolWriteQueueThresholdAvg** (number) Threshold for ThreadpoolWriteQueue metric. __*Default*__: 100 + +__Returns__: +* [WatchOpenSearchDomain](#cdk-watchful-watchopensearchdomain) + #### watchRdsAuroraCluster(title, cluster, options?) @@ -424,6 +502,7 @@ watchScope(scope: Construct, options?: WatchfulAspectProps): void * **ec2ecs** (boolean) Automatically watch ApplicationLoadBalanced EC2 Ecs Services in the scope (using ECS Pattern). __*Default*__: true * **fargateecs** (boolean) Automatically watch ApplicationLoadBalanced Fargate Ecs Services in the scope (using ECS Pattern). __*Default*__: true * **lambda** (boolean) Automatically watch AWS Lambda functions in the scope. __*Default*__: true + * **opensearch** (boolean) Automatically watch OpenSearch Domains in the scope. __*Default*__: true * **rdsaurora** (boolean) Automatically watch RDS Aurora clusters in the scope. __*Default*__: true * **stateMachine** (boolean) Automatically watch AWS state machines in the scope. __*Default*__: true @@ -470,6 +549,7 @@ new WatchfulAspect(watchful: Watchful, props?: WatchfulAspectProps) * **ec2ecs** (boolean) Automatically watch ApplicationLoadBalanced EC2 Ecs Services in the scope (using ECS Pattern). __*Default*__: true * **fargateecs** (boolean) Automatically watch ApplicationLoadBalanced Fargate Ecs Services in the scope (using ECS Pattern). __*Default*__: true * **lambda** (boolean) Automatically watch AWS Lambda functions in the scope. __*Default*__: true + * **opensearch** (boolean) Automatically watch OpenSearch Domains in the scope. __*Default*__: true * **rdsaurora** (boolean) Automatically watch RDS Aurora clusters in the scope. __*Default*__: true * **stateMachine** (boolean) Automatically watch AWS state machines in the scope. __*Default*__: true @@ -706,6 +786,67 @@ Name | Type | Description +## struct WatchOpenSearchOptions + + + + + + +Name | Type | Description +-----|------|------------- +**automatedSnapshotFailureThresholdMax**? | number | Threshold for AutomatedSnapshotFailure metric.
__*Default*__: 1 +**clusterIndexWritesBlockedThresholdMax**? | number | Threshold for ClusterIndexWritesBlocked metric.
__*Default*__: 1 +**clusterStatusRedThresholdMax**? | number | Threshold for ClusterStatus.red metric.
__*Default*__: 1 +**clusterStatusYellowThresholdMax**? | number | Threshold for ClusterStatus.yellow metric.
__*Default*__: 1 +**cpuMaximumThresholdPercent**? | number | Threshold for the Cpu Maximum utilization.
__*Default*__: 80 +**freeStorageSpaceThresholdPercent**? | number | Threshold for FreeStorageSpace metric.
__*Default*__: 25% of storage +**http5XXResponsesThresholdPercent**? | number | Threshold for 5XX Errors metric.
__*Default*__: 10 +**jvmMemoryPressureThresholdMax**? | number | Threshold for the JVMMMemoryPressure metric.
__*Default*__: 80 +**kmsKeyErrorThresholdMax**? | number | Threshold for KMSKeyError.
__*Default*__: 1 +**kmsKeyInaccessibleThresholdMax**? | number | Threshold for KMSKeyInaccessible.
__*Default*__: 1 +**masterCpuMaximumThresholdPercent**? | number | Threshold for the Master Cpu Maximum utilization.
__*Default*__: 50 +**masterJVMMemoryPressureThresholdMax**? | number | Threshold for the MasterJVMMMemoryPressure metric.
__*Default*__: 80 +**masterReachableFromNodeThresholdMin**? | number | Threshold for MasterReachableFromNode metric.
__*Default*__: 1 +**nodesThresholdMin**? | number | Threshold for Nodes metric.
__*Default*__: instanceCount +**shardsActiveThresholdSum**? | number | Threshold for ShardsActive metric.
__*Default*__: 30000 +**threadpoolSearchQueueThresholdAvg**? | number | Threshold for ThreadpoolSearchQueue metric.
__*Default*__: 500 +**threadpoolWriteQueueThresholdAvg**? | number | Threshold for ThreadpoolWriteQueue metric.
__*Default*__: 100 + + + +## struct WatchOpenSearchProps + + + + + + +Name | Type | Description +-----|------|------------- +**domain** | [aws_opensearchservice.Domain](#aws-cdk-lib-aws-opensearchservice-domain) | [aws_opensearchservice.CfnDomain](#aws-cdk-lib-aws-opensearchservice-cfndomain) | +**title** | string | +**watchful** | [IWatchful](#cdk-watchful-iwatchful) | +**automatedSnapshotFailureThresholdMax**? | number | Threshold for AutomatedSnapshotFailure metric.
__*Default*__: 1 +**clusterIndexWritesBlockedThresholdMax**? | number | Threshold for ClusterIndexWritesBlocked metric.
__*Default*__: 1 +**clusterStatusRedThresholdMax**? | number | Threshold for ClusterStatus.red metric.
__*Default*__: 1 +**clusterStatusYellowThresholdMax**? | number | Threshold for ClusterStatus.yellow metric.
__*Default*__: 1 +**cpuMaximumThresholdPercent**? | number | Threshold for the Cpu Maximum utilization.
__*Default*__: 80 +**freeStorageSpaceThresholdPercent**? | number | Threshold for FreeStorageSpace metric.
__*Default*__: 25% of storage +**http5XXResponsesThresholdPercent**? | number | Threshold for 5XX Errors metric.
__*Default*__: 10 +**jvmMemoryPressureThresholdMax**? | number | Threshold for the JVMMMemoryPressure metric.
__*Default*__: 80 +**kmsKeyErrorThresholdMax**? | number | Threshold for KMSKeyError.
__*Default*__: 1 +**kmsKeyInaccessibleThresholdMax**? | number | Threshold for KMSKeyInaccessible.
__*Default*__: 1 +**masterCpuMaximumThresholdPercent**? | number | Threshold for the Master Cpu Maximum utilization.
__*Default*__: 50 +**masterJVMMemoryPressureThresholdMax**? | number | Threshold for the MasterJVMMMemoryPressure metric.
__*Default*__: 80 +**masterReachableFromNodeThresholdMin**? | number | Threshold for MasterReachableFromNode metric.
__*Default*__: 1 +**nodesThresholdMin**? | number | Threshold for Nodes metric.
__*Default*__: instanceCount +**shardsActiveThresholdSum**? | number | Threshold for ShardsActive metric.
__*Default*__: 30000 +**threadpoolSearchQueueThresholdAvg**? | number | Threshold for ThreadpoolSearchQueue metric.
__*Default*__: 500 +**threadpoolWriteQueueThresholdAvg**? | number | Threshold for ThreadpoolWriteQueue metric.
__*Default*__: 100 + + + ## struct WatchRdsAuroraOptions @@ -800,6 +941,7 @@ Name | Type | Description **ec2ecs**? | boolean | Automatically watch ApplicationLoadBalanced EC2 Ecs Services in the scope (using ECS Pattern).
__*Default*__: true **fargateecs**? | boolean | Automatically watch ApplicationLoadBalanced Fargate Ecs Services in the scope (using ECS Pattern).
__*Default*__: true **lambda**? | boolean | Automatically watch AWS Lambda functions in the scope.
__*Default*__: true +**opensearch**? | boolean | Automatically watch OpenSearch Domains in the scope.
__*Default*__: true **rdsaurora**? | boolean | Automatically watch RDS Aurora clusters in the scope.
__*Default*__: true **stateMachine**? | boolean | Automatically watch AWS state machines in the scope.
__*Default*__: true diff --git a/src/aspect.ts b/src/aspect.ts index 2f05ae3f..383371e2 100644 --- a/src/aspect.ts +++ b/src/aspect.ts @@ -3,6 +3,7 @@ import * as apigw from 'aws-cdk-lib/aws-apigateway'; import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; import * as ecs_patterns from 'aws-cdk-lib/aws-ecs-patterns'; import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as opensearch from 'aws-cdk-lib/aws-opensearchservice'; import * as rds from 'aws-cdk-lib/aws-rds'; import * as stepfunctions from 'aws-cdk-lib/aws-stepfunctions'; import { IConstruct } from 'constructs'; @@ -51,6 +52,11 @@ export interface WatchfulAspectProps { */ readonly ec2ecs?: boolean; + /** + * Automatically watch OpenSearch Domains in the scope. + * @default true + */ + readonly opensearch?: boolean; } /** @@ -74,6 +80,7 @@ export class WatchfulAspect implements IAspect { const watchRdsAuroraCluster = this.props.rdsaurora === undefined ? true : this.props.rdsaurora; const watchFargateEcs = this.props.fargateecs === undefined ? true : this.props.fargateecs; const watchEc2Ecs = this.props.ec2ecs === undefined ? true : this.props.ec2ecs; + const watchOpenSearch = this.props.opensearch === undefined ? true : this.props.opensearch; if (watchApiGateway && node instanceof apigw.RestApi) { this.watchful.watchApiGateway(node.node.path, node); @@ -102,6 +109,11 @@ export class WatchfulAspect implements IAspect { if (watchEc2Ecs && node instanceof ecs_patterns.ApplicationLoadBalancedEc2Service) { this.watchful.watchEc2Ecs(node.node.path, node.service, node.targetGroup); } + + if (watchOpenSearch && node instanceof opensearch.Domain || watchOpenSearch && node instanceof opensearch.CfnDomain) { + this.watchful.watchOpenSearch(node.node.path, node); + } + } } diff --git a/src/index.ts b/src/index.ts index 3ae157e9..f82667f0 100644 --- a/src/index.ts +++ b/src/index.ts @@ -7,4 +7,5 @@ export * from './dynamodb'; export * from './lambda'; export * from './state-machine'; export * from './rds-aurora'; -export * from './ecs'; \ No newline at end of file +export * from './ecs'; +export * from './opensearch'; \ No newline at end of file diff --git a/src/monitoring/aws/opensearch/metrics.ts b/src/monitoring/aws/opensearch/metrics.ts new file mode 100644 index 00000000..58cb614a --- /dev/null +++ b/src/monitoring/aws/opensearch/metrics.ts @@ -0,0 +1,354 @@ +import { Aws, Duration } from 'aws-cdk-lib'; +import { MathExpression, Metric, Statistic } from 'aws-cdk-lib/aws-cloudwatch'; + +const enum ClusterMetrics { + ClusterStatusGreen = 'ClusterStatus.green', + ClusterStatusYellow = 'ClusterStatus.yellow', + ClusterStatusRed = 'ClusterStatus.red', + ShardsActive = 'Shards.active', + ShardsUnassigned = 'Shards.unassigned', + ShardsDelayedUnassigned = 'Shards.delayedUnassigned', + ShardsActivePrimary = 'Shards.activePrimary', + ShardsInitializing = 'Shards.initializing', + ShardsRelocating = 'Shards.relocating', + Nodes = 'Nodes', + SearchableDocuments = 'SearchableDocuments', + DeletedDocuments = 'DeletedDocuments', + CPUUtilization = 'CPUUtilization', + FreeStorageSpace = 'FreeStorageSpace', + ClusterUsedSpace = 'ClusterUsedSpace', + ClusterIndexWritesBlocked = 'ClusterIndexWritesBlocked', + JVMMemoryPressure = 'JVMMemoryPressure', + AutomatedSnapshotFailure = 'AutomatedSnapshotFailure', + KMSKeyError = 'KMSKeyError', + KMSKeyInaccessible = 'KMSKeyInaccessible', + InvalidHostHeaderRequests = 'InvalidHostHeaderRequests', + OpenSearchRequests = 'OpenSearchRequests', + HTTP2XX = '2xx', + HTTP3XX = '3xx', + HTTP4XX = '4xx', + HTTP5XX = '5xx', +} +const enum MasterMetrics { + MasterCPUUtilization = 'MasterCPUUtilization', + MasterJVMMemoryPressure = 'MasterJVMMemoryPressure', + MasterReachableFromNode = 'MasterReachableFromNode', + MasterSysMemoryUtilization = 'MasterSysMemoryUtilization', +} + +const enum InstanceMetrics { + SearchLatency = 'SearchLatency', + IndexingLatency = 'IndexingLatency', + SearchRate = 'SearchRate', + IndexingRate = 'IndexingRate', + ThreadpoolBulkQueue = 'ThreadpoolBulkQueue', + ThreadpoolWriteQueue = 'ThreadpoolWriteQueue', + ThreadpoolSearchQueue = 'ThreadpoolSearchQueue', + ThreadpoolIndexQueue = 'ThreadpoolIndexQueue', + ThreadpoolForceMergeQueue = 'ThreadpoolForce_mergeQueue', + ThreadpoolBulkThreads = 'ThreadpoolBulkThreads', + ThreadpoolWriteThreads = 'ThreadpoolWriteThreads', + ThreadpoolSearchThreads = 'ThreadpoolSearchThreads', + ThreadpoolIndexThreads = 'ThreadpoolIndexThreads', + ThreadpoolForceMergeThreads = 'ThreadpoolForce_mergeThreads', + ThreadpoolBulkRejected = 'ThreadpoolBulkRejected', + ThreadpoolWriteRejected = 'ThreadpoolWriteRejected', + ThreadpoolSearchRejected = 'ThreadpoolSearchRejected', + ThreadpoolIndexRejected = 'ThreadpoolIndexRejected', + ThreadpoolForceMergeRejected = 'ThreadpoolForce_mergeRejected', + CoordinatingWriteRejected = 'CoordinatingWriteRejected', + PrimaryWriteRejected = 'PrimaryWriteRejected', + ReplicaWriteRejected = 'ReplicaWriteRejected', +} + +const enum EbsMetrics { + ReadLatency = 'ReadLatency', + WriteLatency = 'WriteLatency', + ReadThroughput = 'ReadThroughput', + WriteThroughput = 'WriteThroughput', + ReadIOPS = 'ReadIOPS', + WriteIOPS = 'WriteIOPS' +} + +const Namespace = 'AWS/ES'; + +/** + * Metrics for OpenSearch + */ +export class OpenSearchMetricFactory { + + metricClusterStatus(domainName: string) { + return { + clusterStatusGreenMetric: this.metric(ClusterMetrics.ClusterStatusGreen, domainName).with({ + statistic: Statistic.MAXIMUM, + period: Duration.minutes(1), + }), + clusterStatusRedMetric: this.metric(ClusterMetrics.ClusterStatusRed, domainName).with({ + statistic: Statistic.MAXIMUM, + period: Duration.minutes(1), + }), + clusterStatusYellowMetric: this.metric(ClusterMetrics.ClusterStatusYellow, domainName).with({ + statistic: Statistic.MAXIMUM, + period: Duration.minutes(1), + }), + }; + } + + metricFreeStorageSpace(domainName: string) { + return this.metric(ClusterMetrics.FreeStorageSpace, domainName).with({ period: Duration.minutes(1) }); + } + + metricClusterIndexWritesBlocked(domainName: string) { + return this.metric(ClusterMetrics.ClusterIndexWritesBlocked, domainName).with({ statistic: Statistic.MAXIMUM }); + } + + metricNodes(domainName: string) { + return this.metric(ClusterMetrics.Nodes, domainName).with({ statistic: Statistic.MAXIMUM, period: Duration.hours(1) }); + } + + metricAutomatedSnapshotFailure(domainName: string) { + return this.metric(ClusterMetrics.AutomatedSnapshotFailure, domainName).with({ statistic: Statistic.MAXIMUM, period: Duration.minutes(1) }); + } + + metricCpuUtilization(domainName: string) { + return this.metric(ClusterMetrics.CPUUtilization, domainName).with({ statistic: Statistic.AVERAGE, period: Duration.minutes(15) }); + } + + metricJVMMemoryPressure(domainName: string) { + return this.metric(ClusterMetrics.JVMMemoryPressure, domainName).with({ statistic: Statistic.MAXIMUM }); + } + + metricKms(domainName: string) { + return { + kmsKeyErrorMetric: this.metric(ClusterMetrics.KMSKeyError, domainName).with( + { + statistic: Statistic.MAXIMUM, + period: Duration.minutes(1), + }), + kmsKeyInaccessibleMetric: this.metric(ClusterMetrics.KMSKeyInaccessible, domainName).with( + { + statistic: Statistic.MAXIMUM, + period: Duration.minutes(1), + }), + }; + } + + metricShards(domainName: string) { + return { + shardsActive: this.metric(ClusterMetrics.ShardsActive, domainName).with({ + statistic: Statistic.SUM, + period: Duration.minutes(1), + }), + shardsUnassigned: this.metric(ClusterMetrics.ShardsUnassigned, domainName).with({ + statistic: Statistic.SUM, + period: Duration.minutes(1), + }), + shardsDelayedUnassigned: this.metric(ClusterMetrics.ShardsDelayedUnassigned, domainName).with({ + statistic: Statistic.SUM, + period: Duration.minutes(1), + }), + shardsActivePrimary: this.metric(ClusterMetrics.ShardsActivePrimary, domainName).with({ + statistic: Statistic.SUM, + period: Duration.minutes(1), + }), + shardsInitializing: this.metric(ClusterMetrics.ShardsInitializing, domainName).with({ + statistic: Statistic.SUM, + period: Duration.minutes(1), + }), + shardsRelocating: this.metric(ClusterMetrics.ShardsRelocating, domainName).with({ + statistic: Statistic.SUM, + period: Duration.minutes(1), + }), + }; + } + + metricHttp5xxPercentage(domainName: string) { + const http5XXMetric = this.metricResponseCodes(domainName)['5xxMetric']; + const openSearchRequestsMetric = this.metricRequests(domainName).openSearchRequests; + + return new MathExpression({ + expression: '100*(http5XXMetric/openSearchRequestsMetric)', + period: Duration.minutes(1), + usingMetrics: { http5XXMetric, openSearchRequestsMetric }, + label: 'HTTP5XXPercentage', + }); + } + + metricThreadpoolQueues(domainName: string) { + return { + threadpoolBulkQueue: this.metric(InstanceMetrics.ThreadpoolBulkQueue, domainName).with( { + statistic: Statistic.AVERAGE, + period: Duration.minutes(1), + }), + threadpoolWriteQueue: this.metric(InstanceMetrics.ThreadpoolWriteQueue, domainName).with( { + statistic: Statistic.AVERAGE, + period: Duration.minutes(1), + }), + threadpoolSearchQueue: this.metric(InstanceMetrics.ThreadpoolSearchQueue, domainName).with( { + statistic: Statistic.AVERAGE, + period: Duration.minutes(1), + }), + threadpoolIndexQueue: this.metric(InstanceMetrics.ThreadpoolIndexQueue, domainName).with( { + statistic: Statistic.AVERAGE, + period: Duration.minutes(1), + }), + threadpoolForceMergeQueue: this.metric(InstanceMetrics.ThreadpoolForceMergeQueue, domainName).with( { + statistic: Statistic.AVERAGE, + period: Duration.minutes(1), + }), + }; + } + + metricThreadpoolThreads(domainName: string) { + return { + threadpoolBulkThreads: this.metric(InstanceMetrics.ThreadpoolBulkThreads, domainName).with( { + statistic: Statistic.AVERAGE, + period: Duration.minutes(1), + }), + threadpoolWriteThreads: this.metric(InstanceMetrics.ThreadpoolWriteThreads, domainName).with( { + statistic: Statistic.AVERAGE, + period: Duration.minutes(1), + }), + threadpoolSearchThreads: this.metric(InstanceMetrics.ThreadpoolSearchThreads, domainName).with( { + statistic: Statistic.AVERAGE, + period: Duration.minutes(1), + }), + threadpoolIndexThreads: this.metric(InstanceMetrics.ThreadpoolIndexThreads, domainName).with( { + statistic: Statistic.AVERAGE, + period: Duration.minutes(1), + }), + threadpoolForceMergeThreads: this.metric(InstanceMetrics.ThreadpoolForceMergeThreads, domainName).with( { + statistic: Statistic.AVERAGE, + period: Duration.minutes(1), + }), + }; + } + + metricThreadpoolRejected(domainName: string) { + return { + threadpoolBulkRejected: this.metric(InstanceMetrics.ThreadpoolBulkRejected, domainName).with( { + period: Duration.minutes(1), + }), + threadpoolWriteRejected: this.metric(InstanceMetrics.ThreadpoolWriteRejected, domainName).with( { + period: Duration.minutes(1), + }), + threadpoolSearchRejected: this.metric(InstanceMetrics.ThreadpoolSearchRejected, domainName).with( { + period: Duration.minutes(1), + }), + threadpoolIndexRejected: this.metric(InstanceMetrics.ThreadpoolIndexRejected, domainName).with( { + period: Duration.minutes(1), + }), + threadpoolForceMergeRejected: this.metric(InstanceMetrics.ThreadpoolForceMergeRejected, domainName).with( { + period: Duration.minutes(1), + }), + }; + } + + metricWriteRejections(domainName: string) { + return { + CoordinatingWriteRejected: this.metric(InstanceMetrics.CoordinatingWriteRejected, domainName), + PrimaryWriteRejected: this.metric(InstanceMetrics.PrimaryWriteRejected, domainName), + ReplicaWriteRejected: this.metric(InstanceMetrics.ReplicaWriteRejected, domainName), + }; + } + + metricRequests(domainName: string) { + return { + invalidHostHeaderRequests: this.metric(ClusterMetrics.InvalidHostHeaderRequests, domainName).with({ + period: Duration.minutes(1), + }), + openSearchRequests: this.metric(ClusterMetrics.OpenSearchRequests, domainName).with({ + period: Duration.minutes(1), + }), + + }; + } + + metricStorage(domainName: string) { + return { + freeStorageSpaceMetric: this.metric(ClusterMetrics.FreeStorageSpace, domainName).with({ + statistic: Statistic.MINIMUM, + period: Duration.minutes(1), + }), + clusterUsedSpaceMetric: this.metric(ClusterMetrics.ClusterUsedSpace, domainName).with({ statistic: Statistic.AVERAGE }), + }; + } + + metricDocuments(domainName: string) { + return { + searchableDocumentsMetric: this.metric(ClusterMetrics.SearchableDocuments, domainName).with({ statistic: Statistic.AVERAGE }), + deletedDocumentsMetric: this.metric(ClusterMetrics.DeletedDocuments, domainName).with({ statistic: Statistic.AVERAGE }), + }; + } + + metricLatency(domainName: string) { + return { + indexingLatencyMetric: this.metric(InstanceMetrics.IndexingLatency, domainName).with({ statistic: Statistic.AVERAGE }), + searchLatencyMetric: this.metric(InstanceMetrics.SearchLatency, domainName).with({ statistic: Statistic.AVERAGE }), + }; + } + + metricRate(domainName: string) { + return { + indexingRateMetric: this.metric(InstanceMetrics.IndexingRate, domainName).with({ statistic: Statistic.AVERAGE }), + searchRateMetric: this.metric(InstanceMetrics.SearchRate, domainName).with({ statistic: Statistic.AVERAGE }), + }; + } + + metricClusterEbsLatency(domainName: string) { + return { + readLatencyMetric: this.metric(EbsMetrics.ReadLatency, domainName).with({ statistic: Statistic.AVERAGE }), + writeLatencyMetric: this.metric(EbsMetrics.WriteLatency, domainName).with({ statistic: Statistic.AVERAGE }), + }; + } + metricClusterEbsThroughput(domainName: string) { + return { + readThroughputMetric: this.metric(EbsMetrics.ReadThroughput, domainName).with({ statistic: Statistic.AVERAGE }), + writeThroughputMetric: this.metric(EbsMetrics.WriteThroughput, domainName).with({ statistic: Statistic.AVERAGE }), + }; + } + metricClusterEbsIOPS(domainName: string) { + return { + readIOPSMetric: this.metric(EbsMetrics.ReadIOPS, domainName).with({ statistic: Statistic.AVERAGE }), + writeIOPSMetric: this.metric(EbsMetrics.WriteIOPS, domainName).with({ statistic: Statistic.AVERAGE }), + }; + } + + metricResponseCodes(domainName: string) { + return { + '2xxMetric': this.metric(ClusterMetrics.HTTP2XX, domainName).with({ period: Duration.minutes(1) }), + '3xxMetric': this.metric(ClusterMetrics.HTTP3XX, domainName).with({ period: Duration.minutes(1) }), + '4xxMetric': this.metric(ClusterMetrics.HTTP4XX, domainName).with({ period: Duration.minutes(1) }), + '5xxMetric': this.metric(ClusterMetrics.HTTP5XX, domainName).with({ period: Duration.minutes(1) }), + }; + } + + metricMasterCpuUtilization(domainName: string) { + return this.metric(MasterMetrics.MasterCPUUtilization, domainName).with({ statistic: Statistic.MAXIMUM, period: Duration.minutes(1) }); + } + + metricMasterJVMMemoryPressure(domainName: string) { + return this.metric(MasterMetrics.MasterJVMMemoryPressure, domainName).with({ statistic: Statistic.MAXIMUM, period: Duration.minutes(1) }); + } + + metricMasterSysMemoryUtilization(domainName: string) { + return this.metric(MasterMetrics.MasterSysMemoryUtilization, domainName).with({ statistic: Statistic.MAXIMUM, period: Duration.minutes(1) }); + } + + metricMasterReachableFromNode(domainName: string) { + return this.metric(MasterMetrics.MasterReachableFromNode, domainName).with({ statistic: Statistic.MINIMUM, period: Duration.hours(1) }); + } + + protected metric(metric: ClusterMetrics | MasterMetrics | InstanceMetrics | EbsMetrics, domainName: string) { + return new Metric({ + metricName: metric, + namespace: Namespace, + period: Duration.minutes(5), + statistic: Statistic.SUM, + dimensionsMap: { + DomainName: domainName, + ClientId: Aws.ACCOUNT_ID, + }, + }); + } +} diff --git a/src/opensearch.ts b/src/opensearch.ts new file mode 100644 index 00000000..faa1b7c9 --- /dev/null +++ b/src/opensearch.ts @@ -0,0 +1,665 @@ +import { Duration } from 'aws-cdk-lib'; +import * as cloudwatch from 'aws-cdk-lib/aws-cloudwatch'; +import * as opensearch from 'aws-cdk-lib/aws-opensearchservice'; +import { Construct } from 'constructs'; +import EBSOptionsProperty = opensearch.CfnDomain.EBSOptionsProperty; +import ClusterConfigProperty = opensearch.CfnDomain.ClusterConfigProperty; +import { IWatchful } from './api'; +import { OpenSearchMetricFactory } from './monitoring/aws/opensearch/metrics'; + +export interface WatchOpenSearchOptions { + /** + * Threshold for ClusterStatus.yellow metric + * + * @default 1 + */ + readonly clusterStatusYellowThresholdMax?: number; + + /** + * Threshold for ClusterStatus.red metric + * + * @default 1 + */ + readonly clusterStatusRedThresholdMax?: number; + + /** + * Threshold for FreeStorageSpace metric. + * + * @default 25% of storage + */ + readonly freeStorageSpaceThresholdPercent?: number; + + /** + * Threshold for ClusterIndexWritesBlocked metric + * + * @default 1 + */ + readonly clusterIndexWritesBlockedThresholdMax?: number; + + /** + * Threshold for Nodes metric + * + * @default instanceCount + */ + readonly nodesThresholdMin?: number; + + /** + * Threshold for AutomatedSnapshotFailure metric + * + * @default 1 + */ + readonly automatedSnapshotFailureThresholdMax?: number; + + /** + * Threshold for the Cpu Maximum utilization + * + * @default 80 + */ + readonly cpuMaximumThresholdPercent?: number; + + /** + * Threshold for the JVMMMemoryPressure metric + * + * @default 80 + */ + readonly jvmMemoryPressureThresholdMax?: number; + + /** + * Threshold for KMSKeyError + * + * @default - 1 + */ + readonly kmsKeyErrorThresholdMax?: number; + + /** + * Threshold for KMSKeyInaccessible + * + * @default - 1 + */ + readonly kmsKeyInaccessibleThresholdMax?: number; + + /** + * Threshold for ShardsActive metric + * + * @default - 30000 + */ + readonly shardsActiveThresholdSum?: number; + + /** + * Threshold for 5XX Errors metric. Percent of requests compared to OpenSearchRequests that are returned as HTTP5XX + * + * @default - 10 + */ + readonly http5XXResponsesThresholdPercent?: number; + + /** + * Threshold for MasterReachableFromNode metric. + * + * @default - 1 + */ + readonly masterReachableFromNodeThresholdMin?: number; + + /** + * Threshold for ThreadpoolWriteQueue metric. + * + * @default - 100 + */ + readonly threadpoolWriteQueueThresholdAvg?: number; + + /** + * Threshold for ThreadpoolSearchQueue metric. + * + * @default - 500 + */ + readonly threadpoolSearchQueueThresholdAvg?: number; + + + /** + * Threshold for the Master Cpu Maximum utilization + * + * @default 50 + */ + readonly masterCpuMaximumThresholdPercent?: number; + + /** + * Threshold for the MasterJVMMMemoryPressure metric + * + * @default 80 + */ + readonly masterJVMMemoryPressureThresholdMax?: number; + +} + +export interface WatchOpenSearchProps extends WatchOpenSearchOptions { + readonly title: string; + readonly watchful: IWatchful; + readonly domain: opensearch.Domain | opensearch.CfnDomain; +} + +export class WatchOpenSearchDomain extends Construct { + private readonly watchful: IWatchful; + private readonly domain: opensearch.Domain | opensearch.CfnDomain; + private readonly metrics: OpenSearchMetricFactory; + private readonly domainName: string; + private readonly cfnDomain: opensearch.CfnDomain; + private readonly dataNodesCount: number; + private readonly masterNodesCount: number; + private readonly warmNodesCount: number; + private readonly totalNodeCount: number; + constructor(scope: Construct, id: string, props: WatchOpenSearchProps) { + super(scope, id); + + this.watchful = props.watchful; + this.domain = props.domain; + this.metrics = new OpenSearchMetricFactory(); + + this.domainName = this.domain.domainName ?? ''; + this.cfnDomain = (this.domain instanceof opensearch.Domain ? this.domain.node.defaultChild as opensearch.CfnDomain : this.domain); + this.dataNodesCount = (this.cfnDomain.clusterConfig as ClusterConfigProperty).instanceCount as number; + this.masterNodesCount = (this.cfnDomain.clusterConfig as ClusterConfigProperty).dedicatedMasterCount as number ?? 0; + this.warmNodesCount = (this.cfnDomain.clusterConfig as ClusterConfigProperty).warmCount as number ?? 0; + this.totalNodeCount = (this.dataNodesCount + this.masterNodesCount + this.warmNodesCount); + + + this.watchful.addSection(props.title, { + links: [ + { title: 'AWS OpenSearch Domain', url: linkForOpenSearch(this.domain) }, + ], + }); + /* eslint-disable max-len */ + const { clusterStatusRedMetric, clusterStatusRedAlarm } = this.createClusterStatusRedMonitor(props.clusterStatusRedThresholdMax); + const { clusterStatusYellowMetric, clusterStatusYellowAlarm } = this.createClusterStatusYellowMonitor(props.clusterStatusYellowThresholdMax); + const { freeStorageSpaceMetric, freeStorageSpaceAlarm } = this.createFreeStorageSpaceMonitor(props.freeStorageSpaceThresholdPercent); + const { clusterIndexWritesBlockedMetric, clusterIndexWritesBlockedAlarm } = this.createClusterIndexWritesBlockedMonitor(props.clusterIndexWritesBlockedThresholdMax); + const { nodesMetric, nodesAlarm } = this.createNodesMonitor(props.nodesThresholdMin); + const { cpuUtilizationMetric, cpuUtilizationAlarm } = this.createCpuUtilizationMonitor(props.cpuMaximumThresholdPercent); + const { JVMMemoryPressureMetric, JVMMemoryPressureAlarm } = this.createJVMMemoryPressureMonitor(props.jvmMemoryPressureThresholdMax); + const { automatedSnapshotFailureMetric, automatedSnapshotFailureAlarm } = this.createAutomatedSnapshotMonitor(props.automatedSnapshotFailureThresholdMax); + const { kmsKeyErrorMetric, kmsKeyErrorAlarm } = this.createKmsKeyErrorMonitor(props.kmsKeyErrorThresholdMax); + const { kmsKeyInaccessibleMetric, kmsKeyInaccessibleAlarm } = this.createKmsKeyInaccessibleMonitor(props.kmsKeyInaccessibleThresholdMax); + const { shardsActiveMetric, shardsActiveAlarm } = this.createShardsActiveMonitor(props.shardsActiveThresholdSum); + const { http5XXPercentageMetric, http5XXPercentageAlarm } = this.createHttp5XXMonitor(props.http5XXResponsesThresholdPercent); + const { threadpoolWriteQueueMetric, threadpoolWriteQueueAlarm } = this.createThreadpoolWriteQueueMonitor(props.threadpoolWriteQueueThresholdAvg); + const { threadpoolSearchQueueMetric, threadpoolSearchQueueAlarm } = this.createThreadpoolSearchQueueMonitor(props.threadpoolSearchQueueThresholdAvg); + /* eslint-enable max-len */ + + this.watchful.addWidgets( + new cloudwatch.AlarmStatusWidget({ + title: 'Alarm List', + width: 12, + height: 5, + alarms: this.node.children.filter((child) => child.constructor.name === 'Alarm') as cloudwatch.IAlarm[], + }), + new cloudwatch.SingleValueWidget({ + title: 'Nodes', + height: 4, + width: 2, + metrics: [this.metrics.metricNodes(this.domainName).with({ period: Duration.minutes(1) })], + }), + new cloudwatch.SingleValueWidget({ + title: 'Cluster status', + height: 4, + width: 7, + metrics: [ + clusterStatusYellowMetric.with({ color: cloudwatch.Color.ORANGE }), + clusterStatusRedMetric.with({ color: cloudwatch.Color.RED }), + this.metrics.metricClusterStatus(this.domainName).clusterStatusGreenMetric.with({ color: cloudwatch.Color.GREEN }), + ], + }), + ); + + this.watchful.addSection('Cluster metrics'); + this.watchful.addWidgets( + new cloudwatch.GraphWidget({ + title: `CPUUtilization/${cpuUtilizationMetric.period.toMinutes()}min`, + width: 8, + left: [cpuUtilizationMetric.with({ period: Duration.minutes(1) })], + leftAnnotations: [cpuUtilizationAlarm.toAnnotation()], + }), + new cloudwatch.GraphWidget({ + title: `ClusterStatus/${clusterStatusYellowMetric.period.toMinutes()}min`, + width: 8, + left: [ + this.metrics.metricClusterStatus(this.domainName).clusterStatusGreenMetric, + clusterStatusYellowMetric, + clusterStatusRedMetric, + ], + leftAnnotations: [clusterStatusYellowAlarm.toAnnotation(), clusterStatusRedAlarm.toAnnotation()], + }), + new cloudwatch.GraphWidget({ + title: `StorageSpace/${freeStorageSpaceMetric.period.toMinutes()}min`, + width: 8, + left: [ + freeStorageSpaceMetric, + this.metrics.metricStorage(this.domainName).clusterUsedSpaceMetric, + ], + leftAnnotations: [freeStorageSpaceAlarm.toAnnotation()], + }), + new cloudwatch.GraphWidget({ + title: `IndexWritesBlocked/${clusterIndexWritesBlockedMetric.period.toMinutes()}min`, + width: 8, + left: [ + clusterIndexWritesBlockedMetric, + ], + leftAnnotations: [clusterIndexWritesBlockedAlarm.toAnnotation()], + }), + new cloudwatch.GraphWidget({ + title: `Nodes/${nodesMetric.period.toMinutes()}min`, + width: 8, + left: [ + nodesMetric.with({ period: Duration.minutes(1) }), + ], + leftAnnotations: [nodesAlarm.toAnnotation()], + }), + new cloudwatch.GraphWidget({ + title: `JVMMemoryPressure/${JVMMemoryPressureMetric.period.toMinutes()}min`, + width: 8, + left: [ + JVMMemoryPressureMetric, + ], + leftAnnotations: [JVMMemoryPressureAlarm.toAnnotation()], + }), + new cloudwatch.GraphWidget({ + title: `Shards/${shardsActiveMetric.period.toMinutes()}min`, + width: 8, + left: [ + shardsActiveMetric, + this.metrics.metricShards(this.domainName).shardsUnassigned, + this.metrics.metricShards(this.domainName).shardsDelayedUnassigned, + this.metrics.metricShards(this.domainName).shardsActivePrimary, + this.metrics.metricShards(this.domainName).shardsInitializing, + this.metrics.metricShards(this.domainName).shardsRelocating, + ], + leftAnnotations: [shardsActiveAlarm.toAnnotation()], + }), + new cloudwatch.GraphWidget({ + title: `HTTP 5XX Errors/${http5XXPercentageMetric.period.toMinutes()}min`, + width: 8, + left: [ + http5XXPercentageMetric, + ], + leftAnnotations: [http5XXPercentageAlarm.toAnnotation()], + }), + new cloudwatch.GraphWidget({ + title: 'HTTP Requests', + width: 8, + left: [ + this.metrics.metricRequests(this.domainName).openSearchRequests, + this.metrics.metricRequests(this.domainName).invalidHostHeaderRequests, + this.metrics.metricResponseCodes(this.domainName)['2xxMetric'], + this.metrics.metricResponseCodes(this.domainName)['3xxMetric'], + this.metrics.metricResponseCodes(this.domainName)['4xxMetric'], + this.metrics.metricResponseCodes(this.domainName)['5xxMetric'], + ], + }), + new cloudwatch.GraphWidget({ + title: `AutomatedSnapshotFailure/${automatedSnapshotFailureMetric.period.toMinutes()}min`, + width: 8, + left: [automatedSnapshotFailureMetric], + leftAnnotations: [automatedSnapshotFailureAlarm.toAnnotation()], + }), + new cloudwatch.GraphWidget({ + title: `KmsKeyError/${kmsKeyErrorMetric.period.toMinutes()}min`, + width: 8, + left: [kmsKeyErrorMetric], + leftAnnotations: [kmsKeyErrorAlarm.toAnnotation()], + }), + new cloudwatch.GraphWidget({ + title: `KmsKeyInaccessible/${kmsKeyInaccessibleMetric.period.toMinutes()}min`, + width: 8, + left: [kmsKeyInaccessibleMetric], + leftAnnotations: [kmsKeyInaccessibleAlarm.toAnnotation()], + }), + ); + this.watchful.addSection('EBS metrics'); + this.watchful.addWidgets( + new cloudwatch.GraphWidget({ + title: `EBS IOPS/${this.metrics.metricClusterEbsIOPS(this.domainName).writeIOPSMetric.period.toMinutes()}min`, + width: 8, + left: [ + this.metrics.metricClusterEbsIOPS(this.domainName).writeIOPSMetric, + this.metrics.metricClusterEbsIOPS(this.domainName).readIOPSMetric, + ], + }), + new cloudwatch.GraphWidget({ + title: `EBS Latency/${this.metrics.metricClusterEbsLatency(this.domainName).writeLatencyMetric.period.toMinutes()}min`, + width: 8, + left: [ + this.metrics.metricClusterEbsLatency(this.domainName).writeLatencyMetric, + this.metrics.metricClusterEbsLatency(this.domainName).readLatencyMetric, + ], + }), + new cloudwatch.GraphWidget({ + title: `EBS Throughput/${this.metrics.metricClusterEbsThroughput(this.domainName).writeThroughputMetric.period.toMinutes()}min`, + width: 8, + left: [ + this.metrics.metricClusterEbsThroughput(this.domainName).writeThroughputMetric, + this.metrics.metricClusterEbsThroughput(this.domainName).readThroughputMetric, + ], + }), + ); + this.watchful.addSection('Instance metrics'); + this.watchful.addWidgets( + new cloudwatch.GraphWidget({ + title: `Search/IndexingLatency/${this.metrics.metricLatency(this.domainName).searchLatencyMetric.period.toMinutes()}min`, + width: 12, + left: [ + this.metrics.metricLatency(this.domainName).searchLatencyMetric, + this.metrics.metricLatency(this.domainName).indexingLatencyMetric, + ], + }), + new cloudwatch.GraphWidget({ + title: `Search/IndexingRate/${this.metrics.metricRate(this.domainName).searchRateMetric.period.toMinutes()}min`, + width: 12, + left: [ + this.metrics.metricRate(this.domainName).searchRateMetric, + this.metrics.metricRate(this.domainName).indexingRateMetric, + ], + }), + new cloudwatch.GraphWidget({ + title: `ThreadpoolQueues/${threadpoolWriteQueueMetric.period.toMinutes()}min`, + width: 6, + left: [ + threadpoolWriteQueueMetric, + threadpoolSearchQueueMetric, + this.metrics.metricThreadpoolQueues(this.domainName).threadpoolBulkQueue, + this.metrics.metricThreadpoolQueues(this.domainName).threadpoolIndexQueue, + this.metrics.metricThreadpoolQueues(this.domainName).threadpoolForceMergeQueue, + ], + leftAnnotations: [threadpoolWriteQueueAlarm.toAnnotation(), threadpoolSearchQueueAlarm.toAnnotation()], + }), + new cloudwatch.GraphWidget({ + title: `ThreadpoolThreads/${this.metrics.metricThreadpoolThreads(this.domainName).threadpoolWriteThreads.period.toMinutes()}min`, + width: 6, + left: [ + this.metrics.metricThreadpoolThreads(this.domainName).threadpoolWriteThreads, + this.metrics.metricThreadpoolThreads(this.domainName).threadpoolSearchThreads, + this.metrics.metricThreadpoolThreads(this.domainName).threadpoolBulkThreads, + this.metrics.metricThreadpoolThreads(this.domainName).threadpoolIndexThreads, + this.metrics.metricThreadpoolThreads(this.domainName).threadpoolForceMergeThreads, + ], + }), + new cloudwatch.GraphWidget({ + title: `ThreadpoolRejected/${this.metrics.metricThreadpoolRejected(this.domainName).threadpoolWriteRejected.period.toMinutes()}min`, + width: 6, + left: [ + this.metrics.metricThreadpoolRejected(this.domainName).threadpoolWriteRejected, + this.metrics.metricThreadpoolRejected(this.domainName).threadpoolSearchRejected, + this.metrics.metricThreadpoolRejected(this.domainName).threadpoolBulkRejected, + this.metrics.metricThreadpoolRejected(this.domainName).threadpoolIndexRejected, + this.metrics.metricThreadpoolRejected(this.domainName).threadpoolForceMergeRejected, + ], + }), + new cloudwatch.GraphWidget({ + title: `WriteRejections/${this.metrics.metricWriteRejections(this.domainName).PrimaryWriteRejected.period.toMinutes()}min`, + width: 6, + left: [ + this.metrics.metricWriteRejections(this.domainName).PrimaryWriteRejected, + this.metrics.metricWriteRejections(this.domainName).ReplicaWriteRejected, + this.metrics.metricWriteRejections(this.domainName).CoordinatingWriteRejected, + ], + }), + ); + + // Add relevant metrics if DedicatedMasterEnabled + if ((this.cfnDomain.clusterConfig as ClusterConfigProperty).dedicatedMasterEnabled) { + /* eslint-disable max-len */ + const { masterCpuUtilizationMetric, masterCpuUtilizationAlarm } = this.createMasterCpuUtilizationMonitor(props.masterCpuMaximumThresholdPercent); + const { masterJVMMemoryPressureMetric, masterJVMMemoryPressureAlarm } = this.createMasterJVMMemoryPressureMonitor(props.masterJVMMemoryPressureThresholdMax); + const { masterReachableFromNodeMetric, masterReachableFromNodeAlarm } = this.createMasterReachableFromNodeMonitor(props.masterReachableFromNodeThresholdMin); + /* eslint-enable max-len */ + + this.watchful.addSection('Master metrics'); + this.watchful.addWidgets( + new cloudwatch.GraphWidget({ + title: `MasterCPUUtilization/${masterCpuUtilizationMetric.period.toMinutes()}min`, + width: 6, + left: [ + masterCpuUtilizationMetric, + ], + leftAnnotations: [masterCpuUtilizationAlarm.toAnnotation()], + }), + new cloudwatch.GraphWidget({ + title: `MasterJVMMemoryPressure/${masterJVMMemoryPressureMetric.period.toMinutes()}min`, + width: 6, + left: [ + masterJVMMemoryPressureMetric, + ], + leftAnnotations: [masterJVMMemoryPressureAlarm.toAnnotation()], + }), + new cloudwatch.GraphWidget({ + title: `MasterReachableFromNode/${masterReachableFromNodeMetric.period.toMinutes()}min`, + width: 6, + left: [ + masterReachableFromNodeMetric, + ], + leftAnnotations: [masterReachableFromNodeAlarm.toAnnotation()], + }), + new cloudwatch.GraphWidget({ + title: `MasterSysMemoryUtilization/${this.metrics.metricMasterSysMemoryUtilization(this.domainName).period.toMinutes()}min`, + width: 6, + left: [ + this.metrics.metricMasterSysMemoryUtilization(this.domainName), + ], + }), + ); + } + } + + private createClusterStatusRedMonitor(clusterStatusRedMaximumThreshold = 1) { + const clusterStatusRedMetric = this.metrics.metricClusterStatus(this.domainName).clusterStatusRedMetric; + const clusterStatusRedAlarm = clusterStatusRedMetric.createAlarm(this, 'ClusterStatusRedAlarm', { + alarmDescription: 'clusterStatusRedAlarm', + threshold: clusterStatusRedMaximumThreshold, + comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD, + evaluationPeriods: 1, + }); + this.watchful.addAlarm(clusterStatusRedAlarm); + return { clusterStatusRedMetric, clusterStatusRedAlarm }; + } + + private createClusterStatusYellowMonitor(clusterStatusYellowMaximumThreshold = 1) { + const clusterStatusYellowMetric = this.metrics.metricClusterStatus(this.domainName).clusterStatusYellowMetric; + const clusterStatusYellowAlarm = clusterStatusYellowMetric.createAlarm(this, 'ClusterStatusYellowAlarm', { + alarmDescription: 'clusterStatusYellowAlarm', + threshold: clusterStatusYellowMaximumThreshold, + comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD, + evaluationPeriods: 1, + }); + this.watchful.addAlarm(clusterStatusYellowAlarm); + return { clusterStatusYellowMetric, clusterStatusYellowAlarm }; + } + + private createFreeStorageSpaceMonitor(freeStorageSpaceThresholdPercent = 25) { + const volumeSize = (this.cfnDomain.ebsOptions as EBSOptionsProperty).volumeSize as number ?? 10; + + const freeStorageSpaceMetric = this.metrics.metricFreeStorageSpace(this.domainName); + const freeStorageSpaceAlarm = freeStorageSpaceMetric.createAlarm(this, 'FreeStorageSpaceAlarm', { + alarmDescription: 'freeStorageSpaceAlarm', + threshold: freeStorageSpaceThresholdPercent/100*volumeSize*this.dataNodesCount*1024, // Percentage of volumeSize to MiB + comparisonOperator: cloudwatch.ComparisonOperator.LESS_THAN_THRESHOLD, + evaluationPeriods: 1, + }); + this.watchful.addAlarm(freeStorageSpaceAlarm); + return { freeStorageSpaceMetric, freeStorageSpaceAlarm }; + } + + private createClusterIndexWritesBlockedMonitor(clusterIndexWritesBlockedThresholdMax = 1) { + const clusterIndexWritesBlockedMetric = this.metrics.metricClusterIndexWritesBlocked(this.domainName); + const clusterIndexWritesBlockedAlarm = clusterIndexWritesBlockedMetric.createAlarm(this, 'ClusterIndexWritesBlockedAlarm', { + alarmDescription: 'clusterIndexWritesBlockedAlarm', + threshold: clusterIndexWritesBlockedThresholdMax, + comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD, + evaluationPeriods: 1, + }); + this.watchful.addAlarm(clusterIndexWritesBlockedAlarm); + return { clusterIndexWritesBlockedMetric, clusterIndexWritesBlockedAlarm }; + } + + private createNodesMonitor(nodesThresholdMin?: number) { + + const nodesMetric = this.metrics.metricNodes(this.domainName); + const nodesAlarm = nodesMetric.createAlarm(this, 'NodesAlarm', { + alarmDescription: 'availableNodesAlarm', + threshold: nodesThresholdMin ?? this.totalNodeCount, + comparisonOperator: cloudwatch.ComparisonOperator.LESS_THAN_THRESHOLD, + evaluationPeriods: 1, + }); + this.watchful.addAlarm(nodesAlarm); + return { nodesMetric, nodesAlarm }; + } + + private createAutomatedSnapshotMonitor(automatedSnapshotFailureThresholdMax = 1) { + const automatedSnapshotFailureMetric = this.metrics.metricAutomatedSnapshotFailure(this.domainName); + const automatedSnapshotFailureAlarm = automatedSnapshotFailureMetric.createAlarm(this, 'AutomatedSnapshotFailureAlarm', { + alarmDescription: 'automatedSnapshotFailureAlarm', + threshold: automatedSnapshotFailureThresholdMax, + comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD, + evaluationPeriods: 1, + }); + this.watchful.addAlarm(automatedSnapshotFailureAlarm); + return { automatedSnapshotFailureMetric, automatedSnapshotFailureAlarm }; + } + + private createCpuUtilizationMonitor(cpuMaximumThresholdPercent = 80) { + const cpuUtilizationMetric = this.metrics.metricCpuUtilization(this.domainName); + const cpuUtilizationAlarm = cpuUtilizationMetric.createAlarm(this, 'CpuUtilizationAlarm', { + alarmDescription: 'cpuUtilizationAlarm', + threshold: cpuMaximumThresholdPercent, + comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_THRESHOLD, + evaluationPeriods: 3, + }); + this.watchful.addAlarm(cpuUtilizationAlarm); + return { cpuUtilizationMetric, cpuUtilizationAlarm }; + } + + private createJVMMemoryPressureMonitor(jvmMemoryPressureThresholdMax = 80) { + const JVMMemoryPressureMetric = this.metrics.metricJVMMemoryPressure(this.domainName); + const JVMMemoryPressureAlarm = JVMMemoryPressureMetric.createAlarm(this, 'JVMMemoryPressureAlarm', { + alarmDescription: 'JVMMemoryPressureAlarm', + threshold: jvmMemoryPressureThresholdMax, + comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD, + evaluationPeriods: 3, + }); + this.watchful.addAlarm(JVMMemoryPressureAlarm); + return { JVMMemoryPressureMetric, JVMMemoryPressureAlarm }; + } + + private createKmsKeyErrorMonitor(kmsKeyErrorThresholdMax = 1) { + const kmsKeyErrorMetric = this.metrics.metricKms(this.domainName).kmsKeyErrorMetric; + const kmsKeyErrorAlarm = kmsKeyErrorMetric.createAlarm(this, 'KmsKeyErrorAlarm', { + alarmDescription: 'kmsKeyErrorAlarm', + threshold: kmsKeyErrorThresholdMax, + comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD, + evaluationPeriods: 1, + treatMissingData: cloudwatch.TreatMissingData.NOT_BREACHING, + }); + this.watchful.addAlarm(kmsKeyErrorAlarm); + return { kmsKeyErrorMetric, kmsKeyErrorAlarm }; + } + + private createKmsKeyInaccessibleMonitor(kmsKeyInaccessibleThresholdMax = 1) { + const kmsKeyInaccessibleMetric = this.metrics.metricKms(this.domainName).kmsKeyInaccessibleMetric; + const kmsKeyInaccessibleAlarm = kmsKeyInaccessibleMetric.createAlarm(this, 'KmsKeyInaccessibleAlarm', { + alarmDescription: 'kmsKeyInaccessibleAlarm', + threshold: kmsKeyInaccessibleThresholdMax, + comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD, + evaluationPeriods: 1, + treatMissingData: cloudwatch.TreatMissingData.NOT_BREACHING, + }); + this.watchful.addAlarm(kmsKeyInaccessibleAlarm); + return { kmsKeyInaccessibleMetric, kmsKeyInaccessibleAlarm }; + } + + private createShardsActiveMonitor(shardsActiveThresholdSum = 30000) { + const shardsActiveMetric = this.metrics.metricShards(this.domainName).shardsActive; + const shardsActiveAlarm = shardsActiveMetric.createAlarm(this, 'ShardsActiveAlarm', { + alarmDescription: 'shardsActiveAlarm', + threshold: shardsActiveThresholdSum, + comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD, + evaluationPeriods: 1, + }); + this.watchful.addAlarm(shardsActiveAlarm); + return { shardsActiveMetric, shardsActiveAlarm }; + } + + private createHttp5XXMonitor(http5XXResponsesThresholdPercent = 10) { + const http5XXPercentageMetric = this.metrics.metricHttp5xxPercentage(this.domainName); + const http5XXPercentageAlarm = http5XXPercentageMetric.createAlarm(this, 'HTTP5XXPercentageAlarm', { + alarmDescription: 'HTTP5XXPercentageAlarm', + threshold: http5XXResponsesThresholdPercent, + comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD, + evaluationPeriods: 1, + treatMissingData: cloudwatch.TreatMissingData.NOT_BREACHING, + }); + this.watchful.addAlarm(http5XXPercentageAlarm); + return { http5XXPercentageMetric, http5XXPercentageAlarm }; + } + + private createThreadpoolWriteQueueMonitor(threadpoolWriteQueueThresholdAvg = 100) { + const threadpoolWriteQueueMetric = this.metrics.metricThreadpoolQueues(this.domainName).threadpoolWriteQueue; + const threadpoolWriteQueueAlarm = threadpoolWriteQueueMetric.createAlarm(this, 'ThreadpoolWriteQueueAlarm', { + alarmDescription: 'threadpoolWriteQueueAlarm', + threshold: threadpoolWriteQueueThresholdAvg, + comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD, + evaluationPeriods: 1, + }); + this.watchful.addAlarm(threadpoolWriteQueueAlarm); + return { threadpoolWriteQueueMetric, threadpoolWriteQueueAlarm }; + } + + private createThreadpoolSearchQueueMonitor(threadpoolSearchQueueThresholdAvg = 500) { + const threadpoolSearchQueueMetric = this.metrics.metricThreadpoolQueues(this.domainName).threadpoolSearchQueue; + const threadpoolSearchQueueAlarm = threadpoolSearchQueueMetric.createAlarm(this, 'ThreadpoolSearchQueueAlarm', { + alarmDescription: 'threadpoolSearchQueueAlarm', + threshold: threadpoolSearchQueueThresholdAvg, + comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD, + evaluationPeriods: 1, + }); + this.watchful.addAlarm(threadpoolSearchQueueAlarm); + return { threadpoolSearchQueueMetric, threadpoolSearchQueueAlarm }; + } + + private createMasterCpuUtilizationMonitor(masterCpuMaximumThresholdPercent = 50) { + const masterCpuUtilizationMetric = this.metrics.metricMasterCpuUtilization(this.domainName); + const masterCpuUtilizationAlarm = masterCpuUtilizationMetric.createAlarm(this, 'MasterCpuUtilizationAlarm', { + alarmDescription: 'masterCpuUtilizationAlarm', + threshold: masterCpuMaximumThresholdPercent, + comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_THRESHOLD, + evaluationPeriods: 3, + }); + this.watchful.addAlarm(masterCpuUtilizationAlarm); + return { masterCpuUtilizationMetric, masterCpuUtilizationAlarm }; + } + + private createMasterJVMMemoryPressureMonitor(masterJVMMemoryPressureThresholdMax = 80) { + const masterJVMMemoryPressureMetric = this.metrics.metricMasterJVMMemoryPressure(this.domainName); + const masterJVMMemoryPressureAlarm = masterJVMMemoryPressureMetric.createAlarm(this, 'MasterJVMMemoryPressureAlarm', { + alarmDescription: 'masterJVMMemoryPressureAlarm', + threshold: masterJVMMemoryPressureThresholdMax, + comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD, + evaluationPeriods: 1, + }); + this.watchful.addAlarm(masterJVMMemoryPressureAlarm); + return { masterJVMMemoryPressureMetric, masterJVMMemoryPressureAlarm }; + } + + private createMasterReachableFromNodeMonitor(masterReachableFromNodeThresholdMin = 1) { + const masterReachableFromNodeMetric = this.metrics.metricMasterReachableFromNode(this.domainName); + const masterReachableFromNodeAlarm = masterReachableFromNodeMetric.createAlarm(this, 'MasterReachableFromNodeAlarm', { + alarmDescription: 'masterReachableFromNodeAlarm', + threshold: masterReachableFromNodeThresholdMin, + comparisonOperator: cloudwatch.ComparisonOperator.LESS_THAN_THRESHOLD, + evaluationPeriods: 1, + }); + this.watchful.addAlarm(masterReachableFromNodeAlarm); + return { masterReachableFromNodeMetric, masterReachableFromNodeAlarm }; + } + +} + +function linkForOpenSearch(domain: opensearch.Domain | opensearch.CfnDomain) { + return `https://console.aws.amazon.com/esv3/home?region=${domain.stack.region}#opensearch/domains/${domain.domainName}`; +} + + diff --git a/src/watchful.ts b/src/watchful.ts index 1a7c50f5..f0ea4c0d 100644 --- a/src/watchful.ts +++ b/src/watchful.ts @@ -6,6 +6,7 @@ import * as dynamodb from 'aws-cdk-lib/aws-dynamodb'; import * as ecs from 'aws-cdk-lib/aws-ecs'; import { ApplicationTargetGroup } from 'aws-cdk-lib/aws-elasticloadbalancingv2'; import * as lambda from 'aws-cdk-lib/aws-lambda'; +import * as opensearch from 'aws-cdk-lib/aws-opensearchservice'; import * as rds from 'aws-cdk-lib/aws-rds'; import * as sns from 'aws-cdk-lib/aws-sns'; import * as sns_subscriptions from 'aws-cdk-lib/aws-sns-subscriptions'; @@ -18,6 +19,7 @@ import { WatchfulAspect, WatchfulAspectProps } from './aspect'; import { WatchDynamoTableOptions, WatchDynamoTable } from './dynamodb'; import { WatchEcsServiceOptions, WatchEcsService } from './ecs'; import { WatchLambdaFunctionOptions, WatchLambdaFunction } from './lambda'; +import { WatchOpenSearchOptions, WatchOpenSearchDomain } from './opensearch'; import { WatchRdsAuroraOptions, WatchRdsAurora } from './rds-aurora'; import { WatchStateMachineOptions, WatchStateMachine } from './state-machine'; import { SectionWidget } from './widget/section'; @@ -192,6 +194,12 @@ export class Watchful extends Construct implements IWatchful { title, watchful: this, ec2Service, targetGroup, ...options, }); } + + public watchOpenSearch(title: string, domain: opensearch.Domain | opensearch.CfnDomain, options: WatchOpenSearchOptions = {}) { + return new WatchOpenSearchDomain(this, Names.uniqueId(domain), { + title, watchful: this, domain, ...options, + }); + } } function linkForDashboard(dashboard: cloudwatch.Dashboard) { diff --git a/test/monitoring/aws/opensearch/__snapshots__/metrics.test.ts.snap b/test/monitoring/aws/opensearch/__snapshots__/metrics.test.ts.snap new file mode 100644 index 00000000..0cef949b --- /dev/null +++ b/test/monitoring/aws/opensearch/__snapshots__/metrics.test.ts.snap @@ -0,0 +1,1473 @@ +// Jest Snapshot v1, https://goo.gl/fbAQLP + +exports[`snapshot test: metricAutomatedSnapshotFailure 1`] = ` +Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "AutomatedSnapshotFailure", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Maximum", + "unit": undefined, +} +`; + +exports[`snapshot test: metricClusterEbsIOPS 1`] = ` +Object { + "readIOPSMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ReadIOPS", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, + "writeIOPSMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "WriteIOPS", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, +} +`; + +exports[`snapshot test: metricClusterEbsLatency 1`] = ` +Object { + "readLatencyMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ReadLatency", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, + "writeLatencyMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "WriteLatency", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, +} +`; + +exports[`snapshot test: metricClusterEbsThroughput 1`] = ` +Object { + "readThroughputMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ReadThroughput", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, + "writeThroughputMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "WriteThroughput", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, +} +`; + +exports[`snapshot test: metricClusterIndexWritesBlocked 1`] = ` +Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ClusterIndexWritesBlocked", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Maximum", + "unit": undefined, +} +`; + +exports[`snapshot test: metricClusterStatus 1`] = ` +Object { + "clusterStatusGreenMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ClusterStatus.green", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Maximum", + "unit": undefined, + }, + "clusterStatusRedMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ClusterStatus.red", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Maximum", + "unit": undefined, + }, + "clusterStatusYellowMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ClusterStatus.yellow", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Maximum", + "unit": undefined, + }, +} +`; + +exports[`snapshot test: metricCpuUtilization 1`] = ` +Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "CPUUtilization", + "namespace": "AWS/ES", + "period": Duration { + "amount": 15, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, +} +`; + +exports[`snapshot test: metricDocuments 1`] = ` +Object { + "deletedDocumentsMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "DeletedDocuments", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, + "searchableDocumentsMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "SearchableDocuments", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, +} +`; + +exports[`snapshot test: metricFreeStorageSpace 1`] = ` +Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "FreeStorageSpace", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, +} +`; + +exports[`snapshot test: metricHttp5xxPercentage 1`] = ` +MathExpression { + "color": undefined, + "expression": "100*(http5XXMetric/openSearchRequestsMetric)", + "label": "HTTP5XXPercentage", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "searchAccount": undefined, + "searchRegion": undefined, + "usingMetrics": Object { + "http5XXMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "5xx", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, + "openSearchRequestsMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "OpenSearchRequests", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, + }, +} +`; + +exports[`snapshot test: metricJVMMemoryPressure 1`] = ` +Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "JVMMemoryPressure", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Maximum", + "unit": undefined, +} +`; + +exports[`snapshot test: metricKms 1`] = ` +Object { + "kmsKeyErrorMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "KMSKeyError", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Maximum", + "unit": undefined, + }, + "kmsKeyInaccessibleMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "KMSKeyInaccessible", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Maximum", + "unit": undefined, + }, +} +`; + +exports[`snapshot test: metricLatency 1`] = ` +Object { + "indexingLatencyMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "IndexingLatency", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, + "searchLatencyMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "SearchLatency", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, +} +`; + +exports[`snapshot test: metricMasterCpuUtilization 1`] = ` +Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "MasterCPUUtilization", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Maximum", + "unit": undefined, +} +`; + +exports[`snapshot test: metricMasterJVMMemoryPressure 1`] = ` +Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "MasterJVMMemoryPressure", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Maximum", + "unit": undefined, +} +`; + +exports[`snapshot test: metricMasterReachableFromNode 1`] = ` +Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "MasterReachableFromNode", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 3600000, + "isoLabel": "H", + "label": "hours", + }, + }, + "region": undefined, + "statistic": "Minimum", + "unit": undefined, +} +`; + +exports[`snapshot test: metricMasterSysMemoryUtilization 1`] = ` +Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "MasterSysMemoryUtilization", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Maximum", + "unit": undefined, +} +`; + +exports[`snapshot test: metricNodes 1`] = ` +Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "Nodes", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 3600000, + "isoLabel": "H", + "label": "hours", + }, + }, + "region": undefined, + "statistic": "Maximum", + "unit": undefined, +} +`; + +exports[`snapshot test: metricRate 1`] = ` +Object { + "indexingRateMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "IndexingRate", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, + "searchRateMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "SearchRate", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, +} +`; + +exports[`snapshot test: metricRequests 1`] = ` +Object { + "invalidHostHeaderRequests": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "InvalidHostHeaderRequests", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, + "openSearchRequests": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "OpenSearchRequests", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, +} +`; + +exports[`snapshot test: metricResponseCodes 1`] = ` +Object { + "2xxMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "2xx", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, + "3xxMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "3xx", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, + "4xxMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "4xx", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, + "5xxMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "5xx", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, +} +`; + +exports[`snapshot test: metricShards 1`] = ` +Object { + "shardsActive": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "Shards.active", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, + "shardsActivePrimary": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "Shards.activePrimary", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, + "shardsDelayedUnassigned": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "Shards.delayedUnassigned", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, + "shardsInitializing": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "Shards.initializing", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, + "shardsRelocating": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "Shards.relocating", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, + "shardsUnassigned": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "Shards.unassigned", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, +} +`; + +exports[`snapshot test: metricStorage 1`] = ` +Object { + "clusterUsedSpaceMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ClusterUsedSpace", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, + "freeStorageSpaceMetric": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "FreeStorageSpace", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Minimum", + "unit": undefined, + }, +} +`; + +exports[`snapshot test: metricThreadpoolQueues 1`] = ` +Object { + "threadpoolBulkQueue": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ThreadpoolBulkQueue", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, + "threadpoolForceMergeQueue": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ThreadpoolForce_mergeQueue", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, + "threadpoolIndexQueue": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ThreadpoolIndexQueue", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, + "threadpoolSearchQueue": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ThreadpoolSearchQueue", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, + "threadpoolWriteQueue": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ThreadpoolWriteQueue", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, +} +`; + +exports[`snapshot test: metricThreadpoolRejected 1`] = ` +Object { + "threadpoolBulkRejected": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ThreadpoolBulkRejected", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, + "threadpoolForceMergeRejected": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ThreadpoolForce_mergeRejected", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, + "threadpoolIndexRejected": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ThreadpoolIndexRejected", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, + "threadpoolSearchRejected": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ThreadpoolSearchRejected", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, + "threadpoolWriteRejected": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ThreadpoolWriteRejected", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, +} +`; + +exports[`snapshot test: metricThreadpoolThreads 1`] = ` +Object { + "threadpoolBulkThreads": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ThreadpoolBulkThreads", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, + "threadpoolForceMergeThreads": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ThreadpoolForce_mergeThreads", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, + "threadpoolIndexThreads": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ThreadpoolIndexThreads", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, + "threadpoolSearchThreads": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ThreadpoolSearchThreads", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, + "threadpoolWriteThreads": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ThreadpoolWriteThreads", + "namespace": "AWS/ES", + "period": Duration { + "amount": 1, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Average", + "unit": undefined, + }, +} +`; + +exports[`snapshot test: metricWriteRejections 1`] = ` +Object { + "CoordinatingWriteRejected": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "CoordinatingWriteRejected", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, + "PrimaryWriteRejected": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "PrimaryWriteRejected", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, + "ReplicaWriteRejected": Metric { + "account": undefined, + "color": undefined, + "dimensions": Object { + "ClientId": "012345679012", + "DomainName": "DummyDomainName", + }, + "label": undefined, + "metricName": "ReplicaWriteRejected", + "namespace": "AWS/ES", + "period": Duration { + "amount": 5, + "unit": TimeUnit { + "inMillis": 60000, + "isoLabel": "M", + "label": "minutes", + }, + }, + "region": undefined, + "statistic": "Sum", + "unit": undefined, + }, +} +`; diff --git a/test/monitoring/aws/opensearch/metrics.test.ts b/test/monitoring/aws/opensearch/metrics.test.ts new file mode 100644 index 00000000..6186ccf9 --- /dev/null +++ b/test/monitoring/aws/opensearch/metrics.test.ts @@ -0,0 +1,424 @@ +import { Aws } from 'aws-cdk-lib'; +import { OpenSearchMetricFactory } from '../../../../src/monitoring/aws/opensearch/metrics'; +const DummyDomainName = 'DummyDomainName'; +const DummyAccountId = '012345679012'; + +Object.defineProperty(Aws, 'ACCOUNT_ID', { + get: jest.fn(), +}); +jest.spyOn(Aws, 'ACCOUNT_ID', 'get').mockReturnValue(DummyAccountId); + + +test('snapshot test: metricClusterStatus', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricClusterStatus(DummyDomainName); + + // THEN + Object.values(metric).forEach(eachMetric => { + expect(eachMetric.metricName).toStrictEqual(eachMetric.metricName); + expect(eachMetric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + }); + expect(metric).toMatchSnapshot(); + +}); + +test('snapshot test: metricFreeStorageSpace', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricFreeStorageSpace(DummyDomainName); + + // THEN + expect(metric.metricName).toStrictEqual('FreeStorageSpace'); + expect(metric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + expect(metric).toMatchSnapshot(); + +}); + +test('snapshot test: metricClusterIndexWritesBlocked', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricClusterIndexWritesBlocked(DummyDomainName); + + // THEN + expect(metric.metricName).toStrictEqual('ClusterIndexWritesBlocked'); + expect(metric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + expect(metric).toMatchSnapshot(); + +}); + +test('snapshot test: metricNodes', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricNodes(DummyDomainName); + + // THEN + expect(metric.metricName).toStrictEqual('Nodes'); + expect(metric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + expect(metric).toMatchSnapshot(); + +}); + +test('snapshot test: metricAutomatedSnapshotFailure', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricAutomatedSnapshotFailure(DummyDomainName); + + // THEN + expect(metric.metricName).toStrictEqual('AutomatedSnapshotFailure'); + expect(metric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricCpuUtilization', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricCpuUtilization(DummyDomainName); + + // THEN + expect(metric.metricName).toStrictEqual('CPUUtilization'); + expect(metric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricJVMMemoryPressure', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricJVMMemoryPressure(DummyDomainName); + + // THEN + expect(metric.metricName).toStrictEqual('JVMMemoryPressure'); + expect(metric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricKms', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricKms(DummyDomainName); + + // THEN + + Object.values(metric).forEach(eachMetric => { + expect(eachMetric.metricName).toStrictEqual(eachMetric.metricName); + expect(eachMetric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + }); + + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricShards', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricShards(DummyDomainName); + + // THEN + Object.values(metric).forEach(eachMetric => { + expect(eachMetric.metricName).toStrictEqual(eachMetric.metricName); + expect(eachMetric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + }); + + expect(metric).toMatchSnapshot(); + +}); + +test('snapshot test: metricHttp5xxPercentage', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricHttp5xxPercentage(DummyDomainName); + + // THEN + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricThreadpoolQueues', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricThreadpoolQueues(DummyDomainName); + + // THEN + Object.values(metric).forEach(eachMetric => { + expect(eachMetric.metricName).toStrictEqual(eachMetric.metricName); + expect(eachMetric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + }); + + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricThreadpoolThreads', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricThreadpoolThreads(DummyDomainName); + + // THEN + Object.values(metric).forEach(eachMetric => { + expect(eachMetric.metricName).toStrictEqual(eachMetric.metricName); + expect(eachMetric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + }); + + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricThreadpoolRejected', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricThreadpoolRejected(DummyDomainName); + + // THEN + + Object.values(metric).forEach(eachMetric => { + expect(eachMetric.metricName).toStrictEqual(eachMetric.metricName); + expect(eachMetric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + }); + + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricWriteRejections', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricWriteRejections(DummyDomainName); + + // THEN + + Object.values(metric).forEach(eachMetric => { + expect(eachMetric.metricName).toStrictEqual(eachMetric.metricName); + expect(eachMetric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + }); + + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricRequests', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricRequests(DummyDomainName); + + // THEN + + Object.values(metric).forEach(eachMetric => { + expect(eachMetric.metricName).toStrictEqual(eachMetric.metricName); + expect(eachMetric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + }); + + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricStorage', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricStorage(DummyDomainName); + + // THEN + + Object.values(metric).forEach(eachMetric => { + expect(eachMetric.metricName).toStrictEqual(eachMetric.metricName); + expect(eachMetric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + }); + + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricDocuments', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricDocuments(DummyDomainName); + + // THEN + + Object.values(metric).forEach(eachMetric => { + expect(eachMetric.metricName).toStrictEqual(eachMetric.metricName); + expect(eachMetric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + }); + + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricLatency', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricLatency(DummyDomainName); + + // THEN + + Object.values(metric).forEach(eachMetric => { + expect(eachMetric.metricName).toStrictEqual(eachMetric.metricName); + expect(eachMetric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + }); + + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricRate', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricRate(DummyDomainName); + + // THEN + + Object.values(metric).forEach(eachMetric => { + expect(eachMetric.metricName).toStrictEqual(eachMetric.metricName); + expect(eachMetric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + }); + + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricClusterEbsLatency', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricClusterEbsLatency(DummyDomainName); + + // THEN + + Object.values(metric).forEach(eachMetric => { + expect(eachMetric.metricName).toStrictEqual(eachMetric.metricName); + expect(eachMetric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + }); + + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricClusterEbsThroughput', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricClusterEbsThroughput(DummyDomainName); + + // THEN + + Object.values(metric).forEach(eachMetric => { + expect(eachMetric.metricName).toStrictEqual(eachMetric.metricName); + expect(eachMetric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + }); + + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricClusterEbsIOPS', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricClusterEbsIOPS(DummyDomainName); + + // THEN + + Object.values(metric).forEach(eachMetric => { + expect(eachMetric.metricName).toStrictEqual(eachMetric.metricName); + expect(eachMetric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + }); + + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricResponseCodes', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricResponseCodes(DummyDomainName); + + // THEN + + Object.values(metric).forEach(eachMetric => { + expect(eachMetric.metricName).toStrictEqual(eachMetric.metricName); + expect(eachMetric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + }); + + expect(metric).toMatchSnapshot(); +}); + + +test('snapshot test: metricMasterCpuUtilization', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricMasterCpuUtilization(DummyDomainName); + + // THEN + expect(metric.metricName).toStrictEqual('MasterCPUUtilization'); + expect(metric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricMasterJVMMemoryPressure', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricMasterJVMMemoryPressure(DummyDomainName); + + // THEN + expect(metric.metricName).toStrictEqual('MasterJVMMemoryPressure'); + expect(metric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricMasterSysMemoryUtilization', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricMasterSysMemoryUtilization(DummyDomainName); + + // THEN + expect(metric.metricName).toStrictEqual('MasterSysMemoryUtilization'); + expect(metric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + expect(metric).toMatchSnapshot(); +}); + +test('snapshot test: metricMasterReachableFromNode', () => { + // GIVEN + const unitToTest = new OpenSearchMetricFactory(); + + // WHEN + const metric = unitToTest.metricMasterReachableFromNode(DummyDomainName); + + // THEN + expect(metric.metricName).toStrictEqual('MasterReachableFromNode'); + expect(metric.dimensions).toStrictEqual({ DomainName: DummyDomainName, ClientId: DummyAccountId }); + expect(metric).toMatchSnapshot(); +}); \ No newline at end of file