CodaHale Aggregated Metrics CloudWatch Reporter

This is a CloudWatch Reporter for the stable version of Dropwizard Metrics (formerly CodaHale & Yammer Metrics). The reporter is an implementation of ScheduledReporter from Dropwizard Metrics v4.1.8

Table of Contents

• Prerequisites
• Testing
• Code Coverage
• AWS Java SDK v1 and v2
• Summary
• Reportable Metrics
• Defaults
• Adding cloudwatch reporter to your project
• Usage
  • Dry run
• Credits
• Changelog
• License

Prerequisites

• Java 1.8

Testing

• ./gradlew test runs unit tests

Code Coverage

A code coverage report will be generated under build/reports/jacoco/html/. You need to open index.html in a browser.

• ./gradlew clean jacocoTestReport

AWS Java SDK v1 and v2

The master branch is now on AWS Java SDK v2.x.x. If you want to submit pull requests for v1.x.x, please use com_amazonaws_aws_cloudwatch_1.x.x branch.

Summary

• This CloudWatchReporter reports the metric data to CloudWatch asynchronously using the CloudWatchAsyncClient (AWS) interface
• Each reportable value in CodeHale Metric is reported as a separate MetricDatum (AWS)
• When reporting Meter, Counter, Histogram and Timer count metrics (getCount()) as MetricDatum (AWS), only the count difference since the last report is reported. This way the counters do not require a reset within the application using this reporter.
• If configured, each Snapshot translated into StatisticSet (AWS) in the most direct way possible.
• If configured, JVM statistic is reported

Reportable Metrics

Currently the only metric values that are reportable through configuration are:

• Values of type Number from Gauge
• Counts from Counter, Histogram, Meter and Timer
• Percentiles from Snapshot in Histogram and Timer
• Arithmetic mean & standard deviation of Snapshot values in Histogram and Timer
• Mean rates from Meter and Timer
• Summaries of Snapshot values in Histogram and Timer as StatisticSet (AWS)

Please note:

• Histogram values (the percentiles, min, max, sum, arithmetic mean & std-dev from Snapshot) are reported raw.
• Timer values (the percentiles, min, max, sum, arithmetic mean & std-dev from Snapshot) are reported after conversion by a duration factor was applied. The duration factor is calculated by converting 1 unit of duration unit type to nanoseconds (see ScheduledReporter)
• Meter values (the 1min rate, 5min rate, 15min rate & mean rate) are reported after conversion by a rate factor was applied. The rate factor is calculated by converting 1 unit of rate unit type to seconds (see ScheduledReporter). The Unit of the sent MetricDatum will default to that rate unit if not changed with builder.withMeterUnitSentToCW().

Metric Dimensions

A dimension is a name/value pair that helps you to uniquely identify a metric. Every metric has specific characteristics that describe it, and you can think of dimensions as categories for those characteristics.

Global Dimensions

The reporter can be configured with a set of global dimensions. These will be added to all reported metrics.

Per Metric Dimensions

The reporter will look for dimensions encoded in the metric name when the metric is reported. To add dimensions to a metric, encode them inside square brackets at the end of the metric name;

"metricName[dimension1:value1,dimension2:value2]"

Use DimensionedName to more easily create metric names with dimensions;

final DimensionedName dimensionedName = DimensionedName.withName("test")
        .withDimension("key1", "val1")
        .withDimension("key2", "val2")
        .withDimension("key3", "val3")
        .build();

metricRegistry.counter(dimensionedName.encode()).inc();

It's also possible to derive a new DimensionedName from an existing one;

final DimensionedName dimensionedName = DimensionedName
        .withName("test")
        .withDimension("key1", "val1")
        .withDimension("key2", "val2")
        .withDimension("key3", "val3")
        .build();

final DimensionedName derivedDimensionedName = dimensionedName
        .withDimension("key3", "replaced_value")
        .withDimension("key4", "val4")
        .build();

metricRegistry.counter(dimensionedName.encode()).inc();
metricRegistry.counter(derivedDimensionedName.encode()).inc();

Since DimensionedName is immutable, the string representation returned by encode() is cached. Multiple calls to encode() will return the same String.

Defaults

The Reporter uses the following defaults which can be configured:

• Rate metrics are in TimeUnit.SECONDS
• Duration metrics are in TimeUnit.MILLISECONDS
• MetricFilter.ALL will be used for the Filter
• Clock.defaultClock() will be used for the Clock (Unconfigurable)
• Metrics are reported using standard resolution (can be changed to high resolution)
• Empty global Dimension (AWS) list
• The reporter adds a Type Dimension (AWS) to each reported metric, e.g:

Type	Metric Name
1-min-mean-rate [per-second]	com.example.component.SomeComponent.timer
snapshot-mean [in-milliseconds]	com.example.component.SomeComponent.timer
snapshot-mean	com.example.component.SomeComponent.histogram
95%	com.example.component.SomeComponent.timer
99.5%	com.example.component.SomeComponent.timer
99.5%	com.example.component.SomeComponent.histogram
count	com.example.component.SomeComponent.counter

The only metrics that are reportable by default are:

• Count values (getCount()) from Meter, Counter, Histogram and Timer
• Percentile values (75%, 95%, 99.9%) from Histogram and Timer

All other metrics have to be configured for reporting by invoking their respective withXXXX() methods on the CloudWatchReporter.Builder instance

Adding cloudwatch reporter to your project

The library artifact ID is dropwizard-metrics-cloudwatch if you want to search for it on Maven Central

The following is an example how to include the library in your project using Gradle:

repositories {
    mavenLocal()
    maven { url "https://repo.maven.apache.org/maven2" }
    jcenter { url "https://jcenter.bintray.com" }
}

dependencies { 
    implementation("io.github.azagniotov:dropwizard-metrics-cloudwatch:2.0.8")
}

Usage

The reporter provides a fine-grained configuration options through its builder to configure what metrics should be reported to CloudWatch. Since AWS costs money, you probably do not want to report all the values from Metric classes or Snapshot, but only what's really useful to you.

    final CloudWatchAsyncClient amazonCloudWatchAsync =
            CloudWatchAsyncClient
                    .builder()
                    .region(Region.US_WEST_2)
                    .build();

    final CloudWatchReporter cloudWatchReporter =
            CloudWatchReporter.forRegistry(metricRegistry, amazonCloudWatchAsync, Main.class.getName())
                    .convertRatesTo(TimeUnit.SECONDS)
                    .convertDurationsTo(TimeUnit.MILLISECONDS)
                    .filter(MetricFilter.ALL)
                    .withPercentiles(Percentile.P75, Percentile.P99)
                    .withOneMinuteMeanRate()
                    .withFiveMinuteMeanRate()
                    .withFifteenMinuteMeanRate()
                    .withMeanRate()
                    .withArithmeticMean()
                    .withStdDev()
                    .withStatisticSet()
                    .withZeroValuesSubmission()
                    .withReportRawCountValue()
                    .withHighResolution()
                    .withMeterUnitSentToCW(StandardUnit.BYTES)
                    .withJvmMetrics()
                    .withGlobalDimensions("Region=us-west-2", "Instance=stage")
                    .withDryRun()
                    .build();

    cloudWatchReporter.start(10, TimeUnit.SECONDS);

Dry run

The reporter can be configured to run in DRY RUN mode by invoking .withDryRun() on the Builder. In that case, the reporter will log.DEBUG the created instance of PutMetricDataRequest (AWS), instead of doing a real POST to CloudWatch.

Credits

• https://github.com/blacklocus/metrics-cloudwatch
• https://github.com/tptodorov/metrics-cloudwatch
• https://github.com/basis-technology-corp/metrics-cloudwatch-reporter
• https://github.com/wavefrontHQ/java/tree/master/dropwizard-metrics/3.1

Changelog

2.0.8

• Changes to build.gradle to ensure that upon publishing an artifact, the dependencies section in a generated POM has a compile scope

2.0.7

• Some clean up

2.0.6

• PR #37 Update Metrics dependency to v4.1.8 (https://github.com/madanepuri)

2.0.5

• PR #34 Allow percentiles to be removed (https://github.com/t-taylor)
• PR #35 Update cloudwatch to 2.11.3 (https://github.com/anilkumarmyla)
• PR #36 Only invoke Gauge.getValue once per report (https://github.com/jebl01)

2.0.4

• PR #33 Update cloudwatch to 2.10.56 (https://github.com/anilkumarmyla)

2.0.3

• PR #32 Dependency upgrades (https://github.com/anilkumarmyla)

2.0.2

• PR #30 Dependency upgrades (https://github.com/anilkumarmyla)

2.0.1

• PR #28 Update to cloudwatch 2.10.13 (https://github.com/anilkumarmyla)

2.0.0

• PR #26 Upgrading AWS Java SDK to v2 (https://github.com/mmccuiston)

1.0.13

• PR #23 Makes reported Unit of Meters configurable (https://github.com/tjarkoGrossmann0815)
• PR #24 Makes it possible to provide dimensions per metric (https://github.com/jebl01)

1.0.12

• PR #22 Provides the possibility to report metrics to AWS using high resolution (https://github.com/jebl01)

1.0.11

• PR #21 Removing logging implementation (https://github.com/anilkumarmyla)

1.0.10

• PR #17 Add option to report raw count values instead of delta values (https://github.com/fyi-coursera)

1.0.9

• PR #16 Update Metrics dependency to v4.0.2 to fix java.lang.IncompatibleClassChangeError (https://github.com/jkgeyti)

1.0.8

• PR #14 Replaced Guava usages with Java 8 native APIs (https://github.com/skuehn)

1.0.7

• PR #11 Upgraded Metrics to v4.0.0 (https://github.com/fr3dch3n)

1.0.6

• Issue #8: Make it configurable to send zero values.

1.0.5

• Upgraded Metrics to v3.2.3 due to #1115
• Upgraded AWS Java SDK to v1.11.179

1.0.4

• Issue #4: Not reporting metric zero values.
• PR #6: Reporting Histogram snapshot raw values as StatisticSet, without applying a conversion by duration factor (https://github.com/williedoran)
• Checking isDebugEnabled when logging debug information

1.0.3

• PR #3: Updated dependencies to latest versions (https://github.com/efenderbosch)

1.0.2

• PR #2: Updated AWS SDK to com.amazonaws:aws-java-sdk-cloudwatch:1.11.86 (https://github.com/MeiSign)
• Reporting Histogram snapshot Arithemtic Mean & StdDev raw values, without applying a conversion by duration factor

1.0.1

• Revisited Javadoc
• Added dependency on metrics-jvm module in order to be able to export JVM metrics
• Code clean-up

1.0.0

• Initial release

License

MIT