Skip to content

Commit

Permalink
Code review feedback
Browse files Browse the repository at this point in the history
  • Loading branch information
kgeckhart committed Dec 21, 2023
1 parent 1951e08 commit d7ed095
Show file tree
Hide file tree
Showing 3 changed files with 34 additions and 25 deletions.
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,11 @@ Main (unreleased)

- Added 'country' mmdb-type to log pipeline-stage geoip. (@superstes)

- In the azure exporter, dimension options will no longer be validated by the Azure API. (@kgeckhart)
- This change will not break any existing configurations and you can opt in to validation via the `validate_dimensions` configuration option.
- Before this change, pulling metrics for azure resources with variable dimensions required one configuration per metric + dimension combination to avoid an error.
- After this change, you can include all metrics and dimensions in a single configuration and the Azure APIs will only return dimensions which are valid for the various metrics.

### Bugfixes

- Update `pyroscope.ebpf` to fix a logical bug causing to profile to many kthreads instead of regular processes https://github.com/grafana/pyroscope/pull/2778 (@korniltsev)
Expand Down
28 changes: 15 additions & 13 deletions docs/sources/flow/reference/components/prometheus.exporter.azure.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,22 +17,24 @@ The exporter supports all metrics defined by Azure Monitor. You can find the com
Metrics for this integration are exposed with the template `azure_{type}_{metric}_{aggregation}_{unit}` by default. As an example,
the Egress metric for BlobService would be exported as `azure_microsoft_storage_storageaccounts_blobservices_egress_total_bytes`.

The exporter offers two options for gathering metrics,
1. (Default) Use an [Azure Resource Graph](https://azure.microsoft.com/en-us/get-started/azure-portal/resource-graph/#overview) query to identify resources for gathering metrics
1. This will make 1 API call per resource identified
1. Subscriptions with a reasonable amount of resources are liable to hit the [12000 requests per hour rate limit](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/request-limits-and-throttling#subscription-and-tenant-limits) azure enforces
1. Set the regions to gather metrics from and get metrics for all resources across those regions
1. This will make an API call per subscription reducing the number of API calls dramatically
1. This approach does not work with all resource types and Azure's does not document which resource types do/do not work
1. A resource type which is not support will produce errors which look like `Resource type: microsoft.containerservice/managedclusters not enabled for Cross Resource metrics`
The exporter offers the following two options for gathering metrics.

1. (Default) Use an [Azure Resource Graph](https://azure.microsoft.com/en-us/get-started/azure-portal/resource-graph/#overview) query to identify resources for gathering metrics.
1. This query will make one API call per resource identified.
1. Subscriptions with a reasonable amount of resources can hit the [12000 requests per hour rate limit](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/request-limits-and-throttling#subscription-and-tenant-limits) Azure enforces.
1. Set the regions to gather metrics from and get metrics for all resources across those regions.
1. This option will make one API call per subscription, dramatically reducing the number of API calls.
1. This approach does not work with all resource types, and Azure does not document which resource types do or do not work.
1. A resource type that is not supported produces errors that look like `Resource type: microsoft.containerservice/managedclusters not enabled for Cross Resource metrics`.
1. If you encounter one of these errors you must use the default Azure Resource Graph based option to gather metrics.

## Authentication

{{< param "PRODUCT_NAME" >}} must be running in an environment with access to Azure. The exporter uses the Azure SDK for go and supports [authentication](https://learn.microsoft.com/en-us/azure/developer/go/azure-sdk-authentication?tabs=bash#2-authenticate-with-azure).

The account used by {{< param "PRODUCT_NAME" >}} needs:

- When using an Azure Resoure Graph query, [read access to the resources that will be queried by Resource Graph](https://learn.microsoft.com/en-us/azure/governance/resource-graph/overview#permissions-in-azure-resource-graph)
- When using an Azure Resource Graph query, [read access to the resources that will be queried by Resource Graph](https://learn.microsoft.com/en-us/azure/governance/resource-graph/overview#permissions-in-azure-resource-graph)
- Permissions to call the [Microsoft.Insights Metrics API](https://learn.microsoft.com/en-us/rest/api/monitor/metrics/list) which should be the `Microsoft.Insights/Metrics/Read` permission

## Usage
Expand Down Expand Up @@ -62,11 +64,11 @@ Omitted fields take their default values.

| Name | Type | Description | Default | Required |
|-------------------------------|----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------|----------|
| `subscriptions` | `list(string)` | List of subscriptions to scrap metrics from. | | yes |
| `subscriptions` | `list(string)` | List of subscriptions to scrape metrics from. | | yes |
| `resource_type` | `string` | The Azure Resource Type to scrape metrics for. | | yes |
| `metrics` | `list(string)` | The metrics to scrape from resources. | | yes |
| `resource_graph_query_filter` | `string` | The [Kusto query][] filter to apply when searching for resources. Cannot be used if `regions` is set. | | no |
| `regions` | `list(string)` | The list of regions for gathering metrics and enables gathering metrics for all resources in the subscription. Cannot be used if `resource_graph_query_filter` is set. | | no |
| `resource_graph_query_filter` | `string` | The [Kusto query][] filter to apply when searching for resources. Can't be used if `regions` is set. | | no |
| `regions` | `list(string)` | The list of regions for gathering metrics and enables gathering metrics for all resources in the subscription. Can't be used if `resource_graph_query_filter` is set. | | no |
| `metric_aggregations` | `list(string)` | Aggregations to apply for the metrics produced. | | no |
| `timespan` | `string` | [ISO8601 Duration][] over which the metrics are being queried. | `"PT1M"` (1 minute) | no |
| `included_dimensions` | `list(string)` | List of dimensions to include on the final metrics. | | no |
Expand All @@ -91,7 +93,7 @@ Tags in `included_resource_tags` will be added as labels with the name `tag_<tag

Valid values for `azure_cloud_environment` are `azurecloud`, `azurechinacloud`, `azuregovernmentcloud` and `azurepprivatecloud`.

`validate_dimensions` is disabled by default to reduce the number of azure exporter instances requires when a `resource_type` has metrics with varying dimensions. When `validate_dimensions` is enabled you will need 1 exporter instance per metric + dimension combination which is more tedious to maintain.
`validate_dimensions` is disabled by default to reduce the number of Azure exporter instances requires when a `resource_type` has metrics with varying dimensions. When `validate_dimensions` is enabled you will need one exporter instance per metric + dimension combination which is more tedious to maintain.

[Kusto query]: https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/
[Azure Monitor essentials]: https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/metrics-supported
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,16 @@ The `azure_exporter_config` block configures the `azure_exporter` integration, a
[`azure-metrics-exporter`](https://github.com/webdevops/azure-metrics-exporter), used to
collect metrics from [Azure Monitor](https://azure.microsoft.com/en-us/products/monitor).

The exporter offers two options for gathering metrics,
1. (Default) Use an [Azure Resource Graph](https://azure.microsoft.com/en-us/get-started/azure-portal/resource-graph/#overview) query to identify resources for gathering metrics
1. This will make 1 API call per resource identified
1. Subscriptions with a reasonable amount of resources are liable to hit the [12000 requests per hour rate limit](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/request-limits-and-throttling#subscription-and-tenant-limits) azure enforces
1. Set the regions to gather metrics from and get metrics for all resources across those regions
1. This will make an API call per subscription reducing the number of API calls dramatically
1. This approach does not work with all resource types and Azure's does not document which resource types do/do not work
1. A resource type which is not support will produce errors which look like `Resource type: microsoft.containerservice/managedclusters not enabled for Cross Resource metrics`
The exporter offers the following two options for gathering metrics.

1. (Default) Use an [Azure Resource Graph](https://azure.microsoft.com/en-us/get-started/azure-portal/resource-graph/#overview) query to identify resources for gathering metrics.
1. This query will make one API call per resource identified.
1. Subscriptions with a reasonable amount of resources can hit the [12000 requests per hour rate limit](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/request-limits-and-throttling#subscription-and-tenant-limits) Azure enforces.
1. Set the regions to gather metrics from and get metrics for all resources across those regions.
1. This option will make one API call per subscription, dramatically reducing the number of API calls.
1. This approach does not work with all resource types, and Azure does not document which resource types do or do not work.
1. A resource type that is not supported produces errors that look like `Resource type: microsoft.containerservice/managedclusters not enabled for Cross Resource metrics`.
1. If you encounter one of these errors you must use the default Azure Resource Graph based option to gather metrics.

## List of Supported Services and Metrics
The exporter supports all metrics defined by Azure Monitor. The complete list of available metrics can be found in the [Azure Monitor documentation](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/metrics-supported).
Expand Down Expand Up @@ -102,12 +104,12 @@ The account used by Grafana Agent needs:

# Optional: The [kusto query](https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/) filter to apply when searching for resources
# This value will be embedded in to a template query of the form `Resources | where type =~ "<resource_type>" <resource_graph_query_filter> | project id, tags`
# Cannot be used if `regions` is set.
# Can't be used if `regions` is set.
[resource_graph_query_filter: <string>]

# Optional: The list of regions for gathering metrics. Enables gather metrics for all resources in the subscription.
# The list of available `regions` to your subscription can be found by running the azure CLI command `az account list-locations --query '[].name'`.
# Cannot be used if `resource_graph_query_filter` is set.
# The list of available `regions` to your subscription can be found by running the Azure CLI command `az account list-locations --query '[].name'`.
# Can't be used if `resource_graph_query_filter` is set.
regions:
[ - <string> ... ]

Expand Down Expand Up @@ -155,7 +157,7 @@ The account used by Grafana Agent needs:
[azure_cloud_environment: <string> | default = "azurecloud"]

# Optional: validation is disabled by default to reduce the number of azure exporter instances required when a `resource_type` has metrics with varying dimensions.
# Choosing to enable `validate_dimensions` will require 1 exporter instance per metric + dimension combination which can be very tedious to maintain.
# Choosing to enable `validate_dimensions` will require one exporter instance per metric + dimension combination which can be very tedious to maintain.
[validate_dimensions: <bool> | default = false]
```
Expand Down

0 comments on commit d7ed095

Please sign in to comment.