Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: Migrating from Grafana Agent Operator to Flow #5259

Merged
merged 49 commits into from
Oct 24, 2023
Merged
Changes from 14 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
0102c60
initial text
captncraig Aug 10, 2023
e5618e8
Merge remote-tracking branch 'origin/main' into cmp_guide
captncraig Sep 20, 2023
fab2321
more text
captncraig Sep 20, 2023
c6d748f
some formatting
captncraig Sep 20, 2023
d8b2155
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 12, 2023
7967605
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 12, 2023
0cffa43
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 12, 2023
fbc8ed1
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 12, 2023
ac297ee
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 12, 2023
187affe
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 12, 2023
b124517
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 13, 2023
69b2273
Merge branch 'main' into cmp_guide
captncraig Oct 13, 2023
22fe64f
word on integrations
captncraig Oct 13, 2023
ce85ab7
link to deploy-agent
captncraig Oct 13, 2023
f1ffefd
Merge branch 'main' into cmp_guide
captncraig Oct 17, 2023
c95705c
tested configs more
captncraig Oct 17, 2023
71afe3d
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 19, 2023
34607ea
add loki example
captncraig Oct 19, 2023
92803d9
Merge branch 'cmp_guide' of github.com:grafana/agent into cmp_guide
captncraig Oct 19, 2023
76870b8
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 19, 2023
e21092c
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 19, 2023
9cd48af
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 19, 2023
09970bf
suggestion
captncraig Oct 19, 2023
e0217ef
Merge branch 'cmp_guide' of github.com:grafana/agent into cmp_guide
captncraig Oct 19, 2023
ad9f319
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 20, 2023
bb4f1ff
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 20, 2023
64bea6e
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 20, 2023
cd9fec1
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 20, 2023
6a65ee5
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 20, 2023
9a9781a
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 20, 2023
e3e359f
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 20, 2023
5b9ce16
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 20, 2023
0cc95d6
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 20, 2023
252494d
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 20, 2023
adf2cb6
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 20, 2023
cca35da
capitalization
captncraig Oct 20, 2023
efb60c9
reviews
captncraig Oct 23, 2023
0858a8f
Merge branch 'main' into cmp_guide
captncraig Oct 23, 2023
4edc051
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 24, 2023
e846b71
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 24, 2023
f24397f
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 24, 2023
88f2f84
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 24, 2023
3afc7e3
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 24, 2023
a8c572a
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 24, 2023
2893983
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 24, 2023
2c7734d
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 24, 2023
c280e3a
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 24, 2023
0f8024e
Update docs/sources/flow/getting-started/migrating-from-operator.md
captncraig Oct 24, 2023
aca0c63
Merge branch 'main' into cmp_guide
captncraig Oct 24, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 128 additions & 0 deletions docs/sources/flow/getting-started/migrating-from-operator.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
---
canonical: https://grafana.com/docs/agent/latest/flow/getting-started/migrating-from-operator/
description: Migrating from Grafana Agent Operator to Grafana Agent Flow
menuTitle: Migrate from Operator
title: Migrating from Grafana Agent Operator to Grafana Agent Flow
weight: 320
---

# Migrating from Grafana Agent Operator to Grafana Agent Flow

With the release of flow, Grafana Agent Operator is no longer the recommended way to deploy Grafana Agent in Kubernetes. Some of the Operator functionality has been moved into Grafana Agent
captncraig marked this conversation as resolved.
Show resolved Hide resolved
itself, and the remaining functionality has been replaced by our Helm Chart.

- The Monitor types (`PodMonitor`, `ServiceMonitor`, `Probe`, and `LogsInstance`) are all supported natively by Grafana Agent in Flow mode. You no longer are
captncraig marked this conversation as resolved.
Show resolved Hide resolved
captncraig marked this conversation as resolved.
Show resolved Hide resolved
required to use the Operator to consume those CRDs for dynamic monitoring in your cluster.
- The parts of the Operator that deploy the agent itself (`GrafanaAgent`, `MetricsInstance`, and `LogsInstance` CRDs) are deprecated. We now recommend
captncraig marked this conversation as resolved.
Show resolved Hide resolved
operator users use the [Grafana Agent Helm Chart](https://grafana.com/docs/agent/latest/flow/setup/install/kubernetes/) to deploy the Agent directly to your clusters.

This guide will provide some steps to get started with Grafana Agent for users coming from Grafana Agent Operator.

## Deploy Grafana Agent with Helm

1. You will need to create a `values.yaml` file, which contains options for how to deploy your agent. You may start with the [default values](https://github.com/grafana/agent/blob/main/operations/helm/charts/grafana-agent/values.yaml) and customize as you see fit, or start with this snippet, which should be a good starting point for what the operator does:
captncraig marked this conversation as resolved.
Show resolved Hide resolved

```yaml
agent:
mode: 'flow'
configMap:
create: true
clustering:
enabled: true
controller:
type: 'statefulset'
replicas: 2
```

This config will deploy Grafana Agent as a `StatefulSet` using the built in [clustering](https://grafana.com/docs/agent/latest/flow/concepts/clustering/) functionality to allow distributing scrapes across all Agent Pods.
captncraig marked this conversation as resolved.
Show resolved Hide resolved
captncraig marked this conversation as resolved.
Show resolved Hide resolved

This is not the only deployment mode possible. For example, you may desire a `DaemonSet` if collecting host level logs or metrics. See [this page](https://grafana.com/docs/agent/latest/flow/setup/deploy-agent/) for more details about different topologies.
captncraig marked this conversation as resolved.
Show resolved Hide resolved

2. Create a flow config file, `agent.river`.

You can add any config you need directly to this file.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be good to link to the guide for migrating static configs to flow ones, e.g.:

Suggested change
You can add any config you need directly to this file.
You can add any config you need directly to this file. See [Migrating from Static](https://grafana.com/docs/agent/latest/flow/getting-started/migrating-from-static/) for how to migrate a static mode config to Flow.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also add an example of agent flow config? If in the first step were are showing an snippet with an specific config example, why not here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If they're operator users, they won't have directly interacted with static config, so I'm not sure how relevant that is. Since all the config comes from crds, there isn't any real "initial config", other than what they add below to correspond with a metrics instance. Maybe I can clarify that at least.


3. Install the grafana helm repository:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
3. Install the grafana helm repository:
3. Install the Grafana Helm repository:


```
helm repo add grafana https://grafana.github.io/helm-charts
helm repo update
```

4. Create a helm relase. You may name the relase anything you like. Here we are installing a release named `grafana-agent` in the `monitoring` namespace.

```
helm upgrade grafana-agent grafana/grafana-agent -i -n monitoring -f values.yaml --set-file agent.configMap.content=agent.river
```

This command uses the `--set-file` flag to pass the config file as a helm value, so that we can continue to edit it as a regular river file.
captncraig marked this conversation as resolved.
Show resolved Hide resolved

## Convert `MetricsIntances` to flow components

A `MetricsInstance` resource primarily defines:

- The remote endpoint(s) Grafana Agent should send metrics to.
- Which `PodMonitor`, `ServiceMonitor`, and `Probe` resources this Agent should discover.

These functions can be done in Grafana Agent Flow with the `prometheus.remote_write`, `prometheus.operator.podmonitors`, `prometheus.operator.servicemonitors`, and `prometheus.operator.probes` components respectively.

This is a river sample that is equivalent to the `MetricsInstance` from our [operator guide](https://grafana.com/docs/agent/latest/operator/deploy-agent-operator-resources/#deploy-a-metricsinstance-resource):
captncraig marked this conversation as resolved.
Show resolved Hide resolved

```river

// read the credentials secret for remote_write authorization
remote.kubernetes.secret "credentials" {
namespace = "monitoring"
name = "primary-credentials-metrics"
Comment on lines +77 to +78
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
namespace = "monitoring"
name = "primary-credentials-metrics"
namespace = "monitoring"
name = "primary-credentials-metrics"

}

prometheus.remote_write "primary" {
endpoint {
url = your_remote_write_URL
basic_auth {
username = nonsensitive(remote.kubernetes.secret.credentials.data["username"])
password = remote.kubernetes.secret.credentials.data["password"]
}
}
}

prometheus.operator.podmonitors "primary" {
forward_to = [prometheus.remote_write.primary.receiver]
// leave out selector to find all podmonitors in the entire cluster
selector {
key = "instance"
operator = "In"
values = ["primary"]
}
}

prometheus.operator.servicemonitors "primary" {
forward_to = [prometheus.remote_write.primary.receiver]
// leave out selector to find all podmonitors in the entire cluster
selector {
key = "instance"
operator = "In"
values = ["primary"]
}
}

```

This config will discover all `PodMonitor`, `ServiceMonitor`, and `Probe` resources in your cluster that match our label selector `instance=primary`. It will then scrape metrics from their targets, and forward them on to your remote write endpoint.

If you are using additional features in your `MetricsInstance` resources, you may need to further customize this config. Please see the documentation for the relevant components for additional information:
captncraig marked this conversation as resolved.
Show resolved Hide resolved

- [remote.kubernetes.secret](https://grafana.com/docs/agent/latest/flow/reference/components/remote.kubernetes.secret)
- [prometheus.remote_write](https://grafana.com/docs/agent/latest/flow/reference/components/prometheus.remote_write)
- [prometheus.operator.podmonitors](https://grafana.com/docs/agent/latest/flow/reference/components/prometheus.operator.podmonitors)
- [prometheus.operator.servicemonitors](https://grafana.com/docs/agent/latest/flow/reference/components/prometheus.operator.servicemonitors)
- [prometheus.operator.probes](https://grafana.com/docs/agent/latest/flow/reference/components/prometheus.operator.probes)
- [prometheus.scrape](https://grafana.com/docs/agent/latest/flow/reference/components/prometheus.scrape)

## Collecting Logs



## Integrations

The `Integration` CRD is not supported with Grafana Agent Flow, however all static mode integrations have an eqivalent component in the [`prometheus.exporter`](https://grafana.com/docs/agent/latest/flow/reference/components) namespace. The reference docs should help convert those integrations to their flow equivalent.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to mention here loki.source.kubernetes_events as it was an integration that was not an exporter? WDYT here documenting prometheus.exporter component configured to run in combination with MetricsIntances replacements?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MetricsInstance didn't specify any integrations, there was a seperate CRD for that which took pretty free-form arguments. What if we added an info box on the static mode integration pages that linked to the matching flow component?