Skip to content

Commit

Permalink
Cherry-pick documentation tweaks back to v0.28 release (#2247)
Browse files Browse the repository at this point in the history
Signed-off-by: Paschalis Tsilias <[email protected]>
Co-authored-by: mattdurham <[email protected]>
Co-authored-by: Robert Fratto <[email protected]>
Co-authored-by: Karen Germond <[email protected]>
  • Loading branch information
4 people authored Sep 29, 2022
1 parent eb475ce commit b5cc08c
Show file tree
Hide file tree
Showing 46 changed files with 1,427 additions and 557 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/scripts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ jobs:
- uses: actions/checkout@v3
- uses: azohra/shell-linter@latest
with:
path: "packaging,production,tools/ci,tools/release"
path: "packaging,production,tools/ci,tools/release,docs/sources/flow/tutorials/assets"
304 changes: 304 additions & 0 deletions docs/sources/assets/concepts_example_pipeline.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
216 changes: 216 additions & 0 deletions docs/sources/assets/flow_referencing_exports_diagram.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/sources/assets/ui_component_detail_page.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/sources/assets/ui_graph_page.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/sources/assets/ui_home_page.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
24 changes: 21 additions & 3 deletions docs/sources/flow/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ Grafana Agent Flow is a _component-based_ experimental revision of Grafana
Agent with a focus on ease-of-use, debuggability, and ability to adapt to the
needs of power users.

Components allow for reusability, composability, and focus on a single task.

* **Reusability** allows for the output of components to be reused as the input for multiple other components.
* **Composability** allows for components to be chained together to form a pipeline.
* **Single task** means the scope of a component is limited to one narrow task and thus has fewer side effects.

> **EXPERIMENTAL**: Grafana Agent Flow is an [experimental][] feature.
> Experimental features are subject to frequent breaking changes and are
> subject for removal if the experiment doesn't work out.
Expand All @@ -24,9 +30,10 @@ needs of power users.
## Features

* Write declarative configurations with a Terraform-inspired configuration
language
* Declare components to configure parts of a pipeline
* Use expressions to bind components together to build a programmable pipeline
language.
* Declare components to configure parts of a pipeline.
* Use expressions to bind components together to build a programmable pipeline.
* Includes a UI for debugging the state of a pipeline.

## Example

Expand Down Expand Up @@ -77,6 +84,17 @@ prometheus.remote_write "default" {
[configuration language]: {{< relref "./config-language/" >}}
[reference documentation]: {{< relref "./reference" >}}

## Current limitations

The goal of Grafana Agent Flow is to eventually support the same use cases that
Grafana Agent does today. Some functionality may be missing while Grafana Agent
Flow is still in development:

* Logging-specific components
* Tracing-specific components
* An equivalent list of integrations
* An equivalent to the scraping service

## Provide feedback

Feedback about Grafana Agent Flow and its configuration language can be
Expand Down
64 changes: 32 additions & 32 deletions docs/sources/flow/concepts/component_controller.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,16 @@ components at runtime.

It is responsible for:

* Reading and validating the config file
* Managing the lifecycle of defined components
* Evaluating the arguments used to configure components
* Reporting the health of defined components
* Reading and validating the config file.
* Managing the lifecycle of defined components.
* Evaluating the arguments used to configure components.
* Reporting the health of defined components.

## Component graph

As discussed in [Components][], a relationship between components is created
when an expression is used to set the argument of one component to an export of
another component.
when an expression is used to set the argument of one component to an exported
field of another component.

The set of all components and the relationships between them define a [directed
acyclic graph][DAG] (DAG), which informs the component controller which
Expand All @@ -31,7 +31,7 @@ For a config file to be valid, components must not reference themselves or
contain a cyclic reference:

```river
// INVALID: local.file.some_file may not reference itself:
// INVALID: local.file.some_file can not reference itself:
local.file "self_reference" {
filename = local.file.self_reference.content
}
Expand All @@ -50,60 +50,60 @@ local.file "b" {
## Component evaluation

A component is _evaluated_ when its expressions are computed into concrete
values and used to configure the component. The component controller is
finished loading once all components have been evaluated and are running.
values. The computed values are then used to configure the component's runtime
behavior. The component controller is finished loading once all components are
evaluated, configured, and running.

The component controller is guaranteed to only evaluate a given component after
evaluating all of the components the given component references. Components
which do not reference other components may be evaluated at any time during the
evaluation process.
The component controller only evaluates a given component after evaluating all
of that component's dependencies. Component that do not depend on other
components can be evaluated at any time during the evaluation process.

## Component reevaluation

As mentioned in [Components][], a component is dynamic: a component may update
As mentioned in [Components][], a component is dynamic: a component can update
its exports any number of times throughout its lifetime.

When a component updates its exports, a _controller reevaluation_ is triggered:
the component controller will reevaluate any component which references the
changed component, any components which reference those components, and so on,
until all affected components have been reevaluated.
the component controller reevaluates any component which references the changed
component, any components which reference those components, and so on, until
all affected components are reevaluated.

## Component health

At any given time, a component may have one of the following health states:
At any given time, a component can have one of the following health states:

1. Unknown: default state, the component isn't running yet
2. Healthy: the component is working as expected
3. Unhealthy: the component is not working as expected
4. Exited: the component has stopped and is no longer running
1. Unknown: default state, the component isn't running yet.
2. Healthy: the component is working as expected.
3. Unhealthy: the component is not working as expected.
4. Exited: the component has stopped and is no longer running.

By default, the component controller determines the health of a component. The
component controller will mark a component has healthy as long as that
component is running and its most recent evaluation was successful.
component controller marks a component as healthy as long as that component is
running and its most recent evaluation succeeded.

Some components may report their own component-specific health information. For
example, the `local.file` component will report itself unhealthy if the file it
Some components can report their own component-specific health information. For
example, the `local.file` component reports itself as unhealthy if the file it
was watching gets deleted.

The overall health of a component is reported by combining the
The overall health of a component is determined by combining the
controller-reported health of the component with the component-specific health
information.

An individual component's health is independent from the health of any other
components it references: a component may be marked as healthy even if it
references the exports of an unhealthy component.
components it references: a component can be marked as healthy even if it
references an exported field of an unhealthy component.

## Handling evaluation failures

When a component fails to evaluate, it is marked as unhealthy with the reason
for why the evaluation failed.

When an evaluation fails, the component continues operating as normal: it
continues using its previous set of evaluated arguments, and it may continue
When an evaluation fails, the component continue operating as normal: it
continues using its previous set of evaluated arguments, and it can continue
exporting new values.

This prevents failure propagation: if your `local.file` component which watches
API keys suddenly stops working, other components will continue using the last
API keys suddenly stops working, other components continues using the last
valid API key until the component returns to a healthy state.

## Updating the config file
Expand Down
44 changes: 25 additions & 19 deletions docs/sources/flow/concepts/components.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,16 @@ collecting Prometheus metrics.

Components are composed of two parts:

* Arguments: settings which configure a component
* Exports: named values which a component exposes to other components
* Arguments: settings which configure a component.
* Exports: named values which a component exposes to other components.

Each component has a name which describes what that component is responsible
for. For example, the `local.file` component is responsible for retrieving the
contents of files on disk.

Components are specified by users by first providing the component's name with
a user-specified label, and then by providing arguments to configure the
component:
Components are specified in the config file by first providing the component's
name with a user-specified label, and then by providing arguments to configure
the component:

```
discovery.kubernetes "pods" {
Expand All @@ -44,18 +44,20 @@ discovery.kubernetes "nodes" {
## Pipelines

Most arguments a user specifies for a component will be some constant value,
such setting a `log_level` attribute to the quoted string `"debug"` (`log_level
= "debug"`).
Most arguments for a component in a config file are constant values, such
setting a `log_level` attribute to the quoted string `"debug"`:

Users may also use _expressions_ to dynamically compute the value of an
argument at runtime. Among other things, expressions may be used to retrieving
the value of an environment variable (`log_level = env("LOG_LEVEL")`) or
referencing the export of another component (`log_level =
local.file.log_level.content`).
```river
log_level = "debug"
```

When a user configures a component's argument to reference an export of another
component, a relationship is created: a component's input (arguments) now
_Expressions_ can be used to dynamically compute the value of an argument at
runtime. Among other things, expressions can be used to retrieve the value of
an environment variable (`log_level = env("LOG_LEVEL")`) or to reference an
exported field of another component (`log_level = local.file.log_level.content`).

When a component's argument references an exported field of another component,
a dependant relationship is created: a component's input (arguments) now
depends on another component's output (exports). The input of the component
will now be re-evaluated any time the exports of the components it references
get updated.
Expand All @@ -75,21 +77,25 @@ An example pipeline may look like this:
component, and sends collected metrics to the `prometheus.remote_write`
component.

A user would use this config file to represent the above pipeline:
<p align="center">
<img src="../../../assets/concepts_example_pipeline.svg" alt="Flow of example pipeline" width="500" />
</p>

The following config file represents the above pipeline:

```river
// Get our API key from disk.
//
// This component has an exported value called "content", holding the content
// This component has an exported field called "content", holding the content
// of the file.
//
// local.file.api_key will watch the file and update its exports any time the
// file changes.
local.file "api_key" {
filename = "/var/data/secrets/api-key"
// Mark this file as sensitive to prevent its value from being shown to
// users.
// Mark this file as sensitive to prevent its value from being shown in the
// UI.
is_secret = true
}
Expand Down
2 changes: 1 addition & 1 deletion docs/sources/flow/concepts/configuration_language.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ River was designed with the following requirements in mind:
## Attributes

_Attributes_ are used to configure individual settings. They always take the
form of `<ATTRIBUTE_NAME> = <ATTRIBUTE_VALUE>`.
form of `ATTRIBUTE_NAME = ATTRIBUTE_VALUE`.

```river
log_level = "debug"
Expand Down
10 changes: 5 additions & 5 deletions docs/sources/flow/config-language/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ weight: 400

# Configuration language

Grafana Agent Flow uses a custom configuration language called River to
Grafana Agent Flow contains a custom configuration language called River to
dynamically configure and connect components.

River aims to reduce errors in configuration files by making configurations
Expand All @@ -28,15 +28,15 @@ local.file "my_file" {
}
// Pattern for creating a labeled block, which the above block follows:
<BLOCK NAME> "<BLOCK LABEL>" {
BLOCK_NAME "BLOCK_LABEL" {
// Block body
<IDENTIFIER> = <EXPRESSION> // Attribute
IDENTIFIER = EXPRESSION // Attribute
}
// Pattern for creating an unlabeled block:
<BLOCK NAME> {
BLOCK_NAME {
// Block body
<IDENTIFIER> = <EXPRESSION> // Attribute
IDENTIFIER = EXPRESSION // Attribute
}
```

Expand Down
38 changes: 19 additions & 19 deletions docs/sources/flow/config-language/components.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,17 +49,17 @@ as well as whether its contents are sensitive or not.

```river
local.file "targets" {
// Required argument
filename = "/etc/agent/targets"
// Required argument
filename = "/etc/agent/targets"
// Optional arguments: Components may have some optional arguments that
// do not need to be defined.
//
// The optional arguments for local.file are is_secret, detector, and
// poll_frequency.
// Optional arguments: Components may have some optional arguments that
// do not need to be defined.
//
// The optional arguments for local.file are is_secret, detector, and
// poll_frequency.
// Exports: a single field named `content`
// It can be referred to as `local.file.targets.content`
// Exports: a single field named `content`
// It can be referred to as `local.file.targets.content`
}
```

Expand All @@ -74,22 +74,22 @@ an expression that ties the target to the value of

```river
prometheus.scrape "default" {
targets = [
{ "__address__" = local.file.targets.content }, // tada!
{ "__address__" = "localhost:9001" },
]
forward_to = [prometheus.remote_write.default.receiver]
scrape_config {
job_name = "default"
}
targets = [
{ "__address__" = local.file.targets.content }, // tada!
{ "__address__" = "localhost:9001" },
]
forward_to = [prometheus.remote_write.default.receiver]
scrape_config {
job_name = "default"
}
}
```

Every time the file contents change, the `local.file` will update its exports,
so the new value will provided to `prometheus.scrape` targets field.

All arguments and exports have an underlying [type]({{< relref "./expressions/types_and_values.md" >}}).
Each argument and exported field has an underlying [type]({{< relref "./expressions/types_and_values.md" >}}).
River will type-check expressions before assigning a value to an attribute; the
documentation of each component will have more information about the ways that
you can wire components together.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ weight: 400

# Function Calls
Function calls is one more River feature that lets users build richer
expresisons.
expressions.

Functions take zero or more arguments as their input and always return a single
value as their output. Functions cannot be constructed by users, but can be
Expand Down
Loading

0 comments on commit b5cc08c

Please sign in to comment.