Skip to content

Commit

Permalink
Merge branch 'main' into basic-auth-for-web-ui
Browse files Browse the repository at this point in the history
  • Loading branch information
hainenber authored Jan 6, 2024
2 parents 6555331 + cbdfc87 commit 2b40e00
Show file tree
Hide file tree
Showing 161 changed files with 2,515 additions and 715 deletions.
37 changes: 30 additions & 7 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,15 @@ internal API changes are not present.
Main (unreleased)
-----------------

v0.39.0-rc.0 (2024-01-05)
-------------------------

### Breaking changes

- `otelcol.receiver.prometheus` will drop all `otel_scope_info` metrics when converting them to OTLP. (@wildum)
- If the `otel_scope_info` metric has labels `otel_scope_name` and `otel_scope_version`,
their values will be used to set OTLP Instrumentation Scope name and version respectively.
- Labels of `otel_scope_info` metrics other than `otel_scope_name` and `otel_scope_version`
their values will be used to set OTLP Instrumentation Scope name and version respectively.
- Labels of `otel_scope_info` metrics other than `otel_scope_name` and `otel_scope_version`
are added as scope attributes with the matching name and version.

- The `target` block in `prometheus.exporter.blackbox` requires a mandatory `name`
Expand All @@ -25,6 +28,10 @@ Main (unreleased)
- This change will not break any existing configurations and you can opt in to validation via the `validate_dimensions` configuration option.
- Before this change, pulling metrics for azure resources with variable dimensions required one configuration per metric + dimension combination to avoid an error.
- After this change, you can include all metrics and dimensions in a single configuration and the Azure APIs will only return dimensions which are valid for the various metrics.

### Features

- A new `discovery.ovhcloud` component for discovering scrape targets on OVHcloud. (@ptodev)

### Enhancements

Expand All @@ -43,6 +50,8 @@ Main (unreleased)
Previously, only `remote.*` and `local.*` components could be referenced
without a circular dependency. (@rfratto)

- Add support for Basic Auth-secured connection with Elasticsearch cluster using `prometheus.exporter.elasticsearch`. (@hainenber)

- Add a `resource_to_telemetry_conversion` argument to `otelcol.exporter.prometheus`
for converting resource attributes to Prometheus labels. (@hainenber)

Expand All @@ -55,7 +64,7 @@ Main (unreleased)

- Added links between compatible components in the documentation to make it
easier to discover them. (@thampiotr)

- Allow defining `HTTPClientConfig` for `discovery.ec2`. (@cmbrad)

- The `remote.http` component can optionally define a request body. (@tpaschalis)
Expand All @@ -69,19 +78,25 @@ Main (unreleased)
- Added 'country' mmdb-type to log pipeline-stage geoip. (@superstes)

- Azure exporter enhancements for flow and static mode, (@kgeckhart)
- Allows for pulling metrics at the Azure subscription level instead of resource by resource
- Disable dimension validation by default to reduce the number of exporter instances needed for full dimension coverage
- Allows for pulling metrics at the Azure subscription level instead of resource by resource
- Disable dimension validation by default to reduce the number of exporter instances needed for full dimension coverage

- Add `max_cache_size` to `prometheus.relabel` to allow configurability instead of hard coded 100,000. (@mattdurham)

- Add support for `http_sd_config` within a `scrape_config` for prometheus to flow config conversion. (@erikbaranowski)
- `discovery.lightsail` now supports additional parameters for configuring HTTP client settings. (@ptodev)
- Add `sample_age_limit` to remote_write config to drop samples older than a specified duration. (@marctc)

- Add support for Basic Auth for agent's web UI, including the APIs. (@hainenber)

### Bugfixes

- Update `pyroscope.ebpf` to fix a logical bug causing to profile to many kthreads instead of regular processes https://github.com/grafana/pyroscope/pull/2778 (@korniltsev)

- Update `pyroscope.ebpf` to produce more optimal pprof profiles for python processes https://github.com/grafana/pyroscope/pull/2788 (@korniltsev)

- In Static mode's `traces` subsystem, `spanmetrics` used to be generated prior to load balancing.
This could lead to inaccurate metrics. This issue only affects Agents using both `spanmetrics` and
This could lead to inaccurate metrics. This issue only affects Agents using both `spanmetrics` and
`load_balancing`, when running in a load balanced cluster with more than one Agent instance. (@ptodev)

- Fixes `loki.source.docker` a behavior that synced an incomplete list of targets to the tailer manager. (@FerdinandvHagen)
Expand All @@ -90,10 +105,18 @@ Main (unreleased)

- Add staleness tracking to labelstore to reduce memory usage. (@mattdurham)

- Fix issue where `prometheus.exporter.kafka` would crash when configuring `sasl_password`. (@rfratto)

### Other changes

- Bump github.com/IBM/sarama from v1.41.2 to v1.42.1

- Attach unique Agent ID header to remote-write requests. (@captncraig)

- Update to v2.48.1 of `github.com/prometheus/prometheus`.
Previously, a custom fork of v2.47.2 was used.
The custom fork of v2.47.2 also contained prometheus#12729 and prometheus#12677.

v0.38.1 (2023-11-30)
--------------------

Expand Down
2 changes: 2 additions & 0 deletions cmd/grafana-agent/entrypoint.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import (
"github.com/go-kit/log"
"github.com/go-kit/log/level"
"github.com/gorilla/mux"
"github.com/grafana/agent/internal/agentseed"
"github.com/grafana/agent/pkg/config"
"github.com/grafana/agent/pkg/logs"
"github.com/grafana/agent/pkg/metrics"
Expand Down Expand Up @@ -98,6 +99,7 @@ func NewEntrypoint(logger *server.Logger, cfg *config.Config, reloader Reloader)
return nil, err
}

agentseed.Init("", logger)
ep.reporter, err = usagestats.NewReporter(logger)
if err != nil {
return nil, err
Expand Down
2 changes: 2 additions & 0 deletions cmd/internal/flowmode/cmd_run.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import (
"github.com/grafana/agent/component"
"github.com/grafana/agent/converter"
convert_diag "github.com/grafana/agent/converter/diag"
"github.com/grafana/agent/internal/agentseed"
"github.com/grafana/agent/pkg/boringcrypto"
"github.com/grafana/agent/pkg/config/instrumentation"
"github.com/grafana/agent/pkg/flow"
Expand Down Expand Up @@ -252,6 +253,7 @@ func (fr *flowRun) Run(configPath string) error {
}

labelService := labelstore.New(l, reg)
agentseed.Init(fr.storagePath, l)

f := flow.New(flow.Options{
Logger: l,
Expand Down
1 change: 1 addition & 0 deletions component/all/all.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import (
_ "github.com/grafana/agent/component/discovery/nerve" // Import discovery.nerve
_ "github.com/grafana/agent/component/discovery/nomad" // Import discovery.nomad
_ "github.com/grafana/agent/component/discovery/openstack" // Import discovery.openstack
_ "github.com/grafana/agent/component/discovery/ovhcloud" // Import discovery.ovhcloud
_ "github.com/grafana/agent/component/discovery/puppetdb" // Import discovery.puppetdb
_ "github.com/grafana/agent/component/discovery/relabel" // Import discovery.relabel
_ "github.com/grafana/agent/component/discovery/scaleway" // Import discovery.scaleway
Expand Down
5 changes: 3 additions & 2 deletions component/discovery/aws/ec2.go
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,9 @@ func (args EC2Arguments) Convert() *promaws.EC2SDConfig {
}

var DefaultEC2SDConfig = EC2Arguments{
Port: 80,
RefreshInterval: 60 * time.Second,
Port: 80,
RefreshInterval: 60 * time.Second,
HTTPClientConfig: config.DefaultHTTPClientConfig,
}

// SetToDefault implements river.Defaulter.
Expand Down
40 changes: 22 additions & 18 deletions component/discovery/aws/lightsail.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"github.com/aws/aws-sdk-go/aws/ec2metadata"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/grafana/agent/component"
"github.com/grafana/agent/component/common/config"
"github.com/grafana/agent/component/discovery"
"github.com/grafana/river/rivertypes"
promcfg "github.com/prometheus/common/config"
Expand All @@ -27,34 +28,37 @@ func init() {

// LightsailArguments is the configuration for AWS Lightsail based service discovery.
type LightsailArguments struct {
Endpoint string `river:"endpoint,attr,optional"`
Region string `river:"region,attr,optional"`
AccessKey string `river:"access_key,attr,optional"`
SecretKey rivertypes.Secret `river:"secret_key,attr,optional"`
Profile string `river:"profile,attr,optional"`
RoleARN string `river:"role_arn,attr,optional"`
RefreshInterval time.Duration `river:"refresh_interval,attr,optional"`
Port int `river:"port,attr,optional"`
Endpoint string `river:"endpoint,attr,optional"`
Region string `river:"region,attr,optional"`
AccessKey string `river:"access_key,attr,optional"`
SecretKey rivertypes.Secret `river:"secret_key,attr,optional"`
Profile string `river:"profile,attr,optional"`
RoleARN string `river:"role_arn,attr,optional"`
RefreshInterval time.Duration `river:"refresh_interval,attr,optional"`
Port int `river:"port,attr,optional"`
HTTPClientConfig config.HTTPClientConfig `river:",squash"`
}

func (args LightsailArguments) Convert() *promaws.LightsailSDConfig {
cfg := &promaws.LightsailSDConfig{
Endpoint: args.Endpoint,
Region: args.Region,
AccessKey: args.AccessKey,
SecretKey: promcfg.Secret(args.SecretKey),
Profile: args.Profile,
RoleARN: args.RoleARN,
RefreshInterval: model.Duration(args.RefreshInterval),
Port: args.Port,
Endpoint: args.Endpoint,
Region: args.Region,
AccessKey: args.AccessKey,
SecretKey: promcfg.Secret(args.SecretKey),
Profile: args.Profile,
RoleARN: args.RoleARN,
RefreshInterval: model.Duration(args.RefreshInterval),
Port: args.Port,
HTTPClientConfig: *args.HTTPClientConfig.Convert(),
}
return cfg
}

// DefaultLightsailSDConfig is the default Lightsail SD configuration.
var DefaultLightsailSDConfig = LightsailArguments{
Port: 80,
RefreshInterval: 60 * time.Second,
Port: 80,
RefreshInterval: 60 * time.Second,
HTTPClientConfig: config.DefaultHTTPClientConfig,
}

// SetToDefault implements river.Defaulter.
Expand Down
2 changes: 2 additions & 0 deletions component/discovery/azure/azure.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ var DefaultArguments = Arguments{
Environment: azure.PublicCloud.Name,
Port: 80,
RefreshInterval: 5 * time.Minute,
FollowRedirects: true,
EnableHTTP2: true,
}

// SetToDefault implements river.Defaulter.
Expand Down
11 changes: 6 additions & 5 deletions component/discovery/consul/consul.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,12 @@ type Arguments struct {
}

var DefaultArguments = Arguments{
Server: "localhost:8500",
TagSeparator: ",",
Scheme: "http",
AllowStale: true,
RefreshInterval: 30 * time.Second,
Server: "localhost:8500",
TagSeparator: ",",
Scheme: "http",
AllowStale: true,
RefreshInterval: 30 * time.Second,
HTTPClientConfig: config.DefaultHTTPClientConfig,
}

// SetToDefault implements river.Defaulter.
Expand Down
2 changes: 2 additions & 0 deletions component/discovery/digitalocean/digitalocean.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ type Arguments struct {
var DefaultArguments = Arguments{
Port: 80,
RefreshInterval: time.Minute,
FollowRedirects: true,
EnableHTTP2: true,
}

// SetToDefault implements river.Defaulter.
Expand Down
94 changes: 94 additions & 0 deletions component/discovery/ovhcloud/ovhcloud.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
package ovhcloud

import (
"fmt"
"time"

"github.com/grafana/agent/component"
"github.com/grafana/agent/component/discovery"
"github.com/grafana/river/rivertypes"
"github.com/prometheus/common/config"
"github.com/prometheus/common/model"
prom_discovery "github.com/prometheus/prometheus/discovery/ovhcloud"
)

func init() {
component.Register(component.Registration{
Name: "discovery.ovhcloud",
Args: Arguments{},
Exports: discovery.Exports{},

Build: func(opts component.Options, args component.Arguments) (component.Component, error) {
return New(opts, args.(Arguments))
},
})
}

// Arguments configure the discovery.ovhcloud component.
type Arguments struct {
Endpoint string `river:"endpoint,attr,optional"`
ApplicationKey string `river:"application_key,attr"`
ApplicationSecret rivertypes.Secret `river:"application_secret,attr"`
ConsumerKey rivertypes.Secret `river:"consumer_key,attr"`
RefreshInterval time.Duration `river:"refresh_interval,attr,optional"`
Service string `river:"service,attr"`
}

// DefaultArguments is used to initialize default values for Arguments.
var DefaultArguments = Arguments{
Endpoint: "ovh-eu",
RefreshInterval: 60 * time.Second,
}

// SetToDefault implements river.Defaulter.
func (args *Arguments) SetToDefault() {
*args = DefaultArguments
}

// Validate implements river.Validator.
func (args *Arguments) Validate() error {
if args.Endpoint == "" {
return fmt.Errorf("endpoint cannot be empty")
}

if args.ApplicationKey == "" {
return fmt.Errorf("application_key cannot be empty")
}

if args.ApplicationSecret == "" {
return fmt.Errorf("application_secret cannot be empty")
}

if args.ConsumerKey == "" {
return fmt.Errorf("consumer_key cannot be empty")
}

switch args.Service {
case "dedicated_server", "vps":
// Valid value - do nothing.
default:
return fmt.Errorf("unknown service: %v", args.Service)
}

return nil
}

// Convert returns the upstream configuration struct.
func (args *Arguments) Convert() *prom_discovery.SDConfig {
return &prom_discovery.SDConfig{
Endpoint: args.Endpoint,
ApplicationKey: args.ApplicationKey,
ApplicationSecret: config.Secret(args.ApplicationSecret),
ConsumerKey: config.Secret(args.ConsumerKey),
RefreshInterval: model.Duration(args.RefreshInterval),
Service: args.Service,
}
}

// New returns a new instance of a discovery.ovhcloud component.
func New(opts component.Options, args Arguments) (*discovery.Component, error) {
return discovery.New(opts, args, func(args component.Arguments) (discovery.Discoverer, error) {
newArgs := args.(Arguments)
return prom_discovery.NewDiscovery(newArgs.Convert(), opts.Logger)
})
}
Loading

0 comments on commit 2b40e00

Please sign in to comment.