retries on SQS and STS requests

Merge pull request #417 from abetterinternet/timg/retry-sqs-and-sts

facilitator: retries on SQS and STS requests

2012-02-16 release

Our informal policy is to deploy to staging on Tuesdays, and then production later in the week, depending on how risky the changes seem to be. This release includes:

• lots of dependency bumps across Go and Rust components courtesy of dependabot
• refactoring of workflow-manager to limit instantaneous memory footprint 0530eac
• structured JSON logging in workflow-manager 6b43ba6

More diagnostics for NCI failures

Includes

• 2 minute timeout on writes as well as reads for GCS client
• log when we fail to get credentials from webidp_provider in aws_credentials.rs.

Increase aggregation and intake worker pools

The two major components of this release are:

#395
#396

GCS uploads timeout hotfix for NCI

This release includes a fix for #391, backported to the version NCI currently has deployed to production, with the tag 0.6.2-b to make cargo happy.

GCS uploads timeout hotfix for NCI

This release includes a fix for #391, backported to the version NCI currently has deployed to production, with the tag 0.6.2-a to make cargo happy.

facilitator ureq update, string manifest parsing

This release includes the following changes:

• #357
• #350

Resilient aggregations

The headline change is #348: we no longer abort an aggregation if individual batches are invalid, and instead will proceed to aggregate all the other batches in the aggregation window.

workflow-manager, facilitator and stackdriver metrics

This release includes improvements to prio-server's metrics reporting.

• #332 to capture GCP PubSub metrics
• #329 to scrape metrics from facilitator workers
• #325 for improved labeling of metrics pushed from workflow-manager instances
• #323 to fix prometheus-server storage

Work around for Fargate network availability

The headline change is #324, which teaches workflow-manager to wait for the Kubernetes API server to be available when creating an in-cluster Kubernetes API client, to work around a race condition in network setup in AWS Fargate.