Overview

A lightweight Go Web Server that receives POST alert messages from Prometheus Alert Manager and sends it to a Microsoft Teams Channel using an incoming webhook url. How light? The docker image is just 7 MB!

Synopsis

Alertmanager doesn't support sending to Microsoft Teams out of the box. Fortunately, they allow you to use a generic webhook_config for cases like this. This project was inspired from idealista's prom2teams which was written in Python.

Why choose Go? Not Python or Ruby or Node?

Why use Go? A Go binary is statically compiled unlike the other simple language (python, ruby, node). Having a static binary means that there is no need for you to install your program's dependencies and these dependencies takes up a lot of space in your docker image! Try it out DevOps folks!

Table of Contents

• Getting Started (Quickstart)
  • Installation
  • Setting up Prometheus Alert Manager
  • Simulating a Prometheus Alerts to Teams Channel
• Sending Alerts to Multiple Teams Channel
  • Creating the Configuration File
  • Setting up Prometheus Alert Manager
• Kubernetes Deployment
• Developers Build Guide

Getting Started (Quickstart)

How it works.

Installation

OPTION 1: Run using docker.

docker run -d -p 2000:2000 \
    --name="promteams" \
    -e TEAMS_INCOMING_WEBHOOK_URL="https://outlook.office.com/webhook/xxx" \
    -e TEAMS_REQUEST_URI=alertmanager \
    docker.io/bzon/prometheus-msteams:v1.0.3

OPTION 2: Run using binary.

Download the binary for your platform from RELEASES, and run it like the following:

./prometheus-msteams server \
	-l localhost \
	-p 2000 \
	-w "https://outlook.office.com/webhook/xxx"

OPTION 3: If you are going to deploy this in a Kubernetes cluster, checkout the Kubernetes Deployment Guide.

Setting up Prometheus Alert Manager

By default, prometheus-msteams creates a request uri handler /alertmanager.

route:
  group_by: ['alertname']
  group_interval: 30s
  repeat_interval: 30s
  group_wait: 30s
  receiver: 'prometheus-msteams'

receivers:
- name: 'prometheus-msteams'
  webhook_configs: # https://prometheus.io/docs/alerting/configuration/#webhook_config 
  - send_resolved: true
    url: 'http://localhost:2000/alertmanager' # the prometheus-msteams proxy

If you don't have Prometheus running yet and you wan't to try how this works,
try stefanprodan's Prometheus in Docker to help you install a local Prometheus setup quickly in a single machine.

Simulating a Prometheus Alerts to Teams Channel

Create the following json data as prom-alert.json.

{
    "version": "4",
    "groupKey": "{}:{alertname=\"high_memory_load\"}",
    "status": "firing",
    "receiver": "teams_proxy",
    "groupLabels": {
        "alertname": "high_memory_load"
    },
    "commonLabels": {
        "alertname": "high_memory_load",
        "monitor": "master",
        "severity": "warning"
    },
    "commonAnnotations": {
        "summary": "Server High Memory usage"
    },
    "externalURL": "http://docker.for.mac.host.internal:9093",
    "alerts": [
        {
            "labels": {
                "alertname": "high_memory_load",
                "instance": "10.80.40.11:9100",
                "job": "docker_nodes",
                "monitor": "master",
                "severity": "warning"
            },
            "annotations": {
                "description": "10.80.40.11 reported high memory usage with 23.28%.",
                "summary": "Server High Memory usage"
            },
            "startsAt": "2018-03-07T06:33:21.873077559-05:00",
            "endsAt": "0001-01-01T00:00:00Z"
        }
    ]
}

curl -X POST -d @prom-alert.json http://localhost:2000/alertmanager

The teams channel should received a message.

Sending Alerts to Multiple Teams Channel

You can configure this application to serve 2 or more request path and each path can use a unique Teams channel webhook url to post.

This can be achieved by supplying the application a configuration file.

Creating the Configuration File

Create a yaml file with the following format.

connectors:
- high_priority_channel: "https://outlook.office.com/webhook/xxxx/aaa/bbb"
- low_priority_channel: "https://outlook.office.com/webhook/xxxx/aaa/ccc"

NOTE: high_priority_channel and low_priority_channel are example handler names.

When running as a docker container, mount the config file in the container and set the CONFIG_FILE environment variable.

docker run -d -p 2000:2000 \
    --name="promteams" \
    -v /tmp/config.yml:/tmp/config.yml \
    -e CONFIG_FILE="/tmp/config.yml" \
    docker.io/bzon/prometheus-msteams:v1.0.3

When running as a binary, use the --config flag.

./prometheus-msteams server \
	-l localhost \
	-p 2000 \
	--config /tmp/config.yml

This will create the request uri handlers /high_priority_channel and /low_priority_channel.

To validate your configuration, see the /config endpoint of the application.

curl localhost:2000/config

[
  {
    "high_priority_channel": "https://outlook.office.com/webhook/xxxx/aaa/bbb"
  },
  {
    "low_priority_channel": "https://outlook.office.com/webhook/xxxx/aaa/ccc"
  }
]

Setting up Prometheus Alert Manager

Considering the prometheus-msteams config file settings, your Alert Manager would have a configuration like the following.

route:
  group_by: ['alertname']
  group_interval: 30s
  repeat_interval: 30s
  group_wait: 30s
  receiver: 'low_priority_receiver'  # default/fallback request handler
  routes:
    - receiver: high_priority_receiver
      match:
        severity: critical
    - receiver: low_priority_receiver
      match:
        severity: warning

receivers:
- name: 'high_priority_receiver'
  webhook_configs:
    - send_resolved: true
      url: 'http://localhost:2000/high_priority_channel' # request handler 1
- name: 'low_priority_receiver'
  webhook_configs:
    - send_resolved: true
      url: 'http://localhost:2000/low_priority_channel' # request handler 2

Kubernetes Deployment

See Helm Guide.

Developers Build Guide

Build

make dep
make darwin # for osx
make linux # for linux

Test

make test

Docker Build

make docker VERSION=v1
make docker-push VERSION=v1