Skip to content

Latest commit

 

History

History
112 lines (74 loc) · 5.08 KB

README.md

File metadata and controls

112 lines (74 loc) · 5.08 KB
Raw

Statistics for Engineers

Shoutout: This work was made possibly by Circonus -- the monitoring system with full histogram support.

Abstract

Gathering all kinds of telemetry data is key to operating reliable distributed systems at scale. Once you have set-up your monitoring systems and recorded all relevant data, the challenge becomes to make sense of it and extract valuable information, like:

  • Are we fulfilling our SLA?
  • How did our query response times change with the last update?

Statistics is the art of extracting information from data. In this tutorial, we address the basic statistical knowledge that helps you at your daily work as a system operator. We will cover probabilistic models, summarizing distributions with mean values, quantiles, and histograms and their relations. Also advanced topics like time series forecasting and scalability analysis will be touched.

The tutorial focuses on practical aspects and will give you hands on knowledge of how to handle, import, analyze, and visualize telemetry data with UNIX command line tools, gnuplot, and the iPython toolkit.

Selected Episodes

  1. Introduction
  2. Visualizing Data
  3. Histograms
  4. Summary Statistics
  5. Quantiles and Outliers
  6. Forecasting
  7. Queuing Theory

Boostrap

If you have access to a machine with docker installed, you can boostrap an interactive working environment with a single command:

$ ./docker.sh
[...]
#
# Data Science 4 Effective Operations
#
# starting jupyter notebook&lab ...
done
#
# Notebook:
# * local url: http://0.0.0.0:9999/?token=F2AlHtJBvHIqoLFEVfbMnUVFkcpFlJuZ
# * public url: http://11.22.33.192:9999/?token=F2AlHtJBvHIqoLFEVfbMnUVFkcpFlJuZ
#
# Lab:
# * local url: http://0.0.0.0:9998/?token=F2AlHtJBvHIqoLFEVfbMnUVFkcpFlJuZ
# * public url: http://11.22.33.192:9998/?token=F2AlHtJBvHIqoLFEVfbMnUVFkcpFlJuZ

Events

Sign-up to the mailing list, to get notified about upcoming Statistics for Engieners events.

This workshop has been held in at a number of events in slightly different forms.

See the corresponding subfolders for the presented content.

If you want to be informed about upcoming events consider watch out for the following hashtag on Twitter: #StatsForEngineers

Monitorama, PDX 2016

CACM

A writeup of the material was published in print by the CACM and the ACM Queue magazine.

Further Reading

Datasets