Skip to content

Latest commit

 

History

History
134 lines (91 loc) · 6.73 KB

README.md

File metadata and controls

134 lines (91 loc) · 6.73 KB

Day 3: Building Reliable Web Services

These are the code-samples, exercises, quizzes, and links needed for this course. To complete these exercises follow the instructions in this doc. In the subdirectories of this section are example solutions. Your solutions do not need to match the provided solutions. The goal of the exercises is to learn and understand why things are implemented so that a participant apply principles to any framework or problem set.

Exercises

Exercise 1 Add auth to your server endpoints

Auth tooling is sometimes the first or last measure of security for your endpoints. There are various methods for adding auth to your server endpoint. When building production services the methods you choose for authentication and authorization will be determined by security professionals, but how you implement them is up to you as a developer.

Add an auth method to your server. You can use any method you like such as a middle-ware, a helper functions or, by manually adding the logic to a single function.

Here are some examples of how to add different kinds of auth in your apps. You can pick one to use as a reference for you code.

Exercise 2 Add middleware to your go server

Middleware: Middleware also refers to the software that separates two or more APIs and provides services such as rate-limiting, authentication, and logging.wikipedia The implementation is typically “built-in” functions. In Go, this tends to be platform style tooling shared across the organization. It allows you to add complex functionality to your endpoints.

Using the same web frameworks you used for your web server or the go standard library, add a middleware function to your server. You can use middleware to add metrics, auth, profiling or custom logic to your programs. In this exercise add logging, retry, rate limiting or replace the auth from exercise 1 with a middleware.

Below are framework docs, they will contain examples of build in middleware that you can add with single line functions. They also show you ways of adding custom middleware to your services.

Web frameworks:

Here is an example of setting it up using the chi framework

	r := chi.NewRouter()

	// add prebuilt middleware for all requests
	r.Use(middleware.Logger)
	r.Use(middleware.RequestID)
	r.Use(middleware.RealIP)
	r.Use(middleware.Recoverer)

Pprof live Demo

Pprof is an incredible profiling tool. It is the only tool currently provided to in the standard library what will let you follow memory hot path. If you plan on using pprof as part of your monitoring suit you will need to install graphviz first.

Pprof YouTube video

For more information check out this talk, Pprof for beginners

Exercise 3 Add some monitoring endpoints to your server

Monitoring is often setup as part of the middleware for commonly used metrics like db calls and http status codes. Often there are other metrics that should be added to track specific business logic and functionality. Expvars are provided by the go standard library as a method for exposing metrics to an endpoint where they can be read via a web browser or consumed by a tracking service.

Prometheus is a very common opensource solution for adding metrics to your web services. It adds metrics to end points that can be scraped into a prometheus instance.

NOTE: In this exercise it is not intended to have a prometheus instance up and running, just to set up the endpoint where you can manually view the metrics.

Using Expvars and/or Prometheus SDK add some custom metrics.

Example

    reg := prometheus.NewRegistry()
	reg.MustRegister(collectors.NewBuildInfoCollector())
	reg.MustRegister(collectors.NewDBStatsCollector(db.GetSqlDB(), "postgres"))
	reg.MustRegister(collectors.NewExpvarCollector(
		map[string]*prometheus.Desc{
			"counter200Code": prometheus.NewDesc("expvar_200Status", "number of status 200 api calls", nil, nil),
			"counter400Code": prometheus.NewDesc("expvar_400status", "number of status 400 api calls", nil, nil),
			"counter500Code": prometheus.NewDesc("expvar_500status", "number of status 500 api calls", nil, nil),
		},
	))

	// add prometheus endpoint at /metrics. The above collectors will be shown
	// in the reverse order they are registered.
	r.Mount("/metrics", promhttp.HandlerFor(
		reg,
		promhttp.HandlerOpts{
			// Opt into OpenMetrics to support exemplars.
			EnableOpenMetrics: true,
		},
	))

Bonus exercise: Add Pprof

Add pprof to your service to see how it uses memory when handling API calls. Run pprof and see what insights are available to you.

First add the pprof driver to your app.

import _ "net/http/pprof"

NOTE: the "_" means that the import is added globally as a backend system. This is common for servers, db drivers, etc

Add a pprof server as its own goroutine in your main function.

// run pprof
go func() {
	http.ListenAndServe("localhost:6060", nil)
}()

Install graphviz on your machine to get the visual insights.

Mac:

brew install graphviz

run pprof while your worker-pool is executing

go tool pprof -http=:18080 http://localhost:6060/debug/pprof/profile?seconds=30

In the default graph each node is a function that your program is running. Size and color indicate how much CPU and time each function is taking.

to access the command-line tool run

go tool pprof http://localhost:6060/debug/pprof/allocs