Documentation: Recommend against multiple rate_limit handlers with identical distributed storage configuration

[divergentdave]

We ran into an issue when using this handler in a more complex configuration, and I think it might be worth mentioning in the distributed rate limiting documentation, or adding an additional example configuration file. We had previously instantiated multiple rate_limit handlers, each in a separate handler chain for various routes. We also had one top-level storage module definition (using Redis). When I snooped on Redis traffic, I noticed that there were multiple goroutines reading and writing storage at the same time, using the same keys (based on the instance ID). I think this meant that different rate_limit handlers were clobbering each others' state, and reading back state from other instances that may correspond to a different route and rate limit handler.

We fixed this by instantiating the rate_limit handler once as a named route, invoking the named route from each route's handler chain, and using variables to link together which route handled the request and which rate limiting zone was used. Alternatively, the problem could be avoided by using different storage configuration in each rate_limit handler, and setting key_prefix to something different in each, for example.

[mholt]

That is a really interesting phenomenon, and a clever workaround! I agree we should probably fix this.

What if we roughly hash the handler's configuration and use that as a prefix or suffix to the storage key/filenames? Exactly identically-configured handlers would collide, but I dunno if that's a problem if they're identical?