Locate: Add histogram for Memorystore request duration

[cristinaleonr]

This PR adds a new metric to keep track of how long it takes for the Locate to issue requests to Memorystore.

---

This change is 

[stephen-soltesz]

Some small questions.

Reviewed 2 of 4 files at r1.
Reviewable status: 0 of 1 approvals obtained (waiting on @cristinaleonr)

---

memorystore/client.go line 53 at r1 (raw file):

	}

	metrics.LocateMemorystoreRequestDuration.WithLabelValues("put", field, "OK").Observe(time.Since(t).Seconds())

Is the same field name every both "expire=true" and "expire=false"? I'm wondering about the potential of mixed use of "OK" and whether that matters.

---

metrics/metrics.go line 63 at r1 (raw file):

			Name:    "locate_memorystore_request_duration",
			Help:    "A histogram of request latency to Memorystore.",
			Buckets: prometheus.LinearBuckets(0, 2, 16), // Buckets from 0 to 30 with steps of size 2.

This feels a little coarse-grain.

But, I guess we're primarily looking for large outliners..

Did you consider smaller steps in the low end? e.g. like steps before 1 in prometheus.DefBuckets?

[cristinaleonr]

Reviewable status: 0 of 1 approvals obtained (waiting on @stephen-soltesz)

---

memorystore/client.go line 53 at r1 (raw file):

Previously, stephen-soltesz (Stephen Soltesz) wrote…

Is the same field name every both "expire=true" and "expire=false"? I'm wondering about the potential of mixed use of "OK" and whether that matters.

I added a suffix to differentiate between "expire=true" and "expire=false".

---

metrics/metrics.go line 63 at r1 (raw file):

Previously, stephen-soltesz (Stephen Soltesz) wrote…

This feels a little coarse-grain.

But, I guess we're primarily looking for large outliners..

Did you consider smaller steps in the low end? e.g. like steps before 1 in prometheus.DefBuckets?

Yes, we're primarily looking for request durations that could contribute to the entries' expiration.

I have added smaller steps before 1.

[cristinaleonr]

@stephen-soltesz PTAL when you get a chance.

Reviewable status: 0 of 1 approvals obtained (waiting on @stephen-soltesz)

[stephen-soltesz]

Thank you.

Reviewed 2 of 3 files at r2, 1 of 1 files at r3.
Reviewable status: complete! 1 of 1 approvals obtained

[cristinaleonr]

Related to #149.