More efficient embedding lookup

[siddhartha-gadgil]

Since it looks like embeddings will be used for a while, it is worth optimizing the search for nearest embeddings to some extent. There is partially written code to cluster embeddings and using a priori bounds find candidate clusters and then search within a cluster (in principle with iterated nesting).

[siddhartha-gadgil]

The code is in EpsilonClusters.lean.

[siddhartha-gadgil]

• The commit 4bd2cd1 has (slightly more than) doubled the speed. So now it takes less than 0.3 seconds for each form of embedding.
• Further, if one restricts to dimension 256, then the time is less than 0.12 seconds.
• So we will not try clustering.
• Instead on switching to newer (Matryoshka) embeddings we can try both simply taking fewer dimensions and two rounds.