Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add testing for pinecone index type #3

Open
12 of 14 tasks
oscarlaird opened this issue Feb 23, 2024 · 0 comments
Open
12 of 14 tasks

Add testing for pinecone index type #3

oscarlaird opened this issue Feb 23, 2024 · 0 comments
Assignees

Comments

@oscarlaird
Copy link
Collaborator

oscarlaird commented Feb 23, 2024

Configuration and Index parameter tests

  • pinecone.api_key is empty
  • spec is empty or unprovided; spec is not valid json
  • pinecone index is empty because all the tuples are in the buffer
  • buffersize parameter actually controls the buffersize (you can check that the buffersize guc is working because by increasing the buffersize we ensure all vectors are in the buffer and thus we'll get an empty pinecone index notice.)

Other tests

  • pinecone.top_k guc really controls number of results from pinecone (you can check by making sure all vectors are in pinecone not buffer and then seeing how this param affects the number of results returned.
  • error raised when trying to build or insert a vector with all zero values (the error is only raised when the buffer is flushed so you should set buffersize to 1 for the insertion part of this test).
  • building from a base table works: the tuples are actually sent to pinecone.
  • CORRECTNESS TESTS. Put a few tuples in pinecone and in the buffer. Verify that the query correctly interleaves the top hits from the buffer and the remote index. Do this for each of the three metrics and verify that the order really is correct. Choose values such that the results would be wrong if using a different metric (e.g. using euclidean instead of cosine when we are supposed to use cosine would give the wrong order).
  • METRIC TESTS. For each metric build an index with that metric type: Use ANALYZE to verify that the index is actually used for queries using this metric (you might need to SET enable_seqscan = 'off'). Use ANALYZE to verify that the index is not used for queries using a different metric. (The operators are <-> <=> <#> for euclid, cosine, inner).
  • select pinecone_indexes();
  • select pinecone_delete_unused_indexes();

Wish list (not implemented)

  • poll pinecone to wait for index to finish building
  • hangs if spec is empty
  • it is possible to connect to an already existing index by directly specifying a host instead of a spec.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants