Rebase & Implement Rerank #80

austin-denoble · 2024-10-16T05:29:11Z

Problem

The InferenceService and Embed operation were implemented in a previous PR: #67

As a part of the 2024-10 release we are also adding Rerank to the client, which will live under InferenceService with Embed.

Solution

This work builds on top of external contributions from @Stosan (#73) Thank you for you diligence in getting this started!

Update InferenceService to expose the new Rerank method.
Add various request and response types to represent the interface.
Add doc comments and an integration test for calling the reranker.

Type of Change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update
Infrastructure change (CI configs, etc)
Non-code change (docs, etc)
None of the above: (explain here)

Test Plan

just test

If you want to use the code yourself, here's an example:

	ctx := context.Background()

	clientParams := pinecone.NewClientParams{
		ApiKey:    "YOUR_API_KEY",
	}

	pc, err := pinecone.NewClient(clientParams)
	if err != nil {
		log.Fatalf("Failed to create Client: %v", err)
	}

	rerankModel := "bge-reranker-v2-m3"
	topN := 2
	retunDocuments := true
	documents := []pinecone.Document{
		{
                    "id": "vec1", 
                    "text": "Apple is a popular fruit known for its sweetness and crisp texture."},
		{
                    "id": "vec2", 
                    "text": "Many people enjoy eating apples as a healthy snack."},
		{
                    "id": "vec3", 
                    "text": "Apple Inc. has revolutionized the tech industry with its sleek designs and user-friendly interfaces."},
		{
                    "id": "vec4", 
                    "text": "An apple a day keeps the doctor away, as the saying goes."},
        }

	ranking, err := pc.Inference.Rerank(ctx, &pinecone.RerankRequest{
		Model:           rerankModel,
		Query:           "i love cats",
		ReturnDocuments: &retunDocuments,
		TopN:            &topN,
		RankFields:      &[]string{"text"},
		Documents:       documents,
	})
	if err != nil {
		log.Fatalf("Failed to rerank: %v", err)
	}
	fmt.Printf("Rerank result: %+v\n", ranking)

To see the specific tasks where the Asana app for GitHub is being used, see below:
- https://app.asana.com/0/0/1208401610014474

…nch, refactor code to use generated inference module, correct a few issues with Documents argument shape and rerank test

Stosan · 2024-10-17T07:03:29Z

This is exciting to read.
Love it with Pinecone!

haruska

👍

haruska · 2024-10-17T14:35:24Z

pinecone/client.go

@@ -1261,7 +1261,7 @@ type InferenceService struct {
 // Parameters:
 //   - ctx: A context.Context object controls the request's lifetime, allowing for the request
 //     to be canceled or to timeout according to the context's deadline.
-//   - in: A pointer to an EmbedRequest object that contains the model t4o use for embedding generation, the
+//   - in: A pointer to an EmbedRequest object that contains the model to use for embedding generation, the


aulorbe · 2024-10-17T16:56:21Z

pinecone/client.go

+//   - Model: "The [model] to use for reranking.
+//   - Query: (Required) The query to compare with documents.
+//   - Documents: (Required) A list of Document objects to be reranked.
+//   - RankFields: (Optional) A list of document fields to use for ranking.


This is SUCH a nit, but I think it might make it even more clear if you said "A list of fields in each Document" instead of "A list of document fields"

aulorbe · 2024-10-17T16:56:51Z

pinecone/client.go

+//   - Query: (Required) The query to compare with documents.
+//   - Documents: (Required) A list of Document objects to be reranked.
+//   - RankFields: (Optional) A list of document fields to use for ranking.
+//   - ReturnDocuments: (Optional) Whether to include documents in the response. Defaults to true.


Oo this is nice! I don't think this is an optional field in TS. Maybe I'll incorporate...

aulorbe · 2024-10-17T16:57:10Z

pinecone/client.go

+//   - Documents: (Required) A list of Document objects to be reranked.
+//   - RankFields: (Optional) A list of document fields to use for ranking.
+//   - ReturnDocuments: (Optional) Whether to include documents in the response. Defaults to true.
+//   - TopN: (Optional) How many documents to return. Defaults to the length of input Documents.


Nit: I'd uppercase Documents, just to reinforce that we're talking about Document objects here

aulorbe · 2024-10-17T16:59:30Z

pinecone/client.go

+//   - Data: A list of documents which have been reranked. The documents are sorted in order of relevance,
+//     with the first being the most relevant.
+//   - Model: The model used to rerank documents.
+//   - Usage: Usage statistics for the reranking operation.


Nit: maybe link out to our page about read and write units? Maybe this guy? https://docs.pinecone.io/guides/organizations/manage-cost/monitor-your-usage Might want to check w/Silas and Jesse re: which pg would be best

I had linked to it elsewhere, but added a link here.

aulorbe · 2024-10-17T17:00:20Z

pinecone/client.go

+//   - TopN: (Optional) How many documents to return. Defaults to the length of input Documents.
+//   - Parameters: (Optional) Additional model-specific parameters for the reranker
+//
+// [model]: https://docs.pinecone.io/guides/inference/understanding-inference#models


aulorbe · 2024-10-17T17:01:14Z

pinecone/client.go

+//	     topN := 2
+//	     retunDocuments := true
+//	     documents := []pinecone.Document{
+//		        {"id": "vec1", "text": "Apple is a popular fruit known for its sweetness and crisp texture."},


Nit: I actually wouldn't use the term vec[whatever] here. I think it's a little confusing, since what's in the "Document" objs we're passing in is text, not an embedding. Maybe just stick with "abc1" or whatever for ID?

aulorbe · 2024-10-17T17:01:47Z

pinecone/client.go

+//
+//	     ranking, err := pc.Inference.Rerank(ctx, &pinecone.RerankRequest{
+//		        Model:           rerankModel,
+//		        Query:           "i love cats",


Nit: I'd make the query relevant to the docs. As a reader, I'm like, why would I ask a q about cats when my docs are about apples :)

aulorbe · 2024-10-17T17:04:33Z

pinecone/client.go

+//   - Index: The index position of the document from the original request. This can be used
+//     to locate the position of the document relative to others described in the request.
+//   - Score: The relevance score of the document indicating how closely it matches the query.
+type RankedDocument struct {


I'd specifiy in the docstring here (or somewhere) that the Docs w/2+ fields in them will only be ranked on 1 field -- either text if they do not pass in a custom field to rank or, or their customField, which has to be present in the Doc objs.

Here's how I did it in TS: https://github.com/pinecone-io/pinecone-ts-client/pull/303/files#diff-77967717b18045071d22a72f646f1c4f3cbddc394751cda28af4dfa8732cbd7fR114 (the backend throws the error, so we just have to catch it)

aulorbe · 2024-10-17T17:06:26Z

pinecone/client_test.go

+		ReturnDocuments: &retunDocuments,
+		TopN:            &topN,
+		RankFields:      &[]string{"text"},
+		Documents: []Document{


Note: this is the situation in which the doc will only be ranked on text, in case the user isn't clear on that re: my comment here

aulorbe

Nice job! A few nits:

I'd capitalize things like Document when you're referring to the objs (or the array of objs) themselves
I'd add tests for every situation outlined by Silas here: https://github.com/pinecone-io/python-plugin-inference/blob/main/tests/integration/inference/test_rerank_scenarios.py. Before I did that, I thought everything was 100% good, but it definitely wasn't.

ssmith-pc

LGTM

@Stosan

## Problem The `InferenceService` and `Embed` operation were implemented in a previous PR: #67 As a part of the `2024-10` release we are also adding `Rerank` to the client, which will live under `InferenceService` with `Embed`. ## Solution This work builds on top of external contributions from @Stosan (#73) Thank you for you diligence in getting this started! - Update `InferenceService` to expose the new `Rerank` method. - Add various request and response types to represent the interface. - Add doc comments and an integration test for calling the reranker. ## Type of Change - [ ] Bug fix (non-breaking change which fixes an issue) - [X] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] This change requires a documentation update - [ ] Infrastructure change (CI configs, etc) - [ ] Non-code change (docs, etc) - [ ] None of the above: (explain here) ## Test Plan `just test` If you want to use the code yourself, here's an example: ```go ctx := context.Background() clientParams := pinecone.NewClientParams{ ApiKey: "YOUR_API_KEY", } pc, err := pinecone.NewClient(clientParams) if err != nil { log.Fatalf("Failed to create Client: %v", err) } rerankModel := "bge-reranker-v2-m3" topN := 2 retunDocuments := true documents := []pinecone.Document{ { "id": "vec1", "text": "Apple is a popular fruit known for its sweetness and crisp texture."}, { "id": "vec2", "text": "Many people enjoy eating apples as a healthy snack."}, { "id": "vec3", "text": "Apple Inc. has revolutionized the tech industry with its sleek designs and user-friendly interfaces."}, { "id": "vec4", "text": "An apple a day keeps the doctor away, as the saying goes."}, } ranking, err := pc.Inference.Rerank(ctx, &pinecone.RerankRequest{ Model: rerankModel, Query: "i love cats", ReturnDocuments: &retunDocuments, TopN: &topN, RankFields: &[]string{"text"}, Documents: documents, }) if err != nil { log.Fatalf("Failed to rerank: %v", err) } fmt.Printf("Rerank result: %+v\n", ranking) ``` --------- Co-authored-by: Sam Ayo <[email protected]>

Stosan and others added 3 commits October 16, 2024 15:35

implemented rerankdocument api

d29451b

Update client_test.go

343b9d4

rebase external changes for rerank against release-candidate/2024 bra…

d76c4aa

…nch, refactor code to use generated inference module, correct a few issues with Documents argument shape and rerank test

austin-denoble force-pushed the adenoble/rebase-rerank branch from 4806323 to d76c4aa Compare October 16, 2024 19:54

add and reformat doc comments for reranking

7df61e9

austin-denoble requested review from haruska and aulorbe October 16, 2024 22:10

austin-denoble marked this pull request as ready for review October 16, 2024 22:18

haruska approved these changes Oct 17, 2024

View reviewed changes

aulorbe reviewed Oct 17, 2024

View reviewed changes

austin-denoble added 2 commits October 18, 2024 12:26

review feedback

be67550

add additional rerank integration tests

8253494

ssmith-pc approved these changes Oct 18, 2024

View reviewed changes

fix link in doc comment

e1c3343

austin-denoble merged commit 1153dff into release-candidate/2024-10 Oct 18, 2024
2 checks passed

austin-denoble deleted the adenoble/rebase-rerank branch October 18, 2024 22:56

austin-denoble mentioned this pull request Oct 23, 2024

Merge release-candidate/2024-10 branch to main #85

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rebase & Implement Rerank #80

Rebase & Implement Rerank #80

austin-denoble commented Oct 16, 2024 •

edited

Loading

Stosan commented Oct 17, 2024

haruska left a comment

haruska Oct 17, 2024

aulorbe Oct 17, 2024

aulorbe Oct 17, 2024

aulorbe Oct 17, 2024

aulorbe Oct 17, 2024

austin-denoble Oct 18, 2024

aulorbe Oct 17, 2024

aulorbe Oct 17, 2024

aulorbe Oct 17, 2024

aulorbe Oct 17, 2024 •

edited

Loading

aulorbe Oct 17, 2024

aulorbe left a comment

ssmith-pc left a comment

Rebase & Implement Rerank #80

Rebase & Implement Rerank #80

Conversation

austin-denoble commented Oct 16, 2024 • edited Loading

Problem

Solution

Type of Change

Test Plan

Stosan commented Oct 17, 2024

haruska left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aulorbe Oct 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aulorbe left a comment

Choose a reason for hiding this comment

ssmith-pc left a comment

Choose a reason for hiding this comment

austin-denoble commented Oct 16, 2024 •

edited

Loading

aulorbe Oct 17, 2024 •

edited

Loading