Release v2.0.0

This version of the Pinecone Go SDK supports version 2024-10 of the Pinecone API. You can read more about versioning here.

Features

Rerank

The InferenceService has a new operation called Rerank which provides users the ability to rerank documents in descending relevance-order against a given query. Reranking documents is a common "second-pass" ranking strategy broadly used in retrieval applications.

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/pinecone-io/go-pinecone/pinecone"
)

func main() {
    ctx := context.Background()

    pc, err := pinecone.NewClient(pinecone.NewClientParams{
        ApiKey: "YOUR_API_KEY",
    })
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    }

    rerankModel := "bge-reranker-v2-m3"
    topN := 4
    returnDocuments := true
    documents := []pinecone.Document{
        {"id": "vec1", "my_field": "Apple is a popular fruit known for its sweetness and crisp texture."},
        {"id": "vec2", "my_field": "Many people enjoy eating apples as a healthy snack."},
        {"id": "vec3", "my_field": "Apple Inc. has revolutionized the tech industry with its sleek designs and user-friendly interfaces."},
        {"id": "vec4", "my_field": "An apple a day keeps the doctor away, as the saying goes."},
    }

    ranking, err := pc.Inference.Rerank(ctx, &pinecone.RerankRequest{
        Model:           rerankModel,
        Query:           "The tech company Apple is known for its innovative products like the iPhone.",
        ReturnDocuments: &returnDocuments,
        TopN:            &topN,
        RankFields:      &[]string{"my_field"},
        Documents:       documents,
    })
    if err != nil {
        log.Fatalf("Failed to rerank: %v", err)
    }
    fmt.Printf("Rerank result: %+v\n", ranking)
}

Import

IndexConnection now exposes additional methods for working with Import operations. An Import is a long-running, asynchronous operation that gives users the ability to import vectors directly from object storage (e.g. S3) into a Pinecone index. It is intended to be used with large-scale jobs. For small-scale jobs (e.g. <1000 vectors), we recommend continuing to use upsert.

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/pinecone-io/go-pinecone/pinecone"
)

func main() {
    ctx := context.Background()

    pc, err := pinecone.NewClient(pinecone.NewClientParams{
        ApiKey: "YOUR_API_KEY",
    })
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    }

    idx, err := pc.DescribeIndex(ctx, "example-name")
    if err != nil {
        log.Fatalf("Failed to describe index \"%s\". Error:%s", idx.Name, err)
    }

    idxConnection, err := pc.Index(pinecone.NewIndexConnParams{Host: idx.Host})
    if err != nil {
        log.Fatalf("Failed to create IndexConnection for Host: %v. Error: %v", idx.Host, err)
    }

    uri := "s3://BUCKET_NAME/PATH/TO/DIR"
    
    // Start an import
    errorMode := "continue" // or "abort"
    importRes, err := idxConnection.StartImport(ctx, uri, nil, (*pinecone.ImportErrorMode)(&errorMode))
    if err != nil {
        log.Fatalf("Failed to start import: %v", err)
    }
    fmt.Printf("Import started with ID: %s", importRes.Id)

    // Describe an import
    importDesc, err := idxConnection.DescribeImport(ctx, importRes.Id)
    if err != nil {
        log.Fatalf("Failed to describe import: %s - %v", importRes.Id, err)
    }
    fmt.Printf("Import ID: %s, Status: %s", importDesc.Id, importDesc.Status)

    // List imports
    limit := int32(10)
    firstImportPage, err := idxConnection.ListImports(ctx, &limit, nil)
    if err != nil {
        log.Fatalf("Failed to list imports: %v", err)
    }
    fmt.Printf("First page of imports: %+v", firstImportPage.Imports)

    paginationToken := firstImportPage.NextPaginationToken
    nextImportPage, err := idxConnection.ListImports(ctx, &limit, paginationToken)
    if err != nil {
        log.Fatalf("Failed to list imports: %v", err)
    }
    fmt.Printf("Second page of imports: %+v", nextImportPage.Imports)

    // Cancel import
    err = idxConnection.CancelImport(ctx, importRes.Id)
    if err != nil {
        log.Fatalf("Failed to cancel import: %s", importRes.Id)
    }
}

Changes Overview

• Refactor /codegen/build-clients.sh to support new apis structure, regenerate for 2024-10 by @austin-denoble in #76
• Implement Import by @austin-denoble in #79
• Implement Rerank by @Stosan & @austin-denoble in #80
• Add Import and Rerank details to README by @aulorbe in #82
• Add EmbedResponseType by @austin-denoble in #83
• Merge release-candidate/2024-10 branch to main by @austin-denoble in #85

New Contributors

• @Stosan made their first contribution in #80

Full Changelog: v1.1.1...v2.0.0