Release v2.0.0

This version of the Pinecone Go SDK supports version 2024-10 of the Pinecone API. You can read more about versioning here.

Features

Rerank

The InferenceService has a new operation called Rerank which provides users the ability to rerank documents in descending relevance-order against a given query. Reranking documents is a common "second-pass" ranking strategy broadly used in retrieval applications.

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/pinecone-io/go-pinecone/pinecone"
)

func main() {
    ctx := context.Background()

    pc, err := pinecone.NewClient(pinecone.NewClientParams{
        ApiKey: "YOUR_API_KEY",
    })
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    }

    rerankModel := "bge-reranker-v2-m3"
    topN := 4
    returnDocuments := true
    documents := []pinecone.Document{
        {"id": "vec1", "my_field": "Apple is a popular fruit known for its sweetness and crisp texture."},
        {"id": "vec2", "my_field": "Many people enjoy eating apples as a healthy snack."},
        {"id": "vec3", "my_field": "Apple Inc. has revolutionized the tech industry with its sleek designs and user-friendly interfaces."},
        {"id": "vec4", "my_field": "An apple a day keeps the doctor away, as the saying goes."},
    }

    ranking, err := pc.Inference.Rerank(ctx, &pinecone.RerankRequest{
        Model:           rerankModel,
        Query:           "The tech company Apple is known for its innovative products like the iPhone.",
        ReturnDocuments: &returnDocuments,
        TopN:            &topN,
        RankFields:      &[]string{"my_field"},
        Documents:       documents,
    })
    if err != nil {
        log.Fatalf("Failed to rerank: %v", err)
    }
    fmt.Printf("Rerank result: %+v\n", ranking)
}

Import

IndexConnection now exposes additional methods for working with Import operations. An Import is a long-running, asynchronous operation that gives users the ability to import vectors directly from object storage (e.g. S3) into a Pinecone index. It is intended to be used with large-scale jobs. For small-scale jobs (e.g. <1000 vectors), we recommend continuing to use upsert.

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/pinecone-io/go-pinecone/pinecone"
)

func main() {
    ctx := context.Background()

    pc, err := pinecone.NewClient(pinecone.NewClientParams{
        ApiKey: "YOUR_API_KEY",
    })
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    }

    idx, err := pc.DescribeIndex(ctx, "example-name")
    if err != nil {
        log.Fatalf("Failed to describe index \"%s\". Error:%s", idx.Name, err)
    }

    idxConnection, err := pc.Index(pinecone.NewIndexConnParams{Host: idx.Host})
    if err != nil {
        log.Fatalf("Failed to create IndexConnection for Host: %v. Error: %v", idx.Host, err)
    }

    uri := "s3://BUCKET_NAME/PATH/TO/DIR"
    
    // Start an import
    errorMode := "continue" // or "abort"
    importRes, err := idxConnection.StartImport(ctx, uri, nil, (*pinecone.ImportErrorMode)(&errorMode))
    if err != nil {
        log.Fatalf("Failed to start import: %v", err)
    }
    fmt.Printf("Import started with ID: %s", importRes.Id)

    // Describe an import
    importDesc, err := idxConnection.DescribeImport(ctx, importRes.Id)
    if err != nil {
        log.Fatalf("Failed to describe import: %s - %v", importRes.Id, err)
    }
    fmt.Printf("Import ID: %s, Status: %s", importDesc.Id, importDesc.Status)

    // List imports
    limit := int32(10)
    firstImportPage, err := idxConnection.ListImports(ctx, &limit, nil)
    if err != nil {
        log.Fatalf("Failed to list imports: %v", err)
    }
    fmt.Printf("First page of imports: %+v", firstImportPage.Imports)

    paginationToken := firstImportPage.NextPaginationToken
    nextImportPage, err := idxConnection.ListImports(ctx, &limit, paginationToken)
    if err != nil {
        log.Fatalf("Failed to list imports: %v", err)
    }
    fmt.Printf("Second page of imports: %+v", nextImportPage.Imports)

    // Cancel import
    err = idxConnection.CancelImport(ctx, importRes.Id)
    if err != nil {
        log.Fatalf("Failed to cancel import: %s", importRes.Id)
    }
}

Changes Overview

• Refactor /codegen/build-clients.sh to support new apis structure, regenerate for 2024-10 by @austin-denoble in #76
• Implement Import by @austin-denoble in #79
• Implement Rerank by @Stosan & @austin-denoble in #80
• Add Import and Rerank details to README by @aulorbe in #82
• Add EmbedResponseType by @austin-denoble in #83
• Merge release-candidate/2024-10 branch to main by @austin-denoble in #85

New Contributors

• @Stosan made their first contribution in #80

Full Changelog: v1.1.1...v2.0.0

Release v1.1.1

Fixes

• Fix normalizeHost, allow for non-secure connections by @austin-denoble in #74

Chores

• [Docs] Return error for serverless index creation failure by @jseldess in #71
• [Chore] Refactor sparse vector integration tests by @austin-denoble in #72

Full Changelog: v1.1.0...v1.1.1

Release v1.1.0

Features

Inference API

You can now try out Pinecone's Inference API, which is currently in public preview. See the README for more details.

ctx := context.Background()

pc, err := pinecone.NewClient(pinecone.NewClientParams{
   ApiKey: "YOUR_API_KEY",
})
if err !=  nil {
   log.Fatalf("Failed to create Client: %v", err)
}

embeddingModel := "multilingual-e5-large"
documents := []string{
   "Turkey is a classic meat to eat at American Thanksgiving."
   "Many people enjoy the beautiful mosques in Turkey."
}
docParameters := pinecone.EmbedParameters{
   InputType: "passage",
   Truncate: "END",
}

docEmbeddingsResponse, err := pc.Inference.Embed(ctx, &pinecone.EmbedRequest{
   Model: embeddingModel,
   TextInputs: documents,
   Parameters: docParameters,
})
if err != nil {
   log.Fatalf("Failed to embed documents: %v", err)
}
fmt.Printf("docs embedding response: %+v", docEmbeddingsResponse)

// << Upsert documents into Pinecone >>

Changes overview

• Refactor Integration Tests by @austin-denoble in #65
• Change "client" to "SDK" by @jseldess in #64
• Add GH release workflow and bump-version action, fix hard-coded client version by @austin-denoble in #66
• Implement Inference API by @austin-denoble in #67
• Fix runner in release workflow by @austin-denoble in #69
• Pass currentVersion in release workflow by @austin-denoble in #70

New Contributors

• @jseldess made their first contribution in #64

Full Changelog: v1.0.0...v1.1.0

Release v1.0.0

Features

API Versioning

This first major release of the Pinecone Go SDK depends on API version 2024-07. This v1 SDK release line will continue to receive fixes as long as the 2024-07 API version is in support. Learn more about Pinecone API versioning here.

Configure Index

You can now configure an index using client.ConfigureIndex with the pinecone.ConfigureIndexParams struct. This can be used to adjust the Replicas or PodType of a pods-based index, or enabling or disabling deletion protection for all index types.

package main

import (
	"context"
	"github.com/pinecone-io/go-pinecone/pinecone"
	"log"
	"os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    } 

    // To scale the size of your pods-based index from "x2" to "x4":
    _, err := pc.ConfigureIndex(ctx, "my-pod-index", pinecone.ConfigureIndexParams{PodType: "p1.x4"})
    if err != nil {
	log.Fatalf("Failed to configure index: %v\n", err)
    }

    // To scale the number of replicas to 4:
    _, err := pc.ConfigureIndex(ctx, "my-pod-index", pinecone.ConfigureIndexParams{Replicas: 4})
    if err != nil {
	log.Fatalf("Failed to configure index: %v\n", err)
    }

Deletion Protection

Use deletion protection to prevent your most important indexes from accidentally being deleted. This feature is available for both serverless and pod indexes.

package main

import (
	"context"
	"github.com/pinecone-io/go-pinecone/pinecone"
	"log"
	"os"
)

func main() {
    ctx := context.Background()

    clientParams := pinecone.NewClientParams{
        ApiKey: os.Getenv("PINECONE_API_KEY"),
    }

    pc, err := pinecone.NewClient(clientParams)
    if err != nil {
        log.Fatalf("Failed to create Client: %v", err)
    }

    // create a new index with deletion protection enabled
    idx, err := pc.CreateServerlessIndex(ctx, &pinecone.CreateServerlessIndexRequest{
        Name:      "my-protected-index",
        Dimension: 3,
        Metric:    pinecone.Cosine,
        Cloud:     pinecone.Aws,
        Region:    "us-east-1",
        DeletionProtection: "enabled",
    })
    if err != nil {
        log.Fatalf("Failed to create index: %v\n", err)
    }

    // To enable deletion protection for an existing index
    _, err := pc.ConfigureIndex(ctx, "my-index", pinecone.ConfigureIndexParams{DeletionProtection: "enabled"})
    if err != nil {
        log.Fatalf("Failed to configure index: %v\n", err)
    }
}

For users of the unstable pre-v1.0.0 SDK: see the README for details on which operations and types have changed. At a high level:

• pinecone.NewClient accepts additional configuration options through pinecone.NewClientParams.
• Creating an IndexConnection when working with the data plane has been simplified to a single client.Index method which takes in a pinecone.NewIndexConnParams struct to configure the connection.
• When working with an IndexConnection, types such as FetchVectorsResponse and QueryVectorsResponse will now contain the Namespace of the source index.

Changes Overview

• update Go documentation link by @junefish in #29
• Update codegen dependencies to fixed versions by @aulorbe in #32
• Add docstrings for index_connection.go by @aulorbe in #31
• Add docstrings for client.go by @aulorbe in #27
• Allow configuration of gRPC.ClientConn through grpc.DialOption, refactor Client.Index(), bump grpc dep to 1.64.0 by @austin-denoble in #35
• Add ConfigureIndex method by @aulorbe in #34
• Add docstrings to models.go by @aulorbe in #37
• Rename Filter to MetadataFilter for clarity by @aulorbe in #39
• Add Reference documentation status badge by @rockwotj in #40
• Add unit tests for index_connection.go by @aulorbe in #38
• Add unit tests for client.go by @aulorbe in #36
• Add codegen/build-clients.sh script, update codegen/apis submodule by @austin-denoble in #49
• Centralize Go test suite by @aulorbe in #48
• Implement support for DeletionProtection by @austin-denoble in #50
• Add Namespace to List, Query, and Fetch vector responses by @austin-denoble in #52
• Update README by @aulorbe in #51
• Update issue templates by @anawishnoff in #54
• Block blank issues without templates by @anawishnoff in #53
• Add error handling for missing required fields in passed structs by @aulorbe in #55
• Add Host normalization when building IndexConnection by @austin-denoble in #56
• Auto-adding needs-triage label to all new issues by @anawishnoff in #57

New Contributors

• @junefish made their first contribution in #29
• @aulorbe made their first contribution in #32
• @rockwotj made their first contribution in #40
• @anawishnoff made their first contribution in #54

Full Changelog: v0.5.0...v1.0.0

v0.5.0

Features

• Optional configuration values for applying custom headers or metadata for REST or gRPC operations by @austin-denoble in #18
• Configure apiKey, additionalHeaders, and host through environment variables by @austin-denoble in #22
• Add error handling to REST operations in Client, add new PineconeError struct by @austin-denoble in #23

Additional Client Configuration options

You can now pass your own http.Client directly through NewClientParams with RestClient, along with setting Headers, and Host values. You can also pass metadata to the underlying gRPC client when creating an IndexConnection by using IndexWithAdditionalMetadata().

import (
"context"

"github.com/pinecone-io/go-pinecone/pinecone"
)

apiKey := "your-api-key"
headers := map[string]string{"X-Your-Header": "header-value"}
client, err := NewClient(NewClientParams{ApiKey: apiKey, Headers: headers})
if err != nil {
  return nil, err
}

// perform client operations
index := client.DescribeIndex(context.Background(), "my-index")

// create an IndexConnection with metadata
metadata := map[string]string{"X-Your-Header": "header-value"}
indexConn := client.IndexWithAdditionalMetadata(index.Host, "my-namespace", metadata)

// perform index operations
indexStats := indexConn.DescribeIndexStats(context.Background())

You can also use environment variables to set ApiKey, Headers, and Host:

• PINECONE_API_KEY
• PINECONE_ADDITIONAL_HEADERS
• PINECONE_CONTROLLER_HOST

Values provided through the environment will be overwritten if you also pass them in NewClientParams to NewClient.

Fixes

• Receive and pass context.Context by value rather than as a pointer in IndexConnection methods by @austin-denoble in #20

Chores

• Add json marshaling annotations to various structs by @austin-denoble in #21
• Bump getPackageVersion() to v0.5.0, and allow : in source tag normalization by @austin-denoble in #24

New Contributors

• @austin-denoble made their first contribution in #20

Full Changelog: v0.4.1...v0.5.0

v0.5-alpha.0

What's Changed

• Management Plane Alpha: Project and API Key Management by @haruska in #15

Full Changelog: v0.4.1...v0.5-alpha.0

v0.4.1

What's Changed

• Adds optional source_tag to User-Agent by @ssmith-pc in #16

Source tag

The SDK now optionally allows setting a source tag when constructing a Pinecone client. The source tag allows requests to be associated with the source tag provided.

import "github.com/pinecone-io/go-pinecone/pinecone"

client, err := pinecone.NewClient(pinecone.NewClientParams{
	ApiKey: "my-api-key",
	SourceTag: "foo",
})

// requests initiated from client connection are associated with source tag "foo"

New Contributors

• @ssmith-pc made their first contribution in #16

Full Changelog: v0.4.0...v0.4.1

v0.4.0

This release is a re-write on the Pinecone GoSDK. The SDK now supports all current actions against the Pinecone APIs. See the README for details on installation and usage.

v0.3.0 Release

This switches the GRPC bindings to use the vector_service.proto API instead of the now-deprecated core.proto API.

v0.2.3 Release

add storage protos