Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: New Client Implementation #52

Merged
merged 5 commits into from
Sep 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
name: Go CI
name: Go Tests

on:
push:
branches: [ master, dev ]
branches:
- master
pull_request:
branches: [ '**' ]

jobs:
build:
Expand All @@ -19,6 +19,9 @@ jobs:
with:
go-version-file: ./go.mod

- name: Get
run: go get ./...

- name: Build
run: go build ./...

Expand Down
23 changes: 23 additions & 0 deletions .github/workflows/golangci-lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: golangci-lint
on:
push:
branches:
- master
pull_request:

permissions:
contents: read

jobs:
golangci:
name: lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: stable
- name: golangci-lint
uses: golangci/golangci-lint-action@v6
with:
version: v1.60
340 changes: 340 additions & 0 deletions .golangci.yml

Large diffs are not rendered by default.

104 changes: 104 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
## Report Issues on GitHub [Issues](https://github.com/qdrant/go-client/issues)

We track public bugs and feature requests using GitHub issues. Please report by [opening a new issue](https://github.com/qdrant/go-client/issues/new).

**Effective Bug Reports** should include:

- A clear summary or background
- Steps to reproduce the issue
- Be as specific as possible
- Include sample code when possible
- What you expect to happen
- What happened
- Additional notes (e.g., why you think the issue occurs or solutions you’ve tried that didn’t work)

## Contributing Code

Follow these steps before submitting a pull request:

### Building the Project

```bash
go build ./...
```

This will download all dependencies and compile the project.

### Running Tests

All test files are in the `qdrant_test` directory and use [Testcontainers Go](https://golang.testcontainers.org/) for integration tests.

Run the following command to execute the test suites:

```bash
go test -v ./...
```

This command pulls a Qdrant Docker image to run integration tests. Ensure Docker is running.

### Formatting and Linting

Ensure your code is free from warnings and follows project standards.

The project uses [Gofmt](https://go.dev/blog/gofmt) for formatting and [golangci-lint](https://github.com/golangci/golangci-lint) for linting.

To format your code:

```bash
gofmt -s -w .
```

To lint your code:

```bash
golangci-lint run
```

### Preparing for a New Release

The client uses generated stubs from upstream Qdrant proto definitions, which are downloaded from [qdrant/qdrant](https://github.com/qdrant/qdrant/tree/master/lib/api/src/grpc/proto).

#### Steps:

1. Download and generate the latest client stubs by running the following command from the project root:

```bash
BRANCH=dev sh tools/sync_proto.sh
```

2. Update the test image value in [`qdrant_test/image_test.go`](https://github.com/qdrant/go-client/blob/new-client/qdrant_test/image_test.go) to `qdrant/qdrant:dev`.

3. Remove the gRPC server definitions from the auto-generated code.

There is currently [no way](https://github.com/golang/protobuf/issues/373) to skip generating Go server definitions.

You’ll need to manually delete them from [`snapshots_service_grpc.pb.go`](https://github.com/qdrant/go-client/blob/new-client/qdrant/snapshots_service_grpc.pb.go), [`points_service_grpc.pb.go`](https://github.com/qdrant/go-client/blob/new-client/qdrant/points_service.pb.go), and [`collections_service_grpc.pb.go`](https://github.com/qdrant/go-client/blob/new-client/qdrant/collections_service_grpc.pb.go).

Remove lines starting from comments like `// CollectionsServer is the server API for Collections service.` until the end of the file. [Here’s an example commit](https://github.com/qdrant/go-client/commit/6d04e31bb2acccf54f964a634df8930533642892).

4. Implement new Qdrant methods in [`points.go`](https://github.com/qdrant/go-client/blob/new-client/qdrant/points.go), [`collections.go`](https://github.com/qdrant/go-client/blob/new-client/qdrant/collections.go), or [`qdrant.go`](https://github.com/qdrant/go-client/blob/new-client/qdrant/qdrant.go) as needed.

5. If there are any new `oneOf` properties in the proto definitions, add helper constructors in [`oneof_factory.go`](https://github.com/qdrant/go-client/blob/new-client/qdrant/oneof_factory.go) following the existing patterns.

6. Submit your pull request and get those approvals.

### Releasing a New Version

Once the new Qdrant version is live:

1. Run the following command:

```bash
BRANCH=master sh tools/sync_proto.sh
```

2. Update the test image value in `qdrant_test/image_test.go` to `qdrant/qdrant:NEW_VERSION`.

3. Merge the pull request.

4. Push a new Git tag to publish the version:

```bash
git tag v1.11.0
git push --tags
```
160 changes: 110 additions & 50 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,75 +1,135 @@
# Golang Qdrant client
<p align="center">
<img height="120" src="https://github.com/user-attachments/assets/a69a26dd-ecfb-46d5-bbe1-3c0b9f0c430a" alt="Qdrant-Go">
</p>

Go client for Qdrant vector search engine
<p align="center">
<b>Go client for the <a href="https://github.com/qdrant/qdrant">Qdrant</a> vector search engine.</b>
</p>

## Install
<p align="center">
<a href="https://pkg.go.dev/github.com/qdrant/go-client"><img src="https://img.shields.io/badge/Docs-godoc-success" alt="Godoc"></a>
<a href="https://github.com/qdrant/go-client/actions/workflows/ci.yml"><img src="https://github.com/qdrant/go-client/actions/workflows/ci.yml/badge.svg?branch=master" alt="Tests"></a>
<a href="https://github.com/qdrant/go-client/blob/master/LICENSE"><img src="https://img.shields.io/badge/License-Apache%202.0-success" alt="Apache 2.0 License"></a>
<a href="https://qdrant.to/discord"><img src="https://img.shields.io/badge/Discord-Qdrant-5865F2.svg?logo=discord" alt="Discord"></a>
<a href="https://qdrant.to/roadmap"><img src="https://img.shields.io/badge/Roadmap-2024-bc1439.svg" alt="Roadmap 2024"></a>
</p>

Go client library with handy utilities for interfacing with [Qdrant](https://qdrant.tech/).

## 📥 Installation

```bash
go get github.com/qdrant/go-client
go get -u github.com/qdrant/go-client
```

## Usage
## 📖 Documentation

Run Qdrant with enabled gRPC interface:
- Usage examples are available throughout the [Qdrant documentation](https://qdrant.tech/documentation/quick-start/) and [API Reference](https://api.qdrant.tech/).
- [Godoc Reference](https://pkg.go.dev/github.com/qdrant/go-client)

```bash
# With env variable
docker run -p 6333:6333 -p 6334:6334 \
-e QDRANT__SERVICE__GRPC_PORT="6334" \
qdrant/qdrant
```
## 🔌 Getting started

### Creating a client

A client can be instantiated with

Or by updating the configuration file:
```go
import "github.com/qdrant/go-client/qdrant"

```yaml
service:
grpc_port: 6334
client, err := qdrant.NewClient(&qdrant.Config{
Host: "localhost",
Port: 6334,
})
```

More info about gRPC in [documentation](https://qdrant.tech/documentation/quick-start/#grpc).
Which creates a client that will connect to Qdrant on <http://localhost:6334>.

### Making requests
Internally, the high-level client uses a low-level gRPC client to interact with
Qdrant. `qdrant.Config` provides additional options to control how the gRPC
client is configured. The following example configures API key authentication with TLS:

```go
package main
import "github.com/qdrant/go-client/qdrant"

client, err := qdrant.NewClient(&qdrant.Config{
Host: "xyz-example.eu-central.aws.cloud.qdrant.io",
Port: 6334,
APIKey: "<paste-your-api-key-here>",
UseTLS: true, // uses default config with minimum TLS version set to 1.3
// TLSConfig: &tls.Config{...},
// GrpcOptions: []grpc.DialOption{},
})
```

### Working with collections

Once a client has been created, create a new collection

```go
import (
"context"
"flag"
"log"
"time"

pb "github.com/qdrant/go-client/qdrant"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
"github.com/qdrant/go-client/qdrant"
)

var (
addr = flag.String("addr", "localhost:6334", "the address to connect to")
)
client.CreateCollection(context.Background(), &qdrant.CreateCollection{
CollectionName: "{collection_name}",
VectorsConfig: qdrant.NewVectorsConfig(&qdrant.VectorParams{
Size: 4,
Distance: qdrant.Distance_Cosine,
}),
})
```

Insert vectors into the collection

```go
operationInfo, err := client.Upsert(context.Background(), &qdrant.UpsertPoints{
CollectionName: "{collection_name}",
Points: []*qdrant.PointStruct{
{
Id: qdrant.NewIDNum(1),
Vectors: qdrant.NewVectors(0.05, 0.61, 0.76, 0.74),
Payload: qdrant.NewValueMap(map[string]any{"city": "London"}),
},
{
Id: qdrant.NewIDNum(2),
Vectors: qdrant.NewVectors(0.19, 0.81, 0.75, 0.11),
Payload: qdrant.NewValueMap(map[string]any{"age": 32}),
},
{
Id: qdrant.NewIDNum(3),
Vectors: qdrant.NewVectors(0.36, 0.55, 0.47, 0.94),
Payload: qdrant.NewValueMap(map[string]any{"vegan": true}),
},
},
})
```

Search for similar vectors

func main() {
flag.Parse()
// Set up a connection to the server.
conn, err := grpc.Dial(*addr, grpc.WithTransportCredentials(insecure.NewCredentials()))
if err != nil {
log.Fatalf("did not connect: %v", err)
}
defer conn.Close()

collections_client := pb.NewCollectionsClient(conn)

// Contact the server and print out its response.
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
r, err := collections_client.List(ctx, &pb.ListCollectionsRequest{})
if err != nil {
log.Fatalf("could not get collections: %v", err)
}
log.Printf("List of collections: %s", r.GetCollections())
}
```go
searchResult, err := client.Query(context.Background(), &qdrant.QueryPoints{
CollectionName: "{collection_name}",
Query: qdrant.NewQuery(0.2, 0.1, 0.9, 0.7),
})
```

Search for similar vectors with filtering condition

```go
searchResult, err := client.Query(context.Background(), &qdrant.QueryPoints{
CollectionName: "test_collection",
Query: qdrant.NewQuery(0.2, 0.1, 0.9, 0.7),
Filter: &qdrant.Filter{
Must: []*qdrant.Condition{
qdrant.NewMatch("city", "London"),
},
},
WithPayload: qdrant.NewWithPayload(true),
})
```

> For authenticated request (using API KEY and TLS) to Qdrant Cloud, please refer to the [authenticated](https://github.com/qdrant/go-client/tree/master/examples/authentication/main.go) example.
## ⚖️ LICENSE

A full example for uploading, searching and filtering can be found in the [`examples`](https://github.com/qdrant/go-client/tree/master/examples) directory.
Apache 2.0 © [2024](https://github.com/qdrant/go-client/blob/master/LICENSE)
Loading