Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add vector database modules (Qdrant, Weaviate, Chroma, pgvector, OpenSearch, Milvus) #2245

Merged
merged 38 commits into from
Feb 20, 2024
Merged
Changes from 1 commit
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
e560ee9
fix: add nolint to the examples template
mdelapenya Feb 16, 2024
15ddbe7
chore: bootstrap qdrant module
mdelapenya Feb 16, 2024
018d0e5
feat: add container methods for endpoints
mdelapenya Feb 18, 2024
b23b9f1
chore: add a full example using the gRPC client
mdelapenya Feb 18, 2024
8f314b7
fix: lint
mdelapenya Feb 18, 2024
f9a07ef
fix: use inner t
mdelapenya Feb 18, 2024
7ce031e
chore: reduce network flakines on GH hosted runners
mdelapenya Feb 18, 2024
033cf3c
Revert "chore: reduce network flakines on GH hosted runners"
mdelapenya Feb 18, 2024
67a5abf
chore: try with daemon host
mdelapenya Feb 19, 2024
b602505
chore: hardcode 127.0.0.1 for testing on GH action
mdelapenya Feb 19, 2024
8085b70
chore: print container logs on error
mdelapenya Feb 19, 2024
5190d2d
Revert "chore: hardcode 127.0.0.1 for testing on GH action"
mdelapenya Feb 19, 2024
7d0f136
Revert "chore: try with daemon host"
mdelapenya Feb 19, 2024
6f4ba10
chore: wait for the listening ports to be available
mdelapenya Feb 19, 2024
e8947e7
feat: new weaviate module
mdelapenya Feb 19, 2024
f78b499
chore: add default values for the weviate module
mdelapenya Feb 19, 2024
0d98089
chore: remove helper test method in qdrant
mdelapenya Feb 19, 2024
0c53d43
feat: add method to connect to the REST endpoint of weaviate
mdelapenya Feb 19, 2024
a1912e1
chore: add example for creating an weaviate client
mdelapenya Feb 19, 2024
d62c2ac
docs: document weaviate methods
mdelapenya Feb 19, 2024
7a30684
chore: bootstrap Chroma module
mdelapenya Feb 19, 2024
ac97472
feat: support for passing env vars to modules
mdelapenya Feb 19, 2024
8d3ccb4
chore: add container method to get the base URL
mdelapenya Feb 19, 2024
3ad5d99
chore: add example on how to connect to Chroma
mdelapenya Feb 19, 2024
2d5b163
docs: document weaviate client
mdelapenya Feb 19, 2024
8d030b3
chore: include heartbeat in the wait strategies
mdelapenya Feb 19, 2024
4404928
chore: add example for collections
mdelapenya Feb 19, 2024
8c47d67
Revert "feat: support for passing env vars to modules"
mdelapenya Feb 19, 2024
c3978e8
feat: include pgvector to postgres module
mdelapenya Feb 19, 2024
50ca5f4
fix: wrong value
mdelapenya Feb 19, 2024
6189787
feat: bootstrap opensearch module
mdelapenya Feb 20, 2024
b65745b
chore: configure opensearch
mdelapenya Feb 20, 2024
855737d
chore: implement options and methods for opensearch
mdelapenya Feb 20, 2024
4ee4d34
chore: bootstrap milvus
mdelapenya Feb 20, 2024
3c00a4f
chore: improve container lifecycle and defaults for embed etcd
mdelapenya Feb 20, 2024
f230862
chore: add full example for creating collections
mdelapenya Feb 20, 2024
bb374ee
fix: lint
mdelapenya Feb 20, 2024
73e0468
chore: rename to HttpHostAddress
mdelapenya Feb 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: include pgvector to postgres module
mdelapenya committed Feb 19, 2024
commit c3978e83b2b97b55a47a5987f61acb3055f48c8e
6 changes: 2 additions & 4 deletions docs/modules/postgres.md
Original file line number Diff line number Diff line change
@@ -82,13 +82,11 @@ It's possible to pass extra parameters to the connection string, e.g. `sslmode=d

### Postgres variants

It's possible to use the Postgres container with Timescale or Postgis, to name a few. You simply need to update the image name and the wait strategy.
It's possible to use the Postgres container with PGVector, Timescale or Postgis, to name a few. You simply need to update the image name and the wait strategy.

<!--codeinclude-->
[Image for PGVector](../../modules/postgres/postgres_test.go) inside_block:pgvector
[Image for Timescale](../../modules/postgres/postgres_test.go) inside_block:timescale
<!--/codeinclude-->

<!--codeinclude-->
[Image for Postgis](../../modules/postgres/postgres_test.go) inside_block:postgis
<!--/codeinclude-->

9 changes: 8 additions & 1 deletion modules/postgres/postgres_test.go
Original file line number Diff line number Diff line change
@@ -53,6 +53,13 @@ func TestPostgres(t *testing.T) {
wait: wait.ForLog("database system is ready to accept connections").WithOccurrence(2).WithStartupTimeout(30 * time.Second),
// }
},
{
name: "Pgvector",
// pgvector {
image: "docker.io/pgvector/pgvector:pg16",
wait: wait.ForLog("database system is ready to accept connections").WithOccurrence(2).WithStartupTimeout(30 * time.Second),
// }
},
}

for _, tt := range tests {
@@ -84,7 +91,7 @@ func TestPostgres(t *testing.T) {
// Ensure connection string is using generic format
id, err := container.MappedPort(ctx, "5432/tcp")
require.NoError(t, err)
assert.Equal(t, fmt.Sprintf("postgres://%s:%s@%s:%s/%s?sslmode=disable&application_name=test", user, password, "localhost", id.Port(), dbname), connStr)
assert.Equal(t, fmt.Sprintf("postgres://%s:%s@%s:%s/%s?sslmode=disable&application_name=test", user, password, "127.0.0.1", id.Port(), dbname), connStr)

// perform assertions
db, err := sql.Open("postgres", connStr)