Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(api): MultiQueryRetriever with WeaviateHybridSearchRetriever #153

Closed
wants to merge 2 commits into from

Conversation

Lee-W
Copy link
Collaborator

@Lee-W Lee-W commented Nov 20, 2023

Contradict to our earlier discussion; I suspect what we want to replace WeaviateHybridSearchRetriever with is MultiQueryRetriever instead of ConversationalRetrievalChain. However, it seems to perform even worse than the MultiQueryRetriever when it comes to what is astro python sdk?. astronomer-providers, airflow things seem to work. For testing, we can use the same set of env var and the exact same way to bring up the test env locally.

Copy link

cloudflare-workers-and-pages bot commented Nov 20, 2023

Deploying with  Cloudflare Pages  Cloudflare Pages

Latest commit: a0e9816
Status: ✅  Deploy successful!
Preview URL: https://4c2ded61.ask-astro.pages.dev
Branch Preview URL: https://change-number-of-doc.ask-astro.pages.dev

View logs

@sunank200
Copy link
Collaborator

@Lee-W Now that we have MultiQueryRetriever with WeaviateHydridSearchRetriever. We should try the following two at the earliest when you start the day as Rahul needs to run the test for these.

We should try the following:

  • Increase the number of documents we retrieved per rewording to 5
  • when joining the docs together, we should rerank using something like cohere reranker

Let me know if you need any you need to connect on this, please


from ask_astro.clients.weaviate_ import docsearch
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

@sunank200 sunank200 force-pushed the hybrid-search branch 2 times, most recently from d25c8f6 to 3430a16 Compare November 20, 2023 20:34
@sunank200 sunank200 changed the title feat(api): replace MultiQueryRetriever with WeaviateHybridSearchRetriever feat(api): MultiQueryRetriever with WeaviateHybridSearchRetriever Nov 21, 2023
@Lee-W
Copy link
Collaborator Author

Lee-W commented Nov 21, 2023

Increase the number of documents we retrieved per rewording to 5

tested with 6. a bit better but might have room to improve

when joining the docs together, we should rerank using something like cohere reranker

not yet have time to work on it today. will continue work on it early tomorrow

@Lee-W
Copy link
Collaborator Author

Lee-W commented Nov 22, 2023

close this one, as changes included in #170

@Lee-W Lee-W closed this Nov 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants