Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch Ollama distro to use ollama embeddings #904

Open
ashwinb opened this issue Jan 30, 2025 · 3 comments
Open

Switch Ollama distro to use ollama embeddings #904

ashwinb opened this issue Jan 30, 2025 · 3 comments
Assignees
Labels
good first issue Good for newcomers

Comments

@ashwinb
Copy link
Contributor

ashwinb commented Jan 30, 2025

🚀 Describe the new functionality needed

We should not need sentence transformers for calculating embeddings if we are using ollama. No reason to have a torch dependency. Specifically, we need to update templates/ollama/run.yaml to point the embedding model to the ollama inference provider.

💡 Why is this needed? What if we don't build it?

Dependency on the transformers package is a two-edged sword due to its complexity.

Other thoughts

Need to ensure that client-sdk tests pass (i.e., the standard all-MiniLM alias we use everywhere "just works" -- if there is a more standard HuggingFace ID for that model, maybe we should use that.)

@ashwinb ashwinb added the good first issue Good for newcomers label Jan 30, 2025
@manojks1999
Copy link

/assign

@manojks1999
Copy link

@ashwinb can I work on this

@ashwinb
Copy link
Contributor Author

ashwinb commented Jan 30, 2025

Thanks @manojks19999 -- please hop on to the Discord if you need any help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants