Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CLI for deploying built-in models #694

Open
maxjakob opened this issue May 14, 2024 · 4 comments
Open

Add CLI for deploying built-in models #694

maxjakob opened this issue May 14, 2024 · 4 comments
Labels
enhancement New feature or request topic:NLP Issue or PR about NLP model support and eland_import_hub_model

Comments

@maxjakob
Copy link

In the context of integration tests and generally in cluster setup automation, the eland_import_hub_model CLI is a great tool to get HuggingFace models deployed in Elasticsearch. There is no a comparable tool for built-in models (most notably ELSER and E5). Users have to do this themselves using with 4 different API calls and a wait condition (example). Adding a CLI tool eland_deploy_builtin_model that works in a similar way (e.g. --model-id .elser_model_v2 --url http://elasticsearch:9200 --start) would reduce the room for error, reduce friction when using built-in models and in turn increase their adoption.

@maxjakob maxjakob added the enhancement New feature or request label May 14, 2024
@joshdevins
Copy link
Member

Should this be in eland or in the main client libs? It's just some API calls.

@pquentin
Copy link
Member

pquentin commented May 15, 2024

The API calls should definitely be in the vectorstore helpers in the client. That will directly improve snippets like https://www.elastic.co/search-labs/tutorials/search-tutorial/semantic-search/elser-model#deploying-the-elser-model which can easily timeout and leave you in a bad state that is difficult to recover from (eg. model deployed but no ingest pipeline).

But it seems to me that there's also value in exposing this through Eland? It would be nice to have a single command to upload both E5 and ELSER in my opinion.

@maxjakob
Copy link
Author

Implementing this in the client's helpers module works. Then we also have a simple programmatic way to do this in Python.

Additionally, I think it's also important to have a CLI for this. This could of course also be through the client package. But since users are already used to deploying HF models with Eland I think it's a good fit to add more model deployment tooling here (also).

@maxjakob
Copy link
Author

This would also remove the necessity for these large code blocks in our blog posts:
https://www.elastic.co/search-labs/blog/serverless-semantic-search-with-elser-in-python#download-and-deploy-elser-model

@pquentin pquentin added the topic:NLP Issue or PR about NLP model support and eland_import_hub_model label Nov 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request topic:NLP Issue or PR about NLP model support and eland_import_hub_model
Projects
None yet
Development

No branches or pull requests

3 participants