Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stateless model APIs #225

Open
PaulCCCCCCH opened this issue Jan 8, 2024 · 0 comments
Open

Stateless model APIs #225

PaulCCCCCCH opened this issue Jan 8, 2024 · 0 comments
Labels
enhancement New feature or request future-work

Comments

@PaulCCCCCCH
Copy link
Owner

I recently went through the workflow again and thought it might be better to implement our APIs in stateless manner, i.e., instead of tracking a current_model at the backend, we can instead pass in a model_id parameters in all model APIs - specifying which model to perform this operation on.

Pros:

  • Managing current_model state is tricky and error prone. Difficult to keep frontend in sync with the backend
  • current_model design has poor extensibiliy, and tightly couples with our current architecture.
  • We can only track one model at a time

Cons:

  • Given our current architecture which assumes user deploy the app in their local machine, it's difficult to run these APIs with different model (e.g., what if the user is making interleaving and frequent calls model-1 and model-2, does the back end pre-load them, or load just in time on demand?)
@PaulCCCCCCH PaulCCCCCCH added enhancement New feature or request future-work labels Jan 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request future-work
Projects
None yet
Development

No branches or pull requests

1 participant