Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update algorithm for issuing image generation tasks #6

Open
gpavelgr opened this issue May 15, 2023 · 2 comments
Open

Update algorithm for issuing image generation tasks #6

gpavelgr opened this issue May 15, 2023 · 2 comments
Assignees

Comments

@gpavelgr
Copy link
Collaborator

gpavelgr commented May 15, 2023

Implement proper logic for issuing tasks for different nodes within the network relying on "our" network of gpu's, Vast.ai network and other providers. The algorithm should prioritize our network first and attempting to access other resources in case of too big load (can be measured in expected delay for receiving an image for submitted request, f.e. longer than 1-2minutes).

After sync on Friday:

  • It looks more like the task of finding cab drivers in Uber
  • Server must orchestrate models

Problem:
different models are optimized for different video cards of miners, if we do batch queries, we need to take this into account

@gpavelgr gpavelgr added the research required Further information is requested label May 15, 2023
@gpavelgr
Copy link
Collaborator Author

Question/decision that affects the task:
How important is the speed of response to the prompt to the user? In general and for the first time (mvp 1-2)

@gpavelgr gpavelgr removed the research required Further information is requested label May 15, 2023
@Savokr
Copy link
Collaborator

Savokr commented Jun 9, 2023

Best solution for now: create a mocked node server that will emulate different pools of nodes (paid Vast.ai, Azure cloud, custom network with different GPUs).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants