Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make easier to create custom schedulers and executors #1118

Merged
merged 1 commit into from
Nov 22, 2024

Conversation

milenkovicm
Copy link
Contributor

@milenkovicm milenkovicm commented Nov 18, 2024

Which issue does this PR close?

Closes #1115

Rationale for this change

Users should be able to customize scheduler and executors with custom object stores, memory managers and so on ...
removing responsibility from ballista to provide fit all solution

What changes are included in this PR?

  • cleanup executor and scheduler exec, move some code so it can be reused
  • examples how to create custom binaries
  • documentation update

Are there any user-facing changes?

Additional context

  • as follow up will update clap dependency
  • there is duplicated code which will be fixed by moving test

@milenkovicm milenkovicm marked this pull request as draft November 18, 2024 19:00
@milenkovicm milenkovicm changed the title Make easier to create schedulers and executors Make easier to create custom schedulers and executors Nov 18, 2024
@milenkovicm milenkovicm force-pushed the feature_custom_binaries branch 6 times, most recently from 9bd1c4d to 9a04097 Compare November 20, 2024 10:34
@milenkovicm milenkovicm marked this pull request as ready for review November 20, 2024 18:14
Copy link
Member

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't review in great detail, but this looks great. I love to see documentation and examples for extending Ballista.

@andygrove andygrove merged commit 169cadd into apache:main Nov 22, 2024
17 checks passed
@milenkovicm milenkovicm deleted the feature_custom_binaries branch November 26, 2024 09:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Easy setup/customization of Scheduler/Executor
2 participants