You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It would be better if we extend BootstrapPipeline that take care of this. All executor have to instantiate this call to execute message. We could rely on this to ensure that the worker had initialized the services. It might require the executor worker to load Saturn config and pass it to the bootstrapper and let the services define a open_in_executor and remove the partly supported on_executor_initialized.
The text was updated successfully, but these errors were encountered:
Right now if a service need to run initialization code in the executor worker, it's somewhat difficult or hackish.
We added
hooks.on_executor_initialized
, but many executor (ARQ, Ray) don't support it.Right now, we actually rely on the executor to call services initializer (ie.
saturn/src/saturn_engine/worker/executors/arq/worker.py
Line 48 in b68c977
It would be better if we extend
BootstrapPipeline
that take care of this. All executor have to instantiate this call to execute message. We could rely on this to ensure that the worker had initialized the services. It might require the executor worker to load Saturn config and pass it to the bootstrapper and let the services define aopen_in_executor
and remove the partly supportedon_executor_initialized
.The text was updated successfully, but these errors were encountered: