You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It could be useful to be able to export or import pipelines from a static file definition like .yaml for the sake of running larger experiments. One such example is running things with Hydra or when you have to run many experiments parallelized over a cluster.
This could be fairly straightforward with most pipeline setups that rely on libraries for their components and involve simple config and spaces.
Some clear problems with this of course are functional elements of a pipeline, for example the config_transform, which typically invoke user based transforms during the configuration of a pipeline, or custom components.
These problems could be possible to solve as hydra seems to do some import magic to make it happen, at the cost of less transparency. This also means exported pipelines will not be transferable from one environment to the next.
The text was updated successfully, but these errors were encountered:
It could be useful to be able to export or import pipelines from a static file definition like
.yaml
for the sake of running larger experiments. One such example is running things with Hydra or when you have to run many experiments parallelized over a cluster.This could be fairly straightforward with most pipeline setups that rely on libraries for their components and involve simple
config
andspace
s.Some clear problems with this of course are functional elements of a pipeline, for example the
config_transform
, which typically invoke user based transforms during the configuration of a pipeline, or custom components.These problems could be possible to solve as hydra seems to do some import magic to make it happen, at the cost of less transparency. This also means exported pipelines will not be transferable from one environment to the next.
The text was updated successfully, but these errors were encountered: