Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cli as an alternative to pangeo-forge-runner #794

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

keewis
Copy link
Contributor

@keewis keewis commented Feb 15, 2025

Currently there is one official way to run recipes: pangeo-forge-runner. As this is pretty hard to use (lots of magic and issues like the one pangeo-forge/pangeo-forge-runner#168 is trying to solve), the ad-hoc way of running recipes appears to be to execute recipe files as scripts, which is not ideal because it couples infrastructure and transformation code.

To work around this, I've tried to sketch out a cli that:

  • requires recipes to be functions that take the pipeline as a parameter
  • instantiates the runner and pipeline from a runtime config file
  • allows passing additional runtime parameters to the function

(but this still needs a lot of work to make things consistent and without surprises)

I've also tried to add a command that creates a skeleton recipe, somewhat like what maturin init does, but this does not work at all, at the moment.

@norlandrhagen, this is the draft I showed you during the meeting this week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant