Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add concept of "Data Pipelines" #21

Open
1 of 2 tasks
acouch opened this issue Jan 2, 2018 · 0 comments
Open
1 of 2 tasks

Add concept of "Data Pipelines" #21

acouch opened this issue Jan 2, 2018 · 0 comments

Comments

@acouch
Copy link
Member

acouch commented Jan 2, 2018

Would like to have an API to apply data to documents. Examples include GA stats, #19, #20 and so on.

Would like to align this with https://github.com/frictionlessdata/datapackage-pipelines if possible though that appears to be a "per file" spec and not one that is something that is applied to a large number of files. Will try and ask them about that.

Might want to have a separate pipelines folder where different pipelines can be declared and then have a pipelines.yml file or something similar where they can be declared with arguments like the collection and frequency.

This maybe should be called "tasks" or "cron" or something but going with "Data Pipelines" for now.

Issue Type

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant