Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce multiple-warcs-per-job #23

Open
tokee opened this issue Nov 17, 2017 · 0 comments
Open

Introduce multiple-warcs-per-job #23

tokee opened this issue Nov 17, 2017 · 0 comments
Assignees

Comments

@tokee
Copy link
Contributor

tokee commented Nov 17, 2017

Due to an internal project, it would help to have the option of starting jobs from arctica with more than 1 WARC as argument. This would also help in cases with very small WARCs to lessen the amount of overhead for job starts.

The tricky part is how to handle failure of one of the WARCs in a batch. Ideally only that WARC would be treated as problematic.

@tokee tokee self-assigned this Nov 17, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant