Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate icav2-copy-batch-utility #293

Closed
alexiswl opened this issue May 15, 2024 · 5 comments · Fixed by #295 or #337
Closed

Deprecate icav2-copy-batch-utility #293

alexiswl opened this issue May 15, 2024 · 5 comments · Fixed by #295 or #337
Assignees
Labels
feature New feature pipeline Workflow/Pipeline Manager

Comments

@alexiswl
Copy link
Member

ICAv2 Copy Batch Utility as a stack is overkill.

As @reisingerf suggested, each service that requires the copy batch utility can generate their own.

Create a copy batch utility as a construct and then have microservices create their own versions of this construct.

@alexiswl alexiswl added the feature New feature label May 15, 2024
@alexiswl alexiswl self-assigned this May 15, 2024
@victorskl
Copy link
Member

(made me shock momentarily first)

Create a copy batch utility as a construct

Composing it as a construct sound good.! 👍

@alexiswl
Copy link
Member Author

It does mean that the cttsov2 service will have five step functions associated with its stack

@victorskl
Copy link
Member

Hear you. Hang in there. Pls yell, if step functions started to become out of hand. We can always regroup and discuss on tech choices...

alexiswl added a commit that referenced this issue May 15, 2024
Refactor bssh and cttso v2 so each service generates its own icav2 copy file service

See #293 for more info
@victorskl victorskl added the pipeline Workflow/Pipeline Manager label May 15, 2024
@brainstorm
Copy link
Member

This decision makes sense, in whatever meeting that happened, I've been thinking the same while working on the ICAv1 copy batch utility stack, more generic constructs are needed and I reckon that AWS S3 Batch Ops can do a good job at that.

@alexiswl
Copy link
Member Author

in whatever meeting that happened

No meeting, just orcabus daydreams in my head.

One other benefits is that the step functions executions are less cluttered with different services.
If I want to see when the cttso executed the copy batch I can easily find executions of this without needing to trawl through the bssh fastq copy executions (because they're different step functions now with different names).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment