Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update HaplotypeCaller_GATK4_VCF to use MEM_SIZE and MEM_UNIT #1448

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

kshakir
Copy link
Contributor

@kshakir kshakir commented Dec 9, 2024

Description

The task HaplotypeCaller_GATK4_VCF had a "Note: In the future this should be done using Cromwell's ${MEM_SIZE} and ${MEM_UNIT} environment variables". This pull request replaces the note with code that uses those environment variables.

Before this change, when this WDL runs on a machine with shared memory, such as on a high-performance computing (HPC) cluster or laptop, the command free returns the full memory of the machine. Instead of the amount of memory assigned to the job should be used. To work around this issue, the MEM_SIZE and MEM_UNIT environment variables are now used when present.

Note to anyone else trying to use this workflow on their HPC/laptop: as of December 2024 you must run a fork of Cromwell that incorporates MEM_SIZE and MEM_UNIT in your backend, otherwise free will still be used incorrectly. For more info on memory and memory-retry issues and Cromwell see these years-old PRs, especially 6766:


Checklist

If you can answer "yes" to the following items, please add a checkmark next to the appropriate checklist item(s) and notify our WARP team by tagging @broadinstitute/warp-admins in a comment on this PR.

  • Did you add inputs, outputs, or tasks to a workflow?
  • Did you modify, delete or move: file paths, file names, input names, output names, or task names?
  • If you made a changelog update, did you update the pipeline version number?

@dsde-jenkins
Copy link
Collaborator

Can one of the admins verify this patch?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants