Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Question] Should string-to-file coerced files ensure files exist at coercion or at use/output #695

Open
stxue1 opened this issue Jun 21, 2024 · 1 comment
Labels
interop Bears on spec compatibility question Further information is requested

Comments

@stxue1
Copy link

stxue1 commented Jun 21, 2024

For string-to-file coercion, MiniWDL is content with a nonexistent file if the variable is never used. For example:

version 1.1
workflow wf {
  input {
    File f = "path/to/file"
  }
  output {
  }
}

For Toil, we run a virtualization function for all file types, which ensures that all declared files exist. This means that the above workflow does not work for us.

MiniWDL only complains after this file variable is ever used. For example:

workflow wf {
  input {
    File f = "path/to/file"
  }
  output {
    File out = f
  }
}

or

workflow wf {
  input {
    File f = "path/to/file"
  }
  call t {input: f=f}
  output {
    File out = t.out
  }
}
task t {
  input {
    File f
  }
  command <<<
    ls ~{f}
  >>>
  output {
    File out = stdout()
  }
}

With MINIWDL__FILE_IO__ALLOW_ANY_INPUT=True, both will fail with an error message similar to ...uses nonexistent file/directory: path/to/file....

As this behavior is ambiguous according to the spec, do you think this behavior should be allowed? Or should it be ambiguous to the runner, since MiniWDL currently permits this behavior?

This is related to openwdl/wdl#667

@stxue1
Copy link
Author

stxue1 commented Jun 21, 2024

Some of the unit tests from the spec depend on this behavior. For example:
https://github.com/openwdl/wdl/blob/9c0b9cf4586508a9e6260cc5c5e562e21f625aac/SPEC.md?plain=1#L1292-L1307

version 1.1

workflow string_to_file {
  String path1 = "/path/to/file"
  File path2 = "/path/to/file"

  # valid - String coerces unambiguously to File
  File path3 = path1

  output {
    Boolean paths_equal = path2 == path3
  }
}

@mlin mlin added question Further information is requested interop Bears on spec compatibility labels Jun 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
interop Bears on spec compatibility question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants