bunny 1.0.3 resolving relative file paths from step directory

[chapmanb]

Milos and all;
I've been testing the 1.0.3 release with the fixed permissions issues for #325 -- thank you for these -- and am running into an issue with relative file paths in the input json. Using this workflow:

https://github.com/bcbio/test_bcbio_cwl/tree/master/somatic

which has relative paths to allow it to be portable:

https://github.com/bcbio/test_bcbio_cwl/blob/master/somatic/somatic-workflow/main-somatic-samples.json#L17

It tries to resolve the files from the steps directory instead of the main directory, causing it not to find the files. Here's an in-context error from Travis:

https://travis-ci.org/chapmanb/bcbio-nextgen/builds/300697388#L5628

and the traceback is below. Thanks so much for any suggestions about how to avoid this.

[2017-11-11 15:37:43.018] [ERROR] Failed to start 72d76181-cbac-4cb6-af77-facb7f229c51. org.rabix.executor.ExecutorException: Execution failed for 72d76181-cbac-4cb6-af77-facb7f229c51. org.rabix.bindings.cwl.processor.CWLPortProcessorException: Failed to set input file properties
org.rabix.executor.ExecutorException: Execution failed for 72d76181-cbac-4cb6-af77-facb7f229c51. org.rabix.bindings.cwl.processor.CWLPortProcessorException: Failed to set input file properties
        at org.rabix.executor.handler.impl.JobHandlerImpl.start(JobHandlerImpl.java:186) ~[rabix-cli.jar:na]
        at org.rabix.executor.execution.command.StartCommand.run(StartCommand.java:32) ~[rabix-cli.jar:na]
        at org.rabix.executor.execution.JobHandlerCommand.run(JobHandlerCommand.java:51) [rabix-cli.jar:na]
        at org.rabix.executor.execution.JobHandlerRunnable.run(JobHandlerRunnable.java:58) [rabix-cli.jar:na]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_121]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_121]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
Caused by: org.rabix.bindings.BindingException: org.rabix.bindings.cwl.processor.CWLPortProcessorException: Failed to set input file properties
        at org.rabix.bindings.cwl.CWLProcessor.preprocess(CWLProcessor.java:108) ~[rabix-cli.jar:na]
        at org.rabix.bindings.cwl.CWLProcessor.preprocess(CWLProcessor.java:77) ~[rabix-cli.jar:na]
        at org.rabix.bindings.cwl.CWLBindings.preprocess(CWLBindings.java:74) ~[rabix-cli.jar:na]
        at org.rabix.executor.handler.impl.JobHandlerImpl.start(JobHandlerImpl.java:147) ~[rabix-cli.jar:na]
        ... 6 common frames omitted
Caused by: org.rabix.bindings.cwl.processor.CWLPortProcessorException: Failed to set input file properties
        at org.rabix.bindings.cwl.processor.callback.CWLPortProcessorHelper.setFileProperties(CWLPortProcessorHelper.java:69) ~[rabix-cli.jar:na]
        at org.rabix.bindings.cwl.CWLProcessor.preprocess(CWLProcessor.java:98) ~[rabix-cli.jar:na]
        ... 9 common frames omitted
Caused by: org.rabix.bindings.cwl.processor.CWLPortProcessorException: Failed to process value [{metadata=null, format=null, dirname=../../testdata/genomes/hg19/seq, nameroot=hg19, path=../../testdata/genomes/hg19/seq/hg19.fa, basename=hg19.fa, size=null, nameext=.fa, contents=null, checksum=null, location=null, secondaryFiles=[{metadata=null, format=null, dirname=../../testdata/genomes/hg19/seq, nameroot=hg19.fa, path=../../testdata/genomes/hg19/seq/hg19.fa.fai, basename=hg19.fa.fai, size=null, nameext=.fai, contents=null, checksum=null, location=null, secondaryFiles=[], class=File}, {metadata=null, format=null, dirname=../../testdata/genomes/hg19/seq, nameroot=hg19, path=../../testdata/genomes/hg19/seq/hg19.dict, basename=hg19.dict, size=null, nameext=.dict, contents=null, checksum=null, location=null, secondaryFiles=[], class=File}], class=File}, {metadata=null, format=null, dirname=../../testdata/genomes/hg19/seq, nameroot=hg19, path=../../testdata/genomes/hg19/seq/hg19.fa, basename=hg19.fa, size=null, nameext=.fa, contents=null, checksum=null, location=null, secondaryFiles=[{metadata=null, format=null, dirname=../../testdata/genomes/hg19/seq, nameroot=hg19.fa, path=../../testdata/genomes/hg19/seq/hg19.fa.fai, basename=hg19.fa.fai, size=null, nameext=.fai, contents=null, checksum=null, location=null, secondaryFiles=[], class=File}, {metadata=null, format=null, dirname=../../testdata/genomes/hg19/seq, nameroot=hg19, path=../../testdata/genomes/hg19/seq/hg19.dict, basename=hg19.dict, size=null, nameext=.dict, contents=null, checksum=null, location=null, secondaryFiles=[], class=File}], class=File}]
        at org.rabix.bindings.cwl.processor.CWLPortProcessor.processValues(CWLPortProcessor.java:56) ~[rabix-cli.jar:na]
        at org.rabix.bindings.cwl.processor.CWLPortProcessor.processInputs(CWLPortProcessor.java:31) ~[rabix-cli.jar:na]
        at org.rabix.bindings.cwl.processor.callback.CWLPortProcessorHelper.setFileProperties(CWLPortProcessorHelper.java:67) ~[rabix-cli.jar:na]
        ... 10 common frames omitted
Caused by: java.nio.file.NoSuchFileException: /home/chapmanb/drive/work/cwl/test_bcbio_cwl/somatic/somatic-workflow/steps/../../testdata/genomes/hg19/seq/hg19.fa
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) ~[na:1.8.0_121]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[na:1.8.0_121]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[na:1.8.0_121]
        at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) ~[na:1.8.0_121]
        at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) ~[na:1.8.0_121]
        at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) ~[na:1.8.0_121]
        at java.nio.file.Files.readAttributes(Files.java:1737) ~[na:1.8.0_121]
        at java.nio.file.Files.size(Files.java:2332) ~[na:1.8.0_121]
        at org.rabix.bindings.cwl.helper.CWLFileValueHelper.buildMissingInfo(CWLFileValueHelper.java:410) ~[rabix-cli.jar:na]
        at org.rabix.bindings.cwl.processor.callback.CWLFilePropertiesProcessorCallback.process(CWLFilePropertiesProcessorCallback.java:28) ~[rabix-cli.jar:na]
        at org.rabix.bindings.cwl.processor.CWLPortProcessor.processValue(CWLPortProcessor.java:70) ~[rabix-cli.jar:na]
        at org.rabix.bindings.cwl.processor.CWLPortProcessor.processValue(CWLPortProcessor.java:115) ~[rabix-cli.jar:na]
        at org.rabix.bindings.cwl.processor.CWLPortProcessor.processValues(CWLPortProcessor.java:54) ~[rabix-cli.jar:na]
        ... 12 common frames omitted

[milos-ljubinkovic]

The issue is that relative paths are resolved against the .cwl file's location instead of the location of the file they are appearing in. Sorry about that, all of our tests have the wf and inputs files in the same folder so we didn't catch this. I created a branch with a fix for this and after some testing will make a 1.0.4 prerelease that includes it.

[chapmanb]

Milos -- thanks so much for looking at this so quickly. I can definitely retest when 1.0.4 is out and will stick to 1.0.2 for now as a workaround.