Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strelk2 bug: unsupported pickle protocol:3 #192

Open
moldach opened this issue Sep 24, 2020 · 1 comment
Open

Strelk2 bug: unsupported pickle protocol:3 #192

moldach opened this issue Sep 24, 2020 · 1 comment
Labels
bug Something isn't working

Comments

@moldach
Copy link

moldach commented Sep 24, 2020

Snakemake version
5.24.2

Describe the bug

I'm getting an error when using the Strelka2 wrapper when using the --use-singularity and --use-conda commands in conjunction but not when --use-conda is used alone.

I'm wondering if this is problem with the wrapper or the singularity image?
If it's the image can you suggest a better one to use please

Logs

Workflow defines that rule get_vep_cache is eligible for caching between workflows (use the --cache argument to enable this).
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Job counts:
	count   jobs
        1	strelka2
        1

[Thu Sep 24 17:16:03 2020]
Job 0: --- Call germline variants with Strelka2.

python /home/moldach/wrappers/SUBSET/.snakemake/scripts/tmpoc6_0bnr.wrapper.py
Activating singularity image /home/moldach/wrappers/SUBSET/.snakemake/singularity/d7617773b315c3abcb29e0484085ed06.simg
Activating conda environment: /home/moldach/wrappers/SUBSET/.snakemake/conda/1995398f
Traceback (most recent call last):
  File "/home/moldach/wrappers/SUBSET/.snakemake/scripts/tmpoc6_0bnr.wrapper.py", line 3, in <module>
    x00sampleqhK\x00N\x86qish\x15]qj(h\x17h\x18eh\x17h\x19h\x1e\x85qkRql(h\x1e)}qmh"h\x17sNtqnbh\x18h\x19h\x1e\x85qoRqp(h\x1e)}qqh"h\x18sNtqrbX\x06\x00\x00\x00sampleqsheubX\x07\x00\x00\x00threadsqtK\x08$
  File "/home/moldach/wrappers/SUBSET/.snakemake/conda/1995398f/lib/python2.7/pickle.py", line 1388, in loads
    return Unpickler(file).load()
  File "/home/moldach/wrappers/SUBSET/.snakemake/conda/1995398f/lib/python2.7/pickle.py", line 864, in load
    dispatch[key](self)
  File "/home/moldach/wrappers/SUBSET/.snakemake/conda/1995398f/lib/python2.7/pickle.py", line 892, in load_proto
    raise ValueError, "unsupported pickle protocol: %d" % proto
ValueError: unsupported pickle protocol: 3
[Thu Sep 24 17:16:18 2020]
Error in rule strelka2:
    jobid: 0
    output: strelka/MTG324
    log: logs/bowtie2/MTG324.log (check log file(s) for error message)
    conda-env: /home/moldach/wrappers/SUBSET/.snakemake/conda/1995398f

RuleException:
CalledProcessError in line 453 of /home/moldach/wrappers/SUBSET/Snakefile:
Command ' singularity exec --home /home/moldach/wrappers/SUBSET  --bind /home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages:/mnt/snakemake /home/moldach/wrappers/SUBSET/.snakemake/singula$
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 2189, in run_wrapper
  File "/home/moldach/wrappers/SUBSET/Snakefile", line 453, in __rule_strelka2
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 529, in _callback
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/concurrent/futures/thread.py", line 57, in run
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 515, in cached_or_run
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 2201, in run_wrapper
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

Minimal example

Singularity container

singularity: "docker://continuumio/miniconda3:4.5.11"

Rule

def getDeduppedBamsIndex(sample):
  return(list(os.path.join(aligns_dict[sample],"{0}.sorted.dedupped.bam.bai".format(sample,pair)) for pair in ['']))

if (config["CALLING_TOOL"]=="strelka2"):
        rule strelka2:
            input:
                    fasta=os.path.join(dirs_dict["REF_DIR"],config["REF_GENOME"]),
                    bam=lambda wildcards: getDeduppedBams(wildcards.sample),
                    bam_index=lambda wildcards: getDeduppedBamsIndex(wildcards.sample),
                    fasta_index=os.path.join(dirs_dict["REF_DIR"],GENOME_INDEX)
            output:
                    temp(directory("strelka/{sample}"))
            log: os.path.join(dirs_dict["LOG_DIR"],config["ALIGN_TOOL"],"{sample}.log")
            message: """--- Call germline variants with Strelka2."""
            threads: 8
            resources:
                    mem=4000,
                    time=100
            params:
                    config_extra="",
                    run_extra=""
            wrapper:
                    "0.65.0/bio/strelka/germline"
            """


@moldach moldach added the bug Something isn't working label Sep 24, 2020
@moldach
Copy link
Author

moldach commented Sep 24, 2020

Re: the error

unsupported pickle protocol: 3

"Perhaps the person who created the files wasn’t aware of the fact that the default protocol for pickle in Python 3 is “3”, but that this is backward-incompatible. "

from this post: https://stackoverflow.com/questions/34571063/i-have-pickled-files-using-protocol-3-in-python3-and-now-i-need-to-unpickle-the/34571081#34571081

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant