TELR freezing and crashing

[Will-Tyler]

Overview

I am a new user of TELR. I am able to install TELR, but TELR does not appear to run correctly.

I first installed TELR using conda. I then ran telr -h, where it appeared to freeze and become unresponsive.

I then tried installing TELR using mamba. I observed the same behavior.

Next I tried creating a conda environment with the envs/telr.yml file. I then installed TELR from source using pip. This time TELR had a segmentation fault when I ran telr -h.

Machine details

I am using a Linux system with two 64-core AMD EPYC 7742 2.25 GHz CPUs and 512 GB of RAM. Conda version is conda 24.1.2.

[cbergman]

Hi @Will-Tyler

Thanks for your message. I tried re-installing TELR from conda today and I was not able to replicate your issue. On Rocky Linux release 8.8 (Green Obsidian) running conda 24.11.1, the following code worked for me:

(base ~$ conda create -n telr_121624
(base) ~$ conda activate telr_121624
(telr_121624) ~$ conda install telr
(telr_121624) ~$ telr -h
usage: telr [-h] -i READS -r REFERENCE -l LIBRARY [--aligner ALIGNER]
            [--assembler ASSEMBLER] [--polisher POLISHER] [-x PRESETS]
            [-p POLISH_ITERATIONS] [-o OUT] [-t THREAD] [-g GAP] [-v OVERLAP]
            [--flank_len FLANK_LEN] [--af_flank_interval AF_FLANK_INTERVAL]
            [--af_flank_offset AF_FLANK_OFFSET]
            [--af_te_interval AF_TE_INTERVAL] [--af_te_offset AF_TE_OFFSET]
            [--different_contig_name] [--minimap2_family] [-k]

Program for detecting non-reference TEs in long read data

required arguments:
  -i READS, --reads READS
                        reads in fasta/fastq format or read alignments in bam
                        format
  -r REFERENCE, --reference REFERENCE
                        reference genome in fasta format
  -l LIBRARY, --library LIBRARY
                        TE consensus sequences in fasta format

optional arguments:
  -h, --help            show this help message and exit
  --aligner ALIGNER     choose method for read alignment, please provide
                        'nglmr' or 'minimap2' (default = 'nglmr')
  --assembler ASSEMBLER
                        Choose the method to be used for local contig assembly
                        step, please provide 'wtdbg2' or 'flye' (default =
                        'wtdbg2')
  --polisher POLISHER   Choose the method to be used for local contig
                        polishing step, please provide 'wtdbg2' or 'flye'
                        (default = 'wtdbg2')
  -x PRESETS, --presets PRESETS
                        parameter presets for different sequencing
                        technologies, please provide 'pacbio' or 'ont'
                        (default = 'pacbio')
  -p POLISH_ITERATIONS, --polish_iterations POLISH_ITERATIONS
                        iterations of contig polishing (default = 1)
  -o OUT, --out OUT     directory to output data (default = '.')
  -t THREAD, --thread THREAD
                        max cpu threads to use (default = '1')
  -g GAP, --gap GAP     max gap size for flanking sequence alignment (default
                        = '20')
  -v OVERLAP, --overlap OVERLAP
                        max overlap size for flanking sequence alignment
                        (default = '20')
  --flank_len FLANK_LEN
                        flanking sequence length (default = '500')
  --af_flank_interval AF_FLANK_INTERVAL
                        5' and 3'flanking sequence interval size used for
                        allele frequency estimation (default = '100')
  --af_flank_offset AF_FLANK_OFFSET
                        5' and 3' flanking sequence offset size used for
                        allele frequency estimation (default = '200')
  --af_te_interval AF_TE_INTERVAL
                        5' and 3' te sequence interval size used for allele
                        frequency estimation (default: '50')
  --af_te_offset AF_TE_OFFSET
                        5' and 3' te sequence offset size used for allele
                        frequency estimation (default: '50')
  --different_contig_name
                        If provided then TELR does not require the contig name
                        to match before and after annotation liftover
                        (default: require contig name to be the same before
                        and after liftover)
  --minimap2_family     If provided then minimap2 will be used to annotate TE
                        families in the assembled contigs (default: use
                        repeatmasker for contig TE annotation)
  -k, --keep_files      If provided then all intermediate files will be kept
                        (default: remove intermediate files)

• the contents of this environment are as follows:

(telr_121624) ~$ conda env export > telr_121624.yml
(telr_121624) ~$ more telr_121624.yml 
name: telr_121624
channels:
  - bioconda
  - r
  - defaults
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=5.1=1_gnu
  - bcftools=1.9=h68d8f2e_9
  - bedtools=2.30.0=h7d7f7ad_2
  - biopython=1.78=py37h7f8727e_0
  - blas=1.0=openblas
  - boost=1.67.0=py37_4
  - bzip2=1.0.8=h5eee18b_6
  - c-ares=1.19.1=h5eee18b_0
  - ca-certificates=2024.11.26=h06a4308_0
  - certifi=2022.12.7=py37h06a4308_0
  - curl=7.87.0=h5eee18b_0
  - flye=2.9=py37hccf3b7b_0
  - gmp=6.1.2=h6c8ec71_1
  - gnutls=3.5.19=h2a4e5f8_1
  - gsl=2.5=h294904e_1
  - hmmer=3.3.2=h87f3376_2
  - htslib=1.9=h244ad75_9
  - icu=58.2=he6710b0_3
  - k8=0.2.5=h9a82719_1
  - keyutils=1.6.1=h166bdaf_0
  - krb5=1.19.3=h3790be6_0
  - ld_impl_linux-64=2.40=h12ee557_0
  - libblas=3.9.0=13_linux64_openblas
  - libboost=1.67.0=h46d08c1_4
  - libcblas=3.9.0=13_linux64_openblas
  - libcurl=7.87.0=h91b91d3_0
  - libdeflate=1.17=h5eee18b_1
  - libedit=3.1.20191231=h46ee950_2
  - libev=4.33=h7f8727e_1
  - libffi=3.2.1=hf484d3e_1007
  - libgcc-ng=11.2.0=h1234567_1
  - libgfortran-ng=7.5.0=ha8ba4b0_17
  - libgfortran4=7.5.0=ha8ba4b0_17
  - libgomp=11.2.0=h1234567_1
  - libnghttp2=1.52.0=ha637b67_1
  - libopenblas=0.3.18=hf726d26_0
  - libssh2=1.10.0=h37d81fd_2
  - libstdcxx-ng=11.2.0=h1234567_1
  - minimap2=2.22=h5bf99c6_0
  - ncurses=6.1=he6710b0_1
  - nettle=3.3=0
  - ngmlr=0.2.7=h9a82719_3
  - numpy=1.17.0=py37h99e49ec_0
  - numpy-base=1.17.0=py37h2f8d375_0
  - openssl=1.1.1w=h7f8727e_0
  - pandas=1.2.4=py37ha9443f7_0
  - pcre=8.45=h295c915_0
  - perl=5.26.2=h14c3975_0
  - perl-archive-tar=2.32=pl526_0
  - perl-carp=1.38=pl526_3
  - perl-compress-raw-bzip2=2.087=pl526he1b5a44_0
  - perl-compress-raw-zlib=2.087=pl526hc9558a2_0
  - perl-exporter=5.72=pl526_1
  - perl-exporter-tiny=1.002001=pl526_0
  - perl-extutils-makemaker=7.36=pl526_1
  - perl-io-compress=2.087=pl526he1b5a44_0
  - perl-io-zlib=1.10=pl526_2
  - perl-list-moreutils=0.428=pl526_1
  - perl-list-moreutils-xs=0.428=pl526_0
  - perl-pathtools=3.75=pl526h14c3975_1
  - perl-scalar-list-utils=1.52=pl526h516909a_0
  - perl-text-soundex=3.05=pl526_1000
  - perl-xsloader=0.24=pl526_0
  - pip=22.3.1=py37h06a4308_0
  - py-boost=1.67.0=py37h04863e7_4
  - pysam=0.17.0=py37h45aed0b_0
  - python=3.7.6=h0371630_2
  - python-dateutil=2.8.2=pyhd3eb1b0_0
  - python_abi=3.7=2_cp37m
  - pytz=2022.7=py37h06a4308_0
  - readline=7.0=h7b6447c_5
  - repeatmasker=4.0.7=pl526_13
  - rmblast=2.6.0=h4422958_0
  - samtools=1.9=h10a08f8_12
  - seqtk=1.3=h5bf99c6_3
  - setuptools=65.6.3=py37h06a4308_0
  - six=1.16.0=pyhd3eb1b0_1
  - sniffles=1.0.12=h8b12597_1
  - sqlite=3.31.1=h7b6447c_0
  - tclap=1.2.1=h470a237_1
  - telr=1.1=pyhdfd78af_0
  - tk=8.6.14=h39e8969_0
  - trf=4.09.1=hec16e2b_2
  - wheel=0.38.4=py37h06a4308_0
  - wtdbg=2.5=h2e03b76_1
  - xz=5.2.10=h5eee18b_1
  - zlib=1.2.13=h5eee18b_1

• could you please try installing again through conda into a new environment? The only difference in our environments that I can see is that you are using conda 24.1.2 where I am using conda 24.1.1, so you might want to try downgrading to conda 24.1.1 if a fresh install doesn't work for you.

[Will-Tyler]

I am using a shared university computer, so I don't know that I can change the conda version.

Here is my environment:

name: telr
channels:
  - defaults
  - bioconda
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=5.1=1_gnu
  - bcftools=1.9=h68d8f2e_9
  - bedtools=2.30.0=h7d7f7ad_2
  - biopython=1.78=py37h7f8727e_0
  - blas=1.0=openblas
  - boost=1.67.0=py37_4
  - bottleneck=1.3.5=py37h7deecbd_0
  - bzip2=1.0.8=h5eee18b_6
  - c-ares=1.19.1=h5eee18b_0
  - ca-certificates=2024.11.26=h06a4308_0
  - certifi=2022.12.7=py37h06a4308_0
  - curl=7.87.0=h5eee18b_0
  - flye=2.9=py37hccf3b7b_0
  - gmp=6.1.2=h6c8ec71_1
  - gnutls=3.5.19=h2a4e5f8_1
  - gsl=2.5=h294904e_1
  - hmmer=3.3.2=h87f3376_2
  - htslib=1.9=h244ad75_9
  - icu=58.2=he6710b0_3
  - k8=0.2.5=h9a82719_1
  - keyutils=1.6.1=h166bdaf_0
  - krb5=1.19.3=h3790be6_0
  - ld_impl_linux-64=2.40=h12ee557_0
  - libblas=3.9.0=16_linux64_openblas
  - libboost=1.67.0=h46d08c1_4
  - libcblas=3.9.0=16_linux64_openblas
  - libcurl=7.87.0=h91b91d3_0
  - libdeflate=1.17=h5eee18b_1
  - libedit=3.1.20191231=h46ee950_2
  - libev=4.33=h7f8727e_1
  - libffi=3.2.1=hf484d3e_1007
  - libgcc-ng=11.2.0=h1234567_1
  - libgfortran-ng=11.2.0=h00389a5_1
  - libgfortran5=11.2.0=h1234567_1
  - libgomp=11.2.0=h1234567_1
  - libnghttp2=1.52.0=ha637b67_1
  - libopenblas=0.3.21=h043d6bf_0
  - libssh2=1.10.0=h37d81fd_2
  - libstdcxx-ng=11.2.0=h1234567_1
  - minimap2=2.22=h5bf99c6_0
  - ncurses=6.1=he6710b0_1
  - nettle=3.3=0
  - ngmlr=0.2.7=h9a82719_3
  - numexpr=2.8.4=py37hd2a5715_0
  - numpy=1.21.5=py37hf838250_3
  - numpy-base=1.21.5=py37h1e6e340_3
  - openssl=1.1.1w=h7f8727e_0
  - packaging=22.0=py37h06a4308_0
  - pandas=1.3.5=py37h8c16a72_0
  - pcre=8.45=h295c915_0
  - perl=5.26.2=h14c3975_0
  - perl-archive-tar=2.32=pl526_0
  - perl-carp=1.38=pl526_3
  - perl-compress-raw-bzip2=2.087=pl526he1b5a44_0
  - perl-compress-raw-zlib=2.087=pl526hc9558a2_0
  - perl-exporter=5.72=pl526_1
  - perl-exporter-tiny=1.002001=pl526_0
  - perl-extutils-makemaker=7.36=pl526_1
  - perl-io-compress=2.087=pl526he1b5a44_0
  - perl-io-zlib=1.10=pl526_2
  - perl-list-moreutils=0.428=pl526_1
  - perl-list-moreutils-xs=0.428=pl526_0
  - perl-pathtools=3.75=pl526h14c3975_1
  - perl-scalar-list-utils=1.52=pl526h516909a_0
  - perl-text-soundex=3.05=pl526_1000
  - perl-xsloader=0.24=pl526_0
  - pip=22.3.1=py37h06a4308_0
  - py-boost=1.67.0=py37h04863e7_4
  - pysam=0.17.0=py37h45aed0b_0
  - python=3.7.6=h0371630_2
  - python-dateutil=2.8.2=pyhd3eb1b0_0
  - python_abi=3.7=2_cp37m
  - pytz=2022.7=py37h06a4308_0
  - readline=7.0=h7b6447c_5
  - repeatmasker=4.0.7=pl526_13
  - rmblast=2.6.0=h4422958_0
  - samtools=1.9=h10a08f8_12
  - seqtk=1.3=h5bf99c6_3
  - setuptools=65.6.3=py37h06a4308_0
  - six=1.16.0=pyhd3eb1b0_1
  - sniffles=1.0.12=h8b12597_1
  - sqlite=3.31.1=h7b6447c_0
  - tclap=1.2.1=h470a237_1
  - telr=1.1=pyhdfd78af_0
  - tk=8.6.14=h39e8969_0
  - trf=4.09.1=hec16e2b_2
  - wheel=0.38.4=py37h06a4308_0
  - wtdbg=2.5=h2e03b76_1
  - xz=5.2.10=h5eee18b_1
  - zlib=1.2.13=h5eee18b_1

Here is the diff:

< name: telr
---
> name: telr_121624
3d2
<   - defaults
4a4,5
>   - r
>   - defaults
14d14
<   - bottleneck=1.3.5=py37h7deecbd_0
31c31
<   - libblas=3.9.0=16_linux64_openblas
---
>   - libblas=3.9.0=13_linux64_openblas
33c33
<   - libcblas=3.9.0=16_linux64_openblas
---
>   - libcblas=3.9.0=13_linux64_openblas
40,41c40,41
<   - libgfortran-ng=11.2.0=h00389a5_1
<   - libgfortran5=11.2.0=h1234567_1
---
>   - libgfortran-ng=7.5.0=ha8ba4b0_17
>   - libgfortran4=7.5.0=ha8ba4b0_17
44c44
<   - libopenblas=0.3.21=h043d6bf_0
---
>   - libopenblas=0.3.18=hf726d26_0
51,53c51,52
<   - numexpr=2.8.4=py37hd2a5715_0
<   - numpy=1.21.5=py37hf838250_3
<   - numpy-base=1.21.5=py37h1e6e340_3
---
>   - numpy=1.17.0=py37h99e49ec_0
>   - numpy-base=1.17.0=py37h2f8d375_0
55,56c54
<   - packaging=22.0=py37h06a4308_0
<   - pandas=1.3.5=py37h8c16a72_0
---
>   - pandas=1.2.4=py37ha9443f7_0
98d95

[cbergman]

• I don't see any obvious things that would cause telr to hang.
• Can you please try re-installing into a new environment using the yml output I posted previously? TELR freezing and crashing #40 (comment)
• e.g.,

conda env create -f environment.yaml

[Will-Tyler]

I did as suggested and got

telr -h
[1]    2851882 segmentation fault (core dumped)  telr -h

I confirmed that the conda environment matches the environment that you posted.

[cbergman]

• From what yyou say, it appears that the behavior of the telr has changed from hanging (with your original conda/mamba install) to giving a segmentation fault (using the yml file I posted above). Segmentation fault is also associated with installing from pip via a static yml present in the telr repo. Is this all correct?
• Given that I am unable to reproduce your errors on my system, I suspect this has to do with differences between OS's or how conda and it's dependencies have been installed, rather than telr itself
• Can you clarify how the conda you are using has been installed? You say you can't switch versions, so this implies you are using a system wide conda. Perhaps you should install your own version of conda in your home directory, eg. https://github.com/bergmanlab/TELR/blob/master/docs/01_Installation.md#install-miniconda-optional and try again.
• Alternatively, can you hand this issue to your system administrator to see if they can track down the source of the problem?