-
Notifications
You must be signed in to change notification settings - Fork 192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
nosetests showing 1 error in python3 (and multiple for python 2.7 with appropriate umi tools version) #450
Comments
Hi. For the python 3.7 test, can you try running |
Hi, thanks for the response. |
Very sorry. Yes, you clearly mentioned that... |
Hmm... can't reproduce with clean conda or pip installation. How are you installing? What do you get when you run the command manually? |
Hi
See attached for output (no error seemingly). I have installed inside docker. I have now tried either inside a venv or using system installed python 3 and get same output to your code below. Here is installation routine part of Dockerfile which gives no error that I can see (I remove python 3.5 from requirements.txt as I have python 3.7 installed):
RUN apt-get update && \
wget https://github.com/CGATOxford/UMI-tools/archive/1.1.1.zip && \
unzip 1.1.1.zip && \
cd /usr/umitools/UMI-tools-1.1.1 && \
sed -e "s/python>=3.5//g" -i requirements.txt && \
pip3 install -r requirements.txt && \
python3 setup.py install
…________________________________
From: Tom Smith <[email protected]>
Sent: 07 January 2021 14:37
To: CGATOxford/UMI-tools <[email protected]>
Cc: Smith, Alexander <[email protected]>; Author <[email protected]>
Subject: Re: [CGATOxford/UMI-tools] nosetests showing 1 error in python3 (and multiple for python 2.7 with appropriate umi tools version) (#450)
Hmm... can't reproduce with clean conda or pip installation.
How are you installing?
What do you get when you run the command manually?
umi_tools dedup --paired --log=test.log --filter-umi --umi-whitelist=/usr/umitools/UMI-tools-1.1.1/tests/umi_whitelist.tsv --umi-whitelist-paired=/usr/umitools/UMI-tools-1.1.1/tests/umi_whitelist.tsv --out-sam --random-seed=123456789 --stdin=/usr/umitools/UMI-tools-1.1.1/tests/whitelist_umi_input.bam |sort > tmp
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FCGATOxford%2FUMI-tools%2Fissues%2F450%23issuecomment-756155183&data=04%7C01%7Calexander.e.smith%40kcl.ac.uk%7C06d2c2e7bb4c4792b1c008d8b319ceed%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637456270714553054%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=AOHw0lxmPRtN1%2FJUxwskCiHR9K%2B%2FWU%2F%2F1nY6%2FiEgXmQ%3D&reserved=0>, or unsubscribe<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAG3NR5SQWM54BYDTL57FELDSYXBLZANCNFSM4VY4ZLNA&data=04%7C01%7Calexander.e.smith%40kcl.ac.uk%7C06d2c2e7bb4c4792b1c008d8b319ceed%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637456270714553054%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UApTYHFXofNtelQP9QROjnDxsnVd4wIorapneikPGEY%3D&reserved=0>.
|
btw, are you able to see the attached?
Dr Alexander Smith
Dept Haematology
King's College NHS trust
Denmark Hill
London SE5 9RS
…________________________________
From: Smith, Alexander <[email protected]>
Sent: 07 January 2021 15:01
To: CGATOxford/UMI-tools <[email protected]>; CGATOxford/UMI-tools <[email protected]>
Cc: Author <[email protected]>
Subject: Re: [CGATOxford/UMI-tools] nosetests showing 1 error in python3 (and multiple for python 2.7 with appropriate umi tools version) (#450)
Hi
See attached for output (no error seemingly). I have installed inside docker. I have now tried either inside a venv or using system installed python 3 and get same output to your code below. Here is installation routine part of Dockerfile which gives no error that I can see (I remove python 3.5 from requirements.txt as I have python 3.7 installed):
RUN apt-get update && \
wget https://github.com/CGATOxford/UMI-tools/archive/1.1.1.zip && \
unzip 1.1.1.zip && \
cd /usr/umitools/UMI-tools-1.1.1 && \
sed -e "s/python>=3.5//g" -i requirements.txt && \
pip3 install -r requirements.txt && \
python3 setup.py install
________________________________
From: Tom Smith <[email protected]>
Sent: 07 January 2021 14:37
To: CGATOxford/UMI-tools <[email protected]>
Cc: Smith, Alexander <[email protected]>; Author <[email protected]>
Subject: Re: [CGATOxford/UMI-tools] nosetests showing 1 error in python3 (and multiple for python 2.7 with appropriate umi tools version) (#450)
Hmm... can't reproduce with clean conda or pip installation.
How are you installing?
What do you get when you run the command manually?
umi_tools dedup --paired --log=test.log --filter-umi --umi-whitelist=/usr/umitools/UMI-tools-1.1.1/tests/umi_whitelist.tsv --umi-whitelist-paired=/usr/umitools/UMI-tools-1.1.1/tests/umi_whitelist.tsv --out-sam --random-seed=123456789 --stdin=/usr/umitools/UMI-tools-1.1.1/tests/whitelist_umi_input.bam |sort > tmp
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FCGATOxford%2FUMI-tools%2Fissues%2F450%23issuecomment-756155183&data=04%7C01%7Calexander.e.smith%40kcl.ac.uk%7C06d2c2e7bb4c4792b1c008d8b319ceed%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637456270714553054%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=AOHw0lxmPRtN1%2FJUxwskCiHR9K%2B%2FWU%2F%2F1nY6%2FiEgXmQ%3D&reserved=0>, or unsubscribe<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAG3NR5SQWM54BYDTL57FELDSYXBLZANCNFSM4VY4ZLNA&data=04%7C01%7Calexander.e.smith%40kcl.ac.uk%7C06d2c2e7bb4c4792b1c008d8b319ceed%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637456270714553054%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UApTYHFXofNtelQP9QROjnDxsnVd4wIorapneikPGEY%3D&reserved=0>.
|
No i'm not sure attachments work in github issues. The problem in the nosetests for the python3 version appears to be that the your umi_tools appears to be outputting a sam file with a header, where as the reference does not contain a header. I wonder if this is a pysam/samtools version issue. Has a new version of pysam/samtools be released since we last did a rebuild? |
Hi again
I think I have a dependency issue. A pip install on my local ubuntu machine (not in docker container) works with no errors from nosetests (python 3.6.9 in venv). I'll investigate further and let you know if I find a culprit...such as pysam.
A
…________________________________
From: Ian Sudbery <[email protected]>
Sent: 07 January 2021 15:56
To: CGATOxford/UMI-tools <[email protected]>
Cc: Smith, Alexander <[email protected]>; Author <[email protected]>
Subject: Re: [CGATOxford/UMI-tools] nosetests showing 1 error in python3 (and multiple for python 2.7 with appropriate umi tools version) (#450)
No i'm not sure attachments work in github issues.
The problem in the nosetests for the python3 version appears to be that the your umi_tools appears to be outputting a sam file with a header, where as the reference does not contain a header.
I wonder if this is a pysam/samtools version issue. Has a new version of pysam/samtools be released since we last did a rebuild?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FCGATOxford%2FUMI-tools%2Fissues%2F450%23issuecomment-756204183&data=04%7C01%7Calexander.e.smith%40kcl.ac.uk%7C6607af6314024f0a866a08d8b324d18f%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637456318013968398%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=e4UFcEBSLaxRU%2FB9WuNGwi%2B7LB7Wi5ROcmiHsH1kgdU%3D&reserved=0>, or unsubscribe<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAG3NR5TSJADMIIJLQ55JSRTSYXKTLANCNFSM4VY4ZLNA&data=04%7C01%7Calexander.e.smith%40kcl.ac.uk%7C6607af6314024f0a866a08d8b324d18f%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637456318013968398%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Wdbd21OVqZAYzJQBfAnXmXYZCGTsizkC95vgns7cO1E%3D&reserved=0>.
|
Hi Tom/Ian
I think this is a docker-related issue I haven't been able to resolve yet. I cannot get the nosetests to run faultlessly (same single error everytime) for several docker image variants, one of which uses the same ubuntu base (18) and venv environment (same modules installed on same python version) as my host machine on which nosetest runs faultlessly. I also get faultless nosetests results on another machine running ubuntu 20 when not using docker too. For docker, the error is same for docker containers build on ubuntu or debian buster/buster-slim in both python 3.6 and 3.7.
Dr Alexander Smith
Dept Haematology
King's College NHS trust
Denmark Hill
London SE5 9RS
…________________________________
From: Smith, Alexander <[email protected]>
Sent: 07 January 2021 16:51
To: CGATOxford/UMI-tools <[email protected]>; CGATOxford/UMI-tools <[email protected]>
Cc: Author <[email protected]>
Subject: Re: [CGATOxford/UMI-tools] nosetests showing 1 error in python3 (and multiple for python 2.7 with appropriate umi tools version) (#450)
Hi again
I think I have a dependency issue. A pip install on my local ubuntu machine (not in docker container) works with no errors from nosetests (python 3.6.9 in venv). I'll investigate further and let you know if I find a culprit...such as pysam.
A
________________________________
From: Ian Sudbery <[email protected]>
Sent: 07 January 2021 15:56
To: CGATOxford/UMI-tools <[email protected]>
Cc: Smith, Alexander <[email protected]>; Author <[email protected]>
Subject: Re: [CGATOxford/UMI-tools] nosetests showing 1 error in python3 (and multiple for python 2.7 with appropriate umi tools version) (#450)
No i'm not sure attachments work in github issues.
The problem in the nosetests for the python3 version appears to be that the your umi_tools appears to be outputting a sam file with a header, where as the reference does not contain a header.
I wonder if this is a pysam/samtools version issue. Has a new version of pysam/samtools be released since we last did a rebuild?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FCGATOxford%2FUMI-tools%2Fissues%2F450%23issuecomment-756204183&data=04%7C01%7Calexander.e.smith%40kcl.ac.uk%7C6607af6314024f0a866a08d8b324d18f%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637456318013968398%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=e4UFcEBSLaxRU%2FB9WuNGwi%2B7LB7Wi5ROcmiHsH1kgdU%3D&reserved=0>, or unsubscribe<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAG3NR5TSJADMIIJLQ55JSRTSYXKTLANCNFSM4VY4ZLNA&data=04%7C01%7Calexander.e.smith%40kcl.ac.uk%7C6607af6314024f0a866a08d8b324d18f%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637456318013968398%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Wdbd21OVqZAYzJQBfAnXmXYZCGTsizkC95vgns7cO1E%3D&reserved=0>.
|
Thats very peculiar. The problem is that one your tests the header is being output, and our reference file doesn't have headers. This doesn't seem like it should be an OS related problem. |
Closing due to inactivity |
Hi
I've not been able to get a completely error free output after new installation and running your nose tests. See below for details of how I have run this and environment set-up before hand. I get 1 error around whitelisting for python3 set-up when running nose tests. And multiple errors for old installation of umi tools (0.5.1) in python 2.7....although i'm not sure if the nose tests are appropriate here (nosetest umi_test.py is aligned for the installation version of umi_tools. See below for details:
In python 3.7
export PYTHONHASHSEED=0
root@feac0438ff76:/usr/umitools# which umi_tools
/opt/venv/bin/umi_tools
root@feac0438ff76:/usr/umitools# umi_tools --version
UMI-tools version: 1.1.1
root@e83a97661f88:/usr/umitools/UMI-tools-1.1.1# nosetests tests/test_umi_tools.py
OUTPUT:
I also ran this on older version of UMI_tools in python 2.7 (version 0.5.1), which gave may more errors using same test conditions. Although I am not sure whether this is valid in python2.7 at all.
The text was updated successfully, but these errors were encountered: