Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Downloading playlist as an archive ends up with corrupted zip file #619

Open
kli0rf opened this issue Mar 23, 2016 · 13 comments
Open

Downloading playlist as an archive ends up with corrupted zip file #619

kli0rf opened this issue Mar 23, 2016 · 13 comments
Assignees
@devsnd
Labels
bug

Comments

@kli0rf
Copy link

kli0rf commented Mar 23, 2016

Some files in music.zip file gets corrupted after download hence cannot be extracted. For exapmle, extracting files with 7zip gives me something like: "0 CRC failed in '08. Never Say.mp3'. File is broken."
This is actual file, as you can see there's no non ascii chars in it's filename.

Running latest (i guess) CherryMusic Server 0.36.0 on Debian Jessie (8.3) x86.
@6arms1leg
Copy link
Collaborator

Hmm... never seen this before, but there are many similar reports about extraction errors with 7-Zip.
Since CherryMusic uses the standard GNU/Linux tools as backend to create *.zip files, I doubt it's a bug in CherryMusic.
Could you please copy such a *.zip file, that 7-Zip reports to be broken, back to your CherryMusic server and check if extraction works there under Debian?
Please test the archive with:
$ unzip -vt music.zip
and
$ gunzip -Vvt music.zip
and
$ bunzip2 -vt music.zip

I predict that all methods will report the archive to be OK (which would make this an issue with 7-Zip, not CherryMusic), but if they don't please post the output here so we can investigate further.

@kli0rf
Copy link
Author

kli0rf commented Apr 3, 2016

Here's the output:

kli0rf@studio:~/tmp$ unzip -vt music.zip
Archive:  music.zip
    testing: 01. It Isn't Meant To Be.mp3   OK
    testing: 02. Desire Be Desire Go.mp3   bad CRC da5f2336  (should be 25a0dcca)
    testing: 03. Alter Ego.mp3        OK
    testing: 04. Lucidity.mp3         OK
    testing: 05. Why Won't You Make Up Your Mind.mp3   OK
    testing: 06. Solitude Is Bliss.mp3   bad CRC e34a5bfd  (should be 1cb5a403)
    testing: 07. Jeremy's Storm.mp3   bad CRC a89f9ea7  (should be 57606159)
    testing: 08. Expectation.mp3      bad CRC 933cb972  (should be 6cc3468e)
    testing: 09. The Bold Arrow Of Time.mp3   OK
    testing: 10. Runway, Houses, City, Clouds.mp3   OK
    testing: 11. I Don't Really Mind.mp3   OK
At least one error was detected in music.zip.
kli0rf@studio:~/tmp$ gunzip -Vvt music.zip
gzip 1.6
Copyright (C) 2007, 2010, 2011 Free Software Foundation, Inc.
Copyright (C) 1993 Jean-loup Gailly.
This is free software.  You may redistribute copies of it under the terms of
the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
There is NO WARRANTY, to the extent permitted by law.

Written by Jean-loup Gailly.

kli0rf@studio:~/tmp$ gunzip -vt music.zip
music.zip:      gzip: music.zip has more than one entry--rest ignored
 OK

kli0rf@studio:~/tmp$ bunzip2 -vt music.zip
  music.zip: bad magic number (file not created by bzip2)

You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.

EDIT: Formatted code properly. Please use Markdown next time. -- 6arms1leg

@kli0rf
Copy link
Author

kli0rf commented Apr 3, 2016

And here's another one:

kli0rf@studio:~/tmp/CD I$ ls -l
total 125072
-rw-r--r-- 1 kli0rf kli0rf 12874693 Apr  3 19:50 01. It Isn't Meant To Be.mp3
-rw-r--r-- 1 kli0rf kli0rf 10651148 Apr  3 19:50 02. Desire Be Desire Go.mp3
-rw-r--r-- 1 kli0rf kli0rf 11510034 Apr  3 19:50 03. Alter Ego.mp3
-rw-r--r-- 1 kli0rf kli0rf 10879958 Apr  3 19:50 04. Lucidity.mp3
-rw-r--r-- 1 kli0rf kli0rf  7978325 Apr  3 19:50 05. Why Won't You Make Up Your Mind.mp3
-rw-r--r-- 1 kli0rf kli0rf  9439062 Apr  3 19:50 06. Solitude Is Bliss.mp3
-rw-r--r-- 1 kli0rf kli0rf 13142175 Apr  3 19:50 07. Jeremy's Storm.mp3
-rw-r--r-- 1 kli0rf kli0rf 14520389 Apr  3 19:50 08. Expectation.mp3
-rw-r--r-- 1 kli0rf kli0rf 10603088 Apr  3 19:50 09. The Bold Arrow Of Time.mp3
-rw-r--r-- 1 kli0rf kli0rf 17407477 Apr  3 19:50 10. Runway, Houses, City, Clouds.mp3
-rw-r--r-- 1 kli0rf kli0rf  9040961 Apr  3 19:50 11. I Don't Really Mind.mp3

kli0rf@studio:~/tmp/CD I$ zip -v test.zip ./*
  adding: 01. It Isn't Meant To Be.mp3 .        (in=12874693) (out=12740592) (deflated 1%)
  adding: 02. Desire Be Desire Go.mp3 . (in=10651148) (out=10580343) (deflated 1%)
  adding: 03. Alter Ego.mp3 .   (in=11510034) (out=11461613) (deflated 0%)
  adding: 04. Lucidity.mp3 .    (in=10879958) (out=10807623) (deflated 1%)
  adding: 05. Why Won't You Make Up Your Mind.mp3       (in=7978325) (out=7906298) (deflated 1%)
  adding: 06. Solitude Is Bliss.mp3     (in=9439062) (out=9384092) (deflated 1%)
  adding: 07. Jeremy's Storm.mp3 .      (in=13142175) (out=12980682) (deflated 1%)
  adding: 08. Expectation.mp3 . (in=14520389) (out=14363629) (deflated 1%)
  adding: 09. The Bold Arrow Of Time.mp3 .      (in=10603088) (out=10448407) (deflated 1%)
  adding: 10. Runway, Houses, City, Clouds.mp3 .        (in=17407477) (out=17280542) (deflated 1%)
  adding: 11. I Don't Really Mind.mp3   (in=9040961) (out=8933725) (deflated 1%)
total bytes=128047310, compressed=126887546 -> 1% savings

kli0rf@studio:~/tmp/CD I$ unzip -vt test.zip
Archive:  test.zip
    testing: 01. It Isn't Meant To Be.mp3   OK
    testing: 02. Desire Be Desire Go.mp3   OK
    testing: 03. Alter Ego.mp3        OK
    testing: 04. Lucidity.mp3         OK
    testing: 05. Why Won't You Make Up Your Mind.mp3   OK
    testing: 06. Solitude Is Bliss.mp3   OK
    testing: 07. Jeremy's Storm.mp3   OK
    testing: 08. Expectation.mp3      OK
    testing: 09. The Bold Arrow Of Time.mp3   OK
    testing: 10. Runway, Houses, City, Clouds.mp3   OK
    testing: 11. I Don't Really Mind.mp3   OK
No errors detected in compressed data of test.zip.

@6arms1leg
Copy link
Collaborator

Well, then I was wrong and we can't blame 7-Zip for it.
I see, manually creating *.zip files from the command line works fine.
In your initial post you wrote

Some files in music.zip file gets corrupted [...]

Are they always the same files, that end up corrupted? Let's say you would do the same procedure again, starting with downloading the exact same playlist in CherryMusic etc.... Do the same files end up corrupted?

@kli0rf
Copy link
Author

kli0rf commented Apr 3, 2016

Yup, always the same files. I just tried the same procedure on another CherryMusic instance running on another Jessie with the same content...

@6arms1leg
Copy link
Collaborator

What's the command line output from CherryMusic when you download a playlist?
Otherwise, I guess I'm out of ideas.
CherryMusic uses ZipStream to create the archives and I remember we had an issue with that in the past (see #405), which sparked three new Issues/Pull requests over at their GitHub project page (see the references at the end of the page in #405).
But this issue doesn't seem to be related so far...

@6arms1leg 6arms1leg added the bug label Apr 8, 2016
@kli0rf
Copy link
Author

kli0rf commented Apr 10, 2016

The only output is:

[160410-16:25] 127.0.0.1 - - [10/Apr/2016:16:25:22] "POST /api/heartbeat HTTP/1.1" 200 14 "http://cherry.domain.tld/" "Mozilla/5.0 (Windows NT 5.1; rv:45.0) Gecko/20100101 Firefox/45.0"
[160410-16:25] 127.0.0.1 - - [10/Apr/2016:16:25:46] "POST /api/downloadcheck HTTP/1.1" 200 14 "http://cherry.domain.tld/" "Mozilla/5.0 (Windows NT 5.1; rv:45.0) Gecko/20100101 Firefox/45.0"
[160410-16:25] 127.0.0.1 - - [10/Apr/2016:16:25:46] "POST /download HTTP/1.1" 200 - "http://cherry.domain.tld/" "Mozilla/5.0 (Windows NT 5.1; rv:45.0) Gecko/20100101 Firefox/45.0"
[160410-16:25] 127.0.0.1 - - [10/Apr/2016:16:25:52] "POST /api/heartbeat HTTP/1.1" 200 14 "http://cherry.domain.tld/" "Mozilla/5.0 (Windows NT 5.1; rv:45.0) Gecko/20100101 Firefox/45.0"

I didn't see anything like "-v" flag in the help page, to make the output more verbose.
I don't know, i guess i can provide you an access to my cherrymusic instance if that would help.

@devsnd
Copy link
Owner

devsnd commented May 23, 2016

Hey @kli0rf,

Thanks for the report. CherryMusic uses a patched python 2 module to create the zip files. It might as well contain a bug, as it already did in the past!

Can you please send me the file in question, so that I can create a testcase to recreate this error? thanks!

@devsnd devsnd self-assigned this May 23, 2016
@kli0rf
Copy link
Author

kli0rf commented May 26, 2016 •
edited
Loading

Hi @devsnd,

do you mean a problematical zip file? If so, here's a file on my debian box (since it's too large to attach it to this conversation).

@kli0rf
Copy link
Author

kli0rf commented Jul 1, 2016

Hi @devsnd,

i've removed the file from my server already. Hope you've successfully downloaded it.
Are there any updates on this case?

@devsnd
Copy link
Owner

devsnd commented Jul 9, 2016

I tried to reproduce it but I was not successful... It seems there is still a subtle bug in the zipstream module.

also, do you use python 2 or 3? I'm using python 3 and never have experienced any of those issues.

BTW: I love that album.

@kli0rf
Copy link
Author

kli0rf commented Jul 12, 2016

I use python 2.7.9... So that's why probably i'm getting this error. Ok, let me try to install python 3. I will let You know.

@kli0rf
Copy link
Author

kli0rf commented Jul 12, 2016

Yup, You're absolutely right. With python3 everything's ok. Thank You! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@devsnd devsnd

Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@devsnd @6arms1leg @kli0rf