Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large zip file breaks with loadAsync #912

Open
velgajski1 opened this issue Oct 31, 2023 · 2 comments
Open

Large zip file breaks with loadAsync #912

velgajski1 opened this issue Oct 31, 2023 · 2 comments

Comments

@velgajski1
Copy link

Tested on Chrome/Edge/Safari.

Error is :
DOMException: The requested file could not be read, typically due to permission problems that have occurred after a reference to a file was acquired.

Same file works with zip.js for example, and it used to work with jszip until few days ago. File size is around 3GB. Tested on smaller (100mb) zip files and for those it still works normally.

Any clues?

@sobjornstad
Copy link

I tried to isolate this by creating a test with the same file in a Node script and got instead the following error:

/.../node_modules/jszip/lib/reader/DataReader.js:26
            throw new Error("End of data reached (data length = " + this.length + ", asked index = " + (newIndex) + "). Corrupted zip ?");
                  ^

Error: End of data reached (data length = 4026263362, asked index = -270818191). Corrupted zip ?
    at NodeBufferReader.checkIndex (/.../node_modules/jszip/lib/reader/DataReader.js:26:19)
    at NodeBufferReader.setIndex (/.../node_modules/jszip/lib/reader/DataReader.js:35:14)
    at ZipEntries.readCentralDir (/.../node_modules/jszip/lib/zipEntries.js:132:21)
    at ZipEntries.load (/.../node_modules/jszip/lib/zipEntries.js:256:14)
    at /.../node_modules/jszip/lib/load.js:48:24

unzip -t says the file is valid, though, which makes me wonder if it's a different manifestation of this bug that for some reason occurs differently when reading from a buffer in Node: #543

(Could always be two different issues though.)

Test script (zero points for elegance, it's multiple levels of Claude-generated code and editing while debugging and is more complicated than it needs to be...):

import JSZip from 'jszip';

import * as fs from 'fs';
import * as path from 'path';

class NodeFile {
  constructor(filePath) {
    const stats = fs.statSync(filePath);
    this.name = path.basename(filePath);
    this.size = stats.size;
    this.type = ''; // MIME type, you may want to implement this
    this.lastModified = stats.mtime.getTime();
    this._path = filePath;
  }

  stream() {
    return fs.createReadStream(this._path);
  }

  slice(start, end, contentType) {
    // This is a simplified implementation
    const stream = fs.createReadStream(this._path, { start, end });
    return new NodeFile(this._path, stream, contentType);
  }
}

function stream2buffer(stream) {
    return new Promise((resolve, reject) => {
        const _buf = [];
        stream.on("data", (chunk) => _buf.push(chunk));
        stream.on("end", () => resolve(Buffer.concat(_buf)));
        stream.on("error", (err) => reject(err));
    });
}

const myFile = new NodeFile("my_file_path");
const buff = stream2buffer(myFile.stream());
JSZip.loadAsync(buff);

@hardik-kapadia
Copy link

Did anyone fine a solution to this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants