[Potential Feature] How to identify an entry as symlink?

[ayushmanchhabra]

export async function unzip(zippedFile, cacheDir) {
  const zip = await yauzl.open(zippedFile);

  let entry = await zip.readEntry();

  while (entry !== null) {
    let entryPathAbs = path.join(cacheDir, entry.filename);

    // Create the directory beforehand to prevent `ENOENT: no such file or directory` errors.
    await fs.promises.mkdir(path.dirname(entryPathAbs), { recursive: true });

    const readStream = await entry.openReadStream();
    const writeStream = fs.createWriteStream(entryPathAbs);
    await stream.promises.pipeline(readStream, writeStream);

    entry = await zip.readEntry();
  }

  await zip.close();
}

This is currently how I use this package to unzip files. Is there a way to know if an entry is a symlink? Currently if an entry is a symlink, it is decompressed as a file with the name of directory as file name and the file path as data in that file.

For example, if Resources is a symlink, it is decompressed as a file with file contents:

Versions/Current/Resources

[overlookmotel]

I can't actually remember! Or perhaps the ZIP spec doesn't actually cover this, and it's been tacked-on as custom extensions by different ZIP implementations.

I'd suggest looking at the ZIP spec and also using a bit of trial and error looking at entry.generalPurposeBitFlag and entry.extraFields - comparing the 2 for normal files and symlinks.

If you find the answer, please let me know and we can maybe add an API for it.

Sorry but I don't have time to figure it out myself right now (and symlinks are probably a niche case), but I hope the above is useful.

[ayushmanchhabra]

I can't actually remember! Or perhaps the ZIP spec doesn't actually cover this, and it's been tacked-on as custom extensions by different ZIP implementations.

I'd suggest looking at the ZIP spec and also using a bit of trial and error looking at entry.generalPurposeBitFlag and entry.extraFields - comparing the 2 for normal files and symlinks.

If you find the answer, please let me know and we can maybe add an API for it.

Sorry but I don't have time to figure it out myself right now (and symlinks are probably a niche case), but I hope the above is useful.

Sounds good, I'll look into it!

[ayushmanchhabra]

According to this Stack Overflow post and how this library has implemented symlinks, we have to get the externalFileAttributes from Zip._readEntryAt to calculate the file mode which would allow us to identify if an entry is a symlink or not.

I think with this information, I can identify symlinks without making any changes to yauzl-promise. In the future, an entry.isSymlink() function would be great!

Update:

I'm not sure it'd be the same on Windows

https://stackoverflow.com/questions/4939802/what-are-the-possible-mode-values-returned-by-powershells-get-childitem-cmdle
This might come in handy. Linking here for future reference.

[overlookmotel]

Hi @ayushmanchhabra. Thanks very much for looking into it. From reading those, I'm not sure it'd be the same on Windows, as it has a very different file permissions model, from what I understand.

I would be happy to accept a PR, as long as there's a test for unzipping a ZIP file containing symlinks, and that test passes on a range of OSes. If you're willing, that would be great.

But otherwise, I'm afraid I'm not going to be able to find the time to do it myself.

Happy to provide any help/guidance you need in that process. Please let me know if you want to.

[ayushmanchhabra]

I'm not sure it'd be the same on Windows, as it has a very different file permissions model

How typical of me to ignore Windows! :P

I would be happy to accept a PR, as long as there's a test for unzipping a ZIP file containing symlinks, and that test passes on a range of OSes. If you're willing, that would be great.

I'm planning to test out the symlinks behaviour in nw-builder. Once it works there, I'd be happy to submit a PR!

[ayushmanchhabra]

I was successfully able to identify symlinks in nw-builder!

Before implementing entry.isSymlink()

function modeFromEntry(entry) {
  const attr = entry.externalFileAttributes >> 16 || 33188;

  return [448 /* S_IRWXU */, 56 /* S_IRWXG */, 7 /* S_IRWXO */]
    .map(mask => attr & mask)
    .reduce((a, b) => a + b, attr & 61440 /* S_IFMT */);
}

async function unzip(zippedFile, cacheDir) {
  const zip = await yauzl.open(zippedFile);
  let entry = await zip.readEntry();

  while (entry !== null) {
    let entryPathAbs = path.join(cacheDir, entry.filename);
    /* Create the directory beforehand to prevent `ENOENT: no such file or directory` errors. */
    await fs.promises.mkdir(path.dirname(entryPathAbs), { recursive: true });
    /* Check if entry is a symbolic link */
    const isSymlink = ((modeFromEntry(entry) & 0o170000) === 0o120000);
    const readStream = await entry.openReadStream();
    
    if (isSymlink) {
      const chunks = [];
      readStream.on("data", (chunk) => chunks.push(chunk));
      await stream.promises.finished(readStream);
      const link = Buffer.concat(chunks).toString('utf8').trim();
      await fs.promises.symlink(link, entryPathAbs)
    } else {
      const writeStream = fs.createWriteStream(entryPathAbs);
      await stream.promises.pipeline(readStream, writeStream);
    }

    // Read next entry
    entry = await zip.readEntry();
  }
}

After implementing entry.isSymlink():

async function unzip(zippedFile, cacheDir) {
  const zip = await yauzl.open(zippedFile);
  let entry = await zip.readEntry();

  while (entry !== null) {
    let entryPathAbs = path.join(cacheDir, entry.filename);
    /* Create the directory beforehand to prevent `ENOENT: no such file or directory` errors. */
    await fs.promises.mkdir(path.dirname(entryPathAbs), { recursive: true });
    const readStream = await entry.openReadStream();
    
    if (entry.isSymlink()) {
      const chunks = [];
      readStream.on("data", (chunk) => chunks.push(chunk));
      await stream.promises.finished(readStream);
      const link = Buffer.concat(chunks).toString('utf8').trim();
      await fs.promises.symlink(link, entryPathAbs)
    } else {
      const writeStream = fs.createWriteStream(entryPathAbs);
      await stream.promises.pipeline(readStream, writeStream);
    }

    // Read next entry
    entry = await zip.readEntry();
  }
}

The user will still have to create symlinks on their own and this should not be implemented by yauzl-promise. Thoughts?