matrix_mod2_dense unable to export to png

[rampaq]

Steps To Reproduce

When trying to save a GF(2) dense matrix with one dimension larger than 1000001, the export will fail. However, 1000000 works fine. Saving to PNG is used for example when passing a matrix to a function decorated with @parallel.

In sage, run:

import os
from sage.matrix.matrix_mod2_dense import to_png, from_png
fn = tmp_filename()
M=MatrixSpace(GF(2),1,1_000_001).random_element()
to_png(M, fn)
print(os.path.getsize(fn))

Output:

libpng warning: Image width exceeds user limit in IHDR
GD Warning: gd-png: fatal libpng error: Invalid IHDR data
GD Warning: gd-png error: setjmp returns error condition
0

Expected Behavior

A PNG file with the matrix should be saved.

Actual Behavior

to_png fails when at least one dimension is larger than or equal to 1000001

Additional Information

1000000 works fine.

import os
from sage.matrix.matrix_mod2_dense import to_png, from_png
fn = tmp_filename()
M=MatrixSpace(GF(2),1,1_000_000).random_element()
to_png(M, fn)
print(os.path.getsize(fn))

>>> 12620

For matrix/image with dimensions (a,b), both libPNG, libGD and M4RI support a*b <= 2^31. A million in a single dimension seems rather arbitrary. I could not find where this user limit is set in Sage source files.

Environment

• OS: Ubuntu 20.4
• Sage Version: 10.4

Checklist

• I have searched the existing issues for a bug report that matches the one I want to file, without success.
• I have read the documentation and troubleshoot guide

[DaveWitteMorris]

I think this is an issue with libpng, not sage, because the same problem has been reported on forums that are not related to sage. For example:

https://stackoverflow.com/questions/78812551/does-libpng-have-an-upper-limit-on-image-width

udoprog/c10t#59

[rampaq]

Thanks for quick reply. I have read on libPNG site that they support up to 2^31 sizes and assumed that thats it. However, in their manual 1, it says:

User limits
The PNG specification allows the width and height of an image to be as
large as 2^31-1 (0x7fffffff), or about 2.147 billion rows and columns.
For safety, libpng imposes a default limit of 1 million rows and columns.
Larger images will be rejected immediately with a png_error() call. If
you wish to change these limits, you can use
png_set_user_limits(png_ptr, width_max, height_max);

Perhaps, we should explicitly override this limit in Sage to the maximum 0x7fffffff?

[user202729]

In m4ri source code:

int mzd_to_png(const mzd_t *A, const char *fn, int compression_level, const char *comment,
               int verbose) {
  FILE *fh = fopen(fn, "wb");

  if (!fh) {
    if (verbose) printf("Could not open file '%s' for writing\n", fn);
    return 1;
  }

  png_structp png_ptr = png_create_write_struct(PNG_LIBPNG_VER_STRING, NULL, NULL, NULL);

  if (!png_ptr) {
    if (verbose) printf("failed to initialise PNG write struct.\n");
    fclose(fh);
    return 3;
  }
  png_set_user_limits(png_ptr, 0x7fffffffL, 0x7fffffffL);

Why does the line not work then? Debugging needed.

---

Apparently we're not using mzd_to_png function, instead a custom function is used

    cdef gdImagePtr im = gdImageCreate(c, r)
    cdef FILE * out = fopen(filename, "wb")
    cdef int black = gdImageColorAllocate(im, 0, 0, 0)
    cdef int white = gdImageColorAllocate(im, 255, 255, 255)
    gdImageFilledRectangle(im, 0, 0, c-1, r-1, white)
    for i from 0 <= i < r:
        for j from 0 <= j < c:
            if mzd_read_bit(A._entries, i, j):
                gdImageSetPixel(im, j, i, black )

    gdImagePng(im, out)
    gdImageDestroy(im)
    fclose(out)

Reported issue to libgd: libgd/libgd#931

[rampaq]

In m4ri source code:

int mzd_to_png(const mzd_t *A, const char *fn, int compression_level, const char *comment,
               int verbose) {
  FILE *fh = fopen(fn, "wb");

  if (!fh) {
    if (verbose) printf("Could not open file '%s' for writing\n", fn);
    return 1;
  }

  png_structp png_ptr = png_create_write_struct(PNG_LIBPNG_VER_STRING, NULL, NULL, NULL);

  if (!png_ptr) {
    if (verbose) printf("failed to initialise PNG write struct.\n");
    fclose(fh);
    return 3;
  }
  png_set_user_limits(png_ptr, 0x7fffffffL, 0x7fffffffL);

I wonder if there is a reason why Sage is not using the original mzd_to_png to export the M4RI matrices? It seems like a more obvious choice as opposed to libgd.