node_icon.diag assumes a network connection and Debian system

[emilazy]

blockdiag/src/blockdiag/tests/diagrams/node_icon.diag

Lines 1 to 6 in ce37f0c

	{
	A -> B;
	A [label = "aaaaaaaaaaaaaaaaa", icon = "/usr/share/pixmaps/debian-logo.png"];
	B [label = "aaaaaaaaaaaaaaaaa", icon = "http://blockdiag.com/favicon.ico"];
	}

These cause problems for us in Nixpkgs, as we of course don’t have a Debian logo lying around on the filesystem and our build and testing environment cannot access external network resources.

I see that the Debian logo is already present in‐tree, so hopefully that could be changed to a local reference. I’m not sure if the URL is critical to the functionality being tested here, but I thought I’d raise it anyway.

(Copied from blockdiag#184.)

[ggrossetie]

I'm not even sure that this file is used in the test suite. Could you please provide which command is failing while creating the Nix package?

[emilazy]

Sure, here’s the failure from the log:

=================================== FAILURES ===================================
_ test_generate_with_separate[/private/tmp/nix-build-python3.12-blockdiag-3.0.0.drv-0/source/src/blockdiag/tests/diagrams/node_icon.diag-svg-options92] _

source = '/private/tmp/nix-build-python3.12-blockdiag-3.0.0.drv-0/source/src/blockdiag/tests/diagrams/node_icon.diag'
file_type = 'svg', options = []

    @pytest.mark.parametrize("source,file_type,options", generate_testdata)
    def test_generate_with_separate(source, file_type, options):
>       generate(source, file_type, options)

src/blockdiag/tests/test_generate_diagram.py:72: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ('/private/tmp/nix-build-python3.12-blockdiag-3.0.0.drv-0/source/src/blockdiag/tests/diagrams/node_icon.diag', 'svg', [])
kwargs = {}
stderr = <_io.TextIOWrapper name="<_io.FileIO name=8 mode='rb+' closefd=True>" mode='r+' encoding='utf-8'>

    def wrap(*args, **kwargs):
        try:
            stderr = sys.stderr
            sys.stderr = StringIO()
    
            func(*args, **kwargs)
    
            if re.search('(ERROR|Traceback)', sys.stderr.getvalue()):
>               raise AssertionError('Caught error')
E               AssertionError: Caught error

src/blockdiag/tests/utils.py:78: AssertionError
----------------------------- Captured stdout call -----------------------------
---[ stderr ] ---
WARNING: icon image not found: /usr/share/pixmaps/debian-logo.png
WARNING: Could not retrieve: http://blockdiag.com/favicon.ico
Traceback (most recent call last):
  File "/nix/store/dh1i1387ibdzw0ala5rkl3s3ylf8i8pa-python3-3.12.4/lib/python3.12/urllib/request.py", line 1344, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/nix/store/dh1i1387ibdzw0ala5rkl3s3ylf8i8pa-python3-3.12.4/lib/python3.12/http/client.py", line 1336, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/nix/store/dh1i1387ibdzw0ala5rkl3s3ylf8i8pa-python3-3.12.4/lib/python3.12/http/client.py", line 1382, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/nix/store/dh1i1387ibdzw0ala5rkl3s3ylf8i8pa-python3-3.12.4/lib/python3.12/http/client.py", line 1331, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/nix/store/dh1i1387ibdzw0ala5rkl3s3ylf8i8pa-python3-3.12.4/lib/python3.12/http/client.py", line 1091, in _send_output
    self.send(msg)
  File "/nix/store/dh1i1387ibdzw0ala5rkl3s3ylf8i8pa-python3-3.12.4/lib/python3.12/http/client.py", line 1035, in send
    self.connect()
  File "/nix/store/dh1i1387ibdzw0ala5rkl3s3ylf8i8pa-python3-3.12.4/lib/python3.12/http/client.py", line 1001, in connect
    self.sock = self._create_connection(
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/dh1i1387ibdzw0ala5rkl3s3ylf8i8pa-python3-3.12.4/lib/python3.12/socket.py", line 829, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/dh1i1387ibdzw0ala5rkl3s3ylf8i8pa-python3-3.12.4/lib/python3.12/socket.py", line 964, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/private/tmp/nix-build-python3.12-blockdiag-3.0.0.drv-0/source/src/blockdiag/utils/images.py", line 119, in open
    stream = io.BytesIO(urlopen(url).read())
                        ^^^^^^^^^^^^
  File "/private/tmp/nix-build-python3.12-blockdiag-3.0.0.drv-0/source/src/blockdiag/utils/images.py", line 37, in urlopen
    tmpfile.write(orig_urlopen(url, *args, **kwargs).read())
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/dh1i1387ibdzw0ala5rkl3s3ylf8i8pa-python3-3.12.4/lib/python3.12/urllib/request.py", line 215, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/dh1i1387ibdzw0ala5rkl3s3ylf8i8pa-python3-3.12.4/lib/python3.12/urllib/request.py", line 515, in open
    response = self._open(req, data)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/dh1i1387ibdzw0ala5rkl3s3ylf8i8pa-python3-3.12.4/lib/python3.12/urllib/request.py", line 532, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/dh1i1387ibdzw0ala5rkl3s3ylf8i8pa-python3-3.12.4/lib/python3.12/urllib/request.py", line 492, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/nix/store/dh1i1387ibdzw0ala5rkl3s3ylf8i8pa-python3-3.12.4/lib/python3.12/urllib/request.py", line 1373, in http_open
    return self.do_open(http.client.HTTPConnection, req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/dh1i1387ibdzw0ala5rkl3s3ylf8i8pa-python3-3.12.4/lib/python3.12/urllib/request.py", line 1347, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 8] nodename nor servname provided, or not known>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/private/tmp/nix-build-python3.12-blockdiag-3.0.0.drv-0/source/src/blockdiag/utils/bootstrap.py", line 54, in run
    return self.build_diagram(parsed)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/nix-build-python3.12-blockdiag-3.0.0.drv-0/source/src/blockdiag/command.py", line 43, in build_diagram
    return super(BlockdiagApp, self).build_diagram(tree)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/nix-build-python3.12-blockdiag-3.0.0.drv-0/source/src/blockdiag/utils/bootstrap.py", line 103, in build_diagram
    drawer.draw()
  File "/private/tmp/nix-build-python3.12-blockdiag-3.0.0.drv-0/source/src/blockdiag/drawer.py", line 89, in draw
    self._draw_background()
  File "/private/tmp/nix-build-python3.12-blockdiag-3.0.0.drv-0/source/src/blockdiag/drawer.py", line 115, in _draw_background
    shape = r(node, self.metrics)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/nix-build-python3.12-blockdiag-3.0.0.drv-0/source/src/blockdiag/noderenderer/base.py", line 34, in __init__
    image_size = images.get_image_size(node.icon)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/nix-build-python3.12-blockdiag-3.0.0.drv-0/source/src/blockdiag/utils/images.py", line 50, in get_image_size
    stream = open(image)
             ^^^^^^^^^^^
  File "/private/tmp/nix-build-python3.12-blockdiag-3.0.0.drv-0/source/src/blockdiag/utils/images.py", line 122, in open
    raise IOError
OSError

=============================== warnings summary ===============================
src/blockdiag/noderenderer/__init__.py:18
  /private/tmp/nix-build-python3.12-blockdiag-3.0.0.drv-0/source/src/blockdiag/noderenderer/__init__.py:18: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources

src/blockdiag/tests/rst/test_blockdiag_directives.py::TestRstDirectives::test_align_option1
  /nix/store/3r2canscw43dabd5mym6acqwkbiprqfw-python3.12-reportlab-4.2.2/lib/python3.12/site-packages/reportlab/lib/rl_safe_eval.py:12: DeprecationWarning: ast.NameConstant is deprecated and will be removed in Python 3.14; use ast.Constant instead
    haveNameConstant = hasattr(ast,'NameConstant')

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED src/blockdiag/tests/test_generate_diagram.py::test_generate_with_separate[/private/tmp/nix-build-python3.12-blockdiag-3.0.0.drv-0/source/src/blockdiag/tests/diagrams/node_icon.diag-svg-options92] - AssertionError: Caught error
=========== 1 failed, 712 passed, 1 deselected, 2 warnings in 21.61s ===========

[ggrossetie]

Thanks!
Indeed, test_generate_diagram.py will convert any file found in tests/diagrams (except a few) 😬

I guess we want to make sure that BlockDiag can resolve an icon from an absolute path and/or from an URL.
One way would be to copy an image to the OS temporary directory and use that destination as an absolute path. We could also start an HTTP server to serve an ico/image (or just mock the HTTP client).

I will take a deeper look in order to make the tests more portable.

[emilazy]

Thanks for considering this! Those all sound like reasonable solutions to me. Note that downstream packages like {seq,act,nw}diag reuse the fixtures, so something like a local HTTP server may require some fiddling to plumb in to those test suites. We need to patch them to handle your existing test suite migration changes anyway, though, so I understand if worrying about that is out of scope for you.

[ggrossetie]

Note that downstream packages like {seq,act,nw}diag reuse the fixtures, so something like a local HTTP server may require some fiddling to plumb in to those test suites.

If the HTTP server is started as part of the test suite (in a dedicated thread) it should work right? In other words, the test suite won't assume that a HTTP server is running. We could use something like: https://pytest-httpserver.readthedocs.io/en/latest/

[emilazy]

Right, I just mean that downstream projects use the blockdiag test generators that work with these fixtures, e.g. test_generat_diagram.py in seqdiag (our patch), so the test generator would have to set up the HTTP server fixture in a way that works with downstream tests like that. I’m not too familiar with pytest (despite leading the nose clean‐up effort in Nixpkgs!), so it’s possible this would Just Work.