Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

files_io doctest-> pytest #908

Merged
merged 8 commits into from
Dec 13, 2023
Merged

files_io doctest-> pytest #908

merged 8 commits into from
Dec 13, 2023

Conversation

mmatera
Copy link
Contributor

@mmatera mmatera commented Aug 19, 2023

Here I started to take some nontrivial decisions:

  • "..." does not work in pytests. I just checked the structure from the output by checking the match against a pattern, or by looking at the head and certain elements expected in the output.
  • In some places of true Doctests, there are private lines (#>) for erasing variables, closing a file or deleting a file created by the test. I am not sure what to do with them.

@@ -528,29 +502,9 @@ class OpenRead(_OpenAction):

>> OpenRead["ExampleData/EinsteinSzilLetter.txt", CharacterEncoding->"UTF8"]
= InputStream[...]
#> Close[%];
>> Close[%];

S> Close[OpenRead["https://raw.githubusercontent.com/Mathics3/mathics-core/master/README.rst"]];
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I keep these sandbox tests for another round, because I need to figure out how to mark them in the pytests. Any idea?

Copy link
Member

@rocky rocky Aug 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a thought right now about sandboxed tests.

As for #> Close[%]; vs. >> Close[%];, if it is instructive that is okay. For example:

>> OpenRead["ExampleData/EinsteinSzilLetter.txt", CharacterEncoding->"UTF8"]
     = InputStream[...]

Streams should be closed when done with them: 
 >> Close[%];

But if it is not, but just done to make testing more robust and for bookkeeping, then keep the #> so we don't pollute the docs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good: I leave then S>for another round. Also, I have reverted #> Close ->>> Close when it is not instructive.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Sphinx, the cleanup is implemented in a special code block directive testcleanup:
https://www.sphinx-doc.org/en/master/usage/extensions/doctest.html

@rocky
Copy link
Member

rocky commented Dec 13, 2023

LGTM - thanks. Sorry I didn't get to this sooner.

@rocky rocky merged commit 2abaead into master Dec 13, 2023
11 checks passed
@rocky rocky deleted the move_private_doctests_to_pytest5 branch December 13, 2023 21:01
None,
),
(
'OpenRead["MathicsNonExampleFile"]',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mmatera There are a problems with this test.

It assumes this file does not exist. Another test in this parameterize loop creates a file with exactly this name.

Instead, the test should ensure that a file does not exist is used. One way to do this is to create a new named temporary directory which is empty and then look for some name there. https://docs.python.org/3/library/tempfile.html#tempfile.TemporaryDirectory can help here.

So probably this should be removed from this parameterize loop and be put in a separate test that has the additional code

None,
),
(
'fd=OpenRead["ExampleData/EinsteinSzilLetter.txt", BinaryFormat -> True, CharacterEncoding->"UTF8"]//Head',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mmatera The sequence of tests from here to line 172 should be put in a separate test. Subsequent tests depend on the actions of earlier tests. If an early test fails, there is no cleanup. I am seeing for example "MathicsNonExampleFIle" appearing in the file system on failure and equally worse, in the mathics-core directory which should not be touched by testing.

In that new separate test we should be doing cleanup when something goes wrong or a test fails. Context managers can help, or possibly the use of a temporary new temporary file or directory.

For tests that are writing files, I do not think we are ensuring that we have access to a writable directory.

@@ -82,6 +81,185 @@ def test_export():
assert data.endswith("</svg>")


"""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Double quotes and comments? Comments with a TODO is probably good enough.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants