Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

files_io doctest-> pytest #908

Merged
merged 8 commits into from
Dec 13, 2023
Merged

files_io doctest-> pytest #908

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
467d03f
files_io doctest-> pytest
mmatera Aug 18, 2023
1b082ca
add test_filesystem
mmatera Aug 19, 2023
513847e
add test vectoranalysis
mmatera Aug 19, 2023
ad0aa08
revert #> Close -> >> Close when is not instructive
mmatera Aug 19, 2023
c5a913d
Merge remote-tracking branch 'origin/master' into move_private_doctes…
mmatera Sep 4, 2023
ee82826
Merge remote-tracking branch 'origin/master' into move_private_doctes…
mmatera Sep 9, 2023
2ed8476
Merge branch 'master' into move_private_doctests_to_pytest5
mmatera Sep 25, 2023
cfe335b
Merge branch 'master' into move_private_doctests_to_pytest5
mmatera Oct 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
214 changes: 21 additions & 193 deletions mathics/builtin/files_io/files.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -204,10 +204,6 @@ class Close(Builtin):
Closing a file doesn't delete it from the filesystem
>> DeleteFile[file];

#> Close["abc"]
: abc is not open.
= Close[abc]

#> Clear[file]
"""

Expand Down Expand Up @@ -277,18 +273,6 @@ class FilePrint(Builtin):
<dd>prints the raw contents of $file$.
</dl>

#> exp = Sin[1];
#> FilePrint[exp]
: File specification Sin[1] is not a string of one or more characters.
= FilePrint[Sin[1]]

#> FilePrint["somenonexistentpath_h47sdmk^&h4"]
: Cannot open somenonexistentpath_h47sdmk^&h4.
= FilePrint[somenonexistentpath_h47sdmk^&h4]

#> FilePrint[""]
: File specification is not a string of one or more characters.
= FilePrint[]
"""

messages = {
Expand Down Expand Up @@ -394,16 +378,6 @@ class Get(PrefixOperator):
## TODO: Requires EndPackage implemented
## 'Get' can also load packages:
## >> << "VectorAnalysis`"

#> Get["SomeTypoPackage`"]
: Cannot open SomeTypoPackage`.
= $Failed

## Parser Tests
#> Hold[<< ~/some_example/dir/] // FullForm
= Hold[Get["~/some_example/dir/"]]
#> Hold[<<`/.\-_:$*~?] // FullForm
= Hold[Get["`/.\\\\-_:$*~?"]]
"""
operator = "<<"
options = {
Expand Down Expand Up @@ -528,29 +502,11 @@ class OpenRead(_OpenAction):

>> OpenRead["ExampleData/EinsteinSzilLetter.txt", CharacterEncoding->"UTF8"]
= InputStream[...]
#> Close[%];

S> Close[OpenRead["https://raw.githubusercontent.com/Mathics3/mathics-core/master/README.rst"]];

#> OpenRead[]
: OpenRead called with 0 arguments; 1 argument is expected.
= OpenRead[]

#> OpenRead[y]
: File specification y is not a string of one or more characters.
= OpenRead[y]
The stream must be closed after using it to release the resource:
>> Close[%];

#> OpenRead[""]
: File specification is not a string of one or more characters.
= OpenRead[]

#> OpenRead["MathicsNonExampleFile"]
: Cannot open MathicsNonExampleFile.
= OpenRead[MathicsNonExampleFile]

#> OpenRead["ExampleData/EinsteinSzilLetter.txt", BinaryFormat -> True, CharacterEncoding->"UTF8"]
= InputStream[...]
#> Close[%];
S> Close[OpenRead["https://raw.githubusercontent.com/Mathics3/mathics-core/master/README.rst"]];
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I keep these sandbox tests for another round, because I need to figure out how to mark them in the pytests. Any idea?

Copy link
Member

@rocky rocky Aug 19, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a thought right now about sandboxed tests.

As for #> Close[%]; vs. >> Close[%];, if it is instructive that is okay. For example:

>> OpenRead["ExampleData/EinsteinSzilLetter.txt", CharacterEncoding->"UTF8"]
     = InputStream[...]

Streams should be closed when done with them: 
 >> Close[%];

But if it is not, but just done to make testing more robust and for bookkeeping, then keep the #> so we don't pollute the docs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good: I leave then S>for another round. Also, I have reverted #> Close ->>> Close when it is not instructive.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Sphinx, the cleanup is implemented in a special code block directive testcleanup:
https://www.sphinx-doc.org/en/master/usage/extensions/doctest.html

"""

summary_text = "open a file for reading"
Expand All @@ -569,11 +525,7 @@ class OpenWrite(_OpenAction):

>> OpenWrite[]
= OutputStream[...]
#> DeleteFile[Close[%]];

#> OpenWrite[BinaryFormat -> True]
= OutputStream[...]
#> DeleteFile[Close[%]];
>> DeleteFile[Close[%]];
"""

summary_text = (
Expand All @@ -594,14 +546,8 @@ class OpenAppend(_OpenAction):

>> OpenAppend[]
= OutputStream[...]
#> DeleteFile[Close[%]];
>> DeleteFile[Close[%]];

#> appendFile = OpenAppend["MathicsNonExampleFile"]
= OutputStream[MathicsNonExampleFile, ...]

#> Close[appendFile]
= MathicsNonExampleFile
#> DeleteFile["MathicsNonExampleFile"]
"""

mode = "a"
Expand Down Expand Up @@ -757,17 +703,7 @@ class PutAppend(BinaryOperator):
| 265252859812191058636308480000000
| 8320987112741390144276341183223364380754172606361245952449277696409600000000000000
| "string"
#> DeleteFile["factorials"];

## writing to dir
#> x >>> /var/
: Cannot open /var/.
= x >>> /var/

## writing to read only file
#> x >>> /proc/uptime
: Cannot open /proc/uptime.
= x >>> /proc/uptime
>> DeleteFile["factorials"];
"""

operator = ">>>"
Expand Down Expand Up @@ -842,20 +778,12 @@ class Read(Builtin):
<li>Word
</ul>

## Malformed InputString
#> Read[InputStream[String], {Word, Number}]
= Read[InputStream[String], {Word, Number}]

## Correctly formed InputString but not open
#> Read[InputStream[String, -1], {Word, Number}]
: InputStream[String, -1] is not open.
= Read[InputStream[String, -1], {Word, Number}]

## Reading Strings
>> stream = StringToStream["abc123"];
>> Read[stream, String]
= abc123
#> Read[stream, String]
>> Read[stream, String]
= EndOfFile
#> Close[stream];

Expand All @@ -865,60 +793,19 @@ class Read(Builtin):
= abc
>> Read[stream, Word]
= 123
#> Read[stream, Word]
= EndOfFile
#> Close[stream];
#> stream = StringToStream[""];
#> Read[stream, Word]
= EndOfFile
#> Read[stream, Word]
>> Read[stream, Word]
= EndOfFile
#> Close[stream];

## Number
>> stream = StringToStream["123, 4"];
>> Read[stream, Number]
= 123
>> Read[stream, Number]
= 4
#> Read[stream, Number]
>> Read[stream, Number]
= EndOfFile
#> Close[stream];
#> stream = StringToStream["123xyz 321"];
#> Read[stream, Number]
= 123
#> Quiet[Read[stream, Number]]
= $Failed

## Real
#> stream = StringToStream["123, 4abc"];
#> Read[stream, Real]
= 123.
#> Read[stream, Real]
= 4.
#> Quiet[Read[stream, Number]]
= $Failed

#> Close[stream];
#> stream = StringToStream["1.523E-19"]; Read[stream, Real]
= 1.523×10^-19
#> Close[stream];
#> stream = StringToStream["-1.523e19"]; Read[stream, Real]
= -1.523×10^19
#> Close[stream];
#> stream = StringToStream["3*^10"]; Read[stream, Real]
= 3.×10^10
#> Close[stream];
#> stream = StringToStream["3.*^10"]; Read[stream, Real]
= 3.×10^10
#> Close[stream];

## Expression
#> stream = StringToStream["x + y Sin[z]"]; Read[stream, Expression]
= x + y Sin[z]
#> Close[stream];
## #> stream = Quiet[StringToStream["Sin[1 123"]; Read[stream, Expression]]
## = $Failed

## HoldExpression:
>> stream = StringToStream["2+2\\n2+3"];
Expand All @@ -930,36 +817,24 @@ class Read(Builtin):

>> Read[stream, Expression]
= 5
>> Close[stream];
#> Close[stream];

Reading a comment however will return the empty list:
>> stream = StringToStream["(* ::Package:: *)"];

>> Read[stream, Hold[Expression]]
= {}

>> Close[stream];
#> Close[stream];

## Multiple types
>> stream = StringToStream["123 abc"];
>> Read[stream, {Number, Word}]
= {123, abc}
#> Read[stream, {Number, Word}]
>> Read[stream, {Number, Word}]
= EndOfFile
#> lose[stream];

#> stream = StringToStream["123 abc"];
#> Quiet[Read[stream, {Word, Number}]]
= $Failed
#> Close[stream];

#> stream = StringToStream["123 123"]; Read[stream, {Real, Number}]
= {123., 123}
#> Close[stream];

#> Quiet[Read[stream, {Real}]]
= Read[InputStream[String, ...], {Real}]

Multiple lines:
>> stream = StringToStream["\\"Tengo una\\nvaca lechera.\\""]; Read[stream]
= Tengo una
Expand Down Expand Up @@ -1232,15 +1107,7 @@ class ReadList(Read):
= {abc123}
>> InputForm[%]
= {"abc123"}

#> ReadList[stream, "Invalid"]
: Invalid is not a valid format specification.
= ReadList[..., Invalid]
#> Close[stream];


#> ReadList[StringToStream["a 1 b 2"], {Word, Number}, 1]
= {{a, 1}}
"""

# TODO
Expand Down Expand Up @@ -1398,10 +1265,6 @@ class SetStreamPosition(Builtin):
>> Read[stream, Word]
= is

#> SetStreamPosition[stream, -5]
: Invalid I/O Seek.
= 10

>> SetStreamPosition[stream, Infinity]
= 16
"""
Expand Down Expand Up @@ -1490,7 +1353,7 @@ class Skip(Read):
>> Skip[stream, Word, 2]
>> Read[stream, Word]
= d
#> Skip[stream, Word]
>> Skip[stream, Word]
= EndOfFile
#> Close[stream];
"""
Expand Down Expand Up @@ -1555,15 +1418,15 @@ class Find(Read):
= in manuscript, leads me to expect that the element uranium may be turned into
>> Find[stream, "uranium"]
= become possible to set up a nuclear chain reaction in a large mass of uranium,
>> Close[stream]
#> Close[stream]
= ...

>> stream = OpenRead["ExampleData/EinsteinSzilLetter.txt", CharacterEncoding->"UTF8"];
>> Find[stream, {"energy", "power"} ]
= a new and important source of energy in the immediate future. Certain aspects
>> Find[stream, {"energy", "power"} ]
= by which vast amounts of power and large quantities of new radium-like
>> Close[stream]
#> Close[stream]
= ...
"""

Expand Down Expand Up @@ -1649,14 +1512,8 @@ class StringToStream(Builtin):
>> strm = StringToStream["abc 123"]
= InputStream[String, ...]

#> Read[strm, Word]
= abc

#> Read[strm, Number]
= 123

#> Close[strm]
= String
The stream must be closed after using it, to release the resource:
>> Close[strm];
"""

summary_text = "open an input stream for reading from a string"
Expand Down Expand Up @@ -1685,14 +1542,6 @@ class Streams(Builtin):

>> Streams["stdout"]
= ...

#> OpenWrite[]
= ...
#> Streams[%[[1]]]
= {OutputStream[...]}

#> Streams["some_nonexistent_name"]
= {}
"""

summary_text = "list currently open streams"
Expand Down Expand Up @@ -1764,11 +1613,12 @@ class Write(Builtin):
= ...
>> Write[stream, 10 x + 15 y ^ 2]
>> Write[stream, 3 Sin[z]]
The stream must be closed in order to use the file again:
>> Close[stream];
>> stream = OpenRead[%];
>> ReadList[stream]
= {10 x + 15 y ^ 2, 3 Sin[z]}
#> DeleteFile[Close[stream]];
>> DeleteFile[Close[stream]];
"""

summary_text = "write a sequence of expressions to a stream, ending the output with a newline (line feed)"
Expand Down Expand Up @@ -1817,37 +1667,15 @@ class WriteString(Builtin):
>> FilePrint[%]
| This is a test 1This is also a test 2

#> DeleteFile[pathname];
>> DeleteFile[pathname];
>> stream = OpenWrite[];
>> WriteString[stream, "This is a test 1", "This is also a test 2"]
>> pathname = Close[stream]
= ...
>> FilePrint[%]
| This is a test 1This is also a test 2

#> DeleteFile[pathname];
#> stream = OpenWrite[];
#> WriteString[stream, 100, 1 + x + y, Sin[x + y]]
#> pathname = Close[stream]
= ...
#> FilePrint[%]
| 1001 + x + ySin[x + y]

#> DeleteFile[pathname];
#> stream = OpenWrite[];
#> WriteString[stream]
#> pathame = Close[stream]
= ...
#> FilePrint[%]

#> WriteString[%%, abc]
#> Streams[%%%][[1]]
= ...
#> pathname = Close[%];
#> FilePrint[%]
| abc
#> DeleteFile[pathname];
#> Clear[pathname];
>> DeleteFile[pathname];


If stream is the string "stdout" or "stderr", writes to the system standard output/ standard error channel:
Expand Down
Loading
Loading