Support for reading in edgelists from a file handle (instead of a file)

[davidjurgens]

Some large edgelists need to be stored on disk in a compressed format. However, zenlib requires pass the edgelist the file name, which prevents reading in these kinds of edge lists without first decompressing them. It would be nice to be able to read them in from an open file handle instead of the handle. This lets the caller perform any necessary file decompression first and then having zenlib read them as normal. For example:

f = os.path.join("mypath", "myfile.elist")
G = zen.edgelist.read(f)

would become

f = os.path.join("mypath", "myfile.elist.gz")
fh = gzip.open(f)
G = zen.edgelist.read(fh)

[druths]

Makes sense to me. What if we made all non-strings treated like file handles?

Sent from my iPhone

On Sep 29, 2014, at 9:00 AM, David Jurgens [email protected] wrote:

Some large edgelists need to be stored on disk in a compressed format. However, zenlib requires pass the edgelist the file name, which prevents reading in these kinds of edge lists without first decompressing them. It would be nice to be able to read them in from an open file handle instead of the handle. This lets the caller perform any necessary file decompression first and then having zenlib read them as normal. For example:

f = os.path.join("mypath", "myfile.elist")
G = zen.edgelist.read(f)

would become

f = os.path.join("mypath", "myfile.elist.gz")
fh = gzip.open(f)
G = zen.edgelist.read(fh)

—
Reply to this email directly or view it on GitHub.

[davidjurgens]

Treating all non-strongs as handles makes sense to me. I'm not sure what
else could reasonably be passed to the method.

On Mon, Sep 29, 2014 at 1:19 PM, druths [email protected] wrote:

Makes sense to me. What if we made all non-strings treated like file
handles?

Sent from my iPhone

On Sep 29, 2014, at 9:00 AM, David Jurgens [email protected]
wrote:

Some large edgelists need to be stored on disk in a compressed format.
However, zenlib requires pass the edgelist the file name, which prevents
reading in these kinds of edge lists without first decompressing them. It
would be nice to be able to read them in from an open file handle instead
of the handle. This lets the caller perform any necessary file
decompression first and then having zenlib read them as normal. For
example:

f = os.path.join("mypath", "myfile.elist")
G = zen.edgelist.read(f)

would become

f = os.path.join("mypath", "myfile.elist.gz")
fh = gzip.open(f)
G = zen.edgelist.read(fh)

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#22 (comment)
.

[druths]

Would you be interested in forking it and making these changes? While you’re at it, you could update all the io.read methods (since they’re supposed to all implement the same protocol).

On Sep 29, 2014, at 11:03 AM, David Jurgens [email protected] wrote:

Treating all non-strongs as handles makes sense to me. I'm not sure what
else could reasonably be passed to the method.

On Mon, Sep 29, 2014 at 1:19 PM, druths [email protected] wrote:

Makes sense to me. What if we made all non-strings treated like file
handles?

Sent from my iPhone

On Sep 29, 2014, at 9:00 AM, David Jurgens [email protected]
wrote:

Some large edgelists need to be stored on disk in a compressed format.
However, zenlib requires pass the edgelist the file name, which prevents
reading in these kinds of edge lists without first decompressing them. It
would be nice to be able to read them in from an open file handle instead
of the handle. This lets the caller perform any necessary file
decompression first and then having zenlib read them as normal. For
example:

f = os.path.join("mypath", "myfile.elist")
G = zen.edgelist.read(f)

would become

f = os.path.join("mypath", "myfile.elist.gz")
fh = gzip.open(f)
G = zen.edgelist.read(fh)

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#22 (comment)
.

—
Reply to this email directly or view it on GitHub.