URL objects are not populated with their parent directory information

[mehdisadeghi]

I use saga.filesystem.Directory objects (with their corresponding sessions) to access remote directory contents. I realized that when I use list() method on a Directory instance the produced saga.url.Url objects do not have their parents host, schema and similar information. Here is how to reproduce:

/tmp/some_directory
    -some_file

dir = saga.filesystem.Directory('/tmp/some_directory')
dir.get_url().get_path() -> /tmp
dir.get_url().get_scheme() -> 'file' or 'sftp' etc
dir.get_url().get_host() -> localhost

url1 = dir.list()[0]
url1.get_path() -> 'some_file'
url1.get_scheme() -> ''
url1.get_host() -> ''

If you confirm this behavior is not desired I will look at the code to fix it.

[andre-merzky]

Hi Mehdi,

this is actually the expected behavior -- this behaves similar to the POSIX ls shell command, which, when applied to a directory like ls /etc/ also returns only the filenames, but not the path. So the returned URLs also only contain the filename, and the application needs to construct the full URL as needed:

(ve)merzky@cameo:~/saga/radical.saga (devel %) $ cat t.py 
#!/usr/bin/env python

import saga
import sys

d1 = saga.filesystem.Directory('/tmp/some_directory')
print d1.url
f1 = d1.list()[0]
print f1
f2 = saga.Url(str(d1.url) + '/' + str(f1))
print f2

(ve)merzky@cameo:~/saga/radical.saga (devel %) $ python t.py 
file://localhost/tmp/some_directory
some_file
file://localhost/tmp/some_directory/some_file

(The SAGA spec is actually not very explicit in this respect -- it would technically allow to return fully qualified URLs...)

[mehdisadeghi]

Hi Andre,

Although it totally makes sense to return the file name and not the full path, there is still one thing which is not clear to me.

The saga.url.Url class defines get_scheme, get_host, get_port, etc., which are empty when I get a Url object from a Directory instance. However, when I construct a `Url' object, while passing in a fully qualified URL, the above mentioned methods will have values, according to the given URL.

From a developer point of view, I was expecting a Url object to know where it comes from and to be able to reconstruct a fully qualified Url on demand. An example use case would be passing such an object to saga.filesystem.Directory.open_dir method, while assuming the right URL would be used. If this does not fit into scopes of saga-python and saga specs, let us close this issue safely.

[andre-merzky]

From a developer point of view, I was expecting a Url object to know where it comes from and to be able to reconstruct a fully qualified Url on demand.

I think I see what you say -- but our URL class is not clever enough for that: it has no notion of the context it was constructed in. But:

An example use case would be passing such an object to saga.filesystem.Directory.open_dir method, while assuming the right URL would be used.

I agree that this should actually work:

import saga

d1 = saga.filesystem.Directory("file://localhost/tmp/foo")
u2 = d1.list()[0]  # == 'bar'
d2 = d1.open_dir(u2)

that seems broken indeed, as the bar gets normalized to a full URL, which then translates to file://localhost/bar, which is not open to interpretation in the context of d1 anymore...

[andre-merzky]

#457 is related...

[vivek-bala]

Hey Andre, is this issue address in any commit or PR?

[andre-merzky]

No, alas this is not addressed as of yet.