-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
meta-extract: Catch non-existing paths earlier #354
Comments
adswa
added a commit
to adswa/datalad-metalad
that referenced
this issue
Mar 2, 2023
…lidation This is a demonstration how one existing command could adopt datalad-next's parameter constraint validation. It changes the baseclass to next's ValidatedInterface, and defines a validator with relevant parameter constraints: Specifically, the constraints are: - The provided datasets exists, or a dataset can be derived from the curdir - The path points to an existing file (ref datalad#354) - The extractorname is a string - The extractorargs is a mapping of key-value pairs This makes a dedicated check whether a file exists obsolete, and it could replace the checks that check_dataset() does (provided an additional constraint option in EnsureDataset() that allows to check for valid dataset IDs - I've created an issue about this in datalad/datalad-next#272). This change would introduce a dependency to datalad-next, and as parts of this PR were only tested with yet unreleased branches of datalad-next, it will not work right now unless you're on the right development version of datalad-next.
adswa
added a commit
to adswa/datalad-metalad
that referenced
this issue
Mar 2, 2023
…lidation This is a demonstration how one existing command could adopt datalad-next's parameter constraint validation. It changes the baseclass to next's ValidatedInterface, and defines a validator with relevant parameter constraints: Specifically, the constraints are: - The provided datasets exists, or a dataset can be derived from the curdir - The path points to an existing file (ref datalad#354) - The extractorname is a string - The extractorargs is a mapping of key-value pairs This makes a dedicated check whether a file exists obsolete, and it could replace the checks that check_dataset() does (provided an additional constraint option in EnsureDataset() that allows to check for valid dataset IDs - I've created an issue about this in datalad/datalad-next#272). This change would introduce a dependency to datalad-next, and as parts of this PR were only tested with yet unreleased branches of datalad-next, it will not work right now unless you're on the right development version of datalad-next.
adswa
added a commit
to adswa/datalad-metalad
that referenced
this issue
Mar 2, 2023
…lidation This is a demonstration how one existing command could adopt datalad-next's parameter constraint validation. It changes the baseclass to next's ValidatedInterface, and defines a validator with relevant parameter constraints: Specifically, the constraints are: - The provided datasets exists, or a dataset can be derived from the curdir - The path points to an existing file (ref datalad#354) - The extractorname is a string - The extractorargs is a mapping of key-value pairs This makes a dedicated check whether a file exists obsolete, and it could replace the checks that check_dataset() does (provided an additional constraint option in EnsureDataset() that allows to check for valid dataset IDs - I've created an issue about this in datalad/datalad-next#272). This change would introduce a dependency to datalad-next, and as parts of this PR were only tested with yet unreleased branches of datalad-next, it will not work right now unless you're on the right development version of datalad-next.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When I attempt a metadata extraction of a non-existing file with a relative path, the first thing that returns an error is a
get_file_info
complaining about a missing "dataset status for dataset":The logic below seems to be careful to not resolve paths:
datalad-metalad/datalad_metalad/extract.py
Lines 267 to 287 in fc20e5c
I think we should check early on if a path points to an existing file, and if not, return a proper error or impossible result early and informatively.
The text was updated successfully, but these errors were encountered: