using the BIDS-schema to simplify creating the dataset layout

[Remi-Gau]

I want the data structures to be formally described (in json/yaml) and not in the MATLAB code, that representation should be shared with pybids (MATLAB has support for JSON - see here

This would rely on using the BIDS schema that exists as a set of yml files for now and should therefore be converted to something that matlab can handle like JSON.

Questions

• where to do the conversion and hosting of the JSON schema?

---

See other comments in the "wish-list" issue #60

#60 (comment)
#60 (comment)
#60 (comment)
#60 (comment)
#60 (comment)

[pvdemael]

matbids already has the BIDS scheme apart from the implementation.

It is a one-on-one port from the (older) pybids implementation and uses json files.

A suggestion is to maybe use the bids-validator json files.

[Remi-Gau]

For the record, here is list of BIDS schemas or 'pseudo-schemas':

• official YML based schema in the specification repo with some doc HERE

• regex based rules of the BIDS-validator are in several JSON files here

• pybids has some rules encoded in several json files HERE to help with folder and filename parsing.

• matbids has equivalent ones HERE

From my understanding and seeing that other tools will progressively try to rely only on the BIDS official schema, I think it makes sense to alreay start moving in that direction.

Question:

• who (which repo) is in charge of converting and hosting a json version of the schema?
• do we want to have an "interface" layer that is just in charge of reading the schema and passes its output to bids.layout or other functions? (Just thinking that this could be a way to keep bids.layout insulated from any changes happening at the schema level (and could also help with testing).

[Remi-Gau]

On converting the schema.

This can be done fairly simply with the python package ruamel

Virtual env (because everything is better with a virtual env)

virtualenv -p python3 convert_schema
source  convert_schema/bin/activate
pip install ruamel.yaml

Example to convert the anat.yml from the specs repo.

from ruamel.yaml import YAML
import json

in_file = 'src/schema/datatypes/anat.yaml'
out_file = 'anat.json'

yaml = YAML(typ='safe')
with open(in_file) as fpi:
    data = yaml.load(fpi)
with open(out_file, 'w') as fpo:
    json.dump(data, fpo, indent=2)

Gives the following content for anat.json.

[
  {
    "suffixes": [
      "T1w",
      "T2w",
      "T1rho",
      "T1map",
      "T2map",
      "T2star",
      "FLAIR",
      "FLASH",
      "PD",
      "PDmap",
      "PDT2",
      "inplaneT1",
      "inplaneT2",
      "angio"
    ],
    "extensions": [
      ".nii.gz",
      ".nii",
      ".json"
    ],
    "entities": {
      "sub": "required",
      "ses": "optional",
      "run": "optional",
      "acq": "optional",
      "ce": "optional",
      "rec": "optional"
    }
  },
  {
    "suffixes": [
      "defacemask"
    ],
    "extensions": [
      ".nii.gz",
      ".nii",
      ".json"
    ],
    "entities": {
      "sub": "required",
      "ses": "optional",
      "run": "optional",
      "acq": "optional",
      "ce": "optional",
      "rec": "optional",
      "mod": "optional"
    }
  }
]

[Remi-Gau]

FYI: there is a kanban on the specs repo about anything scheme related.
https://github.com/bids-standard/bids-specification/projects/4

This includes a PR to cover derivatives in the schema.
bids-standard/bids-specification#645

[Remi-Gau]

One issue that is not addressed by this though is how do we deal with things that are in the BIDS-BEP pipeline but not yet in the schema.

For example I have to deal with MP2RAGE images that are in BEP001 but not yet in the schema.

Do we want to a sort of "fall back" solution when we are dealing with a dataset that is not "vanilla BIDS".

Should we then rely on a JSON file to define "patterns" to deal with those cases a bit like pybids does at the moment for the regular stuff.

[Remi-Gau]

fixed by #124