Support for globbing patterns for stim_file in events.tsv

[Remi-Gau]

Context

• this concerns a dataset containing data and stimuli

• stimuli are movies where each is stored as a .wav file + a series images (one per movie frame) - for practical reasons in terms of stimuli presentation.

tree tmp/stimuli | head

tmp/stimuli
├── S1fa01.jpg
├── S1fa02.jpg
├── S1fa03.jpg
├── S1fa04.jpg
├── S1fa05.jpg
├── S1fa06.jpg
├── S1fa07.jpg
├── S1fa08.jpg
├── S1fa09.jpg
...

• during one "visual" event all the movie frames as presented
• one event "visual" is logged in the events.tsv with stim_name with a stim_name.*.jpg to "glob" for all the files that match this stimulus name.

cat tmp/sub-ctrl666/ses-666/beh/sub-ctrl666_ses-666_task-LipSpeechMVPAvis_run-001_events.tsv

onset   duration        trial_type      stim_file       block   modality        repetition      actor   consonant       vowel   target  key_name
0.529170        1.271757        S2pi    S2pi*.jpg       2.000000        vis     1.000000        S2      p       i       0.000000        n/a
4.797281        1.153619        S1pa    S1fa*.jpg       2.000000        vis     1.000000        S1      f       a       0.000000        n/a
8.946907        1.202744        S2pa    S2pa*.jpg       2.000000        vis     1.000000        S2      p       a       1.000000        n/a
13.163524       1.236411        S1pi    S1pi*.jpg       2.000000        vis     1.000000        S1      p       i       0.000000        n/a
17.397068       1.269708        S2pa    S2fa*.jpg       2.000000        vis     1.000000        S2      f       a       0.000000        n/a
21.664094       1.273934        S2pi    S2pi*.jpg       2.000000        vis     1.000000        S2      p       i       1.000000        n/a

Error

Validator complains that

• files that are in the stimuli folder are not listed in any events files
• stim_file listed in the events files do not exist in the stimuli folder

bids-validator tmp  
   
[email protected]

bids-specification@disable

(node:11657) Warning: Closing directory handle on garbage collection
(Use `node --trace-warnings ...` to show where the warning was created)
        1: [ERR] A stimulus file was declared but not found in the dataset. (code: 52 - STIMULUS_FILE_MISSING)
                ./sub-ctrl666/ses-666/beh/sub-ctrl666_ses-666_task-LipSpeechMVPAvis_run-001_events.tsv
                        @ line: 2 character: 3
                        Evidence: S2pi*.jpg
                ./sub-ctrl666/ses-666/beh/sub-ctrl666_ses-666_task-LipSpeechMVPAvis_run-001_events.tsv
                        @ line: 3 character: 3
                        Evidence: S1fa*.jpg
                ./sub-ctrl666/ses-666/beh/sub-ctrl666_ses-666_task-LipSpeechMVPAvis_run-001_events.tsv
                        @ line: 4 character: 3
                        Evidence: S2pa*.jpg
                ./sub-ctrl666/ses-666/beh/sub-ctrl666_ses-666_task-LipSpeechMVPAvis_run-001_events.tsv
                        @ line: 5 character: 3
                        Evidence: S1pi*.jpg
                ./sub-ctrl666/ses-666/beh/sub-ctrl666_ses-666_task-LipSpeechMVPAvis_run-001_events.tsv
                        @ line: 6 character: 3
                        Evidence: S2fa*.jpg

        Please visit https://neurostars.org/search?q=STIMULUS_FILE_MISSING for existing conversations about this issue.

        1: [WARN] There are files in the /stimuli directory that are not utilized in any _events.tsv file. (code: 77 - UNUSED_STIMULUS)
                ... and 1368 more files having this issue (Use --verbose to see them all).

What I have tried

Out of curiosity I tried to see if storing all the frames of a given movie in a folder and referencing this folder in the stim_file column would work: it did not.

What I do not want to do

I would very much prefer to avoid logging the presentation of every frame in the events.tsv:

• this would "clutter" the events.tsv more than necessary
• because we would not do that if we had a regular movie file (.mp4, .avi...)
• this will be used for fMRI where I doubt we need that level of temporal granularity

tagging @VisLab who might have some suggestions as this may somewhat be related to storing of stimuli and event tagging.

[effigies]

Would it make more sense to concatenate your images into a movie with an appropriate framerate? Something like:

ffmpeg -r $FRAMERATE_IN_HZ -pattern_type glob -i "$PREFIX*.jpg" -vcodec mjpeg -out $PREFIX.mp4

Then s2pi*.jpg becomes s2pi.mp4.

[Remi-Gau]

Depends if we want to make sure that the stimuli shipped with the data are EXACTLY (also in terms of file format) those used during the experiment (in this case I don't really have this option).

If we want to ship something that "corresponds" to the stimuli presented but does not match the format used during the fMRI presentation, yup I could do that.

I suspect there are pros and cons to both but I could defo ship both formats in the stimuli folder.

[effigies]

This is a spec question. If it's permitted, we need a specific syntax, and then we can update the schema/validator to handle it.

[VisLab]

Proposed BEP 044 is aimed to address this issue more generally. See also: bids-standard/bids-specification#153.

[neuromechanist]

Depends if we want to make sure that the stimuli shipped with the data are EXACTLY (also in terms of file format) those used during the experiment (in this case I don't really have this option).

In general, we'd like to represent exactly what happened during an experiment or make it as closely as possible 😅.

In this example, representing events with glob patterns seems ambiguous because additional information is needed in the events.tsv (with its JSON sidecar) to make the events understandable (say to an LLM) beyond doubt.

So, having a movie might be a more transparent solution. BEP044 is provisioning one or multiple events.tsv associated with movies (and any stimulus files, actually) to provide additional annotation. In this case, the annotation would be for every frame pointing to a distinct image.

[effigies]

I'm going to close this in favor of BEP44.