Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refractor derivatives naming convention #159

Merged
merged 22 commits into from
Jun 27, 2024

Conversation

sandrinebedard
Copy link
Member

@sandrinebedard sandrinebedard commented Feb 20, 2024

Description

This PR adresses #156 and #157 mainly and cleans up the datasets. Th main changes include:

  1. Adds preprocessed images with entity space-other in folder derivatives/data_preprocessed: (Add processed images in the derivatives folder #156, Some subjects have different image and label dimensions and resolution #144 )
    • T1w: Reoriented to RPI and resampled to 1x1x1 mm
    • T2w: Reoriented to RPI and resampled to 0.8x0.8x0.8 mm
    • T2star: Reoriented to RPI and root mean square across 4th dimension (if exists)
    • MTS: Reoriented to RPI
    • DWI: Mean image across directions after motion correction
  2. Add labels_seftseg_bin in the derivatives
  3. Add warp T2w disc labels for MTS, T2star and DWI data in derivatives/labels
  4. Renaming of derivatives
    • sub-amu01_T2star_seg-manual.nii.gz --> sub-amu01_space-other_T2star_label-SC_seg.nii.gz (Rename seg-manual to label-SC_seg-manual #141)
    • sub-amu01_T2w_labels-disc-manual.nii.gz --> sub-amu01_T2w_label-discs_dlabel.nii.gz
    • sub-amu01_T1w_softseg.nii.gz --> sub-amu01_space-other_T1w_label-SC_softseg.nii.gz
    • sub-amu01_T1w_softseg_bin.nii.gz --> sub-amu01_space-other_T1w_desc-softseg_label-SC_seg.nii.gz
    • sub-amu01_T1w_csfseg-manual.nii.gz --> sub-amu01_space-other_T1w_label-CSF_seg.nii.gz
    • For discs warped from T2w: sub-amu01_T2w_label-discs_desc-warp_dlabel.nii.gz
    • sub-amu01_T2w_RPI_r_labels-manual.nii.gz --> sub-amu01_space-other_T2w_label-vertebrae_desc-C2C5_dlabel.nii.gz (Two T1w labels for some subjects #147 )
    • sub-amu01_T2star_gmseg.nii.gz --> sub-amu01_space-other_T2star_label-GM_seg.nii.gz
  5. Remove duplicated files as described in Labels with missing suffixes #151
  6. Update json sidecar with updated entities
  7. Added spinal canal segmentations

Remaining tasks

  • UPdate GM segmentation
  • Add in readme link of script for SC canal seg
  • Add README for data_preprocessed with link to the script
  • Add README for labels_softseg_bin with link to the script
  • Update contrast-agnostic script and spine-generic analysis scripts with updated names and create a release

Related Issues

Fixes #156
fixes #157
fixes #147
Fixes #144
Fixes #153
Fixes #141
Fixes #151

@sandrinebedard sandrinebedard marked this pull request as draft February 20, 2024 16:22
@valosekj
Copy link
Contributor

valosekj commented Mar 7, 2024

Cross-referring an older issue: neuropoly/idea-projects#1

sandrinebedard and others added 4 commits March 20, 2024 15:16
* add updated labels softseg bin

* add updated softseg labels

* add preprocessed images

* rm excluded participants

* update with script and release tag
@sandrinebedard sandrinebedard marked this pull request as ready for review April 3, 2024 15:40
@sandrinebedard
Copy link
Member Author

Th PR is now ready for review!

@valosekj
Copy link
Contributor

valosekj commented Apr 5, 2024

Thanks for this massive PR @sandrinebedard! 💯

ad 1)
I randomly selected one subject (sub-balgrist01) and checked its files under data_preprocessed (derivatives/data_preprocessed/sub-balgrist01). Images look good -- they are reoriented to RPI and resampled.
I just noticed that flip-1_mt-off_MTS.nii.gz is missing under derivatives/data_preprocessed:

valosek@macbook-pro:~/data/data-multi-subject$ ls -1 sub-balgrist01/anat/sub-balgrist01_flip-*.nii.gz
sub-balgrist01/anat/sub-balgrist01_flip-1_mt-off_MTS.nii.gz
sub-balgrist01/anat/sub-balgrist01_flip-1_mt-on_MTS.nii.gz
sub-balgrist01/anat/sub-balgrist01_flip-2_mt-off_MTS.nii.gz

valosek@macbook-pro:~/data/data-multi-subject$ ls -1 derivatives/data_preprocessed/sub-balgrist01/anat/sub-balgrist01_flip-*.nii.gz
derivatives/data_preprocessed/sub-balgrist01/anat/sub-balgrist01_flip-1_mt-on_space-other_MTS.nii.gz
derivatives/data_preprocessed/sub-balgrist01/anat/sub-balgrist01_flip-2_mt-off_space-other_MTS.nii.gz

ad 2,3,4,5,6)
looks good!

ad 7)
I found only 10 CSF canal segmetations. Just double-checking that this is correct?

valosek@macbook-pro:~/data/data-multi-subject/derivatives$ find . -name "*CSF*nii.gz" | wc -l
      10

@sandrinebedard
Copy link
Member Author

Thank you for reviewing @valosekj !

I found only 10 CSF canal segmetations. Just double-checking that this is correct?

valosek@macbook-pro:~/data/data-multi-subject/derivatives$ find . -name "*CSF*nii.gz" | wc -l
      10

My bad it should've been written SC canal in the PR description! You should find label-canal_seg

(base) sebeda@joplin:~/datasets/data-multi-subject/derivatives/labels$ find . -name "*canal*nii.gz" | wc -l
10

Regarding flip-1_mt-off_MTS.nii.gz are not used in contrats-agnostic and do not have any labels related to them, that is why this contrast is not there. Should we add them still?

@valosekj valosekj mentioned this pull request Apr 9, 2024
1 task
@sandrinebedard
Copy link
Member Author

Regarding flip-1_mt-off_MTS.nii.gz are not used in contrats-agnostic and do not have any labels related to them, that is why this contrast is not there. Should we add them still?

We decided not to include, since there is not associated labels.

@mguaypaq I think we would be ready to merge!

@mguaypaq
Copy link
Member

Hmm, in addition to the current problems with the Dataset Validator, it looks like it's now running out of disk space when trying to git annex get the dataset. I'll manually run as much of the validation steps as I can, then merge this PR anyway so that we don't have to wait too long.

@mguaypaq
Copy link
Member

mguaypaq commented Apr 10, 2024

Ok:

  • git annex get works fine.
  • bids-validator . works fine (only irrelevant warnings).
  • .github/workflows/lint-tsv participants.tsv works fine.
  • I didn't run sg_params_checker or sg_check_data_consistency.

I'm going to merge this.

@mguaypaq
Copy link
Member

Ah, it looks like I can't bypass branch protections. I guess I'll fix the validator before merging this PR, in that case.

@mguaypaq mguaypaq merged commit af1f9e2 into master Jun 27, 2024
1 check failed
@mguaypaq mguaypaq deleted the sb/156-add-preprocessed-images branch June 27, 2024 20:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants