Skip to content

Latest commit

 

History

History
253 lines (169 loc) · 21.5 KB

README.md

File metadata and controls

253 lines (169 loc) · 21.5 KB
Raw

BEN

A generalized Brain Extraction Net for multimodal MRI data from rodents, nonhuman primates, and humans

Paper | Feature | Replicate demo | MRI data release | Pretrained weight | Interface | Contributing to BEN | Documentation | Contents | Video tutorials

See also: MouseGAN++ logo | Github project link

Z. Yu, X. Han, S. Zhang, J. Feng, T. Peng and X. -Y. Zhang, "MouseGAN++: Unsupervised Disentanglement and Contrastive Representation for Multiple MRI Modalities Synthesis and Structural Segmentation of Mouse Brain," in IEEE Transactions on Medical Imaging, 2022, doi: 10.1109/TMI.2022.3225528.

What's new in BEN (v0.2) ?

We release BEN (version 0.2) during November based on the reviewers' suggestions and our experiences accumulated in clinical practice.

(Old version is moved to 'doc' branch)

New features Location
Add orientation detection (Note: if you want to run MR scans on the original orientation, don't set "-check" parameter in commands) utils folder
Add utils functions (some visualization and postprocessing functions) utils folder
Add Human-T1WI-HCP (baby) pretrained weight dataset_release folder
Optimize BEN pipeline. BEN_DA.py, BEN_infer.py
Visual (segmentation quality) and volumetric (brain volume) reports in automatically generated HTML web page. utils folder
Video tutorials Video tutorials

Near future plans in BEN v0.3

Please refer to list.

Behind the work

Coincidentally, the motivation of BEN's training strategy is somewhat consistent with Cellpose v2. BEN tries to quickly develop a customized model specific to each user application with the help of the AdaBN module.

Overview

🚀 Quick start to use BEN or replicate our experiments in 5 minutes!

Feature

Feature Description Colab link
Transferability & flexibility BEN outperforms traditional SOTA methods and advantageously adapts to datasets from diverse domains across multiple species [1], modalities [2], and MR scanners with different field strengths [3]. [1] Open In Colab
[2] Open In Colab
[3] Open In Colab
Quality assessment Unlike traditional toolboxes, which mainly rely on manual inspection to assess the brain extraction quality, BEN incorporates a quality assessment module to automatically evaluate its brain extraction performance. Open In Colab
Speed 🚀 Inference: Less than 1 sec/scan;
Adaptation to a new imaging center/application: About 10 min for deployment.		 -

Replicate demo

Name Description Related Fig.
in paper		 Snapshot Colab link
1. BEN workflow & architecture BEN renovates the brain extraction workflow to adapt to multiple species, modalities and platforms Fig. 1 & 2 Snapshot on Win This repository
2. ✨ Cross species Deploying BEN to new species requires minimal or even no labels. Fig. 3 Snapshot on Win Cross species Open In Colab
3. Cross modalities / MR scanners Deploying BEN to new modalities/MR platforms requires minimal or even no labels. Fig. S1 & S2 Snapshot on Win Cross modalities Open In Colab
Cross MR scanners Open In Colab
4. Compare with other toolbox BEN outperforms traditional SOTA methods and advantageously adapts to datasets from various domains across multiple species, modalities, and field strengths. Fig. 4 Snapshot on Win Open In Colab
5. Uncertainty & inter-rater variations Compared to traditional toolboxes, BEN provides a measure of uncertainty that potentially reflects rater disagreement Fig. 5 Snapshot on Win Disagree map Open In Colab
Uncertainty Open In Colab
6. ✨ Volumetric quantification for longitudinal studies BEN improves the accuracy of atlas registration and benefits brain volumetric quantification compared with using other toolbox. Fig. 6 & 7 Snapshot on Win Registration comparision Open In Colab
Volumetric quantification Open In Colab

MRI data release | pretrained weight

The details can be found in this folder.

Quick Start Contents

Visit our documentation or video tutorials for installation, tutorials and more.

Installation

An Nvidia GPU is needed for faster inference (less than 1 sec/scan on 1080ti gpu).

Requirements:

  • tensorflow-gpu == 1.15.4
  • Keras == 2.2.4
  • numpy == 1.16
  • SimpleITK == 2.0
  • opencv-python == 4.1
  • scikit-image == 0.16.2

Install dependencies:

git clone https://github.com/yu02019/BEN.git
cd BEN
pip install -r requirement.txt

The target domain data folder looks like this: (Download data from this repository/Colab or put your data here.)

folder_tree

Quick Start

  • All the undermentioned results can be repeated via our tutorial Notebook.
  • New weight will be saved independently for further customized application.

Cross modalities

Results:

  1. Modality: T2WI -> EPI

  2. For this exemplar domain adaptation (DA) task, No label is used (zero-shot).

  3. From top row to the third row: Raw image, Baseline result, BEN's result.

    Alt text

Cross MR scanner with different field strengths

  1. MR scanner with different field strengths: 11.7 T -> 7 T

  2. For this exemplar domain adaptation (DA) task, No label is used (zero-shot).

  3. From top row to the third row: Raw image, Baseline result, BEN's result.

Cross species

  1. Species: Mouse -> Rat

  2. For this exemplar domain adaptation (DA) task, only ONE label is used.

  3. The segmentation results are shown in red, the ground truth are shown in orange.

  4. From top row to the fifth row: Raw image, Zero-shot (0 label used), finetune (1 label used), BEN's result (1 label used), Ground truth.

  5. (Optional) Just do some simple postprocessing here, e.g., only save the top-K largest connected regions.

  6. Compared with other methods, it further shows BEN's advantages

Try your data

Feel free to try your data or deploy BEN to your preprocessing pipeline. Details can be found in notebook and video tutorials. Pretrained weights can be found in dataset_release.

# Update BEN (domain adaptation)
python BEN_DA.py -t train_folder -l label_folder -r raw_image_folder -weight pretrained_weight_path -prefix new_model_name -check check_orientation
# Run inference
python BEN_infer.py -i input_folder -o output_folder -weight model_weight_path -check check_orientation

Video tutorials

To further validate BEN’s generalization, we have evaluated BEN on two new external public ex-vivo MRI datasets (rTg4510 mouse: 25 ex-vivo scans, and C57BL/6 mouse: 15 ex-vivo scans). When only one label is used for BEN adaptation/retraining, impressive performance is achieved on both datasets, despite the fact that BEN was originally designed for in-vivo MRI data.

Dataset Used label Description Automatically generated reports Video link
rTg4510 mouse 1 Ex-vivo scans with obvious distortion and different orientations. youtube
C57BL/6 mouse 1 Ex-vivo scans. There is no obvious gap between the brain and the skull borderlines, making the task difficult. youtube
C57BL/6 mouse 0
(zero-shot)		 Ex-vivo scans. The domain gap exists in ex-vivo MRI data and in-vivo images in our training images could be so large that it compromises the performance. In this case, we suggest users add several labels to update BEN. youtube

Pretrained weight used in tutorials can download from Google Drive

Resources

Interface for neruoimaging toolboxes

The usages and details can be found in this folder.

Name Link
AFNI afni.nimh.nih.gov/afni
ANTs stnava.github.io/ANTs/
FSL fsl.fmrib.ox.ac.uk/fsl/fslwiki
FreeSurfer freesurfer.net
SPM fil.ion.ucl.ac.uk/spm
Nipype pypi.org/project/nipype/

Data release / Pretrained weight / Contributing to BEN

The details can be found in this folder.

Citation

If you find our work / datasets / pretrained models useful for your research, please consider citing:

@article{yu2022generalizable,
  title={A generalizable brain extraction net (BEN) for multimodal MRI data from rodents, nonhuman primates, and humans},
  author={Yu, Ziqi and Han, Xiaoyang and Xu, Wenjing and Zhang, Jie and Marr, Carsten and Shen, Dinggang and Peng, Tingying and Zhang, Xiao-Yong and Feng, Jianfeng},
  journal={Elife},
  volume={11},
  pages={e81217},
  year={2022},
  publisher={eLife Sciences Publications Limited}
}
@dataset{yu_ziqi_2022_6844489,
  author       = {Yu Ziqi and
                  Xu Wenjing and
                  Zhang Xiao-Yong},
  title        = {{A longitudinal MRI dataset of young adult C57BL6J 
                   mouse brain}},
  month        = jul,
  year         = 2022,
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.6844489},
  url          = {https://doi.org/10.5281/zenodo.6844489}
}

Disclaimer: This toolkit is only for research purpose.