DeepD3

We provide DeepD3, a framework for the detection of dendritic spines and dendrites.

With DeepD3, you are able to

• train a deep neural network for dendritic spine and dendrite segmentation
• use pre-trained DeepD3 networks for inference
• build 2D and 3D ROIs
• export results to your favourite biomedical image analysis platform
• use command-line or graphical user interfaces

How to install and run DeepD3

DeepD3 is written in Python. First, please download and install any Python-containing distribution, such as Anaconda. We recommend Python 3.7 and more recent version.

Then, installing DeepD3 is as easy as follows:

pip install deepd3

Now, you have access to almost all the DeepD3 functionalities.

If you want to use the DeepD3 Neural Network inference mode, please install Tensorflow using either of the following commands:

# With CPU support only
conda install tensorflow

# With additional GPU support
conda install tensorflow-gpu

Note: Tensorflow has changed a lot in the recent months, please check back with the most recent installation manual.

If you would like to access DeepD3-GUIs, use the following two shortcuts in your favorite shell:

# Opening the segmentation and ROI building GUI
deepd3-inference

# Opening the training utilities
deepd3-training

Model zoo

We provide a comprehensive training dataset on zenodo and the DeepD3 Website:

• DeepD3_8F.h5 - 8 base filters, original resolution
• DeepD3_16F.h5 - 16 base filters, original resolution
• DeepD3_32F.h5 - 32 base filters, original resolution
• DeepD3_8F_94nm.h5 - 8 base filters, resized to 94 nm xy resolution
• DeepD3_16F_94nm.h5 - 16 base filters, resized to 94 nm xy resolution
• DeepD3_32F_94nm.h5 - 32 base filters, resized to 94 nm xy resolution

Brief description:

• Full (32F) DeepD3 model trained on 94 nm (fixed) or a blend of resolutions (free)
• Medium (16F) DeepD3 model trained on 94 nm (fixed) or a blend of resolutions (free)
• Tiny (8F) DeepD3 mode trained on 94 nm (fixed) or a blend of resolutions (free)

Workflow

Train DeepD3 on your own dataset

Use deepd3-training to start the GUI for generating training sets.

For each of your training set, please provide

• The original stack as e.g. TIF files
• The spine annotations (binary labels) as TIF or MASK files (the latter from pipra)
• The dendrite annotations as SWC file (only tested for SWC-files generated by NeuTube)

Create training data

Click on the button "Create training data". For each of your stacks, import the stack, the spine annotation and the dendrite annotation file. If you dendrite annotation is a SWC file, it will create a 3D reconstruction of the SWC file, which will be stored for later use. If you reload the SWC, it will ask you if you want to keep the 3D reconstruction.

After importing all files, enter the metadata (resolution in x, y and z) and determine the region of interest using the bounding box and the sliders. Shortcuts are B for current plane is z begin and E for z end. You may enable or disable the cropping to the bounding box. If you are happy, save this region as d3data-file.

View training data

Click on the button "View training data" to re-visit any d3data files. You also are able to see and potentially manipulate the metadata associated the d3data file.

Arrange training data

For training, you need to create a d3set. This is an assembly of d3data files. Click on the button "Arrange training data". Then, simply load all relevant data using the "Add data to set" button and select appropriate d3data files. Clicking on "Create dataset" allows you to save your assembly as d3set file.

Actual training

We have prepared a Jupyter notebook in the folder examples. Follow the instructions to train your own deep neural network for DeepD3 use. For professionals, you also may utilize directly the files in model and training to allow highly individualized training. You only should ensure that your model allows arbitrary input and outputs two separate channels (dendrites and spines).

Inference

Open the inference mode using deepd3-inference. Load your stack of choice (we currently support TIF stacks) and specify the XY and Z dimensions. Next, you can segment dendrites and dendritic spines using a DeepD3 model from the model zoo by clicking on Analyze -> Segment dendrite and spines. Afterwards, you may clean the predictions by clicking on Analyze -> Cleaning. Finally, you may build 2D or 3D ROIs using the respective functions in Analyze. To test the 3D ROI building, double click in the stack to a region of interest. A window opens that allows you to play with the hyperparameters and segments 3D ROIs in real-time.

All results can be exported to various file formats. For convenience, DeepD3 saves related data in its "proprietary" hdf5 file (that you can open using any hdf5 viewer/program/library). In particular, you may export the predictions as TIF files, the ROIs to ImageJ file format or a folder, the ROI map to a TIF file, or the ROI centroids to a file.

Most functions can be assessed using a batch command script located in deepd3/inference/batch.py.

How to cite

    @article{10.1371/journal.pcbi.1011774,
        doi = {10.1371/journal.pcbi.1011774},
        author = {Fernholz, Martin H. P. AND Guggiana Nilo, Drago A. AND Bonhoeffer, Tobias AND Kist, Andreas M.},
        journal = {PLOS Computational Biology},
        publisher = {Public Library of Science},
        title = {DeepD3, an open framework for automated quantification of dendritic spines},
        year = {2024},
        month = {02},
        volume = {20},
        url = {https://doi.org/10.1371/journal.pcbi.1011774},
        pages = {1-19},
        number = {2},
    }