Muzhi Zhu1*, Yang Liu1*, Zekai Luo1*, Chenchen Jing1, Hao Chen1, Guangkai Xu1, Xinlong Wang2, Chunhua Shen1
1Zhejiang University, 2Beijing Academy of Artificial Intelligence
NeurIPS 2024
We systematically study four crucial elements of applying the Diffusion Model to Few-shot Semantic Segmentation. For each of these aspects, we propose several reasonable solutions and validate them through comprehensive experiments.
Building upon our observations, we establish the DiffewS framework, which maximally retains the generative framework and effectively utilizes the pre-training prior. Notably, we introduce the first diffusion-based model dedicated to Few-shot Semantic Segmentation, setting the groundwork for a diffusion-based generalist segmentation model.
- Release the weights.
- Release the inference code.
- Release the training code.
Preparing the environment following GenPercept.
Preparing the dataset following Matcher
Download the pre-trained model weights from here.
For academic use, this project is licensed under the 2-clause BSD License. For commercial use, please contact Chunhua Shen.
If you find this project useful in your research, please consider to cite:
@article{zhu2024unleashing,
title={Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentation},
author={Zhu, Muzhi and Liu, Yang and Luo, Zekai and Jing, Chenchen and Chen, Hao and Xu, Guangkai and Wang, Xinlong and Shen, Chunhua},
journal={arXiv preprint arXiv:2410.02369},
year={2024}
}