Official code repository for EUSIPCO 2024 paper "FICUS: Few-shot Image Classification with Unsupervised object Segmentation".
The paper is available at https:/lien eusipco.pdf.
IMT Atlantique Jonathan Lys, Frédéric Lin, Clément Beliveau, Jules Decaestecker Lab-STICC Yassir Bendou, Aymane Abdali,Bastien Pasdeloup
This repository contains the code for out of the box ready to use few-shot classifier for ambiguous images. In this paper we have shown that removing the ambiguity from the the query during few shot classification improves performances. To do so we use a combination of foundation models and spectral methods.
git clone https://github.com/NewS0ul/FICUS.git
cd
python3 -m venv ~//venvFicus
source ~/venvFicus/bin/activate
pip install -r requirement.txt
git clone https://github.com/NewS0ul/FICUS.git
cd
conda create -n Ficus python=3.9
conda activate Ficus
pip install -r requirement.txt
The overview of the pipeline is as follows :
Overview of the proposed methodology. First, eigenmaps are produced using DSM. Each eigenmap (here 2) is treated separately. Using the maps, random points (here 2) are sampled, and used to prompt SAM. For each point, we therefore obtain 3 candidate masks. Out of each group of 3 candidate masks, we keep the one that maximizes IOU with an Otsu thresholding of the map. Redundant masks are then filtered out using NMS. Finally, kept masks are used to compute feature representations of associated crops. A NCM is then applied to return a label for the image.
For all our experiments we have used three datasets : ImageNet , Pascal Voc and Cub
We use two foundation model : dinov2_vit_s_14 for image embdedding and classification and Segment Anything for image segmentation.
- To run the evaluations
python3 main.py -t [type of experiment]
- To run deep spectral method on un image
python3 models/deepSpectralMethods.py
Expected result :
If you find our paper or code repository helpful, please consider citing as follows:
bibtex