Learning Across Domains and Devices: Style-Driven Source-Free Domain Adaptation in Clustered Federated Learning

Official implementation of Learning Across Domains and Devices: Style-Driven Source-Free Domain Adaptation in Clustered Federated Learning

by Donald Shenaj^*,1, Eros Fanì^*,2, Marco Toldo¹, Debora Caldarola², Antonio Tavera², Umberto Micheli^†,1, Marco Ciccone^†,2, Pietro Zanuttigh^†,1 and Barbara Caputo^†,2

Corresponding authors: [email protected], [email protected]

^* Equal contribution. ^† Equal supervision. ¹ Authors supported by University of Padova, Padua, Italy. ² Authors supported by Politecnico di Torino, Turin, Italy.

Citation

If you find our work relevant to your research, or use this code, please cite our WACV 2023 paper:

@inproceedings{shenaj2023ladd,
  title={"Learning Across Domains and Devices: Style-Driven Source-Free Domain Adaptation in Clustered Federated Learning},
  author={Shenaj, Donald and Fan\`i, Eros and Toldo, Marco and Caldarola, Debora and Tavera, Antonio and Michieli, Umberto and Ciccone, Marco and Zanuttigh, Pietro and Caputo, Barbara},
  booktitle={Winter Conference on Applications of Computer Vision (WACV)},
  year={2023},
  organization={IEEE}
}

Summary

In this work we propose a novel realistic scenario for Semantic Segmentation in Federated Learning: Federated source-Free Domain Adaptation (FFreeDA). In FFreeDA, the server can pre-train the model on labeled source data. However, as in the Source-Free Domain Adaptation (SFDA) setting, further accessing the source data is forbidden. Clients can access only their unlabeled target dataset, but cannot share it with other clients nor with the server. Moreover, there are many clients in the system and each of them has only a limited amount of images, to emulate real-world scenarios. Therefore, after the pre-training phase, the training is fully unsupervised. To address the FFreeDA problem, we propose LADD, a novel federated algorithm that assumes the presence of multiple distributions hidden among the clients. LADD partitions the clients into clusters based on the styles of the images belonging to each client, trying to match them with their actual latent distribution. LADD shows excellent performance on all benchmarks with a source dataset (GTA5) and three different targets (Cityscapes, CrossCity, Mapillary), with diversified splits of the data across the clients.

Setup

Preliminary operations

1. Clone this repository.
2. Move to the root path of your local copy of the repository.
3. Create the LADD new conda virtual environment and activate it:

conda env create -f environment.yml
conda activate LADD

Datasets

1. Download the the Cityscapes dataset from here (gtFine_trainvaltest.zip and leftImg8bit_trainvaltest.zip files).
2. Ask for the Crosscity dataset here.
3. Download the Mapillary Vistas dataset from here.
4. Download the GTA5 dataset from here.
5. Extract all the datasets' archives. Then, move the datasets' folders in data/[dataset_name]/data, where [dataset_name] is one of {cityscapes, crosscity, mapillary, gta5}.

data
├── cityscapes
│   ├── data
│   │   ├── leftImg8bit
│   │   │   ├── train
│   │   │   ├── val
│   │   ├── gtFine
│   │   │   ├── train
│   │   │   ├── val
│   ├── splits
├── gta5
│   ├── data
│   │   ├── images
│   │   ├── labels
│   ├── splits
├── crosscity
│   ├── data
│   │   ├── cities
│   ├── splits
├── mapillary
│   ├── data
│   │   ├── training
│   │   ├── testing
│   │   ├── validation
│   ├── splits

Experiments' logging

Make a new wandb account if you do not have one yet, and create a new wandb project.

How to run

In the configs folder, it is possible to find examples of config files for some of the experiments to replicate the results of the paper. Run one of the exemplar configs or a custom one from the root path of your local copy of the repository:

./run.sh [path/to/config]

We provide config files to replicate the experiments in the paper:

1. run a pre-training script — e.g. ./run.sh configs/pretrain_crosscity.txt and take the corresponding exp_id from wandb
2. run our method — e.g. ./run.sh configs/crosscity_LADD_classifier.txt. Make sure to set load_FDA_id=[exp_id_pretrain]

N.B. change the wandb_entity argument with the entity name of your wandb project.

N.B. always leave a blank new line at the end of the config. Otherwise, your last argument will be ignored.

Results

GTA5 → Cityscapes

Setting	Method	mIoU (%)
centralized	Oracle	66.64 ± 0.33
centralized	Source Only	24.05 ± 1.14
centralized	FTDA	65.74 ± 0.48
centralized	MCD	20.55 ± 2.66
centralized	DAFormer	42.31 ± 0.20
FL-UDA	MCD	10.86 ± 0.67
FFreeDA	FedAvg† + Self-Tr.	35.10 ± 0.73
FFreeDA	LADD (cls)	36.49 ± 0.13
FFreeDA	LADD (all)	36.49 ± 0.14

^† Same pretrain as LADD.

GTA5 → Crosscity

Setting	Method	mIoU (%)
centralized	Source Only	26.49 ± 1.46
centralized	MCD	27.15 ± 0.87
FL-UDA	MCD	24.80 ± 1.56
FFreeDA	FedAvg† + Self-Tr.	33.59 ± 1.25
FFreeDA	LADD (cls)	39.87 ± 0.14
FFreeDA	LADD (all)	40.09 ± 0.19

^† Same pretrain as LADD.

GTA5 → Mapillary

Setting	Method	mIoU (%)
centralized	Oracle	61.46 ± 0.21
centralized	Source Only	32.40 ± 0.71
centralized	MCD	31.93 ± 1.89
federated	Oracle	49.91 ± 0.49
FL-UDA	MCD	19.15 ± 0.75
FFreeDA	FedAvg† + Self-Tr.	38.97 ± 0.21
FFreeDA	LADD (cls)	40.16 ± 1.02
FFreeDA	LADD (all)	38.78 ± 1.82

^† Same pretrain as LADD.