Global Model Performance

[ghost]

Hello,

I have created a simple federated scenario for image classification, with a non-standard network. When I try the network non-federated, I get fairly good results. When I train it federated, I receive more or less unmodified global weights and I don't know what I am doing wrong.

Initially I split my dataset into four distinct sets. With the first split I train the global model, and the other three I use for the clients. Each one has a separate distinct split.

I initialize the global model via source_ckpt_file_full_name and set MetaKey.NUM_STEPS_CURRENT_ROUND=1since my data splits (for the clients) are more or less equal.

I use the normal Scatter and Gather approach with InTimeAccumulateWeightedAggregator.
After training, I download the global weights and validate it on all the data. The results are bad, it looks like the global model did not learn anything or very very little. The local models themselves show a learning curve when I use them for validation.

Unfortunately, I cannot share code.

[YuanTingHsieh]

@Chucksete thanks for the interest.

Reading from the comment, you are training the global model on 1/4 of your data first and then do federated fine-tuning with 3 clients.
Since you can't share your code, I will ask the following questions to help clarify your setting.

1. Try the network non-federated, how did you train it? and how did you check the performance?
   If you train and validate using the whole data the model will overfit.

2. How did you split your data? randomly split? does each portion contain the same number of samples?

3. "With the first split I train the global model": how many epochs? what accuracy/metrics are you getting? how did you check the performance

4. When you say "results are bad", what accuracy/metrics are you getting?