doubt about image classifition(V2) code

[Jenny970]

Hi! I wonder from which piece of code this result was derived.

	Top-1	Top-5
DeiT-T	72.2	91.1
+BatchFormerV2	72.7	91.5
DeiT-S	79.8	95.0
+BatchFormerV2	80.4	95.2
DeiT-B	81.7	95.5
+BatchFormerV2	82.2	95.8

[zhihou7]

Hi, @Jenny970,
Do you mean the code for reproducing this result? I insert the batchformerv2 in the 8th layer for ImageNet. For CIFAR100, it is all layers. Actually, I get similar results when if insert batchformerv2 in the 9th or 10th layer. For DeiT-T, we can insert all layers, which however do not improve the performance further. For DeiT-S or DeiT-B, we suffer from nan issues.

        res_blocks = []
        encoder_global = BatchFormerV2(args.add_global, model.num_features)
        if not args.shared_bf:
        first_enc = encoder_global 
        insert_list = [8] # or = list(range(12))
        for i, block in enumerate(model.blocks):
            res_blocks.append(block)
            if i in insert_list:
                if insert_list[0] == i: # first layer, 
                    #TODO extends the batch dimension to construct a twin stream.
                    res_blocks.append(first_enc)
                    first_enc = None
                else:
                    if not args.shared_bf:
                        encoder_global = BatchFormerV2(args.add_global, model.num_features)
                    res_blocks.append(encoder_global)
        # model.norm = torch.nn.Identity()
        if args.add_norm_bt:
            # add here is similar to v1
            model.norm = torch.nn.Sequential(model.norm, encoder_global)
        model.blocks = torch.nn.Sequential(*res_blocks)

[Jenny970]

Does this mean that the code for the image classification part is not in the released code?
if so, can you give me the complete code of image classifition? Thank you very much!

[zhihou7]

Yes. I have not released yet.

That's no problem. I will remove my other tentative experiments on that code and share it with you privately tonight.

[zhihou7]

I have shared it with you via github. I do not test it and it includes some messy codes. But, I am sure it includes the codes for batchformerv2.

Regards,

[Jenny970]

ok， I have received it !
Thank you so much！

[RooKichenn]

I have shared it with you via github. I do not test it and it includes some messy codes. But, I am sure it includes the codes for batchformerv2.

Regards,

Hello, can you also give me a complete code for image classification? I would like to reproduce the result, thank you so much！

[zhihou7]

@RooKichenn, I have shared the code with you.

Besides, I also share the code for SAM-DETR with you (@Jenny970 @RooKichenn). Those codes are actually cleaner. The code for classification is messy because I have changed a lot to balance the mix precision and the nan issue.

For the code on MAE, I also release it with a unique repository: https://github.com/zhihou7/mae_bf/. For those repositories, the code is clean.

Feel free to ask if you have further questions.

[RooKichenn]

@RooKichenn, I have shared the code with you.

Besides, I also share the code for SAM-DETR with you (@Jenny970 @RooKichenn). Those codes are actually cleaner. The code for classification is messy because I have changed a lot to balance the mix precision and the nan issue.

For the code on MAE, I also release it with a unique repository: https://github.com/zhihou7/mae_bf/. For those repositories, the code is clean.

Feel free to ask if you have further questions.

I really appreciate it and thank you for your sharing. I am reading your paper on BatchFormerV2 and would like to reproduce your method and use it in remote sensing image classification, but I haven't found the exact implementation of BatchFormer yet. I will probably have a new understanding after I finish reading the V2 paper. I am currently in my third year of university, so I would appreciate your advice on anything I don't understand about this paper.

[zhihou7]

You are welcome. For image classification, it is better to use batchormerv1, which demonstrates the effectiveness of scarcity tasks. You can use this file (The code is very clean) to get the idea of implementation.

diff baseline.py <(curl https://raw.githubusercontent.com/thuml/Transfer-Learning-Library/master/examples/domain_generalization/image_classification/baseline.py)

If you use batchformerv2, you can also insert it into the last layer.

[RooKichenn]

You are welcome. For image classification, it is better to use batchormerv1, which demonstrates the effectiveness of scarcity tasks. You can use this file (The code is very clean) to get the idea of implementation.

diff baseline.py <(curl https://raw.githubusercontent.com/thuml/Transfer-Learning-Library/master/examples/domain_generalization/image_classification/baseline.py)

If you use batchformerv2, you can also insert it into the last layer.

Thank you so much！

[wanliliuxiansen]

I found the two parameters bboxembed and class_embed of DeformableTransformerDecoder are not set.Can you give me the complete code of DeformableTransformerDecoder? Thank you very much!