Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[New Model] <Amini2024MeanSparse> #191

Open
3 tasks done
mrteymoorian opened this issue Aug 16, 2024 · 5 comments
Open
3 tasks done

[New Model] <Amini2024MeanSparse> #191

mrteymoorian opened this issue Aug 16, 2024 · 5 comments
Labels
new-model Addition of a new model to the Leaderboard/Model Zoo

Comments

@mrteymoorian
Copy link

Paper Information

  • Paper Title: MeanSparse: Post-Training Robustness Enhancement Through Mean-Centered Feature Sparsification
  • Paper URL: https://arxiv.org/pdf/2406.05927
  • Paper authors: Sajjad Amini, Mohammadreza Teymoorianfard, Shiqing Ma, Amir Houmansadr

Leaderboard Claim(s)

Models 1 through 5 represent the new SOA models for CIFAR-10 Linf, CIFAR-10 L2, CIFAR-100 Linf, and ImageNet Linf, respectively.

Model 5, which we previously submitted and is currently ranked second in CIFAR-10 Linf, has been updated with improved results. We request an update to our model's results on the leaderboard to reflect these enhancements.

We have also submitted a pull request.

Model 1

  • Architecture: Meansparse_WRN-94-16
  • Dataset: CIFAR-10
  • Threat Model: Linf
  • eps: 8/255
  • Clean accuracy: 93.63
  • Robust accuracy: 75.28
  • Additional data: false
  • Evaluation method: AutoAttack
  • Checkpoint and code: Checkpoint and code

Model 2

  • Architecture: Meansparse_WRN-70-16
  • Dataset: CIFAR-10
  • Threat Model: L2
  • eps: 0.5
  • Clean accuracy: 95.49
  • Robust accuracy: 87.28
  • Additional data: false
  • Evaluation method: AutoAttack
  • Checkpoint and code: Checkpoint and code

Model 3

  • Architecture: Meansparse_WRN-70-16
  • Dataset: CIFAR-100
  • Threat Model: Linf
  • eps: 8/255
  • Clean accuracy: 75.17
  • Robust accuracy: 44.78
  • Additional data: false
  • Evaluation method: AutoAttack
  • Checkpoint and code: Checkpoint and code

Model 4

  • Architecture: Meansparse_Swin_L
  • Dataset: ImageNet
  • Threat Model: Linf
  • eps: 4/255
  • Clean accuracy: 78.86
  • Robust accuracy: 62.12
  • Additional data: false
  • Evaluation method: AutoAttack
  • Checkpoint and code: Checkpoint and code

Model 5

  • Architecture: meansparse_ra_wrn70_16
  • Dataset: CIFAR-10
  • Threat Model: Linf
  • eps: 8/255
  • Clean accuracy: 93.27
  • Robust accuracy: 72.78
  • Additional data: false
  • Evaluation method: AutoAttack
  • Checkpoint and code: Checkpoint and code

Model Zoo:

  • I want to add my models to the Model Zoo (check if true)
  • I use an architecture that is included among
    those here or in timm. If not, I added the link to the architecture implementation so that it can be added.
  • I agree to release my model(s) under MIT license (check if true) OR under a custom license, located here: (put the custom license URL here if a custom license is needed. If no URL is specified, we assume that you are fine with MIT)
@mrteymoorian mrteymoorian added the new-model Addition of a new model to the Leaderboard/Model Zoo label Aug 16, 2024
@fra31
Copy link
Member

fra31 commented Aug 19, 2024

Hi,

thanks for the submission! I'll have a look in the next days.

@mrteymoorian
Copy link
Author

Hello,

I hope you are well. I wanted to kindly ask if you could review the new models when you have a chance. It's been about two months since we submitted them, and your feedback would be greatly appreciated.

Thank you!

@fra31
Copy link
Member

fra31 commented Oct 16, 2024

Hi,

sorry for the delay. Do you happen to have the logs generated by the evaluation for the new models?

@mrteymoorian
Copy link
Author

Hi,
Unfortunately no. I don't have them now.

@fra31
Copy link
Member

fra31 commented Dec 20, 2024

Added the new models and evaluations with #202, please let me know if it's fine for you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new-model Addition of a new model to the Leaderboard/Model Zoo
Projects
None yet
Development

No branches or pull requests

2 participants