-
Notifications
You must be signed in to change notification settings - Fork 22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add cifar10 resnet benchmark #10
Conversation
@stanleybak Note that we enlarged the ResNet-4B model to make it around 14K neurons, so we now have a relatively large model (which is also scored) in our benchmark suite (for addressing the issue of lacking large models). To compensate the difficulty caused by the larger network, we reduce epsilon for the ResNet-4B model from 2/255 to 1/255. It actually becomes easier than the previous model (CROWN verified accuracy is higher). if you feel we should stick to the previous smaller model, we can switch it back as well. BTW, I just found that some PGD attack accuracy numbers reported on the README are outdated. I will update them tonight so don't merge this right now. But all the models and specifications are ready for your review. |
@stanleybak I further cleaned up the code and also made the PGD based property generation script usable on CPUs (it was depending on a GPU). I have made necessary changes and the PR is ready for your final review. |
please rename your |
Done. |
you also want to do this in the generation script |
Done, sorry for forgetting to change it in the generation script. |
Please kindly find detailed benchmark descriptions including resnet models information and commands for generating random properties in Readme.