Coreset Learning Based Sparse Black-box Adversarial Attack For Video Recognition

we propose a novel frame selection algorithm named CLVA, which is based on the coreset concept of active learning, to address the issues of frame selection efficiency and sparsity in existing video attack models. Specifically, CLVA simulates the process of coreset learning to identify frames with high weight for the attack recognition model. To achieve this, we consider a complete video clip as a dataset and treat all frames within it as a mini-dataset. We combine the coreset search algorithm with the attack and recognition algorithm to find frames with higher weights. The frames with the shortest distance are then extracted as coreset members using the K-Center-Greedy algorithm. We guarantee that the frames we find are optimal solutions by accurately calculating the distance between frames through the feature representation of the model. Finally, we perform a sparse attack on the selected frames.

Environment

git clone https://github.com/machineNo6/CLVA.git
cd CLVA
conda create -n CLVA python=3.7
conda activate CLVA

pip install -r requirements.txt

Quick start

Pretrained Models

Please download our pre-trained models (Extraction Code: qnbw, I3D+HMDB51 model) and put them in ./checkpoints.

The models for the remaining combinations need to be downloaded on your own, or you can contact us to provide them for you.

Dataset

You need to download the data for the UCF-101, HMDB-51, and Kinetics-400 datasets yourself (downloading just one of them is acceptable).

You can find them at the following links: UCF-101 HMDB-51 Kinetics-400

Sparse video attack

python new_coreset.py 6 --gpus 0

Results

Acknowledgement

Thanks to DeepSAVA, Heuristic, SVA, AST, Coreset, for sharing their code.

Related Work

• DeepSAVA: Sparse Adversarial Video Attack with Spatial Transformation
• [Reinforcement Learning Based Sparse Black-box Adversarial Attack on Video Recognition Models]
• Heuristic Black-box Adversarial Attacks on Video Recognition Models
• Sparse Black-box Video Attack with Reinforcement Learning
• Efficient Robustness Assessment via Adversarial Spatial-Temporal Focus on Videos
• Active Learning for Convolutional Neural Networks: A Core-Set Approach