MSU Defenses Video Quality Metrics Benchmark
Check your IQA/VQA metric robustness to adversarial attacks
Powered by
Project head: Aleksandr Gushchin
G&M Lab head: Dr. Dmitriy Vatolin
Measurements, analysis:
Khaled Abud, Georgii Bychkov,
Ekaterina Shumitskaya, Vyacheslav Napadovsky
Anastasia Antsiferova, Sergey Lavrushkin
Khaled Abud, Georgii Bychkov,
Ekaterina Shumitskaya, Vyacheslav Napadovsky
Anastasia Antsiferova, Sergey Lavrushkin
Key features of the Benchmark
- 20+ evaluated defense methods of different types (purification, adversarial training, certified robustness)
- 14 adversarial Wb and BB attacks including FGSM-based, Universal Adversarial Perturbation-based and Perceptual-aware attacks
- Automatic cloud-based pipeline for evaluation
What’s new
- 30.06.2024 Alpha-version of benchmark Benchmark Release
Dataset
Dataset can be found here.
Results
Non-adaptive leaderboard for adversarial purification defenses. Evaluated metrics are averaged across all images and attacks. Defense parameters’ values with the highest correlations for adversarial images are selected:
Methodology and dataset
TBD
Cite us
You can find the full text of our paper through the link.
We will include citation later.
Contacts
We would highly appreciate any suggestions and ideas on how to improve our benchmark. Please contact us via email: vqa@videoprocessing.ai.
See Also
Learning-Based Image Compression Benchmark
The First extensive comparison of Learned Image Compression algorithms
Super-Resolution Quality Metrics Benchmark
Discover 66 Super-Resolution Quality Metrics and choose the most appropriate for your videos
Video Saliency Prediction Benchmark
Explore the best video saliency prediction (VSP) algorithms
Super-Resolution for Video Compression Benchmark
Learn about the best SR methods for compressed videos and choose the best model to use with your codec
Metrics Robustness Benchmark
Check your image or video quality metric for robustness to adversarial attacks
Video Upscalers Benchmark
The most extensive comparison of video super-resolution (VSR) algorithms by subjective quality
Site structure
-
MSU Benchmark Collection
- Defenses for Image Quality Metrics Benchmark
- Learning-Based Image Compression Benchmark
- Super-Resolution Quality Metrics Benchmark
- Video Saliency Prediction Benchmark
- Super-Resolution for Video Compression Benchmark
- Metrics Robustness Benchmark
- Video Upscalers Benchmark
- Video Deblurring Benchmark
- Video Frame Interpolation Benchmark
- HDR Video Reconstruction Benchmark
- No-Reference Video Quality Metrics Benchmark
- Full-Reference Video Quality Metrics Benchmark
- Video Alignment and Retrieval Benchmark
- Mobile Video Codecs Benchmark
- Video Super-Resolution Benchmark
- Shot Boundary Detection Benchmark
- The VideoMatting Project
- Video Completion
- Codecs Comparisons & Optimization
- VQMT
- MSU Datasets Collection
- Metrics Research
- Video Quality Measurement Tool 3D
- Video Filters
- Other Projects