Data and Analysis for Full-Reference Metrics
Leaderboard
The chart below shows the correlation of metrics with subjective scores on our dataset. You can choose the type of correlation and the compression standard of codecs used for compression. We recommend that you focus on Spearman’s rank correlation coefficient.
Correlation type: Compression Standard:
Tabel below shows the correlations of metrics with subjective scores on our dataset. Click on the labels to sort the table.
Correlation type:
To analyze metric performance in more detail, we added a few mutually nonexclusive categories with 196 videos from the dataset: Shaking, Nature, Sports, Gaming / Animation.
Correlation type:
Metrics performance on different compression standards
Below you can see the comparison of the metrics performance on videos compressed by codecs with different compression standards. Click simplify to hide variation of metrics.
Correlation type:
Speed-Quality Plot
The chart below shows Speed-Quality characteristics of metrics.
Correlation type:
Submitting
To submit your method please visit this section of the home page.
Contacts
We would highly appreciate any suggestions and ideas on how to improve our benchmark. Please contact us via email: vqa@videoprocessing.ai.
Cite us
|
@inproceedings{
NEURIPS2022_59ac9f01,
author = {Antsiferova, Anastasia and Lavrushkin, Sergey and Smirnov, Maksim and Gushchin, Aleksandr and Vatolin, Dmitriy and Kulikov, Dmitriy},
booktitle = {Advances in Neural Information Processing Systems},
editor = {S. Koyejo and S. Mohamed and A. Agarwal and D. Belgrave and K. Cho and A. Oh},
pages = {13814--13825},
publisher = {Curran Associates, Inc.},
title = {Video compression dataset and benchmark of learning-based video-quality metrics},
url = {https://proceedings.neurips.cc/paper_files/paper/2022/file/59ac9f01ea2f701310f3d42037546e4a-Paper-Datasets_and_Benchmarks.pdf},
volume = {35},
year = {2022}
}
|
You can find the full text of our paper through the link.
-
MSU Benchmark Collection
- Learning-Based Image Compression Benchmark
- Super-Resolution for Video Compression Benchmark
- Video Colorization Benchmark
- Defenses for Image Quality Metrics Benchmark
- Super-Resolution Quality Metrics Benchmark
- Deinterlacer Benchmark
- Video Saliency Prediction Benchmark
- Metrics Robustness Benchmark
- Video Upscalers Benchmark
- Video Deblurring Benchmark
- Video Frame Interpolation Benchmark
- HDR Video Reconstruction Benchmark
- No-Reference Video Quality Metrics Benchmark
- Full-Reference Video Quality Metrics Benchmark
- Video Alignment and Retrieval Benchmark
- Mobile Video Codecs Benchmark
- Video Super-Resolution Benchmark
- Shot Boundary Detection Benchmark
- The VideoMatting Project
- Video Completion
- Codecs Comparisons & Optimization
- VQMT
- MSU Datasets Collection
- Metrics Research
- Video Quality Measurement Tool 3D
- Video Filters
- Other Projects