Leaderboard of the MSU Video Quality Metrics Benchmark


The chart below shows the correlation of metrics with subjective scores on our dataset. You can choose the type of correlation and compression standard of codecs used for compression. We recommend that you focus on Spearman’s rank correlation coefficient.

Correlation type: Compression Standard:

The table below shows the correlations of metrics with subjective scores on our dataset. Click on the labels to sort the table.

Correlation type:

To analyze metric performance in more detail, we added a few mutually nonexclusive categories with 196 videos from the dataset: Shaking, Nature, Sports, Gaming / Animation.

Correlation type:

Metrics performance on different compression standards

Below you can see the comparison of the metrics performance on videos compressed by codecs with different compression standards. In addition, the chart shows 95% confidence interval for each point.

Correlation type:

Speed-Quality Plot

Chart below shows Speed-Quality characteristics of metrics.

Correlation type:


To submit your method please visit this section of the home page.


We would highly appreciate any suggestions and ideas on how to improve our benchmark. Please contact us via email: vqa@videoprocessing.ai.

Cite us

author = {Antsiferova, Anastasia and Lavrushkin, Sergey and Smirnov, Maksim and Gushchin, Aleksandr and Vatolin, Dmitriy and Kulikov, Dmitriy},
booktitle = {Advances in Neural Information Processing Systems},
editor = {S. Koyejo and S. Mohamed and A. Agarwal and D. Belgrave and K. Cho and A. Oh},
pages = {13814--13825},
publisher = {Curran Associates, Inc.},
title = {Video compression dataset and benchmark of learning-based video-quality metrics},
url = {https://proceedings.neurips.cc/paper_files/paper/2022/file/59ac9f01ea2f701310f3d42037546e4a-Paper-Datasets_and_Benchmarks.pdf},
volume = {35},
year = {2022}

You can find the full text of our paper through the link.

12 Mar 2022
See Also
MSU Image- and video-quality metrics analysis
Description of a project in MSU Graphics and Media Laboratory
Super-Resolution Quality Metrics Benchmark
Discover 66 Super-Resolution Quality Metrics and choose the most appropriate for your videos
Video Saliency Prediction Benchmark
Explore the best video saliency prediction (VSP) algorithms
Super-Resolution for Video Compression Benchmark
Learn about the best SR methods for compressed videos and choose the best model to use with your codec
Metrics Robustness Benchmark
Check your image or video quality metric for robustness to adversarial attacks
Video Upscalers Benchmark
The most extensive comparison of video super-resolution (VSR) algorithms by subjective quality
Site structure