Data and Analysis for Full-Reference Metrics

Leaderboard

The chart below shows the correlation of metrics with subjective scores on our dataset. You can choose the type of correlation and the compression standard of codecs used for compression. We recommend that you focus on Spearman’s rank correlation coefficient.

Correlation type: Compression Standard:

Tabel below shows the correlations of metrics with subjective scores on our dataset. Click on the labels to sort the table.

Correlation type:

To analyze metric performance in more detail, we added a few mutually nonexclusive categories with 196 videos from the dataset: Shaking, Nature, Sports, Gaming / Animation.

Correlation type:

Metrics performance on different compression standards

Below you can see the comparison of the metrics performance on videos compressed by codecs with different compression standards. Click simplify to hide variation of metrics.

Correlation type:

Speed-Quality Plot

The chart below shows Speed-Quality characteristics of metrics.

Correlation type:

Submitting

To submit your method please visit this section of the home page.

Contacts

We would highly appreciate any suggestions and ideas on how to improve our benchmark. Please contact us via email: vqa@videoprocessing.ai.

Cite US

@inproceedings{
{Antsiferova, Anastasia and Yakovenko, Alexander and Safonov, Nickolay and Kulikov, Dmitriy and Gushchin, Alexander and Vatolin, Dmitriy},
title = {Applying Objective Quality Metrics to Video-Codec Comparisons: Choosing the Best Metric for Subjective Quality Estimation},
year={2021},
month = {11},
pages={199-210},
publisher={GraphiCon},
doi = {10.20948/graphicon-2021-3027-199-210},
}
12 Mar 2022
See Also
VQA Dataset
During our work we have created the database for video quality assessment with subjective scores
MSU Video Upscalers Benchmark 2022
The most extensive comparison of video super-resolution (VSR) algorithms by subjective quality
MSU Video Deblurring Benchmark 2022
Learn about the best video deblurring methods and choose the best model
Site structure