MSU Video Super-Resolution Quality Metrics Benchmark

Other MSU datasets

We have compiled another private dataset of crops, consisting of videos from the following MSU datasets:

1. SR Dataset
2. SR+Codecs Dataset
3. VSR Benchmark Dataset
4. VUB Benchmark Dataset

For all GT videos from these datasets, the features were calculated: bitrate, colorfulness, FPS, resolution, spatial (SI) and temporal (TI) information. For spatial complexity, we calculated the average size of x264-encoded I-frames normalized to the uncompressed frame size. For temporal complexity, we calculated the average P-frame size divided by the average I-frame size. Using a simple opponent color space representation we calculated the colorfulness of every video. Bitrate, FPS and resolution were obtained by using ffmpeg. Then we divided the whole collection into 30 clusters using the K-means algorithm, and chose one video from each cluster.

Thus, the final dataset consists of 30 reference (ground-truth, GT) videos, which correspond to 1187 distorted videos.

• The dataset covers a large number of use cases in the field of SR due to the large number of content types

• The dataset contains videos with completely different resolutions, FPS values: 8, 24, 25, 30, 60, as well as high and low spatio-temporal complexity

• Distorted videos were obtained using 46 SR methods, some of them were preprocessed with 5 codecs: aomenc, vvenc, x264, x265, uavs3es with different bitrates and qp values

• The dataset was manually checked for redundancy

Videos from benchmarks are FullHD video crops, since the subjective comparison was made on crops. Therefore, the resolution of all videos in the received dataset is low.

The dataset contains videos with the following resolutions:

1. 480×270

2. 200×170

3. 110×80

4. 320×270

5. 120×90

6. 180×150

7. 130×100

8. 360×270

Leaderboard Table

In this section, you can see the leaderboard of the metrics.