MSU Video Upscalers Benchmark 2022: Quality Enhancement

The most comprehensive comparison of video super resolution (VSR) algorithms by subjective quality

G&M Lab head: 
Dr. Dmitriy Vatolin
Measurements, analysis: 
Nikolai Karetin,
Ivan Molodetskikh

Our benchmark will help you to select the best upscaling approach for your needs. Check the “Methodology” section to learn the details.

Everyone is welcome to participate! Run your favorite super-resolution method on our compact test video and send us the result to see how well it performs. Check the “Submitting” section to learn the details.

What’s new

  • November 9th, 2021: Alpha-version Release

Key features of the Benchmark

Introduction

Our benchmark presents the ranking of video upscalers using crowd-sourced subjective comparison. Over 4300 valid participants have selected the most visually appealing upscaling result in a number of pairwise comparisons.

For the evaluation of upscaling algorithms various metrics (objective measures of quality) also were used, including VMAF developed by Netflix. In addition FPS (frames per second) values were calculated (algorithms ran on NVIDIA Titan Xp).

Check the “Participants” section to learn the details about used algorithms.

4× Leaderboards

Clip:

4× Visualizations

Clip:  Frame:  Error map: 

Model 1:  Model 2:  Model 3:

Drag a red rectangle to the area that you want to see zoomed-in

Source

undef

undef

undef

4× Charts

Clip:  Metric:

2× Leaderboards

Clip:

2× Visualizations

Clip:  Frame:  Error map: 

Model 1:  Model 2:  Model 3:

Drag a red rectangle to the area that you want to see zoomed-in

Source

undef

undef

undef

2× Charts

Clip:  Metric:

Submitting

To add your upscaling method to MSU-VUB 2021, follow these steps:

1. Download input data Download the test video
Alternatively you can download it as png sequence
2. Apply your upscaler Apply your 2× or 4× upscaling method to the downloaded video
Configure it for lossless (png/tiff) output, if possible
3. Send us the result
Send video-upscalers-benchmark@videoprocessing.ai
the following information:
  • Name of the upscaling method that will be specified in our benchmark
  • Download link for a ZIP archive with upscaled frames.
    The link must be valid at least within a month after we receive it.
    Please ensure that you gave us the permission to download the files
  • The exact upscaling steps, commands, versions of used programs, etc.
  • Hardware used for the upscaling and time taken
    so we can compute the method’s FPS

Alternatively you can send us the upscaler's executable
and we will run it ourselves.

See also

If you are interested in detail-preserving upscaling, check the MSU Video Super-Resolution Benchmark.

If you are interested in how compression with different video codec standards affects upscaling, check the MSU Super-Resolution for Video Compression Benchmark.

09 Nov 2021
See Also
MSU Video Quality Measurement Tool: Picture types
VQMT 14.0 Online help: List of all picture types available in VQMT and their aliases
MSU HDR Video Reconstruction Benchmark
The most comprehensive comparison of HDR video reconstruction methods
MSU HDR Video Reconstruction Benchmark Participants
The list of participants of MSU HDR Video ReconstructionBenchmark
MSU HDR Video Reconstruction Benchmark Methodology
Evaluation Methodology of MSU HDR Video Reconstruction Benchmark
MSU HDR Video Reconstruction Benchmark Dataset
MSU HDR Video Reconstruction Benchmark Dataset
MSU Super-Resolution for Video Compression Benchmark 2022
Learn about the best SR methods for compressed videos and choose the best model to use with your codec
Site structure