The most comprehensive comparison of video super resolution (VSR) algorithms by subjective quality

Our benchmark will help you to select the best upscaling approach for your needs. Check the “Methodology” section to learn the details.

Everyone is welcome to participate! Run your favorite super-resolution method on our compact test video and send us the result to see how well it performs. Check the “Submitting” section to learn the details.

What’s new

• November 9th, 2021: Alpha-version Release

Key features of the Benchmark

• Subjective comparison with over 4300 participants
• Visual comparison for all 40 algorithms on all 15 clips
  • Find the upscaler of the best visual quality
• Check how upscalers behave in the most practical upscale use cases
  • Find the method which produces the best results on various content types
• Special calculation of objective metrics robust to object borders displacement
• Easily spot upscaling artifacts with metric error maps visualizations

Introduction

Our benchmark presents the ranking of video upscalers using crowd-sourced subjective comparison. Over 4300 valid participants have selected the most visually appealing upscaling result in a number of pairwise comparisons.

For the evaluation of upscaling algorithms various metrics (objective measures of quality) also were used, including VMAF developed by Netflix. In addition FPS (frames per second) values were calculated (algorithms ran on NVIDIA Titan Xp).

Check the “Participants” section to learn the details about used algorithms.

4× Leaderboards

4× Visualizations

4× Charts

2× Leaderboards

2× Visualizations

2× Charts

Submitting

To add your upscaling method to MSU-VUB 2021, follow these steps:

1. Download input data	Download the test video Alternatively you can download it as png sequence
2. Apply your upscaler	Apply your 2× or 4× upscaling method to the downloaded video Configure it for lossless (png/tiff) output, if possible
3. Send us the result	Send video-upscalers-benchmark@videoprocessing.ai the following information: Name of the upscaling method that will be specified in our benchmark Download link for a ZIP archive with upscaled frames. The link must be valid at least within a month after we receive it. Please ensure that you gave us the permission to download the files The exact upscaling steps, commands, versions of used programs, etc. Hardware used for the upscaling and time taken so we can compute the method’s FPS Alternatively you can send us the upscaler's executable and we will run it ourselves.

See also

If you are interested in detail-preserving upscaling, check the MSU Video Super-Resolution Benchmark.

If you are interested in how compression with different video codec standards affects upscaling, check the MSU Super-Resolution for Video Compression Benchmark.