MSU Benchmark Collection


News

Released benchmarks list

  1. MSU Video Upscalers Benchmark 2021
  2. MSU Video Alignment and Retrieval Benchmark
  3. MSU Super-Resolution for Video Compression Benchmark 2021
  4. MSU Mobile Video Codecs Benchmark
  5. MSU Video Super-Resolution Benchmark 2021
  6. MSU Shot Boundary Detection Benchmark 2020
  7. MSU Deinterlacer Benchmark
  8. The VideoMatting Project
  9. Video Completion
  10. (soon) MSU Video Deblurring Benchmark
  11. (soon) MSU Video Frame Interpolation Benchmark
  12. (soon) MSU Full Reference Video Quality Metric Benchmark
  13. (soon) MSU No Reference Video Quality Metric Benchmark
  14. (soon) MSU Super Precision Benchmark

Super-Resolution Benchmarks

MSU Super-Resolution for Video Compression Benchmark 2021


With the emergence of new video resolution standards, more efficient video encoding and decoding techniques are required. Our benchmark can help determine the best SR models to work with each of the different codec standards. This information will help make video coding with downsampling more effective.

Key features

  • Subjective comparison with more than 1900 valid participants
  • Different objective metrics ranked by their correlation with the subjective assessment
  • 75 SR+codec pairs

MSU Video Super-Resolution Benchmark 2021


Super-Resolution is the process of calculating high-resolution samples from their low-resolution counterparts. Working with images we can utilize natural preferences and make a high-resolution image, which is only in a way similar to the real one. Our benchmark is aimed to find the best algorithms for the restoration of real details with Video Super-Resolution.

Key features

  • Subjective comparison with more than 1900 valid participants
  • ERQAv1.0 metric
  • 22 Methods

MSU Video Upscalers Benchmark


Super-Resolution is the process of calculating high-resolution samples from their low-resolution counterparts. We want to create the most comprehensive comparison of video super-resolution (VSR) algorithms by subjective quality

Key features

  • Subjective comparison with more than 4300 valid participants
  • Check how upscalers behave in the most practical upscale use cases

Other benchmarks

MSU Mobile Video Codecs Benchmark


Measurement of the speed and power efficiency of different codecs on different mobile platforms allows for deeper understanding of their suitability for different devices, and allows manufacturers to fine-tune their codec integration

Key features

  • Speed and power efficiency measurement
  • Video playback time increase by up to 22 hours
  • 147 Android models, 6 compression standards.

MSU Shot Boundary Detection Benchmark 2020


One of the basic steps in video processing is video scene splitting. For example, scene cutting is a necessary step in video annotation and indexing, keyframe searching, and automatic video format changing. Our benchmark is aimed at measuring the performance of video scene splitting algorithms

Key features

  • Extensive and diverse datasets
  • Beautiful and easy-interpreting visualizations

MSU Deinterlacer Benchmark


Deinterlacing is the process of converting interlaced video into a non-interlaced or progressive form. Interlaced video signals are commonly found in analog television, digital television, some DVD titles, and a smaller number of Blu-ray discs. Our benchmark is aimed at measuring the performance of video deinterlacing algorithms

Key features

  • Сhoose deinterlacing method that is the best for your speed and quality requirements
  • Discover the newest deinterlacing methods’ achievements

MSU Video Alignment and Retrieval Benchmark


Often, broadcasted video sequences can have some freeze frames. Because of this, the process of comparing the initial sequence and the result one is very obstructed. Video alignment aims at finding point correspondences between two video sequences to overcome this problem. Our benchmark is aimed at measuring the performance of video alignment algorithms

Key features

  • 4 Methods
  • 3 tracks varying on distortions type
  • 560 test pairs in each track with a total duration of ~2 million frames

The Video­Mat­ting pro­ject


The Video­Mat­ting pro­ject is the first pub­lic ob­jec­tive bench­mark for video-mat­ting meth­ods. We be­lieve our work will help rank ex­ist­ing meth­ods and aid de­vel­op­ers of new meth­ods in im­prov­ing their re­sults.

Key features

  • Green screen dataset
  • Stop motion dataset

Video completion


The Video­Com­ple­tion pro­ject in­tro­duces the first bench­mark for video-com­ple­tion meth­ods. We pre­sent re­sults for dif­fer­ent meth­ods on a range of di­verse test se­quences which are avail­able for view­ing on a player equipped with a mov­able zoom re­gion. We be­lieve that our work can help rank ex­ist­ing meth­ods and as­sist de­vel­op­ers of new gen­eral-pur­pose video-com­ple­tion meth­ods.

Key features

  • 7 video se­quences
  • Different objective metrics

Planned benchmarks

MSU Video Deblurring Benchmark


Blur often obscures the process of extracting details. Our bechmark aims at measuring the performance of details restoration by modern deblurring algorithms

Key features

  • Large subjective comparison
  • Dataset with real blur
  • 2 Tracks: motion deblurring and defocus deblurring

MSU Video Frame Interpolation Benchmark


A low frame rate causes aliasing, yields abrupt motion artifacts, and degrades the video quality. To solve this problem a lot of video frame interpolation algorithms have been created so far. Our benchmark will rank these algorithms and determine which is the best by means of interpolation quality.

Key features

  • Large subjective comparison
  • The most comprehensive comparison of frame interpolation algorithms

Feedback

About our benchmarks

The development of benchmarks is important for many reasons:

MSU Video Upscalers Benchmark 2021
The most comprehensive comparison of video super resolution (VSR) algorithms by subjective quality
MSU Video Alignment and Retrieval Benchmark
Explore the best algorithms in different video alignment tasks
MSU Super-Resolution for Video Compression Benchmark 2021
Learn about the best SR methods for compressed videos and choose the best model to use with your codec
MSU Mobile Video Codecs Benchmark 2021
Discover Android devices with the longest video playback time and find the most power-efficient video-decoder on your Android device and
MSU Video Super-Resolution Benchmark
Discover the newest VSR methods and find the most appropriate method for your tasks
MSU Shot Boundary Detection Benchmark 2020
Discover the best Shot Boundary Detection method for your case
MSU Deinterlacer Benchmark
The most comprehensive comparison of deinterlacing methods
Site structure