MSU Frame Rate Conversion Method
- Algorithm, ideas: Dr. Dmitriy Vatolin
- Algorithm, implementaion: Sergey Grishin
Algorithm allows to double frame rate (fps) of the video stream. This conversion is performed by calculating and inserting the new frames in the stream (see picture).
Double up-conversion of the frame rate
The main advantages of the method are:
- high accuracy of motion tracking and compensation
- constant temporal sharpness level of the frames in converted video stream
This is achieved by construction of compensated frames with quarter pixel accuracy. Such high accuracy is obtained by using of bilinear and bicubic interpolation. Second advantage means that interpolated frames have the same sharpness as the original ones. Note that such the feature is not common for the majority of methods of this kind.
Here we present results of comparison of described (MSU), Frame Repetition (FR) and Linear Interpolation (LI) methods on “akiyo” sequence. Results include PSNR metric visualization of interpolated frames in comparison to original ones on the right side. Following is the color semantics on these images: red color - high difference region, this difference is visually noticeable; yellow - little less than previous difference; green and blue - almost unnoticeable difference; black - no difference.
Performance of MSU, LI and FR methods at "akiyo" sequence
On the pictures below there are the frames from sequences with four times increased frame rate. Such the up-conversion is achieved by repeated using of FRC method in accordance to picture:
Four times frame rate up-conversion
On the next picture there is the comparison of MSU and LI FRC methods by the example of conversion of “waterfall” sequence:
Performance of MSU and LI at "waterfall" sequence
Quality comparison of MSU, LI and FR methods is shown at the graphs below. In vertical direction there is PSNR quality level of interpolated frames in comparison to original ones (the higher values of metric means the higher quality of interpolated frames), in horizontal direction interpolated frame numbers are marked.
Performance quality of MSU, LI and FR at "BBC", "buc", "foreman" sequences
Below there is a comparison of MSU and UCSD (University of California, San Diego) FRC methods. Sequences with four times increased frame rate were used for this comparison:
Comparison of MSU and UCSD methods by the example of sequence "dogs"
On the pictures below it is easy to see the sharpness difference on frames, which were calculated by increasing FPS four times:
Frames of MSU and UCSD methods for sequence "dogs"
Two sequences where the quality difference of MSU and UCSD methods can be clearly seen are available for download. In these sequences each frame contains the same part of frame from sequence processed by FR, MSU and LI. This method of sequence generation makes comparison of FRC methods easy and comfortable:
Frame from "crowd" sequence for comparing FR, MSU and LI methods
Filter unavailable due to contract limitation. You can license Advanced FRC Project result.
- Sequence “dogs” for comparing FR, MSU and UCSD (704x464, Divx 4.12, 2.1 Mb)
- Sequence “crowd” for comparing FR, MSU and UCSD (704x464, Divx 4.12, 2.6 Mb)
MSU Benchmark Collection
- MSU Video Upscalers Benchmark 2021
- MSU Video Alignment and Retrieval Benchmark
- MSU Super-Resolution for Video Compression Benchmark 2021
- MSU Mobile Video Codecs Benchmark 2021
- MSU Video Super-Resolution Benchmark
- MSU Shot Boundary Detection Benchmark 2020
- MSU Deinterlacer Benchmark
- The VideoMatting Project
- Video Completion
- Codecs Comparisons & Optimization
- Video Quality Measurement Tool 3D
MSU Datasets Collection
- Video Filters