MSU Frame Rate Conversion Method

See also:

Algorithm allows to double frame rate (fps) of the video stream. This conversion is performed by calculating and inserting the new frames in the stream (see picture).

fps x2 scheme
Double up-conversion of the frame rate

The main advantages of the method are:

This is achieved by construction of compensated frames with quarter pixel accuracy. Such high accuracy is obtained by using of bilinear and bicubic interpolation. Second advantage means that interpolated frames have the same sharpness as the original ones. Note that such the feature is not common for the majority of methods of this kind.

Examples

Here we present results of comparison of described (MSU), Frame Repetition (FR) and Linear Interpolation (LI) methods on “akiyo” sequence. Results include PSNR metric visualization of interpolated frames in comparison to original ones on the right side. Following is the color semantics on these images: red color - high difference region, this difference is visually noticeable; yellow - little less than previous difference; green and blue - almost unnoticeable difference; black - no difference.

akiyo sequence
Performance of MSU, LI and FR methods at "akiyo" sequence

On the pictures below there are the frames from sequences with four times increased frame rate. Such the up-conversion is achieved by repeated using of FRC method in accordance to picture:

akiyo sequence
Four times frame rate up-conversion

On the next picture there is the comparison of MSU and LI FRC methods by the example of conversion of “waterfall” sequence:

Waterfall.avi MSU x4
Waterfall.avi LI x4
Performance of MSU and LI at "waterfall" sequence

Quality comparison of MSU, LI and FR methods is shown at the graphs below. In vertical direction there is PSNR quality level of interpolated frames in comparison to original ones (the higher values of metric means the higher quality of interpolated frames), in horizontal direction interpolated frame numbers are marked.


Performance quality of MSU, LI and FR at "BBC", "buc", "foreman" sequences

Below there is a comparison of MSU and UCSD (University of California, San Diego) FRC methods. Sequences with four times increased frame rate were used for this comparison:

Dogs.avi MSU x4
Dogs.avi UCSD x4
Comparison of MSU and UCSD methods by the example of sequence "dogs"

On the pictures below it is easy to see the sharpness difference on frames, which were calculated by increasing FPS four times:

MSU & UCSD sharpness
Frames of MSU and UCSD methods for sequence "dogs"

Two sequences where the quality difference of MSU and UCSD methods can be clearly seen are available for download. In these sequences each frame contains the same part of frame from sequence processed by FR, MSU and LI. This method of sequence generation makes comparison of FRC methods easy and comfortable:

Sample frame
Frame from "crowd" sequence for comparing FR, MSU and LI methods

Download

Filter unavailable due to contract limitation. You can license Advanced FRC Project result.

Contacts

E-mail: video@graphics.cs.msu.ru

04 Nov 2004
See Also
MSU Image- and video-quality metrics analysis
Description of a project in MSU Graphics and Media Laboratory
Shot-by-Shot Movie Version Comparison
Some movies are released in two or more versions. We propose an algorithm for fully automatic construction of an editing map of two movie versions.
MSU Video Group / Video data filtering and compression
Video filtering and compression by MSU Video Group
Advanced Frame Rate Converter (AFRC)
FRC algorithm increases the total number of frames in the video sequence
Super-Resolution Quality Metrics Benchmark
Discover 66 Super-Resolution Quality Metrics and choose the most appropriate for your videos
Video Saliency Prediction Benchmark
Explore the best video saliency prediction (VSP) algorithms
Site structure