MSU Frame Rate Conversion Method

• Algorithm, ideas: Dr. Dmitriy Vatolin
• Algorithm, implementaion: Sergey Grishin

See also:

• New FRC project: Advanced FRC Project
• Public FRC filter: MSU Frame Rate Conversion

Algorithm allows to double frame rate (fps) of the video stream. This conversion is performed by calculating and inserting the new frames in the stream (see picture).

Double up-conversion of the frame rate

The main advantages of the method are:

• high accuracy of motion tracking and compensation
• constant temporal sharpness level of the frames in converted video stream

This is achieved by construction of compensated frames with quarter pixel accuracy. Such high accuracy is obtained by using of bilinear and bicubic interpolation. Second advantage means that interpolated frames have the same sharpness as the original ones. Note that such the feature is not common for the majority of methods of this kind.

Examples

Here we present results of comparison of described (MSU), Frame Repetition (FR) and Linear Interpolation (LI) methods on “akiyo” sequence. Results include PSNR metric visualization of interpolated frames in comparison to original ones on the right side. Following is the color semantics on these images: red color - high difference region, this difference is visually noticeable; yellow - little less than previous difference; green and blue - almost unnoticeable difference; black - no difference.

Performance of MSU, LI and FR methods at "akiyo" sequence

On the pictures below there are the frames from sequences with four times increased frame rate. Such the up-conversion is achieved by repeated using of FRC method in accordance to picture:

Four times frame rate up-conversion

On the next picture there is the comparison of MSU and LI FRC methods by the example of conversion of “waterfall” sequence:

Performance of MSU and LI at "waterfall" sequence

Quality comparison of MSU, LI and FR methods is shown at the graphs below. In vertical direction there is PSNR quality level of interpolated frames in comparison to original ones (the higher values of metric means the higher quality of interpolated frames), in horizontal direction interpolated frame numbers are marked.

Below there is a comparison of MSU and UCSD (University of California, San Diego) FRC methods. Sequences with four times increased frame rate were used for this comparison:

Comparison of MSU and UCSD methods by the example of sequence "dogs"

On the pictures below it is easy to see the sharpness difference on frames, which were calculated by increasing FPS four times:

Frames of MSU and UCSD methods for sequence "dogs"

Two sequences where the quality difference of MSU and UCSD methods can be clearly seen are available for download. In these sequences each frame contains the same part of frame from sequence processed by FR, MSU and LI. This method of sequence generation makes comparison of FRC methods easy and comfortable:

Frame from "crowd" sequence for comparing FR, MSU and LI methods

Download

Filter unavailable due to contract limitation. You can license Advanced FRC Project result.

• Sequence “dogs” for comparing FR, MSU and UCSD (704x464, Divx 4.12, 2.1 Mb)
• Sequence “crowd” for comparing FR, MSU and UCSD (704x464, Divx 4.12, 2.6 Mb)

Contacts

E-mail: video@graphics.cs.msu.ru

• New FRC project: Advanced FRC Project
• Public FRC filter: MSU Frame Rate Conversion