Second Annual MSU MPEG-4 AVC/H.264 Video Codec Comparison

For professional users and codec authors

Second H.264/AVC comparison home page!

Part 2. “Best quality” preset

Preset “Best quality”

In this mode codecs tried to show best quality in single-pass mode. Therefore major importance in this section is paid to graphs of quality of compressed sequences in different metrics. All H.264 codecs used main profile of encoding standard.

Below are some Y-PSNR/Bitrate and Delta Y-PSNR/Bitrate graphs.

PSNR (peak-to-peak signal-to-noise ratio) is a classic metric for video quality measurement. For two pictures xi,j and yi,j metric’s value is defined by the following formula:

Despite this metric does not generally reflect human perception of pictures’ distortion, during last decades it is commonly used as main criterion for video codecs’ performance evaluation.

Delta Y-PSNR (or Relative Y-PSNR) are comparative PSNR graphs, where x264 codec was chosen as reference. For every measurement of each codec the difference between this measurement and PSNR value of reference codec with the same bitrate is plotted on these graphs. If corresponding reference value is absent then it is linearly interpolated.


Picture 1.   Y-PSNR. Sequence "foreman"

Picture 2.   Delta Y-PSNR. Sequence "foreman"

Picture 3.   Y-PSNR. Sequence "susi"

Picture 4.   Delta Y-PSNR. Sequence "susi"

Picture 5.   Y-PSNR. Sequence "bbc"

Picture 6.   Delta Y-PSNR. Sequence "bbc"

Picture 7.   Y-PSNR. Sequence "battle"

Picture 8.   Delta Y-PSNR. Sequence "battle"

Picture 9.   Y-PSNR. Sequence "simpsons"

Picture 10. Delta Y-PSNR. Sequence "simpsons"

Picture 11. Y-PSNR. Sequence "matrix"

Picture 12. Delta Y-PSNR. Sequence "matrix"

Picture 13. Y-PSNR. Sequence "concert"

Picture 14. Delta Y-PSNR. Sequence "concert"

Conclusions:

U-PSNR, V-PSNR

Besides luminance component (Y-plane) sometimes it is interesting to look at codecs’ behavior on color components (U and V planes).


Picture 15. U-PSNR. Preset "Best quality". Sequence "foreman"

Picture 16. U-PSNR. Preset "Best quality". Sequence "susi"

Picture 17. U-PSNR. Preset "Best quality". Sequence "bbc"

Picture 18. V-PSNR. Preset "Best quality". Sequence "concert"

Conclusions:

SSIM, VQM

Several metrics recently appeared which aspire to be better approximations of human perception. The most popular among them are SSIM and VQM.

We have used them in our comparison, but they have presented no big differences from PSNR metric.

Below are some graphs of these metrics for different sequences.


Picture 19. Y-SSIM measure. Preset "Best quality". Sequence "foreman"

Picture 20. Y-VQM measure. Preset "Best quality". Sequence "foreman"

Picture 21. Y-SSIM measure. Preset "Best quality". Sequence "bbc"

Picture 22. Y-VQM measure. Preset "Best quality". Sequence "concert"

Blurring measure, blocking measure

During some time our laboratory is developing metrics, which allow measuring blocking and blurring of frames in video sequences. Work on these metrics is in progress, therefore we show only several graphs with them.

Red horizontal line is a blocking measure (or blurring measure) of source sequence. For blurring measure: the more is its value, the lesser is the blurring of frame; for blocking measure: the more is its value, the bigger is blocking of frame.


Picture 23. Y-Blurring measure. Preset "Best quality". Sequence "foreman"

Picture 24. Y-Blocking measure. Preset "Best quality". Sequence "foreman"


Graph type Total number of graph Inserted in this document
Average metric 196 24 (12%)

Bitrate handling

Consider graphs showing how accurately every codec kept requested bit rate. Grades on x-axis show target bitrates (point 0 is 100 kbps, point 9 - 2340 kbps). Grades on y-axis show how many times codec exceeded declared bit rate (real bit rate to requested bit rate ratio).


Picture 25. Bitrate handling. Preset "Best quality". Sequence "foreman"

Picture 26. Bitrate handling. Preset "Best quality". Sequence "susi"

Picture 27. Bitrate handling. Preset "Best quality". Sequence "bbc"

Picture 28. Bitrate handling. Preset "Best quality". Sequence "battle"

Picture 29. Bitrate handling. Preset "Best quality". Sequence "simpsons"

Picture 30. Bitrate handling. Preset "Best quality". Sequence "matrix"

Picture 31. Bitrate handling. Preset "Best quality". Sequence "concert"

Conclusions:

Time

Now let’s consider graphs showing encoding time for each sequence using “Best Quality” preset and average normalized time for this preset.

Average normalized time is obtained as follows. At first, for every sequence codec with largest compression time is found. Compression times of other codecs were divided by compression time of this codec. After this step every codec has value from 0 to 1 for each video sequence.

On the second step every codec gets evaluation calculated as arithmetic average of obtained values for all sequences. And these evaluations are shown on average normalized time graph.


Picture 32. Encoding time. Preset "Best quality". Sequence "foreman"

Picture 33. Encoding time. Preset "Best quality". Sequence "susi"

Picture 34. Encoding time. Preset "Best quality". Sequence "bbc"

Picture 35. Encoding time. Preset "Best quality". Sequence "battle"

Picture 36. Encoding time. Preset "Best quality". Sequence "simpsons"

Picture 37. Encoding time. Preset "Best quality". Sequence "matrix"

Picture 38. Encoding time. Preset "Best quality". Sequence "concert"

Picture 39. Normalized average encoding time. Preset "Best quality"

Conclusions:

10 Mar 2011
See Also
Video Colorization Benchmark
Explore the best video colorization algorithms
Call for HEVC codecs 2019
Fourteen modern video codec comparison
HEVC Video Codecs Comparison 2018 (Thirteen MSU Video Codec Comparison)
13th MSU video codecs comparison
HEVC Video Codecs Comparison 2017 (Twelfth MSU Video Codec Comparison)
12th MSU video codecs comparison
MSU Video Codec Comparisons (6 test of lossless, MPEG-4 and MPEG-4 AVC)
Call for HEVC codecs 2018
Site structure