Second Annual MSU MPEG-4 AVC/H.264 Video Codec Comparison

For professional users and codec authors

Second H.264/AVC comparison home page!

Part 2. “Best quality” preset

Preset “Best quality”

In this mode codecs tried to show best quality in single-pass mode. Therefore major importance in this section is paid to graphs of quality of compressed sequences in different metrics. All H.264 codecs used main profile of encoding standard.

Below are some Y-PSNR/Bitrate and Delta Y-PSNR/Bitrate graphs.

PSNR (peak-to-peak signal-to-noise ratio) is a classic metric for video quality measurement. For two pictures x_i,j and y_i,j metric’s value is defined by the following formula:

Despite this metric does not generally reflect human perception of pictures’ distortion, during last decades it is commonly used as main criterion for video codecs’ performance evaluation.

Delta Y-PSNR (or Relative Y-PSNR) are comparative PSNR graphs, where x264 codec was chosen as reference. For every measurement of each codec the difference between this measurement and PSNR value of reference codec with the same bitrate is plotted on these graphs. If corresponding reference value is absent then it is linearly interpolated.

Conclusions:

• Despite significant compression quality improvement in DivX, it is not a leader. And even more, if DivX is not far from leaders on “susi”, “battle” and “matrix” sequences, on the rest sequences (“foreman”, “bbc”, “simpsons”) it is most likely to be related to trailing codecs. New version of DivX codec shows worse results on “bbc” sequence than its old version 5.2.1; it was not able compress “concert” sequence at all.
• There is no obvious leader among H.264 codecs. In most cases leaders are x264 and Ateme codecs.
• The bad performance of Ateme H.264 codec on low birates is due to a bug, which was fixed by developers after publication of this comparison.
• Elecard codec could not manage with low bitrates of “concert” sequence. It is clearly seen from Y-PSNR graph for this codec.

U-PSNR, V-PSNR

Besides luminance component (Y-plane) sometimes it is interesting to look at codecs’ behavior on color components (U and V planes).

Conclusions:

• In color planes x264 codec shows slightly better result than Ateme codec.
• Elecard codec has some problems with bitrate control on “foreman” sequence. It is shown as “fall” of its curve on UV-PSNR graphs. But this codec shows the best results on low bitrates on this sequence.

SSIM, VQM

Several metrics recently appeared which aspire to be better approximations of human perception. The most popular among them are SSIM and VQM.

We have used them in our comparison, but they have presented no big differences from PSNR metric.

Below are some graphs of these metrics for different sequences.

Blurring measure, blocking measure

During some time our laboratory is developing metrics, which allow measuring blocking and blurring of frames in video sequences. Work on these metrics is in progress, therefore we show only several graphs with them.

Red horizontal line is a blocking measure (or blurring measure) of source sequence. For blurring measure: the more is its value, the lesser is the blurring of frame; for blocking measure: the more is its value, the bigger is blocking of frame.

Bitrate handling

Consider graphs showing how accurately every codec kept requested bit rate. Grades on x-axis show target bitrates (point 0 is 100 kbps, point 9 - 2340 kbps). Grades on y-axis show how many times codec exceeded declared bit rate (real bit rate to requested bit rate ratio).

Conclusions:

• DivX codec keeps low bit rates badly. Frequently it generates files which are several times bigger than the worst H.264 codec (by bit rate handling).
• “bbc” is the hardest sequence for bit rate handling for all codecs.
• Fraunhofer IIS codec keeps bit rate better than others.

Time

Now let’s consider graphs showing encoding time for each sequence using “Best Quality” preset and average normalized time for this preset.

Average normalized time is obtained as follows. At first, for every sequence codec with largest compression time is found. Compression times of other codecs were divided by compression time of this codec. After this step every codec has value from 0 to 1 for each video sequence.

On the second step every codec gets evaluation calculated as arithmetic average of obtained values for all sequences. And these evaluations are shown on average normalized time graph.

Conclusions:

• The fastest codec is ArcSoft codec, the slowest - Fraunhofer IIS codec.
• There is a noticeable work slowdown of Fraunhofer IIS and x264 codecs during bit rate increase.
• Fraunhofer IIS codec is approximately five times slower than all the rest codecs.