Second Annual MSU MPEG-4 AVC/H.264 Video Codec Comparison

For professional users and codec authors

Second H.264/AVC comparison home page!

Part 1. Introduction

Thanks

We would like to express our gratitude to the following companies for providing us with codecs and settings:

We would also like to thank these companies for their technical support and help during our tests.

Overview

Codecs

Codec

Developer

Version

1. DivX

DivXNetworks, Inc

6.0

2. ArcSoft H.264

ArcSoft, Inc.

dev. version for 19.09.05

3. Ateme H.264

Ateme

1.2.1.6

4. ATI H.264

ATI Technologies Inc.

3.1.2

5. Elecard H.264

Elecard LTD

dev. version for 09.09.05

6. Franhofer IIS H.264

Fraunhofer Institute for Integrated Circuits

dev. version for 20.09.05

7. VSS H.264

Vanguard Software Solutions, Inc.

3.0.2.7

8. x264

Development group x264

revision 293

  Note: DivX 6.0 is not H.264 codec. It is MPEG-4 ASP codec, it takes part in this testing only as one of the best representative of previous video codec standard MPEG-4

Sequences

Sequence Number of frames Frames per second Spatial dimension and color space
1.       foreman 300 30 352x288(YV12)
2.       susi 374 25 704x576(YV12)
3.       bbc 374 25 704x576(YV12)
4.       battle 1599 24 704x288(YV12)
5.       simpsons 365 24 720x480(YV12)
6.       matrix 239 25 720x416(YV12)
7.       concert 390 25 1664x1088(YV12)

Goal and testing rules

H.264 Codecs Testing Objectives

The main goal of this document is an evaluation of the quality of new H.264 codecs, using objective metrics for comparison. The comparison was done using the settings provided by the developers for each codec.

Testing rules

Metrics Used in Comparison

During testing following metrics were measured: PSNR (Y, U, V, L, R, G, B components) SSIM (Y component) VQM (Y component) Blurring (Y, U, V components) Blocking (Y, U, V components) Detailed description of these metrics including VQM and SSIM algorithms could be found here: /vqmt/info.html

Number of graphs

Graph type Total number of graph Inserted into this document
Average metric 196 24 (12%)
Relative average 196 7 (3.5%)
Per frame metric 1414 13 (0.9%)
Bitrate Handling 14 14 (100%)
Speed/Quality 1960 8 (0.4%)
Speed 16 16 (100%)
Total 3796 82 (2%)

Sequences

Foreman

Sequence title foreman
Resolution 352x288
Number of frames 300
Color space YV12
Frames per second 30
Source Uncompressed (standard sequence), progressive

Picture 1. Frame 77

Picture 2. Frame 258

This is one of the most famous sequences. It represents a face with very rich mimic. Motion is not very intensive here, but on the other hand it is disordered, not forward. Intricate character of motion creates problems for the motion compensation process. In addition camera is shaking that makes the image unsteady. In the end of the sequence camera suddenly turns to the building site and there follows an almost motionless scene. So this sequence also shows codec’s behavior on a static scene after intensive motion.

Susi

Sequence title susi
Resolution 704x576
Number of frames 374
Color space YV12
Frames per second 25
Source MPEG-2 (40Mbit), Smart Deinterlace
Picture 3. Frame 193

This sequence is characterized by high-level noise and slow motion. In its first part the scene is almost static (the girl only blinks), then there is some motion (she abruptly moves her head) and then the scene becomes almost static again. Noise is suppressed on every second frame due to the B-frames option in MPEG-2 codec.

BBC

Sequence title bbc
Resolution 704x576
Number of frames 374
Color space YV12
Frames per second 25
Source Uncompressed (standard sequence), Smart Deinterlace
Picture 4. Frame 185 Picture 5. Frame 258

This sequence is characterized by pronounced rotary motion. It contains a rotating striped drum with different pictures and photos on it. Quality of the compressed sequence can be evaluated by observing details on these images.

Battle

Sequence title battle
Resolution 704x288
Number of frames 1599
Color space YV12
Frames per second 24
Source MPEG-2 (DVD), FlaskMPEG deinterlace
Picture 6. Frame 839

This sequence is a fragment of the “Terminator-2” movie, which represents its very beginning. In terms of compression this sequence is the most difficult one among all other sequences that took part in the testing. That is because of three main reasons: constant brightness changes (explosions and laser flashes, see the picture above), very quick motion and frequent changes of the scene that make codecs often compress frames as I-frames.

Simpsons

Sequence title simpsons
Resolution 720x480
Number of frames 365
Color space YV12
Frames per second 24
Source MPEG-2 (DVD), progressive
Picture 7. Frame 50

This sequence is a fragment of “Simpsons” cartoon film. This is a classical representative of cartoon films: sketchy movement, great number of monochrome regions with abrupt junctions between them. Previously this sequence was compressed in MPEG-2 with rather low bitrate.

Matrix

Sequence title matrix
Resolution 720x416
Number of frames 239
Color space YV12
Frames per second 25
Source MPEG-2 (DVD), Smart Deinterlace
Picture 8. Frame 226

This sequence is a fragment of “Matrix” movie. Relatively simple movement, quite dim colors and small resolution allows codec to treat this sequence in rather simple way.

Concert

Sequence title concert
Resolution 1664x1088
Number of frames 390
Color space YV12
Frames per second 25
Source MPEG-2 (DVD), Smart Deinterlace
Picture 9. Frame 128

This sequence is a part of HDTV broadcast of symphonic orchestra concert. Sequence’s spatial resolution is very high. At the same time motion is rather simple and sometimes it completely disappears. There are two scene changes in this sequence.

Codecs

DivX 6.0



Picture 10. DivX 6.0

Remarks:

However, despite these drawbacks, codec significantly improved compression quality in comparison with previous version. On the next graph there is a comparison between different versions of DivX (metric is Y-PSNR):

Picture 11. DivX 6.0, DivX 5.21 and DivX 3.22 comparison

 

ArcSoft H.264

Remarks:

 

Ateme H.264

Picture 12. Ateme H.264

Remarks:

 

ATI H.264

Remarks:

 

Elecard H.264

Remarks:

 

Fraunhofer IIS H.264

Picture 13. Fraunhofer IIS H.264

Remarks:

 

VSS H.264

Picture 14. VSS H.264

Remarks:

 

x264

Picture 15. x264

Remarks:

10 Mar 2011
See Also
Video Colorization Benchmark
Explore the best video colorization algorithms
Call for HEVC codecs 2019
Fourteen modern video codec comparison
HEVC Video Codecs Comparison 2018 (Thirteen MSU Video Codec Comparison)
13th MSU video codecs comparison
HEVC Video Codecs Comparison 2017 (Twelfth MSU Video Codec Comparison)
12th MSU video codecs comparison
MSU Video Codec Comparisons (6 test of lossless, MPEG-4 and MPEG-4 AVC)
Call for HEVC codecs 2018
Site structure