First Annual MSU MPEG-4 AVC/H.264 Video Codec Comparison

For professional users and codec authors

Return to first H.264/AVC comparison home page!

Part 4. Final part

Informal codecs comparison

As one can see on the diagrams above different codecs show significantly different results for different sequences. That happens because all the represented sequences have different character (motion, noise e.t.c.). This fact allows to define which codecs do well with any kind of video sequences and which provide good results only for some class of sequences. However it would be interesting to find out general characteristics of each codec on the whole testing set.

We decided that averaging resulting values for all the sequences won’t be correct and suggested an informal estimation where every codec is given some score depending on the results of its measurement.

Informal comparison rules

• If some codec is stably better than all the others in more than one point it is given score 3 regardless of other results.
• If some codec is worse than all the others in more than one point it is given score 1.
• Otherwise score 2 is assigned.
• Y-PSNR, U-PSNR, V-PSNR and Y-difference parameters are estimated using this method.

Informal comparison results

	bankomatdi	battle	bbc3di	foreman	susidi	total	place
Ateme	3	3	3	2	3	14	1
DivX	2	2	2	2	3	11	2
Fraunhofer	1	2	2	2	1	8	4,5
MainConcept	-	1	-	1	-	2	n/a
Moonlight	1	2	1	2	2	8	4,5
MpegableAVC	2	2	2	2	2	10	3
Videosoft	2	2	2	3	1	10	3

	bankomatdi		battle		bbc3di		foreman		susidi		total	place
	U	V	U	V	U	V	U	V	U	V
Ateme	3	3	3	3	3	3	3	3	3	3	30	1
DivX	1	1	2	2	1	1	2	1	2	1	14	5,6
Fraunhofer	1	2	1	1	2	2	1	2	1	1	14	5,6
MainConcept	-	-	2	2	-	-	2	2	-	-	8	n/a
Moonlight	2	2	2	2	2	2	2	2	2	2	20	3
MpegableAVC	2	1	2	2	2	2	2	1	2	1	17	4
Videosoft	2	2	2	2	2	2	2	3	2	2	21	2

	bankomatdi	battle	bbc3di	foreman	susidi	Y-diff	Y	UV/2	total	place
	Y-diff	Y-diff	Y-diff	Y-diff	Y-diff
Ateme	3	2	2	1	2	10	14	15	39	1
DivX	3	3	3	3	3	15	11	7	33	2
Fraunhofer	1	2	2	2	1	8	8	7	23	6
MainConcept	-	3	-	2	-	5	2	4	11	n/a
Moonlight	2	2	2	2	2	10	8	10	28	5
MpegableAVC	2	2	3	2	1	10	10	8.5	28.5	4
Videosoft	2	1	2	2	2	9	10	10.5	29.5	3

Common conclusion

• Standard was adopted not long ago - in the end of 2003. At the same time H.264 algorithm is significantly more complex, than previous standards and requires more tuning time. As a result many codecs, used in this comparison, are works in progress, that require much time before achieving product quality
• In the test start we planned to compare different codec compatibility in between and standard conformance. But simple tests shown that currently it is too early to test codec compatibility.
• Codecs that participated in our testing were optimized for maximum performance on current day computers, and do not employ all H264 format possibilities. Afterward with computers performance growth it would be possible to show better results even without data format changes. We could suggest following analogy: current H264 codecs are approximately on a level of DivX 2.0 - so they are not ready for mass distribution. But if quality increase in next versions would be similar to increase from DivX 3 to DivX 5 than format advantages would be noticeable.