Neural network-based algorithm for classification of stereoscopic video by the production method

• Author: Sergey Zvezdakov
• Supervisor: dr. Dmitriy Vatolin

Introduction

Creating an efficient and correct algorithm for quality assessment of stereoscopic videos requires knowledge of the method used for its making. Stereoscopic videos, made through different production methods, contain various types of artifacts, so it’s necessary to analyze them with different quality assessment algorithms.

Artifacts of stereoscopic shooting

Artifacts of 2D-to-3D conversion

Common artifacts

Proposed method

The algorithm classifies stereoscopic video scenes.

Features which can be used for attribution to one of the three main classes are calculated for an input video. Then, using the information obtained from these features, the classifier predicts the production method of the input.

Due to the fact that flat and low-contrast stereoscopic video is uninformative and difficult to classify, it is excluded from analysis.

Algorithm scheme

Experiments

To find the best parameters, various configurations of features based on VQMT3D were tested:

• VQMT3D — a linear method of support vectors using estimates obtained from the algorithms of the VQMT3D project
• VQMT3D + DH (Linear) — a linear method of support vectors using estimates of the VQMT3D project’s algorithms and an estimate of the blurring of the boundaries of objects on the disparity map
• VQMT3D + DH — a neural network using estimates of the VQMT3D project’s algorithms and estimation of objects’ boundaries blurring on the disparity map
• VQMT3D + DH + DS + FP — a neural network using estimates of the VQMT3D project’s algorithms, objects’ boundaries blurring on the disparity map, smoothness of the disparity histogram, and analysis of keystone distortion

Conversion/shooting classification results

CGI/conversion classification results

CGI/shooting classification results

Results

Within the framework of the VQMT3D project, the proposed algorithm was used to analyze 105 full-length stereoscopic movies.

Converted scene statistics for shot movies

Film	Budget, $K/min	Number of scenes found
47 Ronin	1470	9
A Very Harold & Kumar 3D Christmas	211	19
Bait	301	11
Battle of the Year	181	24
Dark Country	45	2
Dredd	520	48
Drive Angry	480	2
Final Destination 5	434	4
Flying Swords of Dragon Gate	286	14
Fright Night	283	6
Ghosts of the Abyss	213	14
Great Canyon Adventure	Unknown	4
Hansel & Gretel: Witch Hunters	568	5
Hugo	1349	9
Jack the Giant Slayer	1710	2
Journey 2: The Mysterious Island	840	3
Katy Perry: Part of Me	127	36
Life of Pi	944	6
One Direction: This Is Us	93	20
Oz the Great and Powerful	1653	2
Pina	40	2
Piranha 3DD	60	23
Pirates of the Caribbean: On Stranger Tides	1838	19
Prometheus	1048	5
Resident Evil: Afterlife	618	3
Resident Evil: Retribution	677	3
Sanctum	277	1
Saw 3D: The Final Chapter	222	48
Sea Rex 3D: Journey to a Prehistoric World	121	12
Silent Hill: Revelation 3D	210	3
Stalingrad	229	1
Step Up Revolution	333	7
Texas Chainsaw 3D	108	15
The Amazing Spider-Man	1691	2
The Darkest Hour	337	14
The Great Gatsby	739	1
The Hobbit: An Unexpected Journey	1914	2
The Legend of Hercules	707	16
Transformers: Dark of the Moon	1266	11
TRON: Legacy	1360	17
Underworld: Awakening	786	11

Shooted scene statistics for converted movies

Film	Budget, $K/min	Number of scenes found
Gulliver’s Travels	1317	1
The Green Hornet	1008	40
Spy Kids: All the Time in the World in 4D	306	38
Thor	1304	3
Priest	689	1
Piranha 3D	272	3