VQMT3D Project: Report 8 on 3D-video Quality Analysis

• Projects, ideas: Dr. Dmitriy Vatolin
• Implementation: Alexander Bokov, Alexey Fedorov, Alexey Shalpegin, Vladimir Yanushkovsky, Sergey Lavrushkin, Vyacheslav Napadovsky, Denis Sumin, Marat Arsaev, Alexander Voronov

In cooperation with IITP RAS

About the Report

The eighth report introduces several new technical quality metrics that analyze geometric inconsistencies and temporal shifts between S3D views, as well as new ones that detect swapped views and scenes that are more prone to crosstalk while viewing. The report presents detailed (366 pages) evaluation results of 25 S3D films, which were natively captured or post-converted.

Download report

Analysed films

Examined problems:

A Very Harold & Kumar 3D Christmas (2011) Abraham Lincoln: Vampire Hunter (2012) The Avengers (2012) Alice in Wonderland (2010) Conan the Barbarian (2011) The Chronicles of Narnia: The Voyage of the Dawn Treader (2010) Clash of the Titans (2010) Drive Angry (2011) Dolphin Tale (2011) Galapagos: The Enchanted Voyage (1999; released as BD3D in 2010) Green Lantern (2011) Gulliver's Travels (2010) Hugo (2011) Immortals (2011) Into the Deep (1994; released as BD3D in 2010) Pirates of the Caribbean: On Stranger Tides (2011) Pina (2011) Resident Evil: Afterlife (2010) Resident Evil: Retribution (2012) Sanctum (2011) Step Up 3D (2010) Step Up Revolution (2012) The Amazing Spider-Man (2012) The Three Musketeers (2011) Wrath of the Titans (2012)

Scale mismatch Rotation mismatch Temporal shift Channel mismatch Crosstalk

The report comprises five main parts:

1. Firstly, we present our approach to analyzing stereoscopic video for geometric inconsistencies and briefly explain how to understand the charts and visualizations. We then provide the results of our analysis for 25 S3D films.
2. Then we present our results for detection of swapped views. We provide comprehensive information about every scene in which we detected swapped views using our automated technique. The report also presents visualizations of these scenes.
3. The following section is dedicated to temporal shifts between views in stereoscopic video. We list each detected scene and present visualizations.
4. We present the analysis of the crosstalk effect and of how the content may actually affect the amount of perceived crosstalk. It contains brief explanations and visualizations of the most crosstalk-prone frames.
5. The report concludes with an overall technical movie comparison. Here we show charts depicting the average values of our metrics for different films.

Professional stereographers have contributed to the analysis by providing valuable comments and general feedback.

Contributed Stereographers

(in alphabetical order; for main part of the report)

• Nick Brown, Stereographer, Stereo Supervisor
• Ross Copeland, Online Editor / Stereographer & Colourist, Post-Production Consultant
• Markus Lanxinger, Stereographer, Stereoscopic Supervisor
• Daniele Pugni, 3D Motion Graphic Designer Freelancer for Sky 3D Channel
• Jill Smolin, Director of Production Education for 3ality Technica
• Paul Taylor, Supervising Stereographer

Overall evaluation results

In this section we present selected overall comparison charts.

Please, read more about the charts and particular films in the report.

Average Scale Mismatch vs. Film Budget

Channel Mismatch Duration Bar Chart

Temporal Shift Duration Bar Chart

Average Crosstalk vs. Film Budget

Publications

If you want to make a reference to this project, please refer to one of the following publications:

1. Mikhail Erofeev, Dmitriy Vatolin, Alexander Voronov, Alexey Fedorov,
   “Toward an Objective Stereo-Video Quality Metric: Depth Perception of Textured Areas,”
   International Conference on 3D Imaging,
   2012. doi:10.1109/IC3D.2012.6615120 (download)
2. Dmitriy Akimov, Alexey Shestov, Alexander Voronov, Dmitriy Vatolin,
   “Automatic Left-Right Channel Swap Detection,”
   International Conference on 3D Imaging,
   2012. doi:10.1109/IC3D.2012.6615126 (download)
3. Alexander Voronov, Alexey Borisov, Dmitriy Vatolin,
   “System for automatic detection of distorted scenes in stereo video,”
   International Workshop on Video Processing and Quality Metrics for Consumer Electronic (VPQM-2012),
   pp. 138–143, 2012. (download)
4. Alexander Voronov, Dmitriy Vatolin, Denis Sumin, Vyacheslav Napadovsky, Alexey Borisov,
   “Towards Automatic Stereo-video Quality Assessment and Detection of Color and Sharpness Mismatch,”
   International Conference on 3D Imaging,
   2012. doi:10.1109/IC3D.2012.6615121 (download)
5. Alexander Voronov, Dmitriy Vatolin, Denis Sumin, Vyacheslav Napadovsky, Alexey Borisov,
   “Methodology for stereoscopic motion-picture quality assessment,”
   Proc. SPIE 8648, Stereoscopic Displays and Applications XXIV,
   vol. 8648, pp. 864810-1–864810-14, 2013. doi:10.1117/12.2008485 (download)
6. Alexander Bokov, Dmitriy Vatolin, Anton Zachesov, Alexander Belous, Mikhail Erofeev,
   “Automatic detection of artifacts in converted S3D video,”
   Proc. SPIE 9011, Stereoscopic Displays and Applications XXV (March 6, 2014),
   vol. 901112, 2014. doi:10.1117/12.2054330 (download)
7. Stanislav Dolganov, Mikhail Erofeev, Dmitriy Vatolin, Yury Gitman,
   “Detection of stuck-to-background objects in converted S3D movies,”
   2015 International Conference on 3D Imaging, IC3D 2015,
   2015. doi:10.1109/IC3D.2015.7391839 (download)
8. Yury Gitman, Can Bal, Mikhail Erofeev, Ankit Jain, Sergey Matyunin, Kyoung-Rok Lee, Alexander Voronov, Jason Juang, Dmitriy Vatolin, Truong Nguyen,
   “Delivering Enhanced 3D Video,”
   Intel Technology Journal,
   vol. 19, pp. 162–200, 2015. (download)
9. Dmitriy Vatolin, Alexander Bokov, Mikhail Erofeev, Vyacheslav Napadovsky,
   “Trends in S3D-Movie Quality Evaluated on 105 Films Using 10 Metrics,”
   Proceedings of Stereoscopic Displays and Applications XXVII,
   pp. SDA-439.1–SDA-439.10, 2016. doi:10.2352/ISSN.2470-1173.2016.5.SDA-439 (download)
10. Alexander Bokov, Sergey Lavrushkin, Mikhail Erofeev, Dmitriy Vatolin, Alexey Fedorov,
    “Toward fully automatic channel-mismatch detection and discomfort prediction for S3D video,”
    2016 International Conference on 3D Imaging (IC3D),
    2016. doi:10.1109/IC3D.2016.7823462 (download)
11. Dmitriy Vatolin, Sergey Lavrushkin,
    “Investigating and predicting the perceptibility protect of channel mismatch in stereoscopic video,”
    Moscow University Computational Mathematics and Cybernetics,
    vol. 40, pp. 185–191, 2016. doi:10.3103/s0278641916040075 (download)
12. Anastasia Antsiferova, Dmitriy Vatolin,
    “The influence of 3D video artifacts on discomfort of 302 viewers,”
    2017 International Conference on 3D Immersion (IC3D),
    2017. doi:10.1109/IC3D.2017.8251897 (download)
13. Dmitriy Vatolin, Alexander Bokov,
    “Sharpness Mismatch and 6 Other Stereoscopic Artifacts Measured on 10 Chinese S3D Movies,”
    Proceedings of Stereoscopic Displays and Applications XXVIII,
    pp. 137–144, 2017. doi:10.2352/ISSN.2470-1173.2017.5.SDA-340 (download)
14. Sergey Lavrushkin, Vitaliy Lyudvichenko, Dmitriy Vatolin,
    “Local Method of Color-Difference Correction Between Stereoscopic-Video Views,”
    Proceedings of the 2018 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON),
    2018. doi:10.1109/3DTV.2018.8478453 (download)
15. Aidar Khatiullin, Mikhail Erofeev, Dmitriy Vatolin,
    “Fast Occlusion Filling Method For Multiview Video Generation,”
    Proceedings of the 2018 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON),
    2018. doi:10.1109/3DTV.2018.8478562 (download)
16. Sergey Lavrushkin, Dmitriy Vatolin,
    “Channel-Mismatch Detection Algorithm for Stereoscopic Video Using Convolutional Neural Network,”
    Proceedings of the 2018 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON),
    2018. doi:10.1109/3DTV.2018.8478542 (download)
17. Alexander Ploshkin, Dmitriy Vatolin,
    “Accurate Method of Temporal Shift Estimation for 3D Video,”
    Proceedings of the 2018 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON),
    2018. doi:10.1109/3DTV.2018.8478431 (download)
18. Sergey Lavrushkin, Konstantin Kozhemyakov, Dmitriy Vatolin,
    “Neural-Network-Based Detection Methods for Color, Sharpness, and Geometry Artifacts in Stereoscopic and VR180 Videos,”
    International Conference on 3D Immersion (IC3D),
    2020. doi:10.1109/IC3D51119.2020.9376385 (download)
19. Kirill Malyshev, Sergey Lavrushkin, Dmitriy Vatolin,
    “Stereoscopic Dataset from A Video Game: Detecting Converged Axes and Perspective Distortions in S3D Videos,”
    International Conference on 3D Immersion (IC3D),
    2020. doi:10.1109/IC3D51119.2020.9376375 (download)
20. Lavrushkin Sergey, Molodetskikh Ivan, Kozhemyakov Konstantin, Vatolin Dmitriy,
    “Stereoscopic quality assessment of 1,000 VR180 videos using 8 metrics,”
    Electronic Imaging, 3D Measurement and Data Processing,
    2021. doi:10.2352/issn.2470-1173.2021.2.sda-350 (download)

Reports overview

Stereo-analysis project description

Report 1 (S3D shooting quality analysis of 5 movies)	Download (Additional info for bloggers and press)	Pages: 246 Figures: 295
Report 2 (S3D shooting quality analysis of 5 movies)	Download (Additional info for bloggers and press)	Pages: 342 Figures: 442
Report 3 (2D-3D conversion quality analysis of 5 movies)	Download (Additional info for bloggers and press)	Pages: 305 Figures: 336
Report 4 (S3D shooting quality analysis of 5 movies)	Download	Pages: 301 Figures: 402
Report 5 (2D-3D conversion quality analysis of 5 movies)	Download (Additional info for bloggers and press)	Pages: 384 Figures: 404
Report 6 (Stereo Window analysis of 10 movies)	Download (Additional info for bloggers and press)	Pages: 415 Figures: 455
Report 7 (Stereo Window analysis of 10 movies)	Download (Additional info for bloggers and press)	Pages: 333 Figures: 348
Report 8 (Rotate Analysis, Temporal Shift, Channels Swap, Zoom Mismatch in 25 movies)	Download (Additional info for bloggers and press)	Pages: 366 Figures: 361
Report 9 (Temporal Shift, Stuck-to-Background Objects, 2D to S3D conversion in Captured Films)	Download (Additional info for bloggers and press)	Pages: 467 Figures: 529
Report 10 (Overall analysis of 105 movies)	Download	Pages: 211 Figures: 270
Report 11 (Overall analysis of 10 selected Chinese movies)	Download	Pages: 322 Figures: 566
Report 12 (VR180 Quality Analysis)	Download	Pages: 348 Figures: 362

Download

Full report (366 pages) is absolutely free but download is available only for industry professionals (due to publication of real mistakes in the post-production process).

To download the report, please fill-in the request form.
You will get the download link for all reports (Reports #1–11) via e-mail.

All materials in this evaluation are used for research purposes and in accordance with Fair Use conditions. This evaluation is not published widely, but it is available for free under a subscription for stereoscopy and cinema technology professionals.

Feedback

Do you have any feedback on previous reports? What did you find useful, or was anything lacking?

What would you like to see in future reports (analysis of certain films, different metrics, charts and visualizations, etc.)?

We are open to collaboration. If you are interested in working with us, please write your propositions:

Any other feedback that doesn't fit in previous fields:

Contacts

For questions and proposition please contact us 3dmovietest@graphics.cs.msu.ru