VQMT3D Project: Report 12 on VR180 Quality Analysis
- Projects, ideas: Dr. Dmitriy Vatolin
- Implementation: Sergey Lavrushkin, Ivan Molodetskikh, Konstantin Kozhemiakov, Maxim Velikanov, Dmitriy Konovalchuk
In cooperation with IITP RAS
About the Report
We would like to present the 12th report of the VQMT3D project. This is our first report that focuses on a detailed and thorough overall comparison of stereoscopic VR180 videos. The project is led by the CS MSU Graphics & Media Lab (Moscow, Russia) team.
To conduct a large-scale VR180-video analysis, we collected 1,000 videos from YouTube. Most of the videos have 10,000 to 100,000 views and are 5 to 10 minutes long. We performed special one-character search requests to ensure that there is no bias in the collected videos. This report is absolutely free for downloading. If you have any feedback, please fill this form or send us an e-mail — we will be glad to receive your suggestions for future reports.
Over 1,000 videos analysed, including: |
Examined problems: |
|
|
Report contents and diagram examples
The main section of the report presents an overall comparison of the videos we evaluated. It includes charts depicting the average metric values relative to a video’s release date, number of views, likes and dislikes, and duration, as well as charts illustrating metric-value distributions. Alongside the overall charts are examples of automatically detected artifacts.
We compared videos by the following metrics:
- Negative and positive parallax—depth budget (average disparity value for a video);
- Vertical parallax, scale and rotation mismatch—geometric distortions between views, quantified using intelligible values;
- Color and sharpness mismatch—dimensionless values that quantify the strength of the color difference and the noticeability of sharpness mismatch between views.
An example bar chart showing depth budget of the selected YouTube videos.
An example plot showing quantity of videos containing sharpness mismatch, with trend lines.
An example of detected frame with scale mismatch. The enlarged part highlights the distortion.
An example visualisation of focus mismatch between views.
An example visualisation of enlarged fragment with focus mismatch between views.
An example of rotation mismatch.
An example visualisation of color mismatch. Checkerboard overlay of the views (left) and color difference (right).
In the subsections dedicated to each of the analysed metrics we list the 20 worst videos, ranked in accordance with the highest values for the given metric.
The final section describes our plans for continuing the VQMT3D project.
Publications
If you want to make a reference to this project, please refer to one of the following publications:
- Mikhail Erofeev, Dmitriy Vatolin, Alexander Voronov, Alexey Fedorov,
“Toward an Objective Stereo-Video Quality Metric: Depth Perception of Textured Areas,”
International Conference on 3D Imaging,
2012. doi:10.1109/IC3D.2012.6615120 (download) - Dmitriy Akimov, Alexey Shestov, Alexander Voronov, Dmitriy Vatolin,
“Automatic Left-Right Channel Swap Detection,”
International Conference on 3D Imaging,
2012. doi:10.1109/IC3D.2012.6615126 (download) - Alexander Voronov, Alexey Borisov, Dmitriy Vatolin,
“System for automatic detection of distorted scenes in stereo video,”
International Workshop on Video Processing and Quality Metrics for Consumer Electronic (VPQM-2012),
pp. 138–143, 2012. (download) - Alexander Voronov, Dmitriy Vatolin, Denis Sumin, Vyacheslav Napadovsky, Alexey Borisov,
“Towards Automatic Stereo-video Quality Assessment and Detection of Color and Sharpness Mismatch,”
International Conference on 3D Imaging,
2012. doi:10.1109/IC3D.2012.6615121 (download) - Alexander Voronov, Dmitriy Vatolin, Denis Sumin, Vyacheslav Napadovsky, Alexey Borisov,
“Methodology for stereoscopic motion-picture quality assessment,”
Proc. SPIE 8648, Stereoscopic Displays and Applications XXIV,
vol. 8648, pp. 864810-1–864810-14, 2013. doi:10.1117/12.2008485 (download) - Alexander Bokov, Dmitriy Vatolin, Anton Zachesov, Alexander Belous, Mikhail Erofeev,
“Automatic detection of artifacts in converted S3D video,”
Proc. SPIE 9011, Stereoscopic Displays and Applications XXV (March 6, 2014),
vol. 901112, 2014. doi:10.1117/12.2054330 (download) - Stanislav Dolganov, Mikhail Erofeev, Dmitriy Vatolin, Yury Gitman,
“Detection of stuck-to-background objects in converted S3D movies,”
2015 International Conference on 3D Imaging, IC3D 2015,
2015. doi:10.1109/IC3D.2015.7391839 (download) - Yury Gitman, Can Bal, Mikhail Erofeev, Ankit Jain, Sergey Matyunin, Kyoung-Rok Lee, Alexander Voronov, Jason
Juang, Dmitriy Vatolin, Truong Nguyen,
“Delivering Enhanced 3D Video,”
Intel Technology Journal,
vol. 19, pp. 162–200, 2015. (download) - Dmitriy Vatolin, Alexander Bokov, Mikhail Erofeev, Vyacheslav Napadovsky,
“Trends in S3D-Movie Quality Evaluated on 105 Films Using 10 Metrics,”
Proceedings of Stereoscopic Displays and Applications XXVII,
pp. SDA-439.1–SDA-439.10, 2016. doi:10.2352/ISSN.2470-1173.2016.5.SDA-439 (download) - Alexander Bokov, Sergey Lavrushkin, Mikhail Erofeev, Dmitriy Vatolin, Alexey Fedorov,
“Toward fully automatic channel-mismatch detection and discomfort prediction for S3D video,”
2016 International Conference on 3D Imaging (IC3D),
2016. doi:10.1109/IC3D.2016.7823462 (download) - Dmitriy Vatolin, Sergey Lavrushkin,
“Investigating and predicting the perceptibility protect of channel mismatch in stereoscopic video,”
Moscow University Computational Mathematics and Cybernetics,
vol. 40, pp. 185–191, 2016. doi:10.3103/s0278641916040075 (download) - Anastasia Antsiferova, Dmitriy Vatolin,
“The influence of 3D video artifacts on discomfort of 302 viewers,”
2017 International Conference on 3D Immersion (IC3D),
2017. doi:10.1109/IC3D.2017.8251897 (download) - Dmitriy Vatolin, Alexander Bokov,
“Sharpness Mismatch and 6 Other Stereoscopic Artifacts Measured on 10 Chinese S3D Movies,”
Proceedings of Stereoscopic Displays and Applications XXVIII,
pp. 137–144, 2017. doi:10.2352/ISSN.2470-1173.2017.5.SDA-340 (download) - Sergey Lavrushkin, Vitaliy Lyudvichenko, Dmitriy Vatolin,
“Local Method of Color-Difference Correction Between Stereoscopic-Video Views,”
Proceedings of the 2018 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON),
2018. doi:10.1109/3DTV.2018.8478453 (download) - Aidar Khatiullin, Mikhail Erofeev, Dmitriy Vatolin,
“Fast Occlusion Filling Method For Multiview Video Generation,”
Proceedings of the 2018 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON),
2018. doi:10.1109/3DTV.2018.8478562 (download) - Sergey Lavrushkin, Dmitriy Vatolin,
“Channel-Mismatch Detection Algorithm for Stereoscopic Video Using Convolutional Neural Network,”
Proceedings of the 2018 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON),
2018. doi:10.1109/3DTV.2018.8478542 (download) - Alexander Ploshkin, Dmitriy Vatolin,
“Accurate Method of Temporal Shift Estimation for 3D Video,”
Proceedings of the 2018 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON),
2018. doi:10.1109/3DTV.2018.8478431 (download) - Sergey Lavrushkin, Konstantin Kozhemyakov, Dmitriy Vatolin,
“Neural-Network-Based Detection Methods for Color, Sharpness, and Geometry Artifacts in Stereoscopic and VR180 Videos,”
International Conference on 3D Immersion (IC3D),
2020. doi:10.1109/IC3D51119.2020.9376385 (download) - Kirill Malyshev, Sergey Lavrushkin, Dmitriy Vatolin,
“Stereoscopic Dataset from A Video Game: Detecting Converged Axes and Perspective Distortions in S3D Videos,”
International Conference on 3D Immersion (IC3D),
2020. doi:10.1109/IC3D51119.2020.9376375 (download) - Lavrushkin Sergey, Molodetskikh Ivan, Kozhemyakov Konstantin, Vatolin Dmitriy,
“Stereoscopic quality assessment of 1,000 VR180 videos using 8 metrics,”
Electronic Imaging, 3D Measurement and Data Processing,
2021. doi:10.2352/issn.2470-1173.2021.2.sda-350 (download)
Reports overview
Stereo-analysis project homepage
Feedback
Contacts
For questions and propositions please contact us 3dmovietest@graphics.cs.msu.ru
See Also
Super-Resolution for Video Compression Benchmark
Learn about the best SR methods for compressed videos and choose the best model to use with your codec
MSU 3D-video Quality Analysis. Report 11
MSU 3D-video Quality Analysis. Report 10
Detection of stereo window violation
How to find objects that are present only in one view?
Depth continuity estimation in S3D video
How smooth is the depth transition between scenes?
Detection of 3D movie scenes shot on converged axes
Another cause of headaches when watching 3D movies.
Site structure
-
MSU Benchmark Collection
- Super-Resolution for Video Compression Benchmark
- Video Colorization Benchmark
- Defenses for Image Quality Metrics Benchmark
- Learning-Based Image Compression Benchmark
- Super-Resolution Quality Metrics Benchmark
- Video Saliency Prediction Benchmark
- Metrics Robustness Benchmark
- Video Upscalers Benchmark
- Video Deblurring Benchmark
- Video Frame Interpolation Benchmark
- HDR Video Reconstruction Benchmark
- No-Reference Video Quality Metrics Benchmark
- Full-Reference Video Quality Metrics Benchmark
- Video Alignment and Retrieval Benchmark
- Mobile Video Codecs Benchmark
- Video Super-Resolution Benchmark
- Shot Boundary Detection Benchmark
- The VideoMatting Project
- Video Completion
- Codecs Comparisons & Optimization
- VQMT
- MSU Datasets Collection
- Metrics Research
- Video Quality Measurement Tool 3D
- Video Filters
- Other Projects