MSU Graphics & Media Lab Report #2 on stereo films quality assessment
Graphics & Media Lab CMC of Lomonosov Moscow State University has about 10 years’ experience in comprehensive analyzing of compressed video. In March 2013 Graphics & Media Lab (in cooperation with IITP RAS) has released its first report on stereo films quality assessment. This first report presented our evaluation results and overall quality comparison of 5 stereo films.
After a very successful release of Avatar the interest to stereo 3D films endures now a bad slump. Some people even think that the 3D is dying and has no future. This decline of 3D demand has a serious background: many movie goers experience headache and eyestrain while watching a stereo film. The Lab decided to analyze the stereo films, captured with stereoscopic camera systems and to investigate the potential reasons of the irritating annoyance. The main examined problems, discovered during the film analysis:
- excessive horizontal disparity
- vertical disparity
- color mismatch (see Image #1)
- sharpness mismatch
Image #1: Example of color mismatch
We’ve got a lot of positive comments from professional stereographers on our first report and were encouraged to continue the work on stereo film evaluation. We are planning to publish in the 2-nd half of 2013 another three reports and invite the stereographers for cooperation.
Now we have a pleasure to share with the industry professionals our second report on stereo film quality analysis, also including five 3D movies filmed in stereo:
- Dolphin Tale
- Drive Angry
- The Three Musketeers
- Hugo
- Sanctum
In the overall comparison of the examined films we compare 3D movies from the 1-st and 2-nd reports, altogether 10 films.
Depth Budget
Take a look at the depth budget (see the Image #2): 3D films created in 1994 have a big depth budget; the movies from 2011-2012 years have a big negative parallax. ‘Hugo’ has a lot of objects before the screen, on the contrary, in ‘The Three Musketeers’ all objects are behind the screen.
Image #2: Depth Budget vs. Release Date
Vertical Disparity
It’s noticeable that all the recent films (‘Hugo’, ‘The Three Musketeers’, ‘Pirates of the Caribbean’) have a good vertical disparity. It means, that the situation is getting more and more better with the lapse of time (see the Images #3 and #4).
Image #3: Vertical Disparity vs. Release Date
It’s obvious, that the stereo quality depends also on the film budget, namely on the 2D to S3D conversion budget. We have estimated the relation between the film budget and existing artifacts (see the images #4, #6 and #8). But unfortunately these evaluations can’t give an accurate information, because we don’t know the conversion budgets (source of the film budgets is http://www.imdb.com/)
Image #4: Vertical Disparity vs. Film Budget
Color Mismatch
According to the Images #5 and #6 the films ‘Hugo’, ‘Three Musketeers’ and ‘Pirates of the Caribbean’ have the best color measures.
Image #5: Color Mismatch vs. Release Date
Image #6: Color Mismatch vs. Film Budget
Sharpness Mismatch
Image #7 shows a good trend of improvement in the last years: the films ‘Hugo’, ‘Three Musketeers’ and ‘Pirates of the Caribbean’ have a minimal sharpness mismatch measures in comparison with the other evaluated movies.
Image #7: Sharpness Mismatch vs. Release Date
Image #8: Sharpness Mismatch vs. Film Budget
In the screenshots #1-4 below please find the samples of valuable comments from professional stereographers and some more charts and visualisations of distortions:
|
|
|
|
|
|
Acknowledgements
We would like to thank for valuable comments, opinions and feedbacks, which helped us in preparing this report (in alphabetical order):
- Ross Copeland, Online Editor / Stereographer & Colourist, Post Production Consultant
- Clyde DeSouza, Author, Stereographer and Creative Technology Evangelist
- Alfredo Gonzalez, Stereoscopic Consultant / Stereographer
- Srboljub Hetlerovic, Stereographer, VFX Supervisor
- Jon Karafin, Senior Scientist and Director of Production Technology at RealD
- Julian Napierr, Director, Editor, Stereographer
- Celine Tricart, Independent Filmmaker and Stereographer
A key motive of this report is to improve the stereo quality and therefore to reduce eye- and headache. The Graphics & Media Lab hopes, that a high 3D video quality would contribute to a continually increasing number of people, leaving the modern theaters without annoyance and with good impressions from the image quality and storytelling. The report is available to all who may be concerned and interested in improving the quality of stereoscopic content. The next reports on 3D quality will be published in the near future on the website.
Reports overview
Stereo-analysis project description
Contacts
For questions and proposition please contact us 3dmovietest@graphics.cs.msu.ru
-
MSU Benchmark Collection
- Video Colorization Benchmark
- Super-Resolution for Video Compression Benchmark
- Defenses for Image Quality Metrics Benchmark
- Learning-Based Image Compression Benchmark
- Super-Resolution Quality Metrics Benchmark
- Video Saliency Prediction Benchmark
- Metrics Robustness Benchmark
- Video Upscalers Benchmark
- Video Deblurring Benchmark
- Video Frame Interpolation Benchmark
- HDR Video Reconstruction Benchmark
- No-Reference Video Quality Metrics Benchmark
- Full-Reference Video Quality Metrics Benchmark
- Video Alignment and Retrieval Benchmark
- Mobile Video Codecs Benchmark
- Video Super-Resolution Benchmark
- Shot Boundary Detection Benchmark
- The VideoMatting Project
- Video Completion
- Codecs Comparisons & Optimization
- VQMT
- MSU Datasets Collection
- Metrics Research
- Video Quality Measurement Tool 3D
- Video Filters
- Other Projects