Automatic sharpness mismatch detection and compensation in stereo
- Author: Alexander Belous
- Supervisor: Dmitriy Vatolin
A lot of S3D movies contain artifacts despite large budgets and modern post-processing methods. It is caused by insufficiency in automatization of post-processing and high rate of human errors.
Differences in sharpness between views are usually caused by incorrectly calibrated cameras. While watching movies with sharpness mismatch the spectator may lose sense of 3D, which is the primary goal of stereo, or even get a headache.
The algorithm estimates the amount of local sharpness mismatch for each stereo pair, trying to fix it if possible.
The steps of the algorithm:
- Alignment of views using optical flow with a custom comparison function;
- Evaluation of sharpness mismatch metric via comparison of DCT coefficients distribution;
- Reconstruction of both views with fixed sharpness mismatch. The reconstruction is performed in regions of high-confidence optical flow.
We compared our algorithm with a commercial plug-in for Nuke — Ocula 3.0 FocusMatcher on a set of 8 stereo sequences.
Pros over Ocula 3.0:
- Allows restoration of picture with both views’ sharpness shifted, while Ocula only allows restoration of one view based on another
- Does not produce new artifacts in the video
Pictures below illustrate the differences between proposed algorithm and FocusMatcher on a single stereo pair. It is notable that FocusMarcher corrupts the background of semi-transparent objects (red boxes), while proposed algorithm doesn’t have such an effect.
- Our algorithm automatically corrects significant sharpness mismatch while preserving the quality of original views
- Temporal consistency of applied corrections
- Speed of reconstruction: 1/16 FullHD stereo pairs per second on an Intel Core i5. Detection speed: 3 stereo pairs per second
MSU Benchmark Collection
- Super-Resolution Quality Metrics Benchmark
- Video Saliency Prediction Benchmark
- Super-Resolution for Video Compression Benchmark
- Metrics Robustness Benchmark
- Video Upscalers Benchmark
- Video Deblurring Benchmark
- Video Frame Interpolation Benchmark
- HDR Video Reconstruction Benchmark
- No-Reference Video Quality Metrics Benchmark
- Full-Reference Video Quality Metrics Benchmark
- Video Alignment and Retrieval Benchmark
- Mobile Video Codecs Benchmark
- Video Super-Resolution Benchmark
- Shot Boundary Detection Benchmark
- Deinterlacer Benchmark
- The VideoMatting Project
- Video Completion
- Codecs Comparisons & Optimization
- MSU Datasets Collection
- Metrics Research
- Video Quality Measurement Tool 3D
- Video Filters
- Other Projects