Automatic sharpness mismatch detection and compensation in stereo
- Author: Alexander Belous
- Supervisor: Dmitriy Vatolin
A lot of S3D movies contain artifacts despite large budgets and modern post-processing methods. It is caused by insufficiency in automatization of post-processing and high rate of human errors.
Differences in sharpness between views are usually caused by incorrectly calibrated cameras. While watching movies with sharpness mismatch the spectator may lose sense of 3D, which is the primary goal of stereo, or even get a headache.
The algorithm estimates the amount of local sharpness mismatch for each stereo pair, trying to fix it if possible.
The steps of the algorithm:
- Alignment of views using optical flow with a custom comparison function;
- Evaluation of sharpness mismatch metric via comparison of DCT coefficients distribution;
- Reconstruction of both views with fixed sharpness mismatch. The reconstruction is performed in regions of high-confidence optical flow.
We compared our algorithm with a commercial plug-in for Nuke — Ocula 3.0 FocusMatcher on a set of 8 stereo sequences.
Pros over Ocula 3.0:
- Allows restoration of picture with both views’ sharpness shifted, while Ocula only allows restoration of one view based on another
- Does not produce new artifacts in the video
Pictures below illustrate the differences between proposed algorithm and FocusMatcher on a single stereo pair. It is notable that FocusMarcher corrupts the background of semi-transparent objects (red boxes), while proposed algorithm doesn’t have such an effect.
- Our algorithm automatically corrects significant sharpness mismatch while preserving the quality of original views
- Temporal consistency of applied corrections
- Speed of reconstruction: 1/16 FullHD stereo pairs per second on an Intel Core i5. Detection speed: 3 stereo pairs per second
MSU Benchmark Collection
- MSU Video Upscalers Benchmark 2022
- MSU Video Deblurring Benchmark 2022
- MSU Video Frame Interpolation Benchmark 2022
- MSU HDR Video Reconstruction Benchmark 2022
- MSU Super-Resolution for Video Compression Benchmark 2022
- MSU No-Reference Video Quality Metrics Benchmark 2022
- MSU Full-Reference Video Quality Metrics Benchmark 2022
- MSU Video Alignment and Retrieval Benchmark
- MSU Mobile Video Codecs Benchmark 2021
- MSU Video Super-Resolution Benchmark
- MSU Shot Boundary Detection Benchmark 2020
- MSU Deinterlacer Benchmark
- The VideoMatting Project
- Video Completion
- Codecs Comparisons & Optimization
- MSU Datasets Collection
- Metrics Research
- Video Quality Measurement Tool 3D
- Video Filters
- Other Projects