Real-World Stereo Color and Sharpness Mismatch Dataset

G&M Lab head:
Measurements, analysis:
Dr. Dmitriy Vatolin
Egor Chistov, Nikita Alutis, Maxim Velikanov

How to film stereoscopic dataset with real color and sharpness distortions?

Visual description of proposed setup

Key Features

  • 24 videos: 50 frames each with 4K cropped to 960×720 resolution
  • Only real-world stereo color and sharpness mismatch distortions
  • Tune your method and check how it performs on real-world data
  • Published all videos and their sources with scripts to reproduce

Download

Download already processed dataset or its sources if you want to do post-processing by yourself. Please fill out your name and email address. We will automatically send you the download link in 30 seconds.

Dataset Preview

In this section you can observe typical left distorted and left ground truth views from our dataset. Note that these views have different brightness and sharpness characteristics. Right view shares the same characteristics with the left ground truth view.

Drag the separator to see differences between two views

Dataset Scenes

In this section you can observe typical scenes from our dataset. Our dataset contains moving objects and light sources. We prefer objects that have complex textures or produce blicks to ensure dataset complexity.

Press the play button to watch selected scene

Methodology

Color and sharpness mismatches between views of the stereoscopic 3D video can decrease the overall video quality and may cause headaches and other viewer discomfort. To eliminate this problem, there are correction methods that aim to make the views consistent.

We present a real-world video dataset for the color and sharpness correction task in stereoscopic 3D video. We have collected it using a beam-splitter optical device and three cameras that simultaneously capture three different views: the distorted left view, the ground truth of the left view and the right view of the same scene. Beam-splitter rig is popular because it allows one to set a small stereo base. However it reduces polarized light and thus introduces mismatches between stereo views. Our dataset contains the distortions encountered while shooting 3D movies using a beam-splitter rig.

To achieve precise ground truth data it is crucial to film scenes without parallax present between left distorted and left ground truth views. The parallax is a difference in apparent position when an object is viewed along the two lines of sight. During post-processing, we can precisely correct affine mismatches between cameras, namely scale, rotation angle, and translation, but we cannot correct parallax without using complex optical flow algorithms that can affect our ground truth data quality. By using beam-splitter we can set the parallax between left distorted and left ground truth views to zero and also achieve real-world distortions caused by beam-splitter.

Methodology

Dataset filming and post-processing pipeline

For post-processing, we use a simple pipeline. Given the three videos from all cameras, we first horizontally flip the left distorted view, then we match it to the ground truth view using a homography transformation. For the estimation of transformation parameters, we utilize SIFT extractor, Brute-Force matcher, and the MAGSAC++ algorithm. Right view is also matched to the left ground truth view using LoFTR matcher and MAGSAC++ algorithm. Global brightness difference between right and left ground truth view was corrected using monge-kantorovitch linear colour mapping method. Temporal alignment was performed using audio streams captured by cameras. Post-processing scripts are available here.

Other Datasets

In this section you can observe key parameters of multiple datasets for color correction task. Previous datasets are either not large enough either have no real-world distortions.

Dataset Source Data Size Applied Distortions
Niu et al. [1] 18 frame pairs from 2D videos captured within a short period (no longer than 2 seconds) that share the same color feature Photoshop CS6 operators: Saturation, Brightness, Contrast, Hue, Color Balance, and Exposure, each with three severity levels
Lavrushkin et al. [2] 1000 stereo frame pairs without color distortions from S3D movies produced using only rendering Random simplex noise smoothed using domain transform filter
Grogan et al. [3] 15 image pairs from the dataset provided by Hwang et al. [4] Note that this is not a stereo image pairs dataset Different illuminations, camera settings, and color touch up styles
Croci et al. [5] 1035 stereo image pairs from Flick1024, InStereo2K, and IVY LAB Stereoscopic 3D image database Photoshop 2021 operators: Brightness, Color Balance, Contrast, Exposure, Hue, and Saturation, each with six severity levels
Ours 1200 stereo frame pairs were filmed using beam-splitter optical device and three cameras Beam-splitter optical device distortions including different illuminations, and different camera settings

Contact Us

Please contact us by email stereo-mismatch-dataset@videoprocessing.ai if you have any questions. We plan to publish a paper about this dataset later. If you want to cite us right now, please contact us as well.

References

  1. Y. Niu, H. Zhang, W. Guo, and R. Ji, “Image Quality Assessment for Color Correction based on Color Contrast Similarity and Color Value Difference,” in IEEE Transactions on Circuits and Systems for Video Technology, 2016, pp. 849-862.
  2. S. Lavrushkin, V. Lyudvichenko, and D. Vatolin, “Local Method of Color-Difference Correction between Stereoscopic-Video Views,” in 2018-3DTV-Conference: The True Vision-Capture, Transmission and Display of 3D Video (3DTV-CON), 2018 pp. 1-4.
  3. M. Grogan, and R. Dahyot, “L2 Divergence for Robust Colour Transfer,” in Computer Vision and Image Understanding, 2019, pp. 39-49.
  4. Y. Hwang, J. Y. Lee, I. So Kweon, and S. Joo Kim, “Color Transfer using Probabilistic Moving Least Squares,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 3342-3349.
  5. S. Croci, C. Ozcinar, E. Zerman, R. Dudek, S. Knorr, and A. Smolic, “Deep Color Mismatch Correction in Stereoscopic 3D Images,” in 2021 IEEE International Conference on Image Processing (ICIP), 2021, pp. 1749-1753.
22 Nov 2022
See Also
MSU Video Upscalers Benchmark 2022
The most extensive comparison of video super-resolution (VSR) algorithms by subjective quality
Forecasting of viewers’ discomfort
How do distortions in a stereo movie affect the discomfort of viewers?
SAVAM - the database
During our work we have created the database of human eye-movements captured while viewing various videos
MSU Video Deblurring Benchmark 2022
Learn about the best video deblurring methods and choose the best model
Site structure