Automatic detection of artifacts in converted S3D videos

Introduction

Our set of algorithms provides an objective quality assessment for stereoscopic videos converted from a 2D source. In particular, the algorithms are able to detect three types of unwanted effects: edge sharpness mismatch, cardboard effect, and crosstalk noticeability.

Edge sharpness mismatch

A distorted region detected with edge sharpness mismatch metric

Due to the nature of 2D-to-3D conversion, the sharpness of edges may differ between the left-eye and right-eye views. This artifact is typically caused by the following:

Such an effect is rarely natural. Therefore, the edge sharpness mismatch may cause difficulties with scene interpretation and discomfort for the viewer.

Cardboard effect

The cardboard effect is a term referring to an unnatural flattening of objects in perceived visual images. Although this artifact may appear in stereoscopic videos captured by a stereoscopic camera system, it appears more frequently in converted S3D videos.

Crosstalk

Crosstalk is a very common problem across modern stereo display devices. Both viewer’s eyes see a combination of left and right views. This effect is caused solely by display devices, but it is possible to reduce its noticeability during film production.

Crosstalk visibility depends on the technology used for views separation:

Proposed method

For each type of the artifacts we have developed a detection algorithm.

Edge sharpness mismatch algorithm

This complex task can be simplified assuming that background remains the same near edges with sharpness mismatch.

Cardboard effect

In a 2D video the true depth map is not available and the converted video’s disparity cannot be verified. Thus, our goal is to detect completely flat objects and scenes.

Flat scenes are detected in the same way. Scenes that were rotated (with disparity changing linearly) are also considered flat.

Crosstalk noticeability

In highly textured regions the crosstalk can be masked by the human visual system. That is why we use a structural similarity function to compute the noticeability of differences. We blend the left and right views in 85% to 15% proportion.

Experiments

The following images demonstrate the work of our algorithm. The regions with distortions are enlarged, and a thumbnail of the full view is placed in the corner.

Image order: left view, right view, edge sharpness mismatch map. Red color indicates the severity of mismatch.

Image order: left view, right view, edge sharpness mismatch map. Red color indicates the severity of mismatch.

A foreground object is detected as flat and has the same depth as background.

Another example of a flat foreground object.

Visualization of crosstalk noticeability map.

Potential crosstalk noticeability analyzed in 105 movies. The metric shows what movies can be viewed on displays susceptible to crosstalk.

Results

Publications

A. Bokov et al., "Automatic detection of artifacts in converted S3D video," in Stereoscopic Displays and Applications XXV, Vol. 9011, 2014.

06 May 2020
See Also
MSU Image- and video-quality metrics analysis
Description of a project in MSU Graphics and Media Laboratory
MSU 3D-video Quality Analysis. Report 12
MSU 3D-video Quality Analysis. Report 11
MSU 3D-video Quality Analysis. Report 10
Detection of stereo window violation
How to find objects that are present only in one view?
Depth continuity estimation in S3D video
How smooth is the depth transition between scenes?
Site structure