2D to 3D video conversion

Abstract

Stereoscopic 3D visualisation is increasingly embedded into social life through the use of commercially available 3D-TV sets. In this work, a hybrid approach for 2D to 3D conversion is presented to produce stereoscopic 3D video automatically from 2D mono video frames. Each frame is synthesized to stereo pairs. Disparity/depth information required for 3D view is extracted from mono frame sequences based on motion and geometrical cues. Depth estimation of the scene is considered separately for background and foreground. Background geometry of the scene is determined by using geometrical cues such as vanishing point and straight lines in the image. According to this geometry, relevant information on the background depth eld of a single image is estimated to generate a canonic disparity map of the background. For foreground depth estimation, on the other hand, two approaches are presented. First approach is based on detection of moving foreground objects. A depth value is assigned to each object based on its corresponding location in the background depth map. In the second approach, background registration is applied for consecutive frames that are captured by a moving camera. By this method, disparity in foreground regions is distinguished from background disparity that leads to a distinctive 3D e ect on foreground regions. Consequently, depth/disparity information of foreground regions is combined with background canonic disparity map. According to these nal disparity maps, pixels of the original frames are shifted to generate virtual frames to enable 3D views. This work is accompanied by a subjective evaluation on the basis of user test which compare our 3D results with commercially available 3D-TV sets.