Académique Documents
Professionnel Documents
Culture Documents
Terminologies
Broadcasting data conversion image generation stereo displays stereo synthesis stereo vision three-dimensional Displays 3D-TV
Outline
Abstract Introduction System Design Existing System Proposed system Problem definitions Computational model Summary of characteristics of depth representation Challenges Conclusion References
Abstract
Three-dimensional television (3D-TV) is the next major revolution in television. A successful rollout of 3D-TV will require a backward-compatible transmission/distribution system, inexpensive 3D displays, and an adequate supply of high-quality 3D program material. With respect to the last factor, the conversion of 2D images/videos to 3D will play an important role. This paper provides an overview of automatic 2D-to-3D video conversion with a specific look at a number of approaches for both the extraction of depth information from monoscopic images and the generation of stereoscopic images. Some challenging issues for the success of automatic 2D-to-3D video conversion are pointed out as possible research topics for the future.
Introduction
Three-dimensional television (3D-TV) is anticipated to be the next step in the advancement of television. The term 3D in this context denotes stereoscopic, meaning a twoview system is used for visualization. The successful adoption of 3D-TV by the general public will depend not only on technological advances in 3D displays and 3D-TV broadcasting systems, but also on the availability of a wide variety of program content in stereoscopic 3D (S3D) format for 3D-TV services. The potential market is attracting many companies to invest their manpower and money for developing 2D-to-3D conversion techniques. The fundamental principle of 2D-to-3D conversion techniques rests on the fact that stereoscopic viewing involves binocular processing of two slightly dissimilar images.
Introduction (continued..)
Thus, converting 2D images to stereoscopic 3D images involves the underlying principle of horizontal shifting of pixels to create a new image, so that there are horizontal disparities between the original image and a new version of it. Various approaches for 2D-to-3D conversion have been proposed. These approaches can be classified into three schemes, namely: manual, humanassisted and automatic conversion.
System Design
Existing system
The manual scheme:
The manual scheme is to shift the pixels horizontally with an artistically chosen depth value for different regions/objects in the image to generate a new image, where hand drawing produces high quality depth, but is very time consuming and expensive.
Proposed System
Automatic 2D-to-3D Video Conversion:
To convert the vast collection of available 2D material into 3D in an economic manner, an automatic conversion scheme is desired. The automatic conversion scheme exploits depth information originated from a single image or from a stream of images to generate a new projection of the scene with a virtual camera of a slightly different (horizontally shifted) viewpoint. It may be done in real-time or in a more time-consuming off-line process. The quality of the resulting product is related to the level of the processing involved.
Problem definitions
There are two key issues to consider for automatic 2D-to-3D conversion techniques: how to retrieve depth information from a monoscopic image or video
Pictorial depth cues in a 2D image. Visible depth cues: linear perspective, relative and known size, texture gradient, atmospheric scattering, relative height in picture, and interposition.
Although the approach of recovering depth from focus/defocus is relatively simple, it suffers from a major drawback, how to distinguish the foreground from the background when the amount of blur is similar.
Depth fusion
To generate the 3D video, DIBR is used to syntheses the second view video based on the estimated depth map and the 2D video input. Depth image based rendering (DIBR) permits the creation of novel images, using information from depth maps, as if they were captured with a camera from different viewpoints. The DIBR system usually consists of three steps: i)The pre-processing ii)The 3D image warping iii)Hole filling: -> Detect holes -> Fill holes by averaging textures from neighborhood pixels -> Linear interpolation technology
2D video input
Right view
Stereoscopic image synthesis with DIBR. (a) Original color image interview, (b) correspondent depth image, and (c) rendered image without hole-filling. Holes are marked with a green color.
Major challenges in DIBR: Occlusion: Two different points in the image plane at the real view can be warped to the same location in the virtual view. To resolve this, the point with position appear closer to the camera in the virtual view will be used. Disocclusion: Occluded area in the real view may become visible in the virtual view. Disocclusion can be resolved by (1) Hole-filling and (2) Depth Map Pre-processing
Challenges
Even though much research has been done to enable automatic 2D-to-3D conversion, the techniques are still far from mature. Most available products and methods are only successful in certain circumstances. The following are some key challenging issues to be solved. One issue that directly affects the image quality is the occlusion disocclusion problem during the generation of the stereoscopic images. The depth ambiguity from monocular depth cues is one issue that impacts the depth quality. The depth ambiguity originates from the violation of the principles of depth generation. The integration of various depth cues is another issue affecting the success of automatic 2D-to-3D video conversion. The real-time implementation of 2D-to-3D conversion is also a critical issue for the adoption of the proposed techniques by the general public.
Conclusion
This paper summarized current technical advances related to the development of automatic 2D-to-3D video conversion. The underlying principle of the conversion is to horizontally shift the pixels of an original image to generate a new version of it. To enable this conversion, different approaches for the extraction of depth information from monoscopic images and the generation of stereoscopic images were reviewed. A number of challenging issues that have to be solved for the success of automatic 2D-to-3D video conversion were pointed out as possible research topics. With the development of more advanced techniques for 2D-to-3D video conversion, the vast collection of 2D material currently available will be converted into stereoscopic 3D to boost the general public interest in purchasing 3D displays and 3D-TV services.
References
IEEE TRANSACTIONS ON BROADCASTING, VOL. 57, NO. 2, JUNE 2011 S. Yano and I. Yuyama, Stereoscopic HDTV: Experimental system and psychological effects, J. SMPTE, vol. 100, pp. 1418, 1991. 3D video conversion .pdf N. S. Holliman, N. A. Dodgson, and G. Favalora, Three-dimensional display technologies: An analysis of technical performance characteristics, IEEE Trans. Broadcast., 2011, to be published. P. Harman, J. Flack, S. Fox, and M. Dowley, Rapid 2D to3D conversion, in SPIE Conf. Stereoscopic Displays and Virtual Reality Systems IX, 2002, vol. 4660, pp. 7886.