This PDF file contains the front matter associated with SPIE Proceedings Volume 7863, including the Title Page, Copyright information, Table of Contents, Introduction, and the Conference Committee listing

The 3D shape perceived from viewing a stereoscopic movie depends on the viewing conditions, most notably on the screen size and distance, and depth and size distortions appear because of the differences between the shooting and viewing geometries. When the shooting geometry is constrained, or when the same stereoscopic movie must be displayed with different viewing geometries (e.g. in a movie theater and on a 3DTV), these depth distortions may be reduced by novel view synthesis techniques. They usually involve three steps: computing the stereo disparity, computing a disparity-dependent 2D mapping from the original stereo pair to the synthesized views, and finally composing the synthesized views. In this paper, we focus on the second and third step: we examine how to generate new views so that the perceived depth is similar to the original scene depth, and we propose a method to detect and reduce artifacts in the third and last step, these artifacts being created by errors contained in the disparity from the first step.

In this paper, we proposed a visual fatigue monitoring system based on eye-movement and eye-blink detection. It analyzes the eye-movement and number of blinks based on the assumption that saccade movement of the eye decreases and the number of eye blink increases when visual fatigue of viewer is accumulated. The proposed system has an infrared single camera and an infrared light source. Then, the pupil of the eye can be detected by applying binary threshold to Purkinje image. The threshold is automatically selected by two constraints which are the eccentricity of ellipse fitting and the size of the pupil. Finally, total amount of eye movement and the number of eye blinks are measured by using the positions of the pupil. The results were obtained while watching stereoscopic videos after personal calibration procedure. The results show that saccade movement of the eye decreases as the visual fatigue of the viewer is accumulated. However, the number of eye blinks shows large variance along the time axis which implies it is not proper for visual fatigue monitoring system.

The stereoscopic 3D industry has fallen short of achieving acceptable Quality of Experience (QoE) because of various technical limitations, such as excessive disparity, accommodation-convergence mismatch. This study investigates the effect of scene content, camera baseline, screen size and viewing location on stereoscopic QoE in a holistic approach. 240 typical test configurations are taken into account, in which a wide range of disparity constructed from the shooting conditions (scene content, camera baseline, sensor resolution/screen size) was selected from datasets, making the constructed disparities locate in different ranges of maximal disparity supported by viewing environment (viewing location). Second, an extensive subjective test is conducted using a single stimulus methodology, in which 15 samples at each viewing location were obtained. Finally, a statistical analysis is performed and the results reveal that scene content, camera baseline, as well as the interactions between screen size, scene content and camera baseline, have significant impact on QoE in stereoscopic images, while other factors, especially viewing location involved, have almost no significant impact. The generated Mean Opinion Scores (MOS) and the statistical results can be used to design stereoscopic quality metrics and validate their performance.

This paper investigates visual discomfort induced by fast motion of salient object in a stereoscopic video. We have conducted a subjective assessment to investigate the degree of visual discomfort caused by motion characteristics of a controlled graphics object in a video scene. As results of the subjective assessment, we observe the changes of the degree of visual discomfort with varying velocity and direction of object motion. In order to verify the acceptability of our observation for real stereoscopic 3D videos, we exploit the concept of visual saliency to define the salient object motion severely affecting the degree of visual discomfort in a video scene. The salient object motion feature is extracted and a visual comfort model is derived from our observation. Then we predict the degree of visual discomfort by using the extracted motion feature and the visual comfort model. We have conducted a subjective test to compare the predicted visual comfort score with actual subjective score. The experiment results show that the predicted visual comfort score correlates well with the actual subject score.

One of the key issues associated with 3D TVs is the tradeoff between comfort and 3D visual impact. Big disparity is often preferred for strong visual impact but often lead to viewer discomfort depending on viewer's condition, display size and viewing distances. The algorithm proposed in this paper is to provide viewers a tool to adjust disparity according to the environment, contents and their preference in order to have more comfortable and higher quality 3D experiences. More specifically, given a planar stereoscopic display, the algorithm takes in a stereoscopic image pair that causes viewing discomfort/fatigue, and outputs a modified stereoscopic pair that causes less or no viewing discomfort/fatigue. The algorithm fulfills the functions of disparity estimation, occlusion detection, disparity adjustment and view synthesis. A novel pixel weighting mechanism in regularized-block-matching based disparity estimation helps improve the robustness, accuracy and speed of matching. Occlusion detection uses multiple cues in addition to matching errors to improve the accuracy. An accommodation/vergence mismatch visual model is used in disparity adjustment to predict discomfort/fatigue from the disparity information, the viewing conditions and display characteristics. The hole filling is in the disparity map of the new view instead of the new view itself to reduce the blurriness.

The creation of binocular images for stereoscopic display has benefited from significant research and commercial development in recent years. However, perhaps surprisingly, the effect of adding 3D sound to stereoscopic images has rarely been studied. If auditory depth information can enhance or extend the visual depth experience it could become an important way to extend the limited depth budget on all 3D displays and reduce the potential for fatigue from excessive use of disparity. Objective: As there is limited research in this area our objective was to ask two preliminary questions. First what is the smallest difference in forward depth that can be reliably detected using 3D sound alone? Second does the addition of auditory depth information influence the visual perception of depth in a stereoscopic image? Method: To investigate auditory depth cues we use a simple sound system to test the experimental hypothesis that: participants will perform better than chance at judging the depth differences between two speakers a set distance apart. In our second experiment investigating both auditory and visual depth cues we setup a sound system and a stereoscopic display to test the experimental hypothesis that: participants judge a visual stimulus to be closer if they hear a closer sound when viewing the stimulus. Results: In the auditory depth cue trial every depth difference tested gave significant results demonstrating that the human ear can hear depth differences between physical sources as short as 0.25 m at 1 m. In our trial investigating whether audio information can influence the visual perception of depth we found that participants did report visually perceiving an object to be closer when the sound was played closer to them even though the image depth remained unchanged. Conclusion: The positive results in the two trials show that we can hear small differences in forward depth between sound sources and suggest that it could be practical to extend the apparent depth in a stereoscopic image by using 3D sound, providing a controlled way to compensate for the depth budget limits on 3D displays.

The perception of space in the real world is based on multifaceted depth cues, most of them monocular, some binocular. Developing 3D-displays raises the question, which of these depth cues are predominant and should be simulated by computational means in such a panel. Beyond the cues based on image content, such as shadows or patterns, Stereopsis and depth from motion parallax are the most significant mechanisms supporting observers with depth information. We set up a carefully designed test situation, widely excluding undesired other distance hints. Thereafter we conducted a user test to find out, which of these two depth cues is more relevant and whether a combination of both would increase accuracy in a depth estimation task. The trials were conducting utilizing our autostereoscopic "Free2C"-displays, which are capable to detect the user eye position and steer the image lobes dynamically into that direction. At the same time, eye position was used to update the virtual camera's location and thereby offering motion parallax to the observer. As far as we know, this was the first time that such a test has been conducted using an autosteresocopic display without any assistive technologies. Our results showed, in accordance with prior experiments, that both cues are effective, however Stereopsis is by order of magnitude more relevant. Combining both cues improved the precision of distance estimation by another 30-40%.

This paper describes an algorithm for estimating the disparity between 2 images of a stereo pair. The disparity is related to the depth of the objects in the scene. Being able to obtain the depth of the objects in the scene is useful in many applications such as virtual reality, 3D user interfaces, background-foreground segmentation, or depth-image-based synthesis. This last application has motivated the proposed algorithm as part of a system that estimates disparities from a stereo pair and synthesizes new views. Synthesizing virtual views enables the post-processing of 3D content to adapt to user preferences or viewing conditions, as well as enabling the interface with multi-view auto-stereoscopic displays. The proposed algorithm has been designed to fulfill the following constraints: (a) low memory requirements, (b) local and parallelizable processing, and (c) adaptability to a sudden reduction in processing resources. Our solution uses a multi-resolution multi-size-windows approach, implemented as a line-independent process, well-suited for GPU implementation. The multi-resolution approach provides adaptability to sudden reduction in processing capabilities, besides computational advantages; the windows-based image processing algorithm guarantees low-memory requirements and local processing.

Interactive free-viewpoint selection applied to a 3D multi-view video signal is an attractive feature of the rapidly developing 3DTV media. In recent years, significant research has been done on free-viewpoint rendering algorithms which mostly have similar building blocks. In our previous work, we have analyzed the principal building blocks of most recent rendering algorithms and their contribution to the overall rendering quality. We have discovered that the first step, Warping determines the basic quality level of the complete rendering chain. In this paper, we have analyzed the warping step in more detail since it leads to ways for improvement. We have observed that the accuracy of warping is mainly determined by two factors which are sampling and rounding errors when performing pixel-based warping and quantization errors of depth maps. For each error factor, we have proposed a technique that can reduce the errors and thus increase the warping quality. Pixel-based warping errors are reduced by employing supersampling at the reference and virtual images and we decrease depth map errors by creating depth maps with more quantization levels. The new techniques are evaluated with two series of experiments using real-life and synthetic data. From these experiments, we have observed that reducing warping errors may increases the overall rendering quality and that the impact of errors due to pixel-based warping is much larger than that of errors due to depth quantization.

We are developing technologies for FTV in which the viewer can freely change the viewpoint. The free-viewpoint image can be generated by using images captured by an static multi-camera system. However, it is hard to render an object that moves widely in the scene. In this paper, we address this problem by proposing moving camera array and the free-viewpoint image synthesis algorithm. In our synthesis method, we use the temporal and spatial information together, in order to further improve the view generation quality. Experiments using a sequence captured by simulated moving multi-camera systems demonstrate the improvement of view synthesis quality in comparison with conventional view synthesis scheme.

3D Video (3DV) delivery standardization is currently ongoing in MPEG. Now time is to choose 3DV data representation format. What is at stake is the final quality for end-users, i.e. synthesized views' visual quality. We focus on two major rival depth-based formats, namely Multiview Video plus Depth (MVD) and Layered Depth Video (LDV). MVD can be considered as the basic depth-based 3DV format, generated by disparity estimation from multiview sequences. LDV is more sophisticated, with the compaction of multiview data into color- and depth-occlusions layers. We compare final views quality using MVD2 and LDV (both containing two color channels plus two depth components) coded with MVC at various compression ratios. Depending on the format, the appropriate synthesis process is performed to generate final stereoscopic pairs. Comparisons are provided in terms of SSIM and PSNR with respect to original views and to synthesized references (obtained without compression). Eventually, LDV outperforms significantly MVD when using state-of-the-art reference synthesis algorithms. Occlusions management before encoding is advantageous in comparison with handling redundant signals at decoder side. Besides, we observe that depth quantization does not induce much loss on the final view quality until a significant degradation level. Improvements in disparity estimation and view synthesis algorithms are therefore still expected during the remaining standardization steps.

In order to compress the multiview image efficiently, we propose a new basis representation by using directional sampling.Because the multiview image has the static correlation related to the camera position, directional sampling can eliminate the correlation efficiently. We apply directional three dimensional Discrete Cosine Transform (directional 3D-DCT), directional three dimensional Discrete Wavelet Transform (directional 3D-DWT) and directional three dimensional Principal Component Analysis (directional 3D-PCA) to three dimensional data that has vertical, horizontal and view axis. In the experimental results, the method by using directional sampling showed better quality than conventional sampling, not only in the objective evaluation, but also in the subjective evaluation. Furthermore, the proposed method showed better Rate-PSNR than conventional method (disparity compensation).

Multiview displays suffer from two common artifacts - Moiré, caused by aliasing, and ghosting artifacts caused by crosstalk. By measuring the angular brightness function of each TFT element we create so-called brightness mask, which allows us to simulate the display output for a given input image. We consider multiview display as image processing channel and model the artifacts as distortions of the input signal. We test the channel by using a set of signals with various frequency components as input, and analyzing the output in the frequency domain. We derive the so-called bandpass region of the display, where the distortions introduced to the input signals are under certain threshold. Then, we extend the simulations including input signals with varying disparity, and obtain multiple passbands - one for each disparity level. We approximate each passband with a rectangle and store the height and width of that rectangle in a table. We propose an artifact mitigation framework which can be used for realtime processing of textures with known apparent depth. The framework gives the user ability to set so-called "3D-shapness" - a parameter, which controls the trade-off between visibility of details and presence of artifacts. The "3D-sharpness parameter determines what level of distortions is allowed in the final image, regardless of its disparity. The framework uses the approximated width and height of the passband areas in order to design an optimal (for the needed disparity and desired distortion level) anti-aliasing filter. We discuss a methodology for filter design, and show example results, based on measurements of an 8-view display.

In recent years, we have seen several different approaches dealing with multiview compression. First, we can find the H264/MVC extension which generates quite heavy bitstreams when used on n-views autostereoscopic medias and does not allow inter-view reconstruction. Another solution relies on the MVD (MultiView+Depth) scheme which keeps p views (n > p > 1) and their associated depth-maps. This method is not suitable for multiview compression since it does not exploit the redundancy between the p views, moreover occlusion areas cannot be accurately filled. In this paper, we present our method based on the LDV (Layered Depth Video) approach which keeps one reference view with its associated depth-map and the n-1 residual ones required to fill occluded areas. We first perform a global per-pixel matching step (providing a good consistency between each view) in order to generate one unified-color RGB texture (where a unique color is devoted to all pixels corresponding to the same 3D-point, thus avoiding illumination artifacts) and a signed integer disparity texture. Next, we extract the non-redundant information and store it into two textures (a unified-color one and a disparity one) containing the reference and the n-1 residual views. The RGB texture is compressed with a conventional DCT or DWT-based algorithm and the disparity texture with a lossless dictionary algorithm. Then, we will discuss about the signal deformations generated by our approach.

We consider the problem of upsampling a low-resolution depth map generated by a range camera, by using information from one or more additional high-resolution vision cameras. The goal is to provide an accurate high resolution depth map from the viewpoint of one of the vision cameras. We propose an algorithm that first converts the low resolution depth map into a depth/disparity map through coordinate mappings into the coordinate frame of one vision camera, then classifies the pixels into regions according to whether the range camera depth map is trustworthy, and finally refine the depth values for the pixels in the untrustworthy regions. For the last refinement step, both a method based on graph cut optimization and that based on bilateral filtering are examined. Experimental results show that the proposed methods using classification are able to upsample the depth map by a factor of 10 x 10 with much improved depth details, with significantly better accuracy comparing to those without the classification. The improvements are visually perceptible on a 3D auto-stereoscopic display.

Attack! of the S. Mutans is a multi-player game designed to harness the immersion and appeal possible with wide-fieldof- view stereoscopic 3D to combat the tooth decay epidemic. Tooth decay is one of the leading causes of school absences and costs more than $100B annually in the U.S. In 2008 the authors received a grant from the National Institutes of Health to build a science museum exhibit that included a suite of serious games involving the behaviors and bacteria that cause cavities. The centerpiece is an adventure game where five simultaneous players use modified Wii controllers to battle biofilms and bacteria while immersed in environments generated within a 11-foot stereoscopic WUXGA display. The authors describe the system and interface used in this prototype application and some of the ways they attempted to use the power of immersion and the appeal of S3D revolution to change health attitudes and self-care habits.

Multi-perspective image acquisition constitutes a novel method of documenting the entireness of an event, by recording from multiple points of view. In combination with stereoscopic imaging, it captures two modalities of three-dimensional representation, perspective and depth. These modalities are then applied for display in a hexagonal stereoscopic multi-screen projection environment, the RE-ACTOR platform. In this paper the author describes, in form of two case studies, the implementation of two projects for the platform within the performing art context.

Purpose: Machine vision with 3DHD video theoretically offers resolving capacity approaching that of the operating microscope and may have applications in vitreoretinal surgery. Methods: 9 eyes of 9 patients were operated. Surgical visualization was provided by one of two 3DHD video systems. The surgeon did not look through the operating microscope at any time from the beginning to the end of the surgical procedure. Results: 9 pars plana vitrectomies, 8 epiretinal membrane peels, 7 ILM peels, 3 cataract surgeries, one air fluid exchange, and one retrieval of a luxed intraocular lens implant from the voitreous cavity were performed successfully. Anatomic and visual results were excellent. No complications occurred. Surgeon comfort during surgery was superb. Conclusions: 3DHD machine vision may be utilized for surgical visualization in select vitreoretinal patients. The implications and applications of this new technology remain unknown.

In this study, we proposed methods to reduce a black-white (BW) 3D crosstalk and gray-to-gray (GtoG) 3D crosstalk in active retarder 3D (AR3D) technology. To obtain the BW 3D crosstalk to 2.5% without backlight scanning, we first optimized the switching time of AR panel. Then, the BW 3D crosstalk of 1.0% was achieved by scanning the backlight synchronized with the liquid crystal display (LCD) panel and AR panel. Finally, with the over-driving method for various condition of gray-to-gray transition, the average GtoG 3D crosstalk was obtained to be under 1.8% showing clear 3D images. With these concepts of reducing the 3D crosstalk in AR3D technology, we developed AR3D monitor having 23 inches in diagonal with full HD resolution. The luminance of our prototype is 73 nits at 3D mode through polarizer glasses and 300 nits at 2D mode for bare eyes, showing that 24% light efficiency in 3D mode. In conclusion, our prototype shows clear 3D image with full HD resolution and high luminance even though it employs polarizer glasses.

A stereoscopic 35mm motion picture projection system is described consisting of a sideframe format with high effective frame area utilization and high brightness optics. The system has been designed to work within the existing motion picture content creation and distribution infrastructures.

A number of Pulfrich 3-D movies and TV shows have been produced, but the standard implementation has inherent drawbacks. The movie and TV industries have correctly concluded that the standard Pulfrich 3-D implementation is not a useful 3-D technique. Continuously Adjustable Pulfrich Spectacles (CAPS) is a new implementation of the Pulfrich effect that allows any scene containing movement in a standard 2-D movie, which are most scenes, to be optionally viewed in 3-D using inexpensive viewing specs. Recent scientific results in the fields of human perception, optoelectronics, video compression and video format conversion are translated into a new implementation of Pulfrich 3- D. CAPS uses these results to continuously adjust to the movie so that the viewing spectacles always conform to the optical density that optimizes the Pulfrich stereoscopic illusion. CAPS instantly provides 3-D immersion to any moving scene in any 2-D movie. Without the glasses, the movie will appear as a normal 2-D image. CAPS work on any viewing device, and with any distribution medium. CAPS is appropriate for viewing Internet streamed movies in 3-D.

Prolonged use of conventional stereo displays causes viewer discomfort and fatigue because of the vergenceaccommodation conflict. We used a novel volumetric display to examine how viewing distance and the sign of the vergence-accommodation conflict affect discomfort and fatigue. In the first experiment, we presented a fixed conflict at short, medium, and long viewing distances. We compared subjects' symptoms in that condition and one in which there was no conflict. We observed more discomfort and fatigue with a given vergence-accommodation conflict at the longer distances. The second experiment compared symptoms when the conflict had one sign compared to when it had the opposite sign at short, medium, and long distances. We observed greater symptoms with uncrossed disparities at long distances and with crossed disparities at short distances. These findings help define comfortable viewing conditions for stereo displays.

Vergence and accommodative responses in viewing near objects in the real world are behaviorally coupled to maintain clarity and singularity for the object of regard. However virtual stereoscopic stimuli, such as in 3D displays, create non-normal coupling that may cause improper vergence and accommodative responses, possibly resulting in visual discomfort. The present study examined whether the dynamic aspect of current 3D displays is the underlying cause of visual and physical discomfort. To this end, subjects' vergence and accommodative responses were measured while they tracked an approaching 2D or 3D target, and while watching a 2D or 3D movie. The tracking target either moved in steps or continuously, and it was either clear or intentionally blurred. Results show that convergence insufficiency and improper accommodation were greater when a 3D target was moving continuously toward the near position compared to a 2D target and a 3D stimulus moving in steps. Clear targets also resulted in greater vergence and accommodative responses than blurred targets. Viewing 3D movie resulted in greater vergence and accommodation, as well as more severe vision- and motion-related discomfort than 2D movie. These findings suggest that with 3D displays, disparity-induced vision difficulty and internal conflicts cause perceived visual and motion-related discomfort. Characteristics of 3D stimuli, such as the frequency and amplitude of target motion, likely critically affect the severity of reported discomfort symptoms.

Camera separation affects the perceived depth in stereoscopic movies. Through control of the separation and thereby the depth magnitudes, the movie can be kept comfortable but interesting. In addition, the viewing context has a significant effect on the perceived depth, as a larger display and longer viewing distances also contribute to an increase in depth. Thus, if the content is to be viewed in multiple viewing contexts, the depth magnitudes should be carefully planned so that the content always looks acceptable. Alternatively, the content can be modified for each viewing situation. To identify the significance of changes due to the viewing context, we studied the effect of stereoscopic camera base distance on the viewer experience in three different situations: 1) small sized video and a viewing distance of 38 cm, 2) television and a viewing distance of 158 cm, and 3) cinema and a viewing distance of 6-19 meters. We examined three different animations with positive parallax. The results showed that the camera distance had a significant effect on the viewing experience in small display/short viewing distance situations, in which the experience ratings increased until the maximum disparity in the scene was 0.34 - 0.45 degrees of visual angle. After 0.45 degrees, increasing the depth magnitude did not affect the experienced quality ratings. Interestingly, changes in the camera distance did not affect the experience ratings in the case of television or cinema if the depth magnitudes were below one degree of visual angle. When the depth was greater than one degree, the experience ratings began to drop significantly. These results indicate that depth magnitudes have a larger effect on the viewing experience with a small display. When a stereoscopic movie is viewed from a larger display, other experiences might override the effect of depth magnitudes.

High-resolution stereoscopic images are effective for use in virtual reality and teleoperation systems. However, the higher the image resolution, the higher is the cost of computer processing and communication. To reduce this cost, numerous earlier studies have suggested the use of multi-resolution images, which have high resolution in region of interests and low resolution in other areas. However, observers can perceive unpleasant sensations and incorrect depth because they can see low-resolution areas in their field of vision. In this study, we conducted an experiment to research the relationship between the viewing field and the perception of image resolution, and determined respective thresholds of image-resolution perception for various positions of the viewing field. The results showed that participants could not distinguish between the high-resolution stimulus and the decreased stimulus, 63 ppi, at positions more than 8 deg outside the gaze point. Moreover, with positions shifted a further 11 and 13 deg from the gaze point, participants could not distinguish between the high-resolution stimulus and the decreased stimuli whose resolution densities were 42 and 25 ppi. Hence, we will propose the composition of multi-resolution images in which observers do not perceive unpleasant sensations and incorrect depth with data reduction (compression).

A projector array-based 70-inch screen display, which is our first prototype, has a smooth horizontal parallax and gives a dense viewpoint interval that is narrower than half of the interocular distance. Our final goal is to develop advanced autostereoscopy so that viewers are not compelled to wear 3D glasses and can avoid watching under insufficient resolution. We believe that larger screen size, higher image quality, and such natural image appearances as motion parallax and multiple viewer capability are priority targets for professional 3D display applications. By combining a proprietary screen and our developed projector array, we've designed and implemented a kind of autostereoscopic projection display. Enough pixels to render true high definition are assigned for every viewpoint. The initial implementation has more than 100 million pixels. The actual observed horizontal motion parallax is smooth and reduces flipping. This feasibility study clarified the following two factors: the strong requirement of an array friendly feature ready projector, and the existence of some image glitches. The appearances of moires and ghost images are the most significant factors of visual fatigue in our implementation. Some of these problems were tackled and suppressed. The projectors for the array must be prepared to manage color space, brightness, geometric image compensation, and accurate frame synchronization. Extracting and examining the practical problems with an autostereoscopic projection display are the first steps of our feasibility study. Our goal is to establish an autostereoscopic display with natural and superior horizontal parallax.

We propose full-parallax integral imaging display with 360 degree horizontal viewing angle. Two-dimensional (2D) elemental images are projected by a high-speed DMD projector and integrated into three-dimensional (3D) image by a lens array. The anamorphic optic system tailors the horizontal and vertical viewing angles of the integrated 3D images in order to obtain high angular ray density in horizontal direction and large viewing angle in vertical direction. Finally, the mirror screen that rotates in synchronization with the DMD projector presents the integrated 3D images to desired direction accordingly. Full-parallax and 360 degree horizontal viewing angle 3D images with both of monocular and binocular depth cues can be achieved by the proposed method.

With the rapid emergence of autosterereoscopic 3D (AS3D) displays in the market the need for optical characterization of these devices and their components is increasing. This paper reviews various performance attributes of AS3D displays such as viewing distance, banding offset, banding angle and stereoedge, and reviews measurement results from a parallax barrier, lenticular and directional back light approach from both conventional measurement techniques and a new instrument, called the optical inspection station (OIS).

We propose Depth Cube Display (DCD) method using depth map. The structure of the proposed method consists of two parts: A projection part composed of projector for generating image and a Twisted Nematic Liquid Crystal display (TNLCD) as polarization modulating device for adjusting the proper depth and a display part composed of air-spaced stack of selective scattering polarizers which make the incident light to scatter selectively as the polarization of light rays. The image from projector whose depth is determined as passing through the TN-LCD displaying depth map progresses into the stack of selective scattering polarizers and then three-dimensional image is generated. At that time, the polarization of each polarizer is set 0°, 45° and 90° sequentially, and then the incident light rays are scattered by different polarizer as the polarization of these rays. If the light ray has the polarization between those of polarizers, this light ray is scattered by multi polarizers and the image of this ray is generated on gap between polarizers. The proposed method is more simple structure and implemented easily than previous DCD method.

In this paper, we propose a novel method of representing the complex surface of a 3D object for a new aerial 3D display which can draw dots of light at an arbitrary position in a space. The aerial 3D display that we use in this research can create a dot of light at 50 kHz and can draw dots of light by vector scanning. The proposed method can generate point sequence data for the aerial 3D display from 3D surface models consisting of polygonal patches. The 3D surface model is polygonal model which are generally used in computer graphics. The proposed method represents the surface with contours consisting of intersections of an object and cross sections by a sequence of points for vector scanning. In this research, some polygonal models, for example face and hand, are examined at experiments. From the experiments of drawing, the polygonal models can successfully be drawn by the proposed method.

Crosstalk is a critical factor determining the image quality of stereoscopic displays. Also known as ghosting or leakage, high levels of crosstalk can make stereoscopic images hard to fuse and lack fidelity; hence it is important to achieve low levels of crosstalk in the development of high-quality stereoscopic displays. In the wider academic literature, the terms crosstalk, ghosting and leakage are often used interchangeably and unfortunately very few publications actually provide a descriptive or mathematical definition of these terms. Additionally the definitions that are available are sometimes contradictory. This paper reviews how the terms crosstalk, ghosting and associated terms (system crosstalk, viewer crosstalk, gray-to-gray crosstalk, leakage, extinction and extinction ratio, and 3D contrast) are defined and used in the stereoscopic literature. Both descriptive definitions and mathematical definitions are considered.

Crosstalk (also known as "ghosting", "leakage", or "extinction"), a vitally important concept in stereoscopic 3D displays, has not been clearly defined or measured in the stereoscopic literature (Woods). In this paper, a mathematical definition is proposed which uses a "physical" approach. This derivation leads to a clear definition of leftview or right-view crosstalk and shows that 1), when the display's black level is not zero, it must be subtracted out and 2), when the source intensities are equal, crosstalk can be measured using observed intensities totally within the respective view. Next, a simple method of measuring crosstalk is presented, one that relies on only viewing a test chart on the display. No electronic or optical instruments are needed. Results of the use of the chart are presented, as well as optical measurements, which did not agree well with chart results. The main reason for the discrepancy is the difficulty of measuring very low light levels. With wide distribution, this tool can lead to the collection of useful performance information about 3D displays and, therefore, to the production of the best stereoscopic displays.

Crosstalk is a phenomenon in stereoscopy where an image becomes blurry due to leakage of the left image into the right eye and vice versa, and is considered one of the serious problems impairing stereoscopic experience. The current study examines mental/cognitive activity under a various levels of crosstalk through heart activity and forehead blood flow. In the experiment that presented three still natural images and one graphical video with a various crosstalk levels, heart rate showed a decelerative-accelerative-decelerative pattern for all the stimuli up to the intolerably severe level. The result suggests changes in mental state in accordance to the crosstalk level: i.e. orientation response under no perceived crosstalk, active mental elaboration upon noticing crosstalk, and reduced level of elaboration as crosstalk progressed. The pattern, however, did not always agree amongst the physiological measures and the crosstalk ratios. This suggests that the mental state under crosstalked image viewing could be more complex than a simple combination of orientation response and active mental elaboration.

A method to characterize time sequential stereoscopic 3D displays which is based on the measurement of the temporal behavior of the system versus grey levels is presented. OPTISCOPE SA, especially designed for precise measurement of luminance and temporal behavior of LCD displays is used. The transmittance and response time of the shutter glasses is first evaluated. Then the grey to grey response times of the display alone are measured. Finally, the temporal behavior of the complete system is modeled and grey to grey crosstalks are deduced. Resulting luminance on left and right eyes of the observer are deduced and compared to integrated measurements using a spectrophotometer. Quality control method using grey level test patterns and imaging luminance meter is deduced and applied to different types of commercial 3D TVs.

Stereoscopic displays must present separate images to the viewer's left and right eyes. Crosstalk is the unwanted contamination of one eye's image from the image of the other eye. It has been shown to cause distortions, reduce image quality and visual comfort and increase perceived workload when performing visual tasks. Crosstalk also affects one's ability to perceive stereoscopic depth although little consideration has been given to the perception of depth magnitude in the presence of crosstalk. In this paper we extend a previous study (Tsirlin, Allison & Wilcox, 2010, submitted) on the perception of depth magnitude in stereoscopic occluding and non-occluding surfaces to the special case of crosstalk in thin structures. Crosstalk in thin structures differs qualitatively from that in larger objects due to the separation of the ghost and real images and thus theoretically could have distinct perceptual consequences. To address this question we used a psychophysical paradigm, where observers estimated the perceived depth difference between two thin vertical bars using a measurement scale. Our data show that crosstalk degrades perceived depth. As crosstalk levels increased the magnitude of perceived depth decreased, especially for stimuli with larger relative disparities. In contrast to the effect of crosstalk on depth magnitude in larger objects, in thin structures, a significant detrimental effect was found at all disparities. Our findings, when considered with the other perceptual consequences of crosstalk, suggest that its presence in S3D media even in modest amounts will reduce observers' satisfaction.

The present study investigates whether VIMS, which can be induced in 2D images, is affected by stereoscopic presentation. To do this, we conducted an experiment to measure the effects psychologically and physiologically. Visual stimulus was computer graphics that simulates traveling along streets with additional pitch and roll motion for 10 minutes. The stimulus were presented as either stereoscopic, "3D", images or "2D" images. Before/after and during each trial, psychological and physiological measurements for biomedical effects were conducted. As results, psychological measurements indicate effects of stereoscopic presentations on VIMS. First, subjective score of comfort level measured every one minute significantly decreased to uncomfortable level in the 3D than in the 2D condition. Second, subscore of "Nausea" of Simulator Sickness Questionnaire significantly higher in the 3D than in the 2D condition, while the other subscores and the total score also showed the similar tendency. Moreover, physiological measurements also indicate effects of 3D presentations on VIMS. The LF/HF ratio, which is the index of sympathetic nerve activity, clearly increased more in the 3D than in the 2D condition. We conclude that stereoscopic presentation enhances biomedical effects of VIMS. We speculate that stereoscopic images can be efficient reference of spatial orientation.

Conventional stereoscopic displays present images on a single focal plane. The resulting mismatch between the stimuli to the eyes' focusing response (accommodation) and to convergence causes fatigue and poor stereo performance. One promising solution is to distribute image intensity across a number of relatively widely spaced image planes - a technique referred to as depth filtering. Previously, we found this elicits accurate, continuous monocular accommodation responses with image-plane separations as large as 1.1 Diopters, suggesting that a relatively small (i.e. practical) number of image planes is sufficient to eliminate vergence-accommodation conflicts over a large range of simulated distances. However, accommodation responses have been found to overshoot systematically when the same stimuli are viewed binocularly. Here, we examined the minimum image-plane spacing required for accurate accommodation to binocular depth-filtered images. We compared accommodation and vergence responses to step changes in depth for depth-filtered stimuli, using image-plane separations of 0.6-1.2 D, and equivalent real stimuli. Accommodation responses to real and depth-filtered stimuli were equivalent for image-plane separations of ~0.6-0.9 D, but inaccurate thereafter. We conclude that depth filtering can be used to precisely match accommodation and vergence demand in a practical stereoscopic display, using a relatively small number of image planes.

Stereoscopic displays are known to offer a number of key advantages in visualizing complex 3D structures or datasets. The large majority of studies that focus on evaluating stereoscopic displays for professional applications use completion time and/or the percentage of correct answers to measure potential performance advantages. However, completion time and accuracy may not fully reflect all the benefits of stereoscopic displays. In this paper, we argue that perceived workload is an additional valuable indicator reflecting the extent to which users can benefit from using stereoscopic displays. We performed an experiment in which participants were asked to perform a visual path-tracing task within a convoluted 3D wireframe structure, varying in level of complexity of the visualised structure and level of disparity of the visualisation. The results showed that an optimal performance (completion time, accuracy and workload), depend both on task difficulty and disparity level. Stereoscopic disparity revealed a faster and more accurate task performance, whereas we observed a trend that performance on difficult tasks stands to benefit more from higher levels of disparity than performance on easy tasks. Perceived workload (as measured using the NASA-TLX) showed a similar response pattern, providing evidence that perceived workload is sensitive to variations in disparity as well as task difficulty. This suggests that perceived workload could be a useful concept, in addition to standard performance indicators, in characterising and measuring human performance advantages when using stereoscopic displays.

We compare two different modes of interaction with a large stereoscopic display, where the physical pointing device is in a volume distinct from the display volume. In absolute mode, the physical pointer's position exactly maps to the virtual pointer's position in the display volume, analogous to a 2D graphics table and 2D screen. In relative mode, the connection between the physical pointer's motion and the motion of the virtual pointer in the display volume is analogous to that obtained with a 2D mouse and 2D screen. Both statistical analysis and participants' feedback indicated a strong preference for absolute mode over relative mode. This is in contrast to 2D displays where relative mode (mouse) is far more prevalent than absolute mode (tablet). We also compared head-tracking against no head-tracking. There was no statistically-significant advantage to using head-tracking, however almost all participants strongly favoured head-tracking.

A plenoptic camera is a natural multi-view acquisition device also capable of measuring distances by correlating a set of images acquired under different parallaxes. Its single lens and single sensor architecture have two downsides: limited resolution and depth sensitivity. In a very first step and in order to circumvent those shortcomings, we have investigated how the basic design parameters of a plenoptic camera optimize both the resolution of each view and also its depth measuring capability. In a second step, we built a prototype based on a very high resolution Red One® movie camera with an external plenoptic adapter and a relay lens. The prototype delivered 5 video views of 820x410. The main limitation in our prototype is view cross talk due to optical aberrations which reduce the depth accuracy performance. We have simulated some limiting optical aberrations and predicted its impact on the performances of the camera. In addition, we developed adjustment protocols based on a simple pattern and analyzing programs which investigate the view mapping and amount of parallax crosstalk on the sensor on a pixel basis. The results of these developments enabled us to adjust the lenslet array with a sub micrometer precision and to mark the pixels of the sensor where the views do not register properly.

This paper presents a geometric and subjective analysis of stereoscopic versions of close range I3A clusters (Subject- Camera distances below 5 m). The geometry of the stereoscopic pipeline from the scene to the viewer's eyes is a very relevant issue in stereoscopic media. One important factor is the camera separation, because it can be used to control the perceived depth of stereoscopic images. The computational camera separations were compared to subjectively preferred camera separations. Participants evaluated the strength and naturalness of depth sensation and overall viewing experience from the still images with single-stimulus method. Results showed that participants were able to perceive the change of depth range even though the images were shown in random order without a reference depth scale. A mild depth sensation was preferred over strong depth sensations. The computational camera separation differed from the subjectively preferred camera separation when the depth range of the scene was narrow. This result indicates that scenes with narrow depth should not be imaged with a long camera separation just to fill the depth budget of the display.

Unlike the real world, stereoscopic cinemas and display devices have a bordered frame which can unnaturally cut off our view, and create conflicting visual cues. These perceptual conflicts can diminish the 3-D effect and cause visual fatigue for the audience. Presented is a method to resolve these issues, by developing a controllable articulated 3D border.

As the production of stereoscopic content increases, so does the need for post-production tools for that content. Video inpainting has become an important tool for rig removal but there has been little consideration of the problem in stereo. This paper presents an algorithm for stereo video inpainting that builds on existing exemplar-based video completion and also considers the issues of view consistency. Given user selected regions in the sequence which may be in the same location in several frames and in both views, the objective is to fill in this area using all the available picture information. Existing algorithms lack temporal consistency, causing flickering and other artefacts. This paper explores the use of long-term picture information across many frames in order to achieve temporal consistency at the same time as exploiting inter-view dependencies within the same framework.

'View plus depth' is an attractive compact representation format for 3D video compression and transmission. It combines 2D video with depth map sequence aligned in a per-pixel manner to represent the moving 3D scene in interest. Any different-perspective view can be synthesized out if this representation through Depth-Image Based Rendering (DIBR). However, such rendering is prone to disocclusion errors: regions originally covered by foreground objects become visible in the synthesized view and have to be filled with perceptually-meaningful data. In this work, a technique for reducing the perceived artifacts by inpainting the disoccluded areas is proposed. Based on Criminisi's exemplar-based inpainting algorithm, the developed technique recovers the disoccluded areas by using pixels of similar blocks surrounding it. In the original work, a moving window is centered on the boundaries between known and unknown parts ('target window'). The known pixels are used to select windows which are most similar to the target one. When this process is completed, the unknown region of the target patch is filled with a weighted combination of pixels from the selected windows. In the proposed scheme, the priority map, which defines the rule for selecting the order of pixels to be filled, has been modified to meet the requirement for disocclusion hole filling and a better non-local mean estimate has been suggested accordingly. Furthermore, the search for similar patches has also been extended to previous and following frames of the video under processing, thus improving both computational efficiency and resulting quality.

This paper presents a study on the stereoscopic 3D codec for non-real-time 3DTV services. Delivering stereoscopic 3DTV contents via broadcast networks can be based on either real-time or non-real-time scenarios. For a digital television system where its bandwidth is somewhat limited to accommodate the full 3D HD quality video, a complementary enabler is the non-real-time delivery in which 3D video component is downloaded in advance. In the case of ATSC system, MPEG-2, H.264/AVC and any other codec can be used for stereoscopic video based on Non- Real-Time services. In order to put in the best performance, the analysis on the stereoscopic codec, scenarios and preliminary considerations are required. In this paper, the NRT 3D codec scenarios and preliminary considerations are addressed to develop the prerequisites required for the analysis on NRT 3D codec. The experimentation of both independent and inter-view coding are covered to investigate the codec combination for NRT 3D video services based on considerations and scenarios.

Different compression formats for stereo and multiview based 3D video is being standardized and software players capable of decoding and presenting these formats onto different display types is a vital part in the commercialization and evolution of 3D video. However, the number of publicly available software video players capable of decoding and playing multiview 3D video is still quite limited. This paper describes the design and implementation of a GPU-based real-time 3D video playback solution, built on top of cross-platform, open source libraries for video decoding and hardware accelerated graphics. A software architecture is presented that efficiently process and presents high definition 3D video in real-time and in a flexible manner support both current 3D video formats and emerging standards. Moreover, a set of bottlenecks in the processing of 3D video content in a GPU-based real-time 3D video playback solution is identified and discussed.

Observation and interviews with people viewing autostereoscopic 3D imagery provides evidence that there are many human perception considerations required for 3D content creation. A study was undertaken whereby it was witnessed that certain test autostereoscopic imagery elicited a highly emotional response and engagement, while other test autostereoscopic imagery was given only a passing glance. That an image can be viewed with a certain level of stereopsis does not make it compelling. By taking into consideration the manner in which humans perceive depth and the space between objects, 3D content can achieve a level of familiarity and realness that is not possible with single perspective imagery. When human perception issues are ignored, 3D imagery can be undesirable to viewers and a negative bias against 3D imagery can occur. The preparation of 3D content is more important than the display technology. Where human perception, as it is used to interpret reality, is not mimicked in the creation of 3D content, the general public typically express a negative bias against that imagery (where choices are provided). For some, the viewing of 3D content that could not exist naturally, induces physical discomfort.

Many 3D formats exist and will probably co-exist for a long time even if 3D standards are today under definition. The support for multiple 3D formats will be important for bringing 3D into home. In this paper, we propose a novel and effective method to detect whether a video is a 3D video or not, and to further identify the exact 3D format. First, we present how to detect those 3D formats that encode a pair of stereo images into a single image. The proposed method detects features and establishes correspondences between features in the left and right view images, and applies the statistics from the distribution of the positional differences between corresponding features to detect the existence of a 3D format and to identify the format. Second, we present how to detect the frame sequential 3D format. In the frame sequential 3D format, the feature points are oscillating from frame to frame. Similarly, the proposed method tracks feature points over consecutive frames, computes the positional differences between features, and makes a detection decision based on whether the features are oscillating. Experiments show the effectiveness of our method.

3D film and 3D TV are becoming reality. More facilities and devices are now 3D capable. Compared to capture 3D video content directly, 2D to 3D video conversion is a low-cost, backward compatible alternate. There also exists a tremendous amount of monoscopic 2D video content that are of high interest to be displayed on 3D devices with noticeable immersiveness. 2D to 3D video conversion, therefore, has drawn lots of attention recently. In this paper, a low complexity 2D to 3D conversion algorithm is presented. The conversion generates stereo video pairs by 3D warping based on estimated per-pixel depth maps. The depth maps are estimated jointly by motion and color cues. Subjective tests show that the proposed algorithm achieves 3D perception with acceptable artifact.

The increasing popularity for stereoscopic content in the entertainment industry and computer graphics applications and the availability of affordable capture and display systems is in contrast to the actual knowledge of underlying stereoscopic design principles and fundamental concepts. Content creators and educators inexperienced in stereoscopy require integrated, easy to use and flexible tools which can assist in the process of creating the three-dimensional "look" they are after within the limits of a comfortable viewing experience. The proposed framework in this paper, a custom stereoscopic export plug-in for the popular 3D modeling application Google Sketchup and a flexible stereoscopic format conversion and display engine, allows for stereoscopic previsualisation in near real-time in a format of choice. The user interface can recommend stereoscopic settings according to the scene, camera and display properties, calculates corresponding values according to manual entries but also leaves unrestricted control over all parameters. The display engine allows for different stereoscopic formats to be shown and saves the result in form of images with metadata for reference. Particular attention is put on usability, accessibility and tight integration.

The relationship between color and lightness appearance and the perception of depth has been studied since a while in the field of perceptual psychology and psycho-physiology. It has been found that depth perception affects the final object color and lightness appearance. In the stereoscopy research field, many studies have been proposed on human physiological effects, considering e.g. geometry, motion sickness, etc., but few has been done considering lightness and color information. Goal of this paper is to realize some preliminar experiments in Virtual Reality in order to determine the effects of depth perception on object color and lightness appearance. We have created a virtual test scene with a simple 3D simultaneous contrast configuration. We have created three different versions of this scene, each with different choices of relative positions and apparent size of the objects. We have collected the perceptual responses of several users after the observation of the test scene in the Virtual Theater of the University of Milan, a VR immersive installation characterized by a semi-cylindrical screen that covers 120° of horizontal field of view from an observation distance of 3.5 m. We present a description of the experiments setup and procedure, and we discuss the obtained results.

Coarse integral volumetric imaging (CIVI) combines multiview and volumetric display solutions and presents undistorted floating 3D image by correcting distortion of volumetric image for each view. In the conventional CIVI with limited viewing angle, distortions of image planes can be approximated to be parabolic in the direction of depth, while those in horizontal and vertical directions can be ignored. When the viewing angle becomes wider, however, this approximation cannot realize presentation of undistorted image. To cope with the strong distortions, the method the authors propose calculate z-coordinate of the generated real image is in detail and depicts each pixel on the display panel of the corresponding depth. Also distortions in horizontal and vertical directions are corrected by using texture mapping. To attain precise correction in vertical, horizontal and depth directions, optical paths of light rays between the display panel and each viewpoint are calculated with an optical simulator. Color aberration can also be corrected by mapping red, green and blue textures separately based on the result of the optical simulation.

Coarse integral imaging (CII), where each elemental lens is large enough to cover pixels far more than the number of views, can show clear floating 3D image when distortion is corrected. One of the major problems left to be solved for CII is suppression of pseudo images that appear around the right image to be presented. In this paper we propose two methods to suppress pseudo images. We first propose use of a lens array with a small F number. When a lens array composed of elemental lenses whose F number is small is set in front of the display panel, pseudo images can be erased by total internal reflection on the outskirt of the large aperture lens because the angle of incidence of the light ray that generates pseudo images becomes larger. The second method we propose is use of a lens array behind the display panel paired with segmented backlight. When convex lenses are set in front of the backlight with limited aperture, leak of ray out to adjacent elemental lenses can be avoided. Since the backlight area is reduced, this method can also reduce consumption of electric power without diminishing brightness of the right image.

In general, free-viewpoint image is generated by captured images by a camera array aligned on a straight line or circle. A camera array is able to capture synchronized dynamic scene. However, camera array is expensive and requires great care to be aligned exactly. In contrast to camera array, a handy camera is easily available and can capture a static scene easily. We propose a method that generates free-viewpoint images from a video captured by a handheld camera in a static scene. To generate free-viewpoint images, view images from several viewpoints and information of camera pose/positions of these viewpoints are needed. In a previous work, a checkerboard pattern has to be captured in every frame to calculate these parameters. And in another work, a pseudo perspective projection is assumed to estimate parameters. This assumption limits a camera movement. However, in this paper, we can calculate these parameters by "Structure From Motion". Additionally, we propose a selection method for reference images from many captured frames. And we propose a method that uses projective block matching and graph-cuts algorithm with reconstructed feature points to estimate a depth map of a virtual viewpoint.

Human binocular depth perception, the most important element brought by 3DTV, is proved to be closely connected to not only the content acquisition (camera focal length, camera baseline and etc.) but also the viewing environment (viewing distance, screen size and etc.). Conventional 3D stereography rule in the literature usually consider the general viewing condition and basic human factors to guide the content acquisition, such as assuming human inter-pupil baseline as the maximum disparity. A lot of new elements or problems of stereoscopic viewing was not considered or precisely defined so that advanced shooting rule is needed to guarantee the overall quality of stereoscopic video. In this paper, we proposed a new stereoscopic video shooting rule considering two most important issues in 3DTV: stereoscopic distortion and comfortable viewing zone. Firstly, a mathematic model mapping the camera space to visualization space is established in order to geometrically estimate the stereoscopic depth distortion. Depth and shape distortion factors are defined and used to describe the stereoscopic distortion. Secondly, comfortable viewing zone (or Depth of focus) is considered to reduce the problem of visual discomfort and visual fatigue. The new shooting rule is to optimize the camera parameters (focal length, camera baseline and etc.) in order to control depth and shape distortion and also guarantee that the perceived scene is located in comfortable viewing zone as possible. However, in some scenarios, the above two conditions cannot be fulfill simultaneously, even sometimes contradict with each other so that priority should be decided. In this paper, experimental stereoscopic synthetic content generation with various sets of camera parameters and various sets of scenes representing different depth range are presented. Justification of the proposed new shooting rule is based on 3D concepts (depth rendering, visual comfort and visual experience) subjective video assessment. The results of this study will provide a new method to propose camera parameters based on management of new criteria (shape distortion and depth of focus) in order to produce optimized stereoscopic images and videos.

To reduce the number of views required for a super multi-view (SMV) display, two or more views are generated with an interval smaller than the pupil diameter, only around a viewer's left and right eyes. The positions of the views are changed referring to the viewer's eye positions to increase viewing freedom. The reduced-view SMV display is implemented using a lenticular-type three-dimensional (3D) display. A cylindrical lens constituting a lenticular lens projects a group of pixels of a flat-panel display to generate a group of views. The pixel group generating the left view group and that generating the right view group through an identical cylindrical lens are spatially separated to separate the view groups. The left and right pixel groups for different cylindrical lenses are interlaced horizontally on the flat-panel display. A prototype reduced-view SMV display was constructed. Each view group consisted of eight views, and the interval of the views was 2.6 mm. An LCD panel with a slanted subpixel arrangement was used. The screen size was 2.57 in, and the 3D resolution was 256 × 192 pixels. A USB camera was attached to the SMV display to detect the viewer's position. The frame rate of face detection and image updating was 30 Hz.

The methods available for delivering stereoscopic (3D) display using glasses can be classified as time-multiplexing and spatial-multiplexing. With both methods, intrinsic visual artifacts result from the generation of the 3D image pair on a flat panel display device. In the case of the time-multiplexing method, an observer perceives three artifacts: flicker, the Mach-Dvorak effect, and a phantom array. These only occur under certain conditions, with flicker appearing in any conditions, the Mach-Dvorak effect during smooth pursuit eye movements (SPM), and a phantom array during saccadic eye movements (saccade). With spatial-multiplexing, the artifacts are temporal-parallax (due to the interlaced video signal), binocular rivalry, and reduced spatial resolution. These artifacts are considered one of the major impediments to the safety and comfort of 3D display users. In this study, the implications of the artifacts for the safety and comfort are evaluated by examining the psychological changes they cause through subjective symptoms of fatigue and the depth sensation. Physiological changes are also measured as objective responses based on analysis of heart and brain activation by visual artifacts. Further, to understand the characteristics of each artifact and the combined effects of the artifacts, four experimental conditions are developed and tested. The results show that perception of artifacts differs according to the visual environment and the display method. Furthermore visual fatigue and the depth sensation are influenced by the individual characteristics of each artifact. Similarly, heart rate variability and regional cerebral oxygenation changes by perception of artifacts in conditions.

We examined the image quality, image clarity and viewing comfort of the 2D images rendered on a fixed 3D autostereoscopic display by using three different rendering schemes. Furthermore, opinions on the daily use of the content were asked. The display used in the study had a lenticular lens as a stereo structure, and both photos of natural scenes and artificial content were used as the test stimuli. Statistical analysis showed no significant differences between the different schemes. When images were divided into subgroups according to the amount of details, significant differences between image quality, image clarity, and viewing comfort scores were found. Moreover, the results indicate that increase in detail levels may decrease evaluated image quality, clarity and viewing comfort. Finally, for all experimental conditions better image quality, image clarity and more comfortable viewing experience had some positive influence on decisions of daily use. In conclusion, amount of details in the content seem to affect on the user experiences of 2D content shown on a 3D display.

The added value of stereoscopy is an important factor for stereoscopic product development and content production. Previous studies have shown that 'image quality' does not encompass the added value of stereoscopy, and thus the attributes naturalness and viewing experience have been used to evaluate stereoscopic content. The objective of this study was to explore what the added value of stereoscopy may consist of and what are the content properties that contribute to the magnitude of the added value. The hypothesis was that interestingness is a significant component of the added value. A subjective study was conducted where the participants evaluated three attributes of the stimuli in the consumer photography domain: viewing experience, naturalness of depth and interestingness. In addition to the no-reference direct scaling method a novel method, the recalled attention map, was introduced and used to study attention in stereoscopic images. In the second part of our study, we use eye tracking to compare the salient regions in monoscopic and stereoscopic conditions. We conclude from the subjective results that viewing experience and naturalness of depth do not cover the entire added value of stereoscopy, and that interestingness brings a new dimension into the added value research. The eye tracking data analysis revealed that the fixation maps are more consistent between participants in stereoscopic viewing than in monoscopic viewing and from this we conclude that stereoscopic imagery is more effective in directing the viewer's attention.

Broadcasting of high definition (HD) stereobased 3D (S3D) TV are planned, or has already begun, in Europe, the US, and Japan. Specific data processing operations such as compression and temporal and spatial resampling are commonly used tools for saving network bandwidth when IPTV is the distribution form, as this results in more efficient recording and transmission of 3DTV signals, however at the same time it inevitably brings quality degradations to the processed video. This paper investigated observers quality judgments of state of the art video coding schemes (simulcast H.264/AVC or H.264/MVC), with or without added temporal and spatial resolution reduction of S3D videos, by subjective experiments using the Absolute Category Rating method (ACR) method. The results showed that a certain spatial resolution reduction working together with high quality video compressing was the most bandwidth efficient way of processing video data when the required video quality is to be judged as "good" quality. As the subjective experiment was performed in two different laboratories in two different countries in parallel, a detailed analysis of the interlab differences was performed.

In this paper, we present a new approach for dense stereo matching which is mainly oriented towards the recovery of depth map of an observed scene. The extraction of depth information from the disparity map is well understood, while the correspondence problem is still subject to errors. In our approach, we propose optimizing correlation based technique by detecting and rejecting mismatched points that occur in the commonly challenging image regions such as textureless areas, occluded portions and discontinuities. The missing values are completed by incorporating edges detection to avoid that a window contains more than one object. It is an efficient method for selecting a variable window size with adaptive shape in order to get accurate results at depth discontinuities and in homogeneous areas while keeping a low complexity of the whole system. Experimental results using the Middlebury datasets demonstrate the validity of our presented approach. The main domain of applications for this study is the design of new functionalities within the context of mobile devices.

In this paper, a subjective study is presented which aims to measure the minimum perceivable depth difference on an autostereoscopic display in order to provide an indication for visual fatigue. The developed experimental setup was used to compare the subject's performance before and after 3D excitation on an autostereoscopic display. By comparing the results to a verification session with 2D excitation, the effect of 3D visual fatigue can be isolated. It was seen that it is possible to reach the threshold of acuity for stereo disparity on that autostereoscopic display. It was also found that the measured depth acuity is slightly higher after 3D viewing than after 2D viewing.

new 3D video format which consists of one full resolution mono video and half resolution left/right videos is proposed. The proposed 3D video format can generate high quality virtual views from small amount of input data while preserving the compatibility for legacy mono and frame compatible stereo video systems. The center view video is the same with normal mono video data, but left/right views are frame compatible stereo video data. This format was tested in terms of compression efficiency, rendering capability, and backward compatibility. Especially we compared view synthesis quality when virtual views are made from full resolution two views or one original view and the other half resolution view. For frame compatible stereo format, experiments were performed on interlaced method. The proposed format gives BD bit-rate gains of 15%.

The current renaissance of 3D movies has drawn more and more attention from the audience. Three-dimensional television (3DTV) has been expected to be the next advance in television. Studies have shown that different people have different comfort range of depth in a 3D content, especially in 3DTV scenario, wherein much smaller screen sizes and viewing distances in home setup than in theater put more restrictions on the 3D content fed into the 3DTV. As a result, the version of the 3D content sent to home will not satisfy all the people in one family. In this paper, we try to solve this problem by providing a prediction of viewing discomfort of certain input content by certain viewer. Our method is based on the Disparity Discomfort Profile (DDP) built through subjective test for each viewer. The input content is analyzed by studying its disparity distribution. The prediction of discomfort is performed by matching the disparity distribution with the viewer's DDP. Then a mechanism to allow the viewers to adjust the depth range according to their visual comfort profile or viewing preference is used to minimize the discomfort. Experiments show promising results of the proposed method.

Three-dimensional holographic system using active shutters for head mounted display application is proposed. Conventional three-dimensional head mounted display suffers from eye-fatigue since it only provides binocular disparity, not monocular depth cues like accommodation. The proposed method presents two holograms of a 3D scene to corresponding eyes using active shutters. Since a holography delivered to each eye has full three-dimensional information, not only the binocular depth cues but also monocular depth cues are presented, eliminating eye-fatigue. The application to the head mounted display also greatly relaxes the viewing angle requirement that is one of the main issues of the conventional holographic displays. In presentation, the proposed optical system will be explained in detail with experimental results.

An integral 3DTV system needs high-density elemental images to increase the reconstructed 3D image's resolution, viewing zone, and depth representability. The dual green pixel-offset method, which uses two green channels of images, is a means of achieving ultra high-resolution imagery. We propose a precise and easy method for detecting the pixel-offset distance when a lens array is mounted in front of the integral imaging display. In this method, pattern luminance distributions based on sinusoidal waves are displayed on each panel of green channels. The difference between phases (amount of phase variation) of these patterns is conserved when the patterns are sampled and transformed to a lower frequency by aliasing with the lens array. This allows the pixel-offset distance of the display panel to be measured in a state of magnification. The relation between the contrast and the amount of phase variation of the pattern is contradicted in relation to the pattern frequency. We derived a way to find the optimal spatial frequency of the pattern by regarding the product of the contrast and amount of phase variation of the patterns as an indicator of accuracy. We also evaluated the pixel-offset detection method in an experiment with the developed display system. The results demonstrate that the resolution characteristics of the projected image were refined. We believe that this method can be used to improve the resolution characteristics of the depth direction of integral imaging.

The multi-view three-dimensional (3D) visualization by means of a 3D display requires reproduction of scene light fields. The complete light field of a scene can be reproduced from the images of a scene ideally taken from infinite viewpoints. However, capturing the images of a scene from infinite viewpoints is not feasible for practical applications. Therefore, in this work, we propose a sparse camera image capture system and an image based virtual image generation method for 3D imaging applications. We show a resulting virtual image produced by the proposed algorithm for generating in-between view of two real images captured with our multi-camera image capture system.

A multilayer liquid crystal display (LCD) is a display device constructed by stacking multiple liquid crystal layers on top of a light source. As shown in a previous study, a multilayer LCD can deliver varying images depending on the viewers'eye positions, and can be used for auto-stereoscopic 3D viewing. However, undesirable blurring is sometimes observed in the images that a viewer receives from the display. Such blurring is notable especially around objects in the scene that are far away from the viewer. To address this problem, we propose to put a convex lens in front of the layers of liquid crystal. The lens refracts the beams of light, thus bringing the effects of moving the objects to nearer positions. Through a simulation-based study, we show that an optimal choice exists for the focal length of the lens, which reduces the local image blurring while not compromising the overall image quality.

We have developed a new volumetric 3-D display using the multi-varifocal lens and high-speed 2-D display. Floating clear 3-D image in space can be successfully obtained. Our volumetric 3-D image is composed of many 2-D layered images by using multi-varifocal lens. Many 2-D images can be layered by changing their depth position using the discrete focal length change of multi-varifocal lens. The high-speed multi-varifocal lens is composed of several sets of a birefringent lens and a polarization switching device. The total lens power is the sum of the lens powers of these sets. The number of lens sets, N, can yield 2^N variations of total focal lengths. In order to re-position many 2-D images within afterimage time, high-speed 2-D display is newly constructed by compact projector array using LED light sources. These projector images are projected to the same position of one screen. By switching these projectors quickly, 2-D images on the screen can be displayed at high-speed. This high-speed 2-D display can successfully provide bright and clear 2-D layered images by using point light sources of LED.

Simple 3D physics simulations with stereoscopic display were created for a part of introductory physics e-Learning. First, cameras to see the 3D world can be made controllable by the user. This enabled to observe the system and motions of objects from any position in the 3D world. Second, cameras were made attachable to one of the moving object in the simulation so as to observe the relative motion of other objects. By this option, it was found that users perceive the velocity and acceleration more sensibly on stereoscopic display than on non-stereoscopic 3D display. Simulations were made using Adobe Flash ActionScript, and Papervison 3D library was used to render the 3D models in the flash web pages. To display the stereogram, two viewports from virtual cameras were displayed in parallel in the same web page. For observation of stereogram, the images of two viewports were superimposed by using 3D stereogram projection box (T&TS CO., LTD.), and projected on an 80-inch screen. The virtual cameras were controlled by keyboard and also by Nintendo Wii remote controller buttons. In conclusion, stereoscopic display offers learners more opportunities to play with the simulated models, and to perceive the characteristics of motion better.

Watermarking already imposed itself as an effective and reliable solution for conventional multimedia content protection (image/video/audio/3D). By persistently (robustly) and imperceptibly (transparently) inserting some extra data into the original content, the illegitimate use of data can be detected without imposing any annoying constraint to a legal user. The present paper deals with stereoscopic image protection by means of watermarking techniques. That is, we first investigate the peculiarities of the visual stereoscopic content from the transparency and robustness point of view. Then, we advance a new watermarking scheme designed so as to reach the trade-off between transparency and robustness while ensuring a prescribed quantity of inserted information. Finally, this method is evaluated on two stereoscopic image corpora (natural image and medical data).

We developed a new multi-view camera system composed of 9 cameras for capturing multi-view images and utilizing them in research and exhibition. The system is composed of cameras, rig, convergence controller, LANC signal controller, IEEE 1394 signal interface. And, we made a control program that can manage a record function and other various camera parameters at PC. By this program, we can shoot multi-view scenes manipulating each element camera independently or all 9 cameras totally. After the capturing, we transform the image into the glass-less multi-view display format, and then we can enjoy natural multi-view images at lenticular display.

Multi-view video coding (MVC) is a video coding standard developed by MPEG and VCEG for multi-view video. It showed average PSNR gain of 1.5dB compared with view-independent coding by H.264/AVC. However, because resolutions of multi-view video are getting higher for more realistic 3D effect, high performance video codec is needed. MVC adopted hierarchical B-picture structure and inter-view prediction as core techniques. The hierarchical B-picture structure removes the temporal redundancy, and the inter-view prediction reduces the inter-view redundancy by compensated prediction from the reconstructed neighboring views. Nevertheless, MVC has inherent limitation in coding efficiency, because it is based on H.264/AVC. To overcome the limit, an enhanced video codec for multi-view video based on Key Technology Area (KTA) is proposed. KTA is a high efficiency video codec by Video Coding Expert Group (VCEG), and it was carried out for coding efficiency beyond H.264/AVC. The KTA software showed better coding gain than H.264/AVC by using additional coding techniques. The techniques and the inter-view prediction are implemented into the proposed codec, which showed high coding gain compared with the view-independent coding result by KTA. The results presents that the inter-view prediction can achieve higher efficiency in a multi-view video codec based on a high performance video codec such as HEVC.

Modern consumer 3D TV sets are able to show video content in two different modes: 2D and 3D. In 3D mode, stereo pair comes from external device such as Blue-ray player, satellite receivers etc. The stereo pair is split into left and right images that are shown one after another. The viewer sees different image for left and right eyes using shutter-glasses properly synchronized with a 3DTV. Besides, some devices that provide TV with a stereo content are able to display some additional information by imposing an overlay picture on video content, an On-Screen-Display (OSD) menu. Some OSDs are not always 3D compatible and lead to incorrect 3D reproduction. In this case, TV set must recognize the type of OSD, whether it is 3D compatible, and visualize it correctly by either switching off stereo mode, or continue demonstration of stereo content. We propose a new stable method for detection of 3D incompatible OSD menus on stereo content. Conventional OSD is a rectangular area with letters and pictograms. OSD menu can be of different transparency levels and colors. To be 3D compatible, an OSD is overlaid separately on both images of a stereo pair. The main problem in detecting OSD is to distinguish whether the color difference is due to OSD presence, or due to stereo parallax. We applied special techniques to find reliable image difference and additionally used a cue that usually OSD has very implicit geometrical features: straight parallel lines. The developed algorithm was tested on our video sequences database, with several types of OSD with different colors and transparency levels overlaid upon video content. Detection quality exceeded 99% of true answers.

Recently stereoscopy has increased a lot its popularity and various technologies are spreading in theaters and homes allowing observation of stereoscopic images and movies, becoming affordable even for home users. However there are some golden rules that users should follow to ensure a better enjoyment of stereoscopic images, first of all the viewing condition should not be too different from the ideal ones, which were assumed during the production process. To allow the user to perceive stereo depth instead of a flat image, two different views of the same scene are shown to the subject, one is seen just through his left eye and the other just through the right one; the vision process is making the work of merging the two images in a virtual three-dimensional scene, giving to the user the perception of depth. The two images presented to the user were created, either from image synthesis or from more traditional techniques, following the rules of perspective. These rules need some boundary conditions to be explicit, such as eye separation, field of view, parallax distance, viewer position and orientation. In this paper we are interested in studying how the variation of the viewer position and orientation from the ideal ones expressed as specified parameters in the image creation process, is affecting the correctness of the reconstruction of the three-dimensional virtual scene.

In this research we investigate the maximum displayable angular resolutions in the horizontal and vertical directions of a planar 3-D object displayed by a glasses-free tabletop 3-D display that is using a conic-shaped optical device and an array of projectors. We estimate initially the resolutions of the (1) slit-like views corresponding to each projector's image and then continue with the resolution for the (2) final 3-D image obtained after the integration of the slits. Our experimental results represent a first attempt in estimating the differences between these resolutions that can benefit (1) future tunings of the tabletop 3-D display as well as (2) optimization of the displaying process.

In the stereoscopic frame-compatible format, the separate high-definition left and high-definition right views are reduced in resolution and packed to fit within the same video frame as a conventional two-dimensional high-definition signal. This format has been suggested for 3DTV since it does not require additional transmission bandwidth and entails only small changes to the existing broadcasting infrastructure. In some instances, the frame-compatible format might be used to deliver both 2D and 3D services, e.g., for over-the-air television services. In those cases, the video quality of the 2D service is bound to decrease since the 2D signal will have to be generated by up-converting one of the two views. In this study, we investigated such loss by measuring the perceptual image quality of 1080i and 720p up-converted video as compared to that of full resolution original 2D video. The video was encoded with either a MPEG-2 or a H.264/AVC codec at different bit rates and presented for viewing with either no polarized glasses (2D viewing mode) or with polarized glasses (3D viewing mode). The results confirmed a loss of video quality of the 2D video up-converted material. The loss due to the sampling processes inherent to the frame-compatible format was rather small for both 1080i and 720p video formats; the loss became more substantial with encoding, particularly for MPEG-2 encoding. The 3D viewing mode provided higher quality ratings, possibly because the visibility of the degradations was reduced.

The market of stereoscopic 3D TV grows up fast recently; however, for 3D TV really taking off, the interoperability of shutter glasses (SG) to view different TV sets must be solved, so we developed a measurement method with ideal shutter glasses (ISG) to separate time-sequential stereoscopic displays and SG. For measuring the crosstalk from time-sequential stereoscopic 3D displays, the influences from SG must be eliminated. The advantages are that the sources to crosstalk are distinguished, and the interoperability of SG is broadened. Hence, this paper proposed ideal shutter glasses, whose non-ideal properties are eliminated, as a platform to evaluate the crosstalk purely from the display. In the ISG method, the illuminance of the display was measured in time domain to analyze the system crosstalk SCT of the display. In this experiment, the ISG method was used to measure SCT with a high-speed-response illuminance meter. From the time-resolved illuminance signals, the slow time response of liquid crystal leading to SCT is visualized and quantified. Furthermore, an intriguing phenomenon that SCT measured through SG increases with shortening view distance was observed, and it may arise from LC leakage of the display and shutter leakage at large view angle. Thus, we measured how LC and shutter leakage depending on view angle and verified our argument. Besides, we used the ISG method to evaluate two displays.

Horizontal image translation (HIT) is an electronic process for shifting the left-eye and right-eye images horizontally as a way to alter the stereoscopic characteristics and alignment of 3D content after signals have been captured by stereoscopic cameras. When used cautiously and with full awareness of the impact on other interrelated aspects of the stereography, HIT is a valuable tool in the post production process as a means to modify stereoscopic content for more comfortable viewing. Most commonly it is used to alter the zero parallax setting (ZPS), to compensate for stereo window violations or to compensate for excessive positive or negative parallax in the source material. As more and more cinematic 3D content migrates to television distribution channels the use of this tool will likely expand. Without proper attention to certain guidelines the use of HIT can actually harm the 3D viewing experience. This paper provides guidance on the most effective use and describes some of the interrelationships and trade-offs. The paper recommends the adoption of the cinematic 2K video format as a 3D source master format for high definition television distribution of stereoscopic 3D video programming.