Right-eye Images Research Articles

A 3D reconstruction based on an unsupervised domain adaptive for binocular endoscopy.

In minimally invasive surgery, endoscopic image quality plays a crucial role in surgery. Aiming at the lack of a real parallax in binocular endoscopic images, this article proposes an unsupervised adaptive neural network. The network combines adaptive smoke removal, depth estimation of binocular endoscopic images, and the 3D display of high-quality endoscopic images. We simulated the smoke generated during surgery by artificially adding fog. The training images of U-Net fused by Laplacian pyramid are introduced to improve the network’s ability to extract intermediate features. We introduce Convolutional Block Attention Module to obtain the optimal parameters of each layer of the network. We utilized the disparity transformation relationship between left- and right-eye images to combine the left-eye images with disparity in HS-Resnet to obtain virtual right-eye images as labels for self-supervised training. This method extracts and fuses the parallax images at different scale levels of the decoder, making the generated parallax images more complete and smoother. A large number of experimental research results show that the scheme can remove the smoke generated during the operation, effectively reconstruct the 3D image of the tissue structure of the binocular endoscope, and at the same time, preserve the contour, edge, detail, and texture of the blood vessels in the medical image. Compared with the existing similar schemes, various indicators have been greatly improved. It has good clinical application prospects.

Backlight system using an interleaved Fresnel lens array that attains a uniform luminance and two-dimensional directional light control.

A novel (to the best of our knowledge) Fresnel lens array is proposed to realize a uniform directional backlight with two-dimensional directionality. Autostereoscopic display with the proposed lens array improves image output quality and relieves the viewer's posture restriction without the need for any additional eye aid. In the proposed lens array, tiny prisms composing two adjacent linear Fresnel lenses are interleaved so that the two lenses may be virtually overlapped and work independently. The widths of the elemental prisms vary depending on the distance from the center of each lens. Thus, the light passing through the two lenses is mixed, which results in higher and more uniform luminance intensity. A prototype of an autostereoscopic display based on the time-multiplexed directional backlight method and made with the proposed lens array attained uniform luminance as well as low cross-talk between left-eye and right-eye images.

Autostereoscopic display for multiviewers positioned at different distances using time-multiplexed layered directional backlight.

In this paper, an autostereoscopic display system based on a time-multiplexed directional backlight using a large aperture Fresnel lens is proposed. High-resolution stereoscopy for multiple viewers positioned at different distances from the screen is achieved in the proposed system by layering polymer dispersed liquid crystal screens behind the Fresnel lens. The screens with segmented electrodes are electrically controlled to change the position of light diffusion, while the time-multiplexed backlight is projected by a digital mirror device projector at a high refresh rate. The light is diffused at the conjugate focal points of the observers' eyes to deliver directional light to each eye. The right-eye image and the left-eye image are alternated on the LCD panel in front of the lens to synchronize with the backlight.

Optimal Display Positions for Heads-Up Surgery to Minimize Crosstalk.

PurposeTo determine optimal display positions during heads-up surgery (HUS) to minimize crosstalk.MethodsThree three-dimensional (3D) displays were analyzed to evaluate the relationship between display position and amount of crosstalk. First, the 3D displays were calibrated to show a completely white image in the right eye and completely black image in the left eye. Images were captured through the polarized filter, which corresponded to the left-eye image. The amount of crosstalk in the left eye was measured as white areas on the black background that originated from the right-eye image. The amount of crosstalk was measured at different display distances and heights to estimate the non-crosstalk display positions for each display.ResultsVarying amounts of crosstalk (0%–70.3%) were observed for different display distances and heights. The crosstalk almost always started from the corner of the display, although the starting area varied according to the type of display. The minimum distance of non-crosstalk position was 1.26 meters away from display 1, 1.24 meters away from display 2, and 1.8 meters away from display 3. With regard to display height, the optimal center-of-display heights for displays 1, 2, and 3 were 72 mm below, 18 mm above, and 101 mm above eye level, respectively.ConclusionsThe amount of crosstalk differed according to display positions and displays.Translational RelevanceThe optimal HUS display settings differ among displays; therefore, each surgeon should carefully evaluate individual display characteristics before using HUS in practice.

2153-PUB: Deep Learning and Laterality of Retinal Images

Background: Retinal fundus images, obtained from routine eye screenings, contain important information, not only concerning microvascular lesions of the retina, but potentially also of the progression and manifestation of various other diabetes complications. Recent advances in the field of machine learning, so-called deep learning neural networks (DNN), make it possible to automatically detect retinal lesions from fundus photos. Our overall aim is to develop a DNN to detect retinal vascular lesions. In this abstract, we describe a pre-processing step consisting of application of a DNN. Methods: For each eye, 5 retinal images of parts of the eye are taken, and collected into a joint mosaic image, used for image grading. There is no automated labelling of the images to identify left or right eyes. To distinguish left- from right eye images, we used a set of 1,916 unique, high-definition retina mosaic fundus images from diabetes patients followed at Steno Diabetes Center Copenhagen (SDCC) between 2003-2017. We trained and validated a DNN, using 964 left-eye (training set: 713, validation: 251) and 952 right-eye images (training set: 706, validation: 246). A DNN is a method that finds the optimal algorithm for relating a large set of input (mosaic fundus images) to output (left or right eye). The DNN used was based on the InceptionV3 architecture and run for 100 epochs with 100 gradient descent steps pr. epoch. Results: The DNN classification obtained a validation AUC-ROC of 0.92 and a validation accuracy of 0.86. Conclusions: The DNN successfully distinguished left-eye from right-eye images. We plan to use the DNN to identify right- and left-eye images in the SDCC image database comprising ∼ 80,000 left-eye and right-eye images. We will subsequently develop a DNN for automated detection of referable diabetic retinopathy of retinal fundus images, for use in the clinic. Disclosure L. Díaz: None. D. Vistisen: Stock/Shareholder; Self; Novo Nordisk A/S. M.E. Jørgensen: Research Support; Self; Amgen, AstraZeneca, Boehringer Ingelheim Pharmaceuticals, Inc., Sanofi-Aventis. Stock/Shareholder; Self; Novo Nordisk A/S. M. Valerius: None. J.N. Hajari: None. H. Lund-Andersen: None. S. Byberg: None.

Анализ методов сжатия стереоскопических изображений

Virtual Reality (VR) and Augmented Reality (AR) Head-Mounted Displays (HMDs) have been emerging in the last years and they are gaining an increased popularity in many industries. HMDs are generally used in entertainment, social interaction, education, but their use for work is also increasing in domains such as medicine, modeling and simulation. Despite the recent release of many types of HMDs, two major problems are hindering their widespread adoption in the mainstream market: the extremely high costs and the user experience issues [1]. The illusion of a 3D display in HMDs is achieved with a technique called stereoscopy. Applications of stereoscopic imagining are such that data transfer rates and—in mobile applications—storage quickly become a bottleneck. Therefore, efficient image compression techniques are required. Standard image compression techniques are not suitable for stereoscopic images due to the discrete differences that occur between the compressed and uncompressed images. The issue is that the loss in lossy image compression may blur the minute differences between the left-eye and right-eye images that are crucial in establishing the illusion of 3D perception. However, in order to achieve more efficient coding, there are various coding techniques that can be adapted to stereoscopic images. Stereo image compression techniques that can be found in the literature utilize discrete Wavelet transformation and the morphological compression algorithm applied to the transform coefficients. This paper provides an overview and comparison of available techniques for the compression of stereoscopic images, as there is still no technique that is accepted as best for all criteria. We want to test the techniques with users who would actually be potential users of HMDs and therefore would be exposed to these techniques. Also, we focused our research on low-priced, consumer grade HMDs which should be available for larger population.

Development of 3DTV Emission Multiplexer and Reception-Signal Status Analyzer for ATSC 8-VSB & MDTV Hybrid 3DTV Services

ABSTRACT This paper describes the development of a real-time three-dimensional television (3DTV) emission multiplexer and reception – signal status analyzing system for Advanced Television Systems Committee 8-vestigial side-band (ATSC 8-VSB) & mobile digital television (MDTV) hybrid 3DTV service. Recently, this system has been adopted as a standard (ATSC A/104 Part 5) of the terrestrial 3DTV broadcasting service under the name of service compatible-main and mobile hybrid (SC-MMH) 3DTV. Compared to the existing ATSC MDTV 2D systems, the SC-MMH 3DTV system requires 8-VSB (left-eye images) and MDTV (right-eye images) frame synchronization. 3DTV shall receive both 8-VSB and MDTV signals simultaneously. For a seamless SC-MMH 3DTV service within these service constraints, broadcasters should deploy transmitters and measuring services over the coverage area. In this paper, we introduce the implementation of SC-MMH 3DTV emission multiplexer and reception-signal status analyzer to configure the transmitter and signal measurements more efficiently. The options of the proposed system are verified by the SC-MMH 3DTV field test in a real terrestrial broadcasting environment.

Critical Binocular Asymmetry Measure for the Perceptual Quality Assessment of Synthesized Stereo 3D Images in View Synthesis

In human vision, excessive binocular asymmetry between the left- and right-eye images can be very problematic to perceive single binocular vision (causing visual discomfort) in the viewing of stereoscopic images. In this paper, we propose a critical binocular asymmetry (CBA) measure for objectively assessing the perceptual quality of the synthesized stereo 3D images generated through a depth-image-based rendering (DIBR) process. The proposed method detects critical regions that are likely to induce excessive binocular asymmetry. In particular, this paper considers view extrapolation since it introduces much more artifacts due to the lack of data. We measure structural similarity on the critical regions between the left and right images to quantify the perceptual effects of binocular asymmetry. The effectiveness of the proposed quality measure has been successfully evaluated by subjective assessment experiments using various types of synthesized stereoscopic images generated by four different DIBR-based view synthesis algorithms. We demonstrate the validity of the proposed quality measure by the comparison of subjective ratings and existing objective methods. Experimental results show that the combined use of the proposed binocular asymmetry measure and existing quality measures substantially improves the performance of the quality measures by explicitly considering the perceptual effects of CBA.

Acute Alcohol Drinking Promotes Piecemeal Percepts during Binocular Rivalry.

Binocular rivalry refers to perceptual alternation when two eyes view different images. One of the potential percepts during binocular rivalry is a spatial mosaic of left- and right-eye images, known as piecemeal percepts, which may result from localized rivalries between small regions in the left- and right-eye images. It is known that alcohol increases inhibitory neurotransmission, which may reduce the number of alternations during binocular rivalry. However, it is unclear whether alcohol affects rivalry dynamics in the same manner for both coherent percepts (i.e., percepts of complete left or right images) and piecemeal percepts. To address this question, the present study measured the dynamics of binocular rivalry before and after 15 moderate-to-heavy social drinkers consumed an intoxicating dose of alcohol versus a placebo beverage. Both simple rivalrous stimuli consisting of gratings with different orientations, and complex stimuli consisting of a face or a house were tested to examine alcohol effects on rivalry as a function of stimulus complexity. Results showed that for both simple and complex stimuli, alcohol affects coherent and piecemeal percepts differently. More specifically, alcohol reduced the number of coherent percepts but not the mean dominance duration of coherent percepts. In contrast, for piecemeal percepts, alcohol increased the mean dominance duration but not the number of piecemeal percepts. These results suggested that alcohol drinking may selectively affect the dynamics of transitional period of binocular rivalry by increasing the duration of piecemeal percepts, leading to a reduction in the number of coherent percepts. The differential effect of alcohol on the dynamics of coherent and piecemeal percepts cannot be accounted for by alcohol’s effect on a common inhibitory mechanism. Other mechanisms, such as increasing neural noise, are needed to explain alcohol’s effect on the dynamics of binocular rivalry.

Pooled, but not single-neuron, responses in macaque V4 represent a solution to the stereo correspondence problem.

Binocular disparity is an important cue for depth perception. To correctly represent disparity, neurons must find corresponding visual features between the left- and right-eye images. The visual pathway ascending from V1 to inferior temporal cortex solves the correspondence problem. An intermediate area, V4, has been proposed to be a critical stage in the correspondence process. However, the distinction between V1 and V4 is unclear, because accumulating evidence suggests that the process begins within V1. In this article, we report that the pooled responses in macaque V4, but not responses of individual neurons, represent a solution to the correspondence problem. We recorded single-unit responses of V4 neurons to random-dot stereograms of varying degrees of anticorrelation. To achieve gradual anticorrelation, we reversed the contrast of an increasing proportion of dots as in our previous psychophysical studies, which predicted that the neural correlates of the solution to correspondence problem should gradually eliminate their disparity modulation as the level of anticorrelation increases. Inconsistent with this prediction, the tuning amplitudes of individual V4 neurons quickly decreased to a nonzero baseline with small anticorrelation. By contrast, the shapes of individual tuning curves changed more gradually so that the amplitude of population-pooled responses gradually decreased toward zero over the entire range of graded anticorrelation. We explain these results by combining multiple energy-model subunits. From a comparison with the population-pooled responses in V1, we suggest that disparity representation in V4 is distinctly advanced from that in V1. Population readout of V4 responses provides disparity information consistent with the correspondence solution.

A bi-modal model to simulate auditory expectation for reverberation time and direct-to-reverberant energy from visual feedback

A bi-modal model is presented that predicts the psychophysical results of Valente and Braasch [Acustica, 2008]. The model simulates expectation of room-acoustical qualities due to visual cues. The visual part of the model estimates angles of incidence and delays of the first two side reflections of a given frontal sound source. To this end, a stereoscopic image is used to determine azimuth angles and distances for the two frontal room corners. The distance estimates are derived by using the angular differences between the left- and right-eye images of each corner. The model then calculates the room volume by reconstructing a rectangular room from these data, assuming a range of possibilities for the missing room coordinates. In a next step, logarithmic fits of volume to expected reverberation time and of volume to direct-to-reverberant energy ratio predict the expected value ranges for these two parameters. Using a feedback structure, the visually-derived acoustic parameters become input to an auditory Precedence-Effect model, where they are used to pre-adjust inhibition parameters for two acoustic side reflections. These inhibition parameters are consequently refined in the course of the analysis of the incoming sound. [Work supported by NSF #1229391/#1320059 and ERC FP7-ICT-2013-C-#618075.]

Enhancement of the perceived image quality of a 2D/3D convertible directional-backlight unit system using a double-slit barrier array and an active diffuser.

In this paper, a method to construct a directional-backlight unit system to separate the left-eye image and the right-eye image by adopting a double-slit barrier array is proposed to enhance the perceived quality of the realized 3D image. Additionally, the two-dimensional/three-dimensional convertible function is also realized by adopting an active diffuser to the spatial interlacing directional-backlight system.

Optimal Backlight Modulation With Crosstalk Control in Stereoscopic Display

Crosstalk between the left-eye and right-eye images is one of the main artifacts affecting the visual quality of stereoscopic liquid crystal display (LCD) systems. In this paper, a novel technique, called Optimal Backlight Modulation (OBM), is proposed to reduce crosstalk by taking the advantage of newly emerged light-emitting diode (LED) backlight panel. Unlike most existing techniques which solely aim at eliminating the effect of crosstalk, ignoring image brightness and contrast, OBM strikes a better balance by maximizing the output luminance while confining the crosstalk within a given level. A simple closed-form approximation of the optimization problem can be easily employed and solved in real time on LCD control hardware. Simulation results show that the proposed OBM algorithm provides the same or higher luminance while reducing the crosstalk by 60% compared with the other tested methods.

Cross-matching: a modified cross-correlation underlying threshold energy model and match-based depth perception

Three-dimensional visual perception requires correct matching of images projected to the left and right eyes. The matching process is faced with an ambiguity: part of one eye's image can be matched to multiple parts of the other eye's image. This stereo correspondence problem is complicated for random-dot stereograms (RDSs), because dots with an identical appearance produce numerous potential matches. Despite such complexity, human subjects can perceive a coherent depth structure. A coherent solution to the correspondence problem does not exist for anticorrelated RDSs (aRDSs), in which luminance contrast is reversed in one eye. Neurons in the visual cortex reduce disparity selectivity for aRDSs progressively along the visual processing hierarchy. A disparity-energy model followed by threshold nonlinearity (threshold energy model) can account for this reduction, providing a possible mechanism for the neural matching process. However, the essential computation underlying the threshold energy model is not clear. Here, we propose that a nonlinear modification of cross-correlation, which we term “cross-matching,” represents the essence of the threshold energy model. We placed half-wave rectification within the cross-correlation of the left-eye and right-eye images. The disparity tuning derived from cross-matching was attenuated for aRDSs. We simulated a psychometric curve as a function of graded anticorrelation (graded mixture of aRDS and normal RDS); this simulated curve reproduced the match-based psychometric function observed in human near/far discrimination. The dot density was 25% for both simulation and observation. We predicted that as the dot density increased, the performance for aRDSs should decrease below chance (i.e., reversed depth), and the level of anticorrelation that nullifies depth perception should also decrease. We suggest that cross-matching serves as a simple computation underlying the match-based disparity signals in stereoscopic depth perception.

Improved Tile Format of Stereoscopic Video for 3-D TV Broadcasting

Tile format is a new frame packing arrangement for the first generation of 3-D TV broadcasting services. It puts the left- and right-eye images of each stereoscopic image pair into a single composition frame that allows the service provider to reuse existing production and distribution infrastructure for offering 3-D TV services. Different from frame compatible formats, the tile format composition does not down-sample the left- and right-eye images and, thus, provides a higher video quality for both 3-D and 2-D display. The problem with the tile format is that it creates artificial edges in the right-eye image and introduces visible tiling artifacts around the artificial edges after compression. This brief paper presents a simple modification to the tile format composition method that significantly reduces the tiling artifacts. Experimental results show that local PSNRs around artificial edges are improved by over 1 dB.

Viewing zone of an autostereoscopic display with a directional backlight using a convex lens array

When a directional backlight to each eye alternates synchronously with the alternation of left-eye and right-eye images on the display panel, the viewer can see a stereoscopic image without wearing special goggles. One way to realize a directional backlight is to place a convex lens array in front of dot matrix light sources to generate collimated light. To implement this method, however, defocusing and field curvature of the lens should be taken into account. The viewing zone of an autostereoscopic display with a directional backlight using a convex lens array is analyzed based on optical simulations.

Recurrent Connectivity Can Account for the Dynamics of Disparity Processing in V1

Disparity tuning measured in the primary visual cortex (V1) is described well by the disparity energy model, but not all aspects of disparity tuning are fully explained by the model. Such deviations from the disparity energy model provide us with insight into how network interactions may play a role in disparity processing and help to solve the stereo correspondence problem. Here, we propose a neuronal circuit model with recurrent connections that provides a simple account of the observed deviations. The model is based on recurrent connections inferred from neurophysiological observations on spike timing correlations, and is in good accord with existing data on disparity tuning dynamics. We further performed two additional experiments to test predictions of the model. First, we increased the size of stimuli to drive more neurons and provide a stronger recurrent input. Our model predicted sharper disparity tuning for larger stimuli. Second, we displayed anticorrelated stereograms, where dots of opposite luminance polarity are matched between the left- and right-eye images and result in inverted disparity tuning in the disparity energy model. In this case, our model predicted reduced sharpening and strength of inverted disparity tuning. For both experiments, the dynamics of disparity tuning observed from the neurophysiological recordings in macaque V1 matched model simulation predictions. Overall, the results of this study support the notion that, while the disparity energy model provides a primary account of disparity tuning in V1 neurons, neural disparity processing in V1 neurons is refined by recurrent interactions among elements in the neural circuit.

Color asymmetry in 3D imaging: Influence on the viewing experience

Because the chain of events in 3D imaging is vulnerable to different context- and technology-specific variables, it is important to understand the extent to which users can accept feature-specific differences between scenes without a decrease in the (observed) image quality. Twenty participants were asked to view natural stereoscopic still images and evaluate how different combinations of color asymmetries affect the overall viewing experience, the naturalness of the image and the depth perception. As expected, an increase in color asymmetry between the viewer's left-eye and right-eye images decreased the image quality evaluation scores. Certain color-channel-specific changes, such as a decrease in blue values, were more acceptable than others, and some content-specific features, such as a brownish or greenish background, were less sensitive to changes compared to close-up images with brighter objects and backgrounds.

Light polarization in support of stereoscopic display

The encoding of three-dimensional image pairs by polarization was proposed as early as the 1890s, perhaps stimulated by the popularity of stereoscopic photography, the proliferation of devices for viewing side-by-side stereoscopic images, and the invention of the anaglyph. The introduction of inexpensive sheet polarizing material gave rise to new three-dimensional technologies, starting in the 1930s with 16-mm black-and-white motion pictures projected by paired projectors equipped with orthogonally oriented polarizing filters. Further advances included the introduction of color, the concept of printing left- and right-eye images on a common carrier, and most recently the development of digital photography and the utilization of polarizers in both two- and three-dimensional digital color display.

A Digital Blind Watermarking for Depth-Image-Based Rendering 3D Images

The content protection for image-based 3D data is getting more importance with the advance of low cost 3D display devices. The depth-image-based rendering (DIBR) 3D image is one of the image-based 3D data which consists of the center image and the depth image generated by the content provider. The left-eye image and the right-eye image are rendered from the center image and the depth image at the content consumer side. The blind watermarking for DIBR 3D image is rarely studied in the literature. In this paper, a novel blind multiple watermarking scheme is proposed to deal with the content protection problem of DIBR 3D images. Besides the usual requirement of mutual orthogonality among reference patterns for multiple watermark embedding, we found that proper embedding order plays an even more important role in watermarking the DIBR 3D images. Experimental results show that the proposed scheme is robust against the JPEG compression and noise adding attacks. More interestingly, it is found that the proposed watermarking can also tolerate large range variations of the depth image during rendering.