Visual Depth Research Articles

Using visual and auditory cues to locate out-of-view objects in head-mounted augmented reality

When looking for an object in a complex visual scene, Augmented Reality (AR) can assist search with visual cues persistently pointing in the target’s direction. The effectiveness of these visual cues can be reduced if they are placed at a different visual depth plane to the target they are indicating. To overcome this visual-depth problem, we test the effectiveness of adding simultaneous spatialized auditory cues that are fixed at the target’s location. In an experiment we manipulated which cue(s) were available (visual-only vs. visual + auditory), and which disparity plane relative to the target the visual cue was displayed on. Results show that participants were slower at finding targets when the visual cue was placed on a different disparity plane to the target. However, this slowdown in search performance could be substantially reduced with auditory cueing. These results demonstrate the importance of AR cross-modal cueing under conditions of visual uncertainty and show that designers should consider augmenting visual cues with auditory ones.

A real-time personalized face modeling method for peking opera with depth vision device

Personalized modeling of Peking Opera faces is a challenging question in computer vision due to the artistry of painted faces, which is more challenging on natural human faces. To address this issue, in this paper, we propose a personalized modeling method for Peking Opera faces based on a depth camera device. Firstly, to solve the abnormal deformation of unrelated areas on 3D standard Peking Opera faces, a method that divide the Action Unit (AU) according to the face painting is designed, and the shape bases are constructed using local deformation based on the Laplacian algorithm. Secondly, to complete the linear fusion of different shape bases and model a natural and vivid personalized Peking Opera face, we design a method that calculates the weight of different shape bases based on the BlendShape algorithm to solve the conflicts between different shape bases. In addition, a real-time interactive method is proposed to calculate the weight value of each shape base based on 3D information of facial feature points captured by a depth camera. The results show that our method can achieve personalized modeling of Peking Opera faces with vivid expression in real-time, while reducing the complexity of the vectorization pattern database used in the existing personalized modeling of Peking Opera faces.

Visual perception and camouflage response to 3D backgrounds and cast shadows in the European cuttlefish, Sepia officinalis.

To conceal themselves on the seafloor, European cuttlefish, Sepia officinalis, express a large repertoire of body patterns. Scenes with 3D relief are especially challenging because it is not possible either to directly recover visual depth from the 2D retinal image or for the cuttlefish to alter its body shape to resemble nearby objects. Here, we characterised cuttlefish camouflage responses to 3D relief, and to cast shadows, which are complementary depth cues. Animals were recorded in the presence of cylindrical objects of fixed (15 mm) diameter, but varying in height, greyscale and strength of cast shadows, and to corresponding 2D pictorial images. With the cylinders, the cuttlefish expressed a '3D' body pattern, which is distinct from previously described Uniform, Mottle and Disruptive camouflage patterns. This pattern was insensitive to variation in object height, contrast and cast shadow, except when shadows were most pronounced, in which case the body patterns resembled those used on the 2D backgrounds. This suggests that stationary cast shadows are not used as visual depth cues by cuttlefish, and that rather than directly matching the 2D retinal image, the camouflage response is a two-stage process whereby the animal first classifies the physical environment and then selects an appropriate pattern. Each type of pattern is triggered by specific cues that may compete, allowing the animal to select the most suitable camouflage, so the camouflage response is categorical rather than continuously variable. These findings give unique insight into how an invertebrate senses its visual environment to generate the body pattern response.

Deep Learning Techniques Applied to Predict and Measure Finger Movement in Patients with Multiple Sclerosis

This research focuses on the development of a system for measuring finger joint angles based on camera image and is intended for work within the field of medicine to track the movement and limits of hand mobility in multiple sclerosis. Measuring changes in hand mobility allows the progress of the disease and its treatment process to be monitored. A static RGB camera without depth vision was used in the system developed, with the system receiving only the image from the camera and no other input data. The research focuses on the analysis of each image in the video stream independently of other images from that stream, and 12 measured hand parameters were chosen as follows: 3 joint angles for the index finger, 3 joint angles for the middle finger, 3 joint angles for the ring finger, and 3 joint angles for the pinky finger. Convolutional neural networks were used to analyze the information received from the camera, and the research considers neural networks based on different architectures and their combinations as follows: VGG16, MobileNet, MobileNetV2, InceptionV3, DenseNet, ResNet, and convolutional pose machine. The final neural network used for image analysis was a modernized neural network based on MobileNetV2, which obtained the best mean absolute error value of 4.757 degrees. Additionally, the mean square error was 67.279 and the root mean square error was 8.202 degrees. This neural network analyzed a single image from the camera without using other sensors. For its part, the input image had a resolution of 512 by 512 pixels, and was processed by the neural network in 7–15 ms by GPU Nvidia 2080ti. The resulting neural network developed can measure finger joint angle values for a hand with non-standard parameters and positions.

Visual function and mobility after multifocal versus monofocal intraocular lens implantation

ABSTRACT Clinical relevance: Mobility and fall risk may be important considerations in choosing between intraocular lenses. Background: Fall risk in older adults increases when wearing multifocal spectacles, but little is known about mobility among individuals with different types of intraocular lenses. This study compared visual function, fall risk and balance control following bilateral implantation of monofocal or multifocal intraocular lenses. Methods: This was a non-randomised, cross-sectional study involving adults with bilateral intraocular lenses. Participants completed questionnaires concerning physical functioning, fall history and balance-related confidence. Binocular visual acuity, contrast sensitivity (Pelli-Robson chart and computerized testing), depth perception and glare sensitivity were assessed. Physical performance measures included the Sensory Organization Test, preferred gait speed, Dynamic Gait Index and wayfinding in a virtual environment. Results: Fifteen participants (mean ± standard deviation, 67.1 ± 6.8 years) had monofocal intraocular lenses and 14 participants (68.1 ± 6.1 years) had multifocal intraocular lenses. Contrast sensitivity in the monofocal group was significantly better than that in the multifocal group (p = 0.02) at intermediate and high spatial frequencies. Contrast sensitivity of the monofocal group also was less affected by glare than the contrast sensitivity of the multifocal group, at an intermediate spatial frequency (p = 0.02). However, the multifocal group had significantly better Dynamic Gait Index scores (p = 0.04), even after controlling for perceived physical function. Conclusions: The participants with monofocal intraocular lenses generally had better contrast sensitivity than did those with multifocal intraocular lenses. However, the scores on a mobility test that is associated with fall risk were worse for those with monofocal lenses.

Optimal allocation of quantized human eye depth perception for multi-focal 3D display design.

Creating immersive 3D stereoscopic, autostereoscopic, and lightfield experiences are becoming the center point of optical design of future head mounted displays and lightfield displays. However, despite the advancement in 3D and light field displays, there is no consensus on what are the necessary quantized depth levels for such emerging displays at stereoscopic or monocular modalities. Here we start from psychophysical theories and work toward defining and prioritizing quantized levels of depth that would saturate the human depth perception. We propose a general optimization framework, which locates the depth levels in a globally optimal way for band limited displays. While the original problem is computationally intractable, we manage to find a tractable reformulation as maximally covering a region of interest with a selection of hypographs corresponding to the monocular depth of field profiles. The results indicate that on average 1731 stereoscopic and 7 monocular depth levels (distributed optimally from 25 cm to infinity) would saturate the visual depth perception. Such that adding further depth levels adds negligible improvement. Also the first 3 depth levels should be allocated at (148), then (83, 170), then (53, 90, 170) distances respectively from the face plane to minimize the monocular error in the entire population. The study further discusses the 3D spatial profile of the quantized stereoscopic and monocular depth levels. The study provides fundamental guidelines for designing optimal near eye displays, light-field monitors, and 3D screens.

THE SENSE OF ENTRANCE TO A PLACE IN KASHAN HISTORICAL HOUSES

The sense of place is like a mental perception of people individuals and the conscious feeling of the environment and the entrance as important signs cause the formation of mental image and the continuity of sense of belonging to the place in people. The character that the entrance takes is an important factor for the connection between inside and outside the building. The research question is what are the criteria for recognizing entrance architecture? And how have these criteria been able to improve the continuity of the sense of belonging to the place in the inhabitants of the historical context of Kashan? In the architecture of Kashan’s houses, entrance space as a conceptual element has high importance and reflects the architectural, cultural, and social values. This important element, which is based on behavioral patterns, has lost its identity today and has been reduced scale to a separating outside from inside. Studies show that integration and integrity, the radius of vision, and visual depth are three important indices designed based on them. This research through case study and using Depth map software has examined these characteristics at the entrance of nine historical houses in Kashan. The results show that the design of these entrances has the highest degree of coherence and a maximum radius of view and due to high visual depth have high privacy.

A novel autonomous learning framework to enhance sEMG-based hand gesture recognition using depth information

Hand gesture recognition using surface electromyography (sEMG) has been one of the most efficient motion analysis techniques in human–computer interaction in the last few decades. In particular, multichannel sEMG techniques have achieved stable performance in hand gesture recognition. However, the general solution of collecting and labeling large data manually leads to time-consuming implementation. A novel learning method is therefore needed to facilitate efficient data collection and preprocessing. In this paper, a novel autonomous learning framework is proposed to integrate the benefits of both depth vision and EMG signals, which automatically label the class of collected EMG data using depth information. It then utilizes a multiple layer neural network (MNN) classifier to achieve real-time recognition of the hand gestures using only the sEMG. The overall framework is demonstrated in an augmented reality application by the recognition of 10 hand gestures using the Myo armband and an HTC VIVE PRO. The results show prominent performance by introducing depth information for real-time data labeling.

Motion parallax via head movements modulates visuo-motor control in pigeons.

Although it has been proposed that birds acquire visual depth cues through dynamic head movements, behavioral evidence on how birds use motion parallax depth cues caused by self-motion is lacking. This study investigated whether self-generated motion parallax modulates pecking motor control and visual size perception in pigeons (Columba livia). We trained pigeons to peck a target on a touch monitor and to classify it as small or large. To manipulate motion parallax of the target, we changed the target position on the monitor according to the bird's head position in real time using a custom-built head tracker with two cameras. Pecking motor control was affected by the manipulation of motion parallax: when the motion parallax signified the target position farther than the monitor surface, the head position just before pecking to target was near the monitor surface, and vice versa. By contrast, motion parallax did not affect how the pigeons classified target sizes, implying that motion parallax might not contribute to size constancy in pigeons. These results indicate that motion parallax via head movements modulates pecking motor control in pigeons, suggesting that head movements of pigeons have the visual function of accessing motion parallax depth cues.

Corneal endothelial cell loss after trabeculectomy and phacoemulsification in one or two steps: a prospective study

ObjectiveThe objective of this study was to analyse the results of the surgical treatment of coexisting cataract and glaucoma and its effects on corneal endothelial cell density (CECD).MethodsWe include two longitudinal prospective studies: one randomised that included 40 eyes with open angle glaucoma that received one- (n = 20) or two-step (n = 20) phacotrabeculectomy and another that included 20 eyes that received phacoemulsification. We assess the impact of surgery on different clinical variables and in particular in CECD using Confoscan 4™ confocal microscopy and semiautomatic counting methods.ResultsPhacoemulsification and phacotrabeculectomy, but not trabeculectomy, increase significantly best-corrected visual acuity and anterior chamber depth and trabeculectomy and one- or two-step phacotrabeculectomy decreased similarly the intraocular pressure. We document percentages of endothelial cell loss of 3.1%, 17.9%, 31.6% and 42.6% after trabeculectomy, phacoemulsification and one- or two-step phacotrabeculectomy, respectively. The coefficient of variation did not increase significantly after surgery but the percentage of hexagonality decreased significantly after phacoemulsification and after two-step phacotrabeculectomy.ConclusionsTrabeculectomy, phacoemulsification and phacotrabeculectomy are surgical techniques that cause morphological changes and decrease the densities of the corneal endothelial cells. Trabeculectomy produces lesser endothelial cell loss than phacoemulsification, and phacoemulsification lesser cell loss than phacotrabeculectomy. Two-step phacotrabeculectomy (trabeculectomy followed 3 months later by phacoemulsification) causes more cell loss than one-step phacotrabeculectomy, and this could be due to the cumulative effects of two separate surgical traumas or to a negative conditioning lesion effect of the first surgery. For the treatment of coexisting glaucoma and cataract, one-step phacotrabeculectomy is the treatment of choice.

The effect of active hand movement on visually perceived depth direction.

This study investigated the effect of active hand movement on the perception of 3-D depth change. In Experiment 1, the 3-D height of an object synchronously changed with the participant’s hand movement, but the 3-D height of the object was incongruent with the distance moved by the hand. The results showed no effect of active hand movement on perceived depth. This was inconsistent with the results of a previous study conducted in a similar setting with passive hand movement. It was speculated that this contradiction appeared because the conflict between the distance moved by the hand and visual depth changes were more easily detected in the active movement situation. Therefore, it was assumed that in a condition where this conflict was hard to detect, active hand movement might affect visual depth perception. To examine this hypothesis, Experiment 2 examined whether information from hand movement would resolve the ambiguity in the depth direction of a shaded visual shape. In this experiment, the distance moved by the hand could (logically) accord with either of two depth directions (concave or convex). Moreover, the discrepancy in the distances between visual and haptic perception could be ambiguous because shading cues are unreliable in estimating absolute depth. The results showed that perceived depth directions were affected by the direction of active hand movement, thus supporting the hypothesis. Based on these results, simulations based on a causal inference model were performed, and it was found that these simulations could replicate the qualitative aspects of the experimental results.

Mesoscopic cortical network reorganization during recovery of optic nerve injury in GCaMP6s mice

As the residual vision following a traumatic optic nerve injury can spontaneously recover over time, we explored the spontaneous plasticity of cortical networks during the early post-optic nerve crush (ONC) phase. Using in vivo wide-field calcium imaging on awake Thy1-GCaMP6s mice, we characterized resting state and evoked cortical activity before, during, and 31 days after ONC. The recovery of monocular visual acuity and depth perception was evaluated in parallel. Cortical responses to an LED flash decreased in the contralateral hemisphere in the primary visual cortex and in the secondary visual areas following the ONC, but was partially rescued between 3 and 5 days post-ONC, remaining stable thereafter. The connectivity between visual and non-visual regions was disorganized after the crush, as shown by a decorrelation, but correlated activity was restored 31 days after the injury. The number of surviving retinal ganglion cells dramatically dropped and remained low. At the behavioral level, the ONC resulted in visual acuity loss on the injured side and an increase in visual acuity with the non-injured eye. In conclusion, our results show a reorganization of connectivity between visual and associative cortical areas after an ONC, which is indicative of spontaneous cortical plasticity.

Recent Progress on Ultrathin Metalenses for Flat Optics.

SummaryAs technology advances, electrical devices such as smartphones have become more and more compact, leading to a demand for the continuous miniaturization of optical components. Metalenses, ultrathin flat optical elements composed of metasurfaces consisting of arrays of subwavelength optical antennas, provide a method of meeting those requirements. Moreover, metalenses have many other distinctive advantages including aberration correction, active tunability, and semi-transparency, compared to their conventional refractive and diffractive counterparts. Therefore, over the last decade, great effort has been focused on developing metalenses to investigate and broaden the capabilities of metalenses for integration into future applications. Here, we discuss recent progress on metalenses including their basic design principles and notable characteristics such as aberration correction, tunability, and multifunctionality.

Soundtrack loudness as a depth cue in stereoscopic 3D media

Assisted by the technological advances of the past decades, stereoscopic 3D cues are being integrated increasingly in both interactive and non-interactive media. Arguably, the main focus of this effort is placed on the creation of an increased sense of visual depth. Considering that human perception relies heavily on the audiovisual integration rather than on visual information alone, it is rather surprising that relatively little attention has been given so far to the potential effect of the soundtrack on 3D depth perception in contrast to the evident interest towards the study of realistic 3D audio spatialisation techniques and technologies. The multisensory nature of human perception suggests that the potential of sound design as a means to influence depth perception in the stereoscopic 3D visual environments may be worthy of further exploration. This study reports on our research into the possibilities of using alterations of the volume levels of the soundtrack in order to influence the perception of visual depth while viewing stereoscopic 3D animation clips. Based on previous findings indicating that the volume level of the soundtrack may be related to the perception of visual depth, a series of experiments further explored the effectiveness of this auditory cue. Results suggest that, under certain conditions, differences in volume levels of the soundtrack could influence the judgement of visual depth in a way that frequently contradicts real-life expectations. It is suggested that different, more metaphorical perceptual mechanisms may be in play when viewing stereoscopic 3D presentations than in real life. In this context, we conclude that stereoscopic 3D media can benefit from further exploration of the effectiveness of certain auditory cues as a means to influence the perception of depth within the stereoscopic 3D environment.

An Object Recognition and Volume Calculation Method Based on Yolov3 and Depth Vision

In recent years, the logistics industry is in its full swing. The demand for logistics is increasing throughout the country, and this rising trend will continue during the following decades. With the substantial increase of workloads, traditional manual ways are increasingly unable to meet the new needs of the logistics industry, therefore new production and transportation modes are becoming imperative. Considering this, a new method was proposed to help detect, recognize and calculate the volume of rectangular objects automatically. In this method, a YOLO-based model was firstly applied to detect the object. Secondly, an RGB-D camera was used to obtain the depth information. Finally, after the length, width and height values were estimated from a single surface, the volume of the object can be calculated. Experimental results showed that the method is simpler and more accurate as compared with traditional three-dimensional reconstruction-based methods.

The Effects of Depth Cues and Vestibular Translation Signals on the Rotation Tolerance of Heading Tuning in Macaque Area MSTd.

When the eyes rotate during translational self-motion, the focus of expansion (FOE) in optic flow no longer indicates heading, yet heading judgements are largely unbiased. Much emphasis has been placed on the role of extraretinal signals in compensating for the visual consequences of eye rotation. However, recent studies also support a purely visual mechanism of rotation compensation in heading-selective neurons. Computational theories support a visual compensatory strategy but require different visual depth cues. We examined the rotation tolerance of heading tuning in macaque area MSTd using two different virtual environments, a frontoparallel (2D) wall and a 3D cloud of random dots. Both environments contained rotational optic flow cues (i.e., dynamic perspective), but only the 3D cloud stimulus contained local motion parallax cues, which are required by some models. The 3D cloud environment did not enhance the rotation tolerance of heading tuning for individual MSTd neurons, nor the accuracy of heading estimates decoded from population activity, suggesting a key role for dynamic perspective cues. We also added vestibular translation signals to optic flow, to test whether rotation tolerance is enhanced by non-visual cues to heading. We found no benefit of vestibular signals overall, but a modest effect for some neurons with significant vestibular heading tuning. We also find that neurons with more rotation tolerant heading tuning typically are less selective to pure visual rotation cues. Together, our findings help to clarify the types of information that are used to construct heading representations that are tolerant to eye rotations.

Visual cortical GABA and depth of amblyopia are negatively correlated

Visual cortical GABA and depth of amblyopia are negatively correlated

Evaluation of daylight distribution and space visual quality at medical centers through spatial layout

ABSTRACT This research investigates the impact of the layout of the interior spaces, as an independent variable, on the distribution of daylight factor, uniformity rate, drift magnitude, maximum radial line of sight and visual mean depth as dependent variables in medical centers. A quantitative-qualitative research methodology was adopted, including library studies and computer-assisted simulations using Relux software. Also, Depthmap software was utilized to study the indices of space visual quality through Isovist analyses under some scenarios. Results indicated that the appropriate and utilization-oriented layout of interior elements takes a significant part in improving the space visual quality experienced by the users. In this study, an L-type layout with openings oriented toward the light source and vertical partitions along the path through which the daylight got into space was presented as a suitable plan for medical centers as far as optimal and appropriate distribution of daylight within standard ranges were concerned. This layout provided a uniformity rate of 0.5 and a daylight factor close to 5%, with the indicators of space visual quality being more favorable, as compared to the other layouts considered in this study.

Visual Search Fixation Strategies in a 3D Image Set: An Eye-Tracking Study

AbstractIn this study, we explore whether the inclusion of monocular depth within a pseudo-3D picture gallery negatively affects visual search strategy and performance. Experimental design facilitated control of (i) the number of visible depth planes and (ii) the presence of semantic sorting. Our results show that increasing the number of visual depth planes facilitates efficiency in search, which in turn results in a decreased response time to target selection and a reduction in participant average pupil dilation—used for measuring cognitive load. Furthermore, results identified that search strategy is based on sorting, which implies that an appropriate management of semantic associations can increase search efficiency by decreasing the number of potential targets.

Weak Memories: Archives of Futurability

The Portuguese documentary filmmaker Susana de Sousa Dias offers a comprehensive overview of her career over the past fifteen years, beginning with Natureza Morta (Still Life, 2005) through her current project, Estação Total (Total Station, working title). She explains that her films aim to reflect on authoritarian systems—especially the forty-eight year rule of Portuguese dictator António de Oliveira Salazar—and to shed light on the present through the materials of the past. In terms of her cinematic praxis, she discusses the vital importance of two techniques: “montage within the shot”—that is, a montage with visual, spatial, and temporal depth—and “decelerated movement,” which describes her approach to time and duration. Her most recent film, Fordlandia Malaise (2019), extends her work on repressive regimes to encompass current geopoliticals tructures with their underlying colonialist and imperialist heritage, whether derived from a European origin or from the ruth less capitalism of the New World.