Visual Processing System Research Articles

Blind image quality assessment with semantic information

No-reference (NR) image quality assessment (IQA) aims to evaluate the quality of an image without reference image, which is greatly desired in the automatic visual signal processing system. Distortions degrade the visual contents and affect the semantics acquisition during the process of human perception. Although the existing methods evaluate the quality of images based on the structure, texture, or statistical characteristics, and deliver high quality prediction accuracy, they do not take the spatial semantics into account. From the perspective of human perception, distortions decrease the structural semantics that represent the structural information, and disturb the spatial semantics that describe the contents of images. Therefore, we attempt to measure the image quality by its degradation of semantics in an image. To extract the semantics of an image, a semantic network is proposed. The network contains convolutional neural networks (CNN) and Long Short-Term Memory (LSTM) that correspond to structural semantics and spatial semantics, respectively. CNN can be regarded as a coarse imitation of human visual mechanism to obtain the structural information, and LSTM can express the contents of an image. Then, by measuring the degradations of different semantics on images, a novel NR IQA is introduced. The proposed approach is evaluated on the databases of LIVE, CSIQ, TID2013, and LIVE multiply distorted database as well as LIVE in the wild image quality challenge database, and the results show superior performance to other state-of-the-art NR IQA methods. Furthermore, we explore the generalization capability of the proposed approach, and the experimental results indicate the proposed approach has a high robustness.

Taking into consideration explanations of perception-action interactions that may be “less dramatic, but more reflective of what happens in the real world”

Bruce Bridgeman and colleagues reported the first experiments providing evidence of two functionally distinct visual-processing systems. We summarize that work and subsequent research that resulted in modifications of this view. Then, we describe studies of stimulus-response correspondence effects that provide evidence for distinct representations of responses. More recently, Bridgeman and colleagues examined whether "action affects perception", concluding that the phenomena can be more accurately construed as "information affects memory". Although unconvinced about claims of action-affects-perception and embodied cognition, Bridgeman and colleagues concluded that processing of visual information in hand-space is facilitated and cited a phenomenon as supporting evidence. We discuss findings indicating that this phenomenon is due to general spatial coding principles. We think that all researchers should proceed in the manner of Bridgeman of developing novel explanations, devising critical tests between them and alternative possible explanations, and accepting the explanation that best conforms to the results, even if that explanation is a "less dramatic" option.

A machine learning framework for full-reference 3D shape quality assessment

To decide whether the perceived quality of a mesh is influenced by a certain modification such as compression or simplification, a metric for estimating the visual quality of 3D meshes is required. Today, machine learning and deep learning techniques are getting increasingly popular since they present efficient solutions to many complex problems. However, these techniques are not much utilized in the field of 3D shape perception. We propose a novel machine learning-based approach for evaluating the visual quality of 3D static meshes. The novelty of our study lies in incorporating crowdsourcing in a machine learning framework for visual quality evaluation. We deliberate that this is an elegant way since modeling human visual system processes is a tedious task and requires tuning many parameters. We employ crowdsourcing methodology for collecting data of quality evaluations and metric learning for drawing the best parameters that well correlate with the human perception. Experimental validation of the proposed metric reveals a promising correlation between the metric output and human perception. Results of our crowdsourcing experiments are publicly available for the community.

Use and Usefulness of Dynamic Face Stimuli for Face Perception Studies-a Review of Behavioral Findings and Methodology.

Faces that move contain rich information about facial form, such as facial features and their configuration, alongside the motion of those features. During social interactions, humans constantly decode and integrate these cues. To fully understand human face perception, it is important to investigate what information dynamic faces convey and how the human visual system extracts and processes information from this visual input. However, partly due to the difficulty of designing well-controlled dynamic face stimuli, many face perception studies still rely on static faces as stimuli. Here, we focus on evidence demonstrating the usefulness of dynamic faces as stimuli, and evaluate different types of dynamic face stimuli to study face perception. Studies based on dynamic face stimuli revealed a high sensitivity of the human visual system to natural facial motion and consistently reported dynamic advantages when static face information is insufficient for the task. These findings support the hypothesis that the human perceptual system integrates sensory cues for robust perception. In the present paper, we review the different types of dynamic face stimuli used in these studies, and assess their usefulness for several research questions. Natural videos of faces are ecological stimuli but provide limited control of facial form and motion. Point-light faces allow for good control of facial motion but are highly unnatural. Image-based morphing is a way to achieve control over facial motion while preserving the natural facial form. Synthetic facial animations allow separation of facial form and motion to study aspects such as identity-from-motion. While synthetic faces are less natural than videos of faces, recent advances in photo-realistic rendering may close this gap and provide naturalistic stimuli with full control over facial motion. We believe that many open questions, such as what dynamic advantages exist beyond emotion and identity recognition and which dynamic aspects drive these advantages, can be addressed adequately with different types of stimuli and will improve our understanding of face perception in more ecological settings.

Attenuation of Retinal Vascular Development in Neonatal Mice Subjected to Hypoxic-Ischemic Encephalopathy

A significant proportion of children that survive hypoxic-ischemic encephalopathy (HIE) develop visual impairment. These visual deficits are generally attributed to injuries that occur in the primary visual cortex and other visual processing systems. Recent studies suggested that neuronal damage might also occur in the retina. An important structure affecting the viability of retinal neurons is the vasculature. However, the effects of HIE on the retinal neurovasculature have not been systemically evaluated. Here we investigated whether exposure of postnatal day 9 (P9) neonatal mice to HIE is sufficient to induce neurovascular damage in the retina. We demonstrate that the blood vessels on the surface of the retina, from mice subjected to HIE, were abnormally enlarged with signs of degeneration. The intermediate and deep vascular layers in these retinas failed to form normally, particularly in the periphery. All the vascular damages observed here were irreversible in nature up to 100 days post HIE. We also observed loss of retinal neurons, together with changes in both astrocytes and Müller cells mainly in the inner retina at the periphery. Collectively, our findings suggest that HIE results in profound alterations in the retinal vasculature, indicating the importance of developing therapeutic strategies to protect neurovascular dysfunction not only in the brain but also in the retina for infants exposed to HIE.

Enhanced Robotic Hand–Eye Coordination Inspired From Human-Like Behavioral Patterns

Robotic hand–eye coordination is recognized as an important skill to deal with complex real environments. Conventional robotic hand–eye coordination methods merely transfer stimulus signals from robotic visual space to hand actuator space. This paper introduces a reverse method. Build another channel that transfers stimulus signals from robotic hand space to visual space. Based on the reverse channel, a human-like behavior pattern: “Stop-to-Fixate,” is imparted to the robot, thereby giving the robot an enhanced reaching ability. A visual processing system inspired by the human retina structure is used to compress visual information so as to reduce the robot’s learning complexity. In addition, two constructive neural networks establish the two sensory delivery channels. The experimental results demonstrate that the robotic system gradually obtains a reaching ability. In particular, when the robotic hand touches an unseen object, the reverse channel successfully drives the visual system to notice the unseen object.

Experiment Study on Fluidelastic Instability of Tube Bundles Consisting of Different Frequency Tubes With Visual Image Processing System

Fluidelastic instability (FEI) is the most harmful vibration mechanism for heat exchangers. Due to the inevitable manufacturing precision and assembly error, natural frequencies of tubes are not equal in the ideal condition. In order to describe the dispersion characteristic of tube bundles, a new factor named dispersion ratio is proposed in this paper. A series of tubes experiments in normal square and rotated triangular array with pitch ratio s = 1.4 and s = 1.28 were designed and conducted with high-speed camera and visual image processing system. Results show that FEI behaviors of tubes were greatly affected by tubes array geometry, pitch ratio, and dispersion ratio. Reduced critical velocity (Vcr) increased with dispersion ratio in normal square array but no obvious phenomenon was observed in rotated triangular array.

Perceptual averaging of facial expressions requires visual awareness and attention.

Humans, as highly social animals, are regularly exposed to the faces of conspecifics-often more than one at a time. This feature of social living is important for understanding face perception, not just because it means that information from faces is available in bulk, but also because it changes the way individuals are perceived. For instance, when two faces are seen nearby one another, they tend to look like each other. This phenomenon of perceptual averaging is robust when both faces are seen and attended. But in everyday life, some faces may not receive the full benefit of attention, or they may not be visible at all. We evaluated whether perceptual averaging of relatively complex and simple information on faces, including facial expression and head orientation, can still occur even in these circumstances. In particular, we used object-substitution masking (OSM) and a dual-task designed to disrupt visual awareness and attention, respectively, during evaluations of briefly presented face pairs. Disruptions of awareness or attention eliminated averaging of facial expression, whereas orientation averaging persisted in spite of these challenges. These results demonstrate boundary conditions for the process of perceptual averaging. More generally, they provide insight into how the visual system processes multitudes of objects, both simple and complex, both with and without attention and awareness.

Coding Schemes in the Archerfish Optic Tectum

Many studies have yielded valuable knowledge on the early visual system but it is biased since the studies have focused on terrestrial mammals alone. Here, to better account for visual systems in different environments and animal classes, we studied the structure of early visual processing in the archerfish which harnesses its extreme visual ability to hunt by shooting water jets at prey hanging on vegetation above the water. Thus, the archerfish provides a unique opportunity to study visual processing in a vertebrate which is an expert vision-guided predator with a very different brain structure than mammals. The receptive field structures in the archerfish (both sexes) optic tectum, the main visual processing region in the fish brain, were measured and linear non-linear cascades were used to analyze their properties. The findings indicate that the spatial receptive field structures lie on a continuum between circular and elliptical shapes. In addition, the cells' functional properties display a richness of response characteristics, since many cells could be captured by more than a single linear filter. Finally, the non-linear response functions that link linear filters and neuronal responses were found to be similar to the non-linear functions of models that describe terrestrial mammalian single cell activity. Overall our results help to better understand the early visual processing system across vertebrates.

From percept to concept in the ventral temporal lobes: Graded hemispheric specialisation based on stimulus and task

The left and right ventral anterior temporal lobes (vATL) have been implicated as key regions for the representation of conceptual knowledge. However, the nature and degree of hemispheric specialisation in their function is unclear. To address this issue, we investigated hemispheric specialisation in the ventral temporal lobes using a distortion-corrected spin-echo fMRI protocol that enhanced signal in vATL. We employed an orthogonal manipulation of stimulus (written words vs pictured objects) and task (naming vs recognition). Words elicited left-lateralised vATL activation while objects elicited bilateral activation with no hemispheric bias. In contrast, posterior ventral temporal cortex exhibited a rightward bias for objects as well as a leftward bias for words. Naming tasks produced left-lateralised activation in vATL while activity for recognition was equal in left and right vATLs. These findings are incompatible with proposals that left and right ATLs are strongly modular in function, since these predict rightward as well as leftward biases. Instead, they support an alternative model in which (a) left and right ATL together form a bilateral, integrated system for the representation of concepts and (b) within this system, graded hemispheric specialisation emerges as a consequence of differential connectivity with other neural systems. On this view, greater left vATL activation for written word processing develops as a consequence of the inputs this region receives from left-lateralised visual word processing system in posterior temporal cortex. Greater left vATL activation during naming tasks is most likely due to connectivity with left-lateralised speech output systems in prefrontal and motor cortices.

Motor Symptoms of Autoimmune Encephalopathies

Autoimmune encephalopathies are clinically and immunologically heterogeneous disorders. At least 20 types of autoimmune encephalopathies have been discovered, with the most common type being Hashimoto encephalopathy. In clinical situations, we often observe that patients with autoimmune encephalopathy are misdiagnosed because they exhibit signs similar to those observed in functional psychogenic movement, conversion, or somatoform disorders. We clinically examined over 100 patients with autoimmune encephalopathy. These patients primarily demonstrated motor disturbances, mostly with give-way weakness, sensory abnormalities, and involuntary movements such as tremor entrainment, dystonia, or coarse involuntary movement. In addition, we observed memory loss, psychogenic non-epileptic seizures, epilepsy, and autonomic symptoms in our patients. To diagnose autoimmune encephalopathies, we propose that a combination of neurological symptoms indicating "diffuse brain damage" be used. "Diffuse brain damage" is a proof of several symptoms, such as give-way weakness; motor symptoms such as paralysis, smoothness disorder of exercise, involuntary movements, and difficulty to sustain; abnormal sensations such as pain, abnormal perception of various parts, and impaired vibration sensation; deterioration of higher order functions such as memory and learning ability; and impairment of the visual processing system and various visual abnormalities. As patients with autoimmune encephalitis exhibit diffuse involvement, the presence of these symptoms was entirely understandable. Over three such abnormal findings could indicate diffuse brain damage. Owing to the regular understanding in neurology, most patients tend to be diagnosed with somatoform disorders. Thus, physicians should not diagnose somatoform disorders without first excluding autoimmune encephalopathy.

Perception of contextual size illusions by honeybees in restricted and unrestricted viewing conditions.

How different visual systems process images and make perceptual errors can inform us about cognitive and visual processes. One of the strongest geometric errors in perception is a misperception of size depending on the size of surrounding objects, known as the Ebbinghaus or Titchener illusion. The ability to perceive the Ebbinghaus illusion appears to vary dramatically among vertebrate species, and even populations, but this may depend on whether the viewing distance is restricted. We tested whether honeybees perceive contextual size illusions, and whether errors in perception of size differed under restricted and unrestricted viewing conditions. When the viewing distance was unrestricted, there was an effect of context on size perception and thus, similar to humans, honeybees perceived contrast size illusions. However, when the viewing distance was restricted, bees were able to judge absolute size accurately and did not succumb to visual illusions, despite differing contextual information. Our results show that accurate size perception depends on viewing conditions, and thus may explain the wide variation in previously reported findings across species. These results provide insight into the evolution of visual mechanisms across vertebrate and invertebrate taxa, and suggest convergent evolution of a visual processing solution.

The contribution of foveal and peripheral visual information to ensemble representation of face race.

The brain can only attend to a fraction of all the information that is entering the visual system at any given moment. One way of overcoming the so-called bottleneck of selective attention (e.g., J. M. Wolfe, Võ, Evans, & Greene, 2011) is to make use of redundant visual information and extract summarized statistical information of the whole visual scene. Such ensemble representation occurs for low-level features of textures or simple objects, but it has also been reported for complex high-level properties. While the visual system has, for example, been shown to compute summary representations of facial expression, gender, or identity, it is less clear whether perceptual input from all parts of the visual field contributes equally to the ensemble percept. Here we extend the line of ensemble-representation research into the realm of race and look at the possibility that ensemble perception relies on weighting visual information differently depending on its origin from either the fovea or the visual periphery. We find that observers can judge the mean race of a set of faces, similar to judgments of mean emotion from faces and ensemble representations in low-level domains of visual processing. We also find that while peripheral faces seem to be taken into account for the ensemble percept, far more weight is given to stimuli presented foveally than peripherally. Whether this precision weighting of information stems from differences in the accuracy with which the visual system processes information across the visual field or from statistical inferences about the world needs to be determined by further research.

3D computer vision based on machine learning with deep neural networks: A review

AbstractRecent advances in the field of computer vision can be attributed to the emergence of deep learning techniques, in particular convolutional neural networks. Neural networks, partially inspired by the brain's visual cortex, enable a computer to “learn” the most important features of the images it is shown in relation to a specific, specified task. Given sufficient data and time, (deep) convolutional neural networks offer more easily designed, more generalizable, and significantly more accurate end‐to‐end systems than is possible with previously employed computer vision techniques. This review paper seeks to provide an overview of deep learning in the field of computer vision with an emphasis on recent progress in tasks involving 3D visual data. Through a backdrop of the mammalian visual processing system, we hope to also provide inspiration for future advances in automated visual processing.

Magnocellular Bias in Exogenous Attention to Biologically Salient Stimuli as Revealed by Manipulating Their Luminosity and Color.

Exogenous attention is a set of mechanisms that allow us to detect and reorient toward salient events-such as appetitive or aversive-that appear out of the current focus of attention. The nature of these mechanisms, particularly the involvement of the parvocellular and magnocellular visual processing systems, was explored. Thirty-four participants performed a demanding digit categorization task while salient (spiders or S) and neutral (wheels or W) stimuli were presented as distractors under two figure-ground formats: heterochromatic/isoluminant (exclusively processed by the parvocellular system, Par trials) and isochromatic/heteroluminant (preferentially processed by the magnocellular system, Mag trials). This resulted in four conditions: SPar, SMag, WPar, and WMag. Behavioral (RTs and error rates in the task) and electrophysiological (ERPs) indices of exogenous attention were analyzed. Behavior showed greater attentional capture by SMag than by SPar distractors and enhanced modulation of SMag capture as fear of spiders reported by participants increased. ERPs reflected a sequence from magnocellular dominant (P1p, ≃120 msec) to both magnocellular and parvocellular processing (N2p and P2a, ≃200 msec). Importantly, amplitudes in one N2p subcomponent were greater to SMag than to SPar and WMag distractors, indicating greater magnocellular sensitivity to saliency. Taking together, results support a magnocellular bias in exogenous attention toward distractors of any nature during initial processing, a bias that remains in later stages when biologically salient distractors are present.

Chinese-character crowding-I. Effects of structural similarity.

Crowding impedes the identification of flanked objects in peripheral vision. Prior studies have shown crowding strength decreases with target-flanker similarity. Research on crowding in Chinese-character recognition has been scarce in the literature. We aimed to fill the research gap by examining the effects of structural similarity on Chinese-character crowding. Regularity in within-character configuration, i.e., orthographic legality, of flankers was manipulated in Experiment 1. Target-flanker similarity in orthographic legality did not affect crowding strength, measured as contrast threshold elevation. Crowding weakened only when the strokes in the flankers were scrambled. Contour integrity of flankers was manipulated by randomly perturbing the phase spectra of the stimulus images in Experiments 2a and 2b. Crowding by perturbed-phase flankers remained robust but was weaker compared with intact-phase flankers. Target-flanker similarity in contour integrity modulated crowding strength. Our findings were consistent with the postulation that faulty integration of low-level visual features contributed to crowding of Chinese characters. Studies on Chinese-character recognition and crowding can provide important insights into how the visual system processes complex daily objects.

Insight into the Role of Photoreception and Light Intervention for Sleep and Neuropsychiatric Behaviour in the Elderly.

Light exerts influences on many physiological and behavioural functions in humans. These functions can be described as image-forming (IF) and non-image forming (NIF) visual processes, both originating in the retina of the eye. Image-forming refers to vision; the process of detecting and distinguishing shapes and colour of objects. Non-image forming refers to detecting level of light intensity or brightness of ambient space, which affects basal physiology such as cycles of rest and activity or the endocrine system. Rod and cone photoreceptors in the outer retinal layer are most important for imageforming vision, while non-image forming functions depend upon additional input from the photopigment melanopsin, which is expressed in retinal ganglion cells (RGC) that makes these cells photosensitive (pRGC). Projections of these pRGCs convey light-induced electrical impulses to a number of brain regions. Visual acuity and colour contrast naturally diminishes with age but dementia often has major effects on the visual processing systems, which impact on the quality of life. The ability of humans to manipulate their light exposure has the immediate potential to either create problems with human physiology (as in shift workers) or to compensate physiological disadvantages (of IF and NIF visual impairment). This mini-review describes the impact of aging on the function of the eye with respect to nonimage forming effects of light, summarises light intervention studies for sleep and neuropsychiatric symptoms and considers implications from photoreceptor-weighted light intensities for biologically effective light intervention and lighting solutions for patients with dementia.

Sunlight irradiance and habituation of visual evoked potentials in migraine: The environment makes its mark.

Background Migraine is a complex multifactorial disease that arises from the interaction between a genetic predisposition and an enabling environment. Habituation is considered as a fundamental adaptive behaviour of the nervous system that is often impaired in migraine populations. Given that migraineurs are hypersensitive to light, and that light deprivation is able to induce functional changes in the visual cortex recognizable through visual evoked potentials habituation testing, we hypothesized that regional sunlight irradiance levels could influence the results of visual evoked potentials habituation studies performed in different locations worldwide. Methods We searched the literature for visual evoked potentials habituation studies comparing healthy volunteers and episodic migraine patients and correlated their results with levels of local solar radiation. Results After reviewing the literature, 26 studies involving 1291 participants matched our inclusion criteria. Deficient visual evoked potentials habituation in episodic migraine patients was reported in 19 studies. Mean yearly sunlight irradiance was significantly higher in locations of studies reporting deficient habituation. Correlation analyses suggested that visual evoked potentials habituation decreases with increasing sunlight irradiance in migraine without aura patients. Conclusion Results from this hypothesis generating analysis suggest that variations in sunlight irradiance may induce adaptive modifications in visual processing systems that could be reflected in visual evoked potentials habituation, and thus partially account for the difference in results between studies performed in geographically distant centers. Other causal factors such as genetic differences could also play a role, and therefore well-designed prospective trials are warranted.

ViPS: A novel visual processing system architecture for medical imaging

Abstract Imaging has become an essential tool in modern medicine science. Numerous powerful platforms to register, store, analyze and process medical imaging applications appear in recent years. In this article, we have designed an advanced visual processing system (ViPS) that stores and processes complex and multi-dimensional medical imaging application. The ViPS provides a user-friendly programming environment and high-performance architecture for data acquisition, registration, storage, analysis and performs segmentation, filtering, and recognition of complex real-time complex and multidimensional medical images or videos. The proposed architecture is highly reliable concerning cost, performance, and power. ViPS is designed and evaluated on a Xilinx Virtex-7 FPGA VC707 Evaluation Kit. The performance of ViPS is compared with the Heterogeneous Multi-Processing Odroid XU3 board and GPU Jetson TK1 Embedded Development Kit with 192 CUDA cores based graphic systems. In comparison to the Heterogeneous Multi-core and GPU based Graphics systems, the results show that ViPS improves 2.4 and 1.4 times system performance respectively, for iridology application. While executing real-time complex images reconstruction at 2× and 1.25× of higher frame rate, the ViPS achieves 15.2× and 5.26× of performance improvement while running various image processing algorithms. The ViPS gets 3.01× and 1.13× of speedup for video processing algorithms and draws 1.55 and 2.27 times less dynamic power.

Natural Inspired Intelligent Visual Computing and Its Application to Viticulture.

This paper presents an investigation of natural inspired intelligent computing and its corresponding application towards visual information processing systems for viticulture. The paper has three contributions: (1) a review of visual information processing applications for viticulture; (2) the development of natural inspired computing algorithms based on artificial immune system (AIS) techniques for grape berry detection; and (3) the application of the developed algorithms towards real-world grape berry images captured in natural conditions from vineyards in Australia. The AIS algorithms in (2) were developed based on a nature-inspired clonal selection algorithm (CSA) which is able to detect the arcs in the berry images with precision, based on a fitness model. The arcs detected are then extended to perform the multiple arcs and ring detectors information processing for the berry detection application. The performance of the developed algorithms were compared with traditional image processing algorithms like the circular Hough transform (CHT) and other well-known circle detection methods. The proposed AIS approach gave a Fscore of 0.71 compared with Fscores of 0.28 and 0.30 for the CHT and a parameter-free circle detection technique (RPCD) respectively.