Visual Research Articles

Blind Quality Assessment of Dense 3D Point Clouds with Structure Guided Resampling

Objective quality assessment of 3D point clouds is essential for the development of immersive multimedia systems in real-world applications. Despite the success of perceptual quality evaluation for 2D images and videos, blind/no-reference metrics are still scarce for 3D point clouds with large-scale irregularly distributed 3D points. Therefore, in this paper, we propose an objective point cloud quality index with Structure Guided Resampling (SGR) to automatically evaluate the perceptually visual quality of dense 3D point clouds. The proposed SGR is a general-purpose blind quality assessment method without the assistance of any reference information. Specifically, considering that the human visual system (HVS) is highly sensitive to structure information, we first exploit the unique normal vectors of point clouds to execute regional pre-processing which consists of keypoint resampling and local region construction. Then, we extract three groups of quality-related features, including: 1) geometry density features; 2) color naturalness features; 3) angular consistency features. Both the cognitive peculiarities of the human brain and naturalness regularity are involved in the designed quality-aware features that can capture the most vital aspects of distorted 3D point clouds. Extensive experiments on several publicly available subjective point cloud quality databases validate that our proposed SGR can compete with state-of-the-art full-reference, reduced-reference, and no-reference quality assessment algorithms.

Fully optically controlled Li-ion-mediated artificial vision reflection arc system

Artificial vision systems have been extensively reported for processing unstructured data, performing pattern recognition, and performing visual simulations. Photonic synapses have become the core of most artificial vision systems due to their advantages such as low energy consumption and ultra-fast signal transmission. However, their integration is difficult, the integration difficulty of photon synapses is higher than that of crossbar arrays. Moreover, due to the hybrid drive of light and external power supply, low energy consumption and weight update under full optical control cannot be achieved. Here, we designed a fully light-controlled multifunctional artificial vision reflex arc system. The solar cell and oscillator generate presynaptic voltage signals that are then transmitted to a lithium-ion-based memristor for the stimulation of different wavelengths and light intensities. The results show that our artificial vision system can distinguish wavelength and light intensity information and has good weight update symmetry and repeatability. Finally, the deformation experiment of the shape memory alloy demonstrated that the system can be used for controlling artificial muscles under different light signals.

Heads-up 3-Dimensional Visualization to Enhance Video Endoscopy During Vitreoretinal Surgery

Purpose: To describe 2 cases in which video endoscopy was combined with heads-up 3-dimensional (3D) visualization. Methods: The imaging modalities of the heads-up 3D visualization system and video endoscopy were combined by converting the endoscope’s S-Video output into a high-definition multimedia interface. The technique was used in 2 cases that included lens dislocation and endophthalmitis. Results: In the 2 cases, the heads-up split-screen view simplified the surgical setup. Simultaneously viewing the endoscopic image and widefield 3D image anecdotally appeared to improve ergonomics, help keep the probe oriented, and assist in guiding surgical movements endoscopically. Conclusions: These cases show the advantages of a heads-up 3D system to simultaneously view endoscopic and widefield images during ophthalmic endoscopy in vitreoretinal surgery.

Uncommon opsin’s retinal isomer is involved in mammalian sperm thermotaxis

In recent years it became apparent that, in mammals, rhodopsin and other opsins, known to act as photosensors in the visual system, are also present in spermatozoa, where they function as highly sensitive thermosensors for thermotaxis. The intriguing question how a well-conserved protein functions as a photosensor in one type of cells and as a thermosensor in another type of cells is unresolved. Since the moiety that confers photosensitivity on opsins is the chromophore retinal, we examined whether retinal is substituted in spermatozoa with a thermosensitive molecule. We found by both functional assays and mass spectrometry that retinal is present in spermatozoa and required for thermotaxis. Thus, starvation of mice for vitamin A (a precursor of retinal) resulted in loss of sperm thermotaxis, without affecting motility and the physiological state of the spermatozoa. Thermotaxis was restored after replenishment of vitamin A. Using reversed-phase ultra-performance liquid chromatography mass spectrometry, we detected the presence of retinal in extracts of mouse and human spermatozoa. By employing UltraPerformance convergence chromatography, we identified a unique retinal isomer in the sperm extracts—tri-cis retinal, different from the photosensitive 11-cis isomer in the visual system. The facts (a) that opsins are thermosensors for sperm thermotaxis, (b) that retinal is essential for thermotaxis, and (c) that tri-cis retinal isomer uniquely resides in spermatozoa and is relatively thermally unstable, suggest that tri-cis retinal is involved in the thermosensing activity of spermatozoa.

Bio-inspired foveal super-resolution method for multi-focal-length images based on local gradient constraints

Most existing super-resolution (SR) imaging systems, inspired by the bionic compound eye, utilize image registration and reconstruction algorithms to overcome the angular resolution limitations of individual imaging systems. This article introduces a multi-aperture multi-focal-length imaging system and a multi-focal-length image super-resolution algorithm, mimicking the foveal imaging of the human eye. Experimental results demonstrate that with the proposed imaging system and an SR imaging algorithm inspired by the human visual system, the proposed method can enhance the spatial resolution of the foveal region by up to 4 × compared to the original acquired image. These findings validate the effectiveness of the proposed imaging system and computational imaging algorithm in enhancing image texture and spatial resolution.

Spatial–Temporal Analysis-Based Video Quality Assessment: A Two-Stream Convolutional Network Approach

In system processing, video inevitably suffers from distortion, which leads to quality degradation and affects the user experience. Therefore, it is of great importance to design an accurate and effective objective video quality assessment (VQA) method. In this paper, by considering the multi-dimensional characteristics for video and visual perceptual mechanism, a two-stream convolutional network for VQA is proposed based on spatial–temporal analysis, named TSCNN-VQA. Specifically, for feature extraction, TSCNN-VQA first extracts spatial and temporal features by two different convolutional neural network branches, respectively. After that, the spatial–temporal joint feature fusion is constructed to obtain the joint spatial–temporal features. Meanwhile, the TSCNN-VQA also integrates an attention module to guarantee that the process conforms to the mechanism that the visual system perceives video information. Finally, the overall quality is obtained by non-linear regression. The experimental results in both the LIVE and CSIQ VQA datasets show that the performance indicators obtained by TSCNN-VQA are higher than those of existing VQA methods, which demonstrates that TSCNN-VQA can accurately evaluate video quality and has better consistency with the human visual system.

Modeling Positions and Orientations of Cantaloupe Flowers for Automatic Pollination

An automatic system for cantaloupe flower pollination in greenhouses is proposed to meet the requirements of automatic pollination. The system consists of a mobile platform, robotic manipulator, and camera that reaches the flowers to detect and recognise their external features. The main task of the vision system is to detect the position and orientation of the flower in Cartesian coordinates, allowing the manipulator to reach the pose and perform pollination. A comprehensive method to ensure the accuracy of the pollination process is proposed that accurately determines the position and orientation of cantaloupe flowers in real environments. The vision system is used to capture images, detect the flower, and recognise its state according to its external features, such as size, colour, and shape, thereby providing appropriate nozzle access during pollination. The proposed approach begins with a segmentation method designed to precisely locate and segment the target cantaloupe flowers. Subsequently, a mathematical model is used to determine the key points that are important for establishing the growth orientation of each flower. Finally, an inverse-projection method is employed to convert the position of the flower from a two-dimensional (2D) image into a three-dimensional (3D) space, providing the necessary position for the pollination robot. The experimental process is conducted in a laboratory and proves the efficacy of the cantaloupe flower segmentation method, yielding precision, recall, and F1 scores of 87.91%, 90.76%, and 89.31%, respectively. Furthermore, the accuracy of the growth-orientation prediction method reaches approximately 86.7%. Notably, positional errors in 3D space predominantly fall within the allowable range, resulting in a successful pollination rate of up to 83.1%.

The Development of Socially Directed Attention: An Functional Magnetic Resonance Imaging Study in Infant Monkeys.

Socially guided visual attention, such as gaze following and joint attention, represents the building block of higher-level social cognition in primates, although their neurodevelopmental processes are still poorly understood. Atypical development of these social skills has served as early marker of autism spectrum disorder and Williams syndrome. In this study, we trace the developmental trajectories of four neural networks underlying visual and attentional social engagement in the translational rhesus monkey model. Resting-state fMRI (rs-fMRI) data and gaze following skills were collected in infant rhesus macaques from birth through 6 months of age. Developmental trajectories from subjects with both resting-state fMRI and eye-tracking data were used to explore brain-behavior relationships. Our findings indicate robust increases in functional connectivity (FC) between primary visual areas (primary visual cortex [V1] - extrastriate area 3 [V3] and V3 - middle temporal area, ventral motion areas middle temporal area - AST, as well as between TE and amygdala (AMY) as infants mature. Significant FC decreases were found in more rostral areas of the pathways, such as areas temporal area occipital part - TE in the ventral object pathway, V3 - lateral intraparietal (LIP) of the dorsal visual attention pathway and V3 - temporo-parietal area of the ventral attention pathway. No changes in FC were found between cortical areas LIP-FEF and temporo-parietal area - Area 12 of the dorsal and ventral attention pathways or between AST-AMY and AMY-insula. Developmental trajectory of gaze following revealed a period of dynamic changes with gradual increases from 1 to 2 months, followed by slight decreases from 3 to 6 months. Exploratory association findings across the 6-month period showed that infants with higher gaze following had lower FC between primary visual areas V1-V3, but higher FC in the dorsal attention areas V3-LIP, both in the right hemisphere. Together, the first 6 months of life in rhesus macaques represent a critical period for the emergence of gaze following skills associated with maturational changes in FC of socially guided attention pathways.

Calibration method for binocular vision system with large field of view based on small target image splicing

In vision measurement, camera calibration has a significant impact on measurement precision. The classical target-based calibration methods require the target to occupy more than one-third of the field of view. A small-size target that does not meet the requirements results in poor calibration accuracy, while an appropriate large-size target is difficult to manufacture and inconvenient to operate. In view of the above problem, we propose a flexible and accurate calibration method based on small target image splicing to calibrate the binocular vision system with a large field of view. The spliced images and virtual large targets are constructed to extend the target size, providing better flexibility for calibration. Moreover, an optimization objective function integrating two constraints in the imaging plane and measurement space is presented to improve the calibration accuracy during the parameter optimization process. The simulation experiments and actual experiments are carried out to test the performance of the proposed method. The results demonstrate that the calibration accuracy of the proposed method using a small target is equivalent to that of Zhang’s method using a large target. Additionally, when using a same-size target, the parameter error of the proposed method is less than that of Zhang’s method, and the proposed method reduces the distance measurement error from 1.169 mm to 0.208 mm compared to Zhang’s method.

A nasal chemosensation–dependent critical window for somatosensory development

Nasal chemosensation is considered the evolutionarily oldest mammalian sense and, together with somatosensation, is crucial for neonatal well-being before auditory and visual pathways start engaging the brain. Using anatomical and functional approaches in mice, we reveal that odor-driven activity propagates to a large part of the cortex during the first postnatal week and enhances whisker-evoked activation of primary whisker somatosensory cortex (wS1). This effect disappears in adult animals, in line with the loss of excitatory connectivity from olfactory cortex to wS1. By performing neonatal odor deprivation, followed by electrophysiological and behavioral work in adult animals, we identify a key transient regulation of nasal chemosensory information necessary for the development of wS1 sensory-driven dynamics and somatosensation. Our work uncovers a cross-modal critical window for nasal chemosensation–dependent somatosensory functional maturation.

Distinct feedforward and feedback pathways for cell-type specific attention effects.

Selective attention is thought to depend on enhanced firing activity in extrastriate areas. Theories suggest that this enhancement depends on selective inter-areal communication via gamma (30-80Hz) phase-locking. To test this, we simultaneously recorded from different cell types and cortical layers of macaque V1 and V4. We find that while V1-V4 gamma phase-locking between local field potentials increases with attention, the V1 gamma rhythm does not engage V4 excitatory-neurons, but only fast-spiking interneurons in L4 of V4. By contrast, attention enhances V4 spike-rates in both excitatory and inhibitory cells, most strongly in L2/3. The rate increase in L2/3 of V4 precedes V1 in time. These findings suggest enhanced signal transmission with attention does not depend on inter-areal gamma phase-locking and show that the endogenous gamma rhythm has cell-type- and layer-specific effects on downstream target areas. Similar findings were made in the mouse visual system, based on opto-tagging of identified interneurons.

Eye development influences horn size but not patterning in horned beetles.

Understanding the origin of novel morphological traits is a long-standing objective in evolutionary developmental biology. We explored the developmental genetic mechanisms that underpin the formation of a textbook example of evolutionary novelties, the cephalic horns of beetles. Previous work has implicated the gene regulatory networks associated with compound eye and ocellar development in horn formation and suggested that horns and compound eyes may influence each other's sizes. Therefore, we investigated the functional significance of genes central to visual system formation in the initiation, patterning, and size determination of head horns across three horned beetle species. We find that while the downregulation of canonical eye patterning genes reliably reduces or eliminates compound eye formation, it does not alter the position or shape of head horns yet does result in an increase in relative horn length. We discuss the implications of our results for our understanding of the genesis of cephalic horns in particular and evolutionary novelties in general.

Semantic scene segmentation for indoor autonomous vision systems: leveraging an enhanced and efficient U-NET architecture

Semantic scene segmentation for indoor autonomous vision systems: leveraging an enhanced and efficient U-NET architecture

Bio-Inspired Crystalline Core-Shell Guanine Spherulites.

Spherical particles with diameters within the wavelength of visible light, known as spherulites, manipulate light uniquely due to their spatial organization and their structural birefringence. Most of the known crystalline spherulites are branched, and composed of metals, alloys, and semi-crystalline polymers. Recently, a different spherulite architecture is discovered in the vision systems of decapod crustaceans - core-shell spherulites composed of highly birefringent ( ) organic single-crystal platelets, with exceptional optical properties. These metastructures, which efficiently scatter light even in dense aqueous environments, have no synthetic equivalence and serve as a natural proof-of-concept as well as synthetic inspiration for thin scattering media. Here, the synthesis of core-shell spherulites composed of guanine crystal platelets (( ) is presented in a two-step emulsification process in which a water/oil/water emulsion and induced pH changes are used to promote interfacial crystallization. Carboxylic acids neutralize the dissolved guanine salts to form spherulites composed of single, radially stacked, β-guanine platelets, which are oriented tangentially to the spherulite surface. Using Mie theory calculations and forward scattering measurements from single spherulites, it is found that due to the single-crystal properties and orientation, the synthetic spherulites possess a high tangential refractive index, similarly to biogenicparticles.

A Drive Interrupted: Stroke of the Anterior Choroidal Artery – A Case Report

The anterior choroidal artery (AChA) is the most distal branch of the internal carotid artery (ICA) The ACHA is significant because it supplies important structures in the brain, including the optic tract, anterior portion of cerebral peduncle, lateral geniculate body, uncus, globus pallidus, posterior and superficial areas of the thalamus, and the retrolenticular and posterior portions of the internal capsule on the same side as the artery. Isolated strokes involving the AChA are rare and can result in HHH Syndrome, consisting of contralateral hemiplegia, hemisensory loss, and homonymous hemianopia. Features which distinguish an AChA infarction from larger arterial pathology are lack of headache and lack of depressed level of consciousness in subacute infarction, and usually lack of aphasia acutely. The etiology remains controversial, with proposed mechanisms including cardioembolic, large-vessel atherosclerosis, dissection of the ICA, small-vessel occlusion, and cryptogenic causes. Herein, the authors report a case of an isolated AChA infarction resulting in a right-sided, pure motor hemiparesis with no sensory or vision loss, highly suggestive of cardioembolic origin, with the evaluation of the patient, and eventual treatment strategy.

Selective Neural Entrainment Reveals Hierarchical Tuning to Linguistic Regularities in Reading

ABSTRACT Reading is both a visual and a linguistic task, and as such it relies on both general-purpose, visual mechanisms and more abstract, meaning-oriented processes. Disentangling the roles of these resources is of paramount importance in reading research. The present study capitalizes on the coupling of fast periodic visual stimulation and MEG recordings to address this issue and investigate the role of different kinds of visual and linguistic units in the visual word identification system. We compared strings of pseudo-characters; strings of consonants (e.g., sfcl); readable, but unattested strings (e.g., amsi); frequent, but non-meaningful chunks (e.g., idge); suffixes (e.g., ment); and words (e.g., vibe); and looked for discrimination responses with a particular focus on the ventral, occipito-temporal regions. The results revealed sensitivity to alphabetic, readable, familiar, and lexical stimuli. Interestingly, there was no discrimination between suffixes and equally frequent, but meaningless endings, thus highlighting a lack of sensitivity to semantics. Taken together, the data suggest that the visual word identification system, at least in its early processing stages, is particularly tuned to form-based regularities, most likely reflecting its reliance on general-purpose, statistical learning mechanisms that are a core feature of the visual system as implemented in the ventral stream.

Oxide Based Pentachromatic‐Vision Inspired Optoelectronic Synaptic Transistor with Large Conduction States Over 512

AbstractOptoelectronic neuromorphic devices based on oxide semiconductors have been potentially investigated to mimic the functions of human visual synapses. However, the challenge comes from the wide bandgap characteristics of numerous oxide semiconductors, which restricts the response range of the device under ultra‐violet (UV) region. Strategies for widening the response range are mostly focused on artificially generating the defect states, however, most of them results in mimicking the tetrachromatic visual system from UV to visible light range. To be used for industries such as robotics, or autonomous vehicles, mimicking the tetrachromatic vision system should be overcome up to near‐infrared (NIR) region. Here, a facile solution processed indium‐gallium‐zinc‐oxide and silver oxide structured optoelectronic synaptic transistor is fabricated not only to mimic the function of human synapses, but to overcome the tetrachromatic human visual system up to the NIR region. The device not only showed photoresponse characteristics under the entire 405 to 830 nm wavelength region, but also showed significant synaptic behaviors with over 512 conduction states under a reasonable incident light power density of 4.5 mW cm−2. The results will offer a useful facile method for fabricating optoelectronic synaptic transistors that can overcome the tetrachromatic vision systems.

Research on load monitoring technology of mine hoist based on machine vision

A hoist load monitoring method based on machine vision technology is proposed in this paper to address the frequent overloading accidents of mine hoists and the low safety and reliability of existing contact load monitoring technologies. The depth image of the skip undergoes time domain and spatial bilateral filtering algorithms for noise reduction, followed by conditional filtering and downsampling algorithms to remove redundant point cloud data. Point cloud recognition, extraction, segmentation, and alignment algorithms are then applied to quickly generate a skip point cloud model. A surface reconstruction optimization process combining greedy projection triangulation algorithm and void repair algorithm is proposed to obtain a smooth and complete sealing of the skip. The closed surface model volume is calculated using VTK volume function. Based on single-rope winding hoist, a load visual monitoring system is constructed for relevant experimental research. Results show that this method can accurately measure the loaded coal volume with a relative error range of 0.05%–4.13%, meeting practical application requirements while providing an effective way for non-contact accurate measurement of hoist loads in mines.

Nighttime Harvesting of OrBot (Orchard RoBot)

The Robotics Vision Lab of Northwest Nazarene University has developed the Orchard Robot (OrBot), which was designed for harvesting fruits. OrBot is composed of a machine vision system to locate fruits on the tree, a robotic manipulator to approach the target fruit, and a gripper to remove the target fruit. Field trials conducted at commercial orchards for apples and peaches during the harvesting season of 2021 yielded a harvesting success rate of about 85% and had an average harvesting cycle time of 12 s. Building upon this success, the goal of this study is to evaluate the performance of OrBot during nighttime harvesting. The idea is to have OrBot harvest at night, and then human pickers continue the harvesting operation during the day. This human and robot collaboration will leverage the labor shortage issue with a relatively slower robot working at night. The specific objectives are to determine the artificial lighting parameters suitable for nighttime harvesting and to evaluate the harvesting viability of OrBot during the night. LED lighting was selected as the source for artificial illumination with a color temperature of 5600 K and 10% intensity. This combination resulted in images with the lowest noise. OrBot was tested in a commercial orchard using twenty Pink Lady apple trees. Results showed an increased success rate during the night, with OrBot gaining 94% compared to 88% during the daytime operations.

Impact of early physiotherapy intervention in managing homonymous hemianopia due to posterior cerebral artery infarct—a case report

Homonymous hemianopia field loss can develop as a result of injury to any part of the retro chiasmal visual pathway and is frequently fatal for subjects. The most common cause of homonymous hemianopia in older persons is ischemic infarction of the occipital lobe and its causes are multifactorial, frequently providing a diagnostic challenge to the clinician and signaling the necessity for a thorough systemic health review. Hemianopia is a loss of half of the visual field in each eye in which the inner or nasal half of one eye and the outer or temporal half of the other eye is affected. We report a case of a 72-year-old male subject who visited our department with complaints of weakness on the right side with difficulty seeing on the right side for 3 days. After undergoing various investigations, it was found that there is an acute infarct in the posterior cerebral artery territory supplying the left medial temporal lobe, medial occipital lobe, left thalamus, and splenium of the corpus callosum. After a physiotherapeutic examination provisionally, he was diagnosed with right hemiparesis and homonymous hemianopia, and thus, we administered an early physical therapy intervention which was found to be effective.