Artificial Vision Research Articles

Wide-angle simulated artificial vision enhances spatial navigation and object interaction in a naturalistic environment

Objective. Vision restoration approaches, such as prosthetics and optogenetics, provide visual perception to blind individuals in clinical settings. Yet their effectiveness in daily life remains a challenge. Stereotyped quantitative tests used in clinical trials often fail to translate into practical, everyday applications. On the one hand, assessing real-life benefits during clinical trials is complicated by environmental complexity, reproducibility issues, and safety concerns. On the other hand, predicting behavioral benefits of restorative therapies in naturalistic environments may be a crucial step before starting clinical trials to minimize patient discomfort and unmet expectations.Approach. To address this, we leverage advancements in virtual reality technology to conduct a fully immersive and ecologically valid task within a physical artificial street environment. As a case study, we assess the impact of the visual field size in simulated artificial vision for common outdoor tasks.Main results. We show that a wide visual angle (45°) enhances participants' ability to navigate and solve tasks more effectively, safely, and efficiently. Moreover, it promotes their learning and generalization capability. Concurrently, it changes the visual exploration behavior and facilitates a more accurate mental representation of the environment. Further increasing the visual angle beyond this value does not yield significant additional improvements in most metrics.Significance. We present a methodology combining augmented reality with a naturalistic environment, enabling participants to perceive the world as patients with retinal implants would and to interact physically with it. Combining augmented reality in naturalistic environments is a valuable framework for low vision and vision restoration research.

Technological prototype with artificial intelligence for military security in river environments

Maritime and river security is one of the main concerns of military forces due to the large number of illicit activities that occur. Not to mention the extensive areas that must be monitored, and the weather conditions that can occur. Currently, technologies have become fundamental to leave aside manual surveillance for intelligent systems that allow remote sensing, traffic control, and object detection. Based on the aforementioned problems, the purpose of this research was to design a technological prototype with artificial vision based on an artificial intelligence model to detect water vessels and people in river environments as a support tool for military security. The prototype used at hardware level a Raspberry Pi 3 card and four pre-trained models based on R-CNN, YOLO, EfficientDet and SSD (single shot multibox). The best-performing model was the Mobilnet V2 SSD, with an mAP of 0.83 and an FPS of 5. Finally, this tool can contribute to strengthening the strategic, tactical, and operational capabilities of actors in the military intelligence sector, aimed at protecting sovereignty and territorial integrity to establish an environment of security in society.

Guidance device for visually impaired people based on ultrasonic signals and open hardware

<span>Visual impairment is a complex challenge that affects people of all ages, and it is estimated that around 2.2 billion people worldwide lack adequate access to medical treatment and support. In Latin America, there is a lack of attention to people with visual disabilities, evidenced by poor urban infrastructure and lack of compliance with inclusion laws. Some projects stand out for the use of prototypes with artificial vision technology, global positioning system (GPS) and smart canes. Therefore, the objective of the project is to use ultrasonic sensors and a low-cost electronic device coupled to canes, for obstacle detection and mobility using an open hardware embedded system. The results confirmed the efficiency in the detection and operation of the ultrasonic sensor by activating the light emitting diode (LED), the buzzer and the vibrating motor according to the programmed distances. Challenges were identified, such as adapting the sensor to the tilt of the cane and the importance of accurate calibration of the ultrasonic sensor. The system met its objectives by detecting objects in a range of 2 to 50 cm and providing sound alerts to improve the perception of blind people.</span>

A Computationally Efficient Neuronal Model for Collision Detection with Contrast Polarity-Specific Feed-Forward Inhibition

Animals utilize their well-evolved dynamic vision systems to perceive and evade collision threats. Driven by biological research, bio-inspired models based on lobula giant movement detectors (LGMDs) address certain gaps in constructing artificial collision-detecting vision systems with robust selectivity, offering reliable, low-cost, and miniaturized collision sensors across various scenes. Recent progress in neuroscience has revealed the energetic advantages of dendritic arrangements presynaptic to the LGMDs, which receive contrast polarity-specific signals on separate dendritic fields. Specifically, feed-forward inhibitory inputs arise from parallel ON/OFF pathways interacting with excitation. However, none of the previous research has investigated the evolution of a computational LGMD model with feed-forward inhibition (FFI) separated by opposite polarity. This study fills this vacancy by presenting an optimized neuronal model where FFI is divided into ON/OFF channels, each with distinct synaptic connections. To align with the energy efficiency of biological systems, we introduce an activation function associated with neural computation of FFI and interactions between local excitation and lateral inhibition within ON/OFF channels, ignoring non-active signal processing. This approach significantly improves the time efficiency of the LGMD model, focusing only on substantial luminance changes in image streams. The proposed neuronal model not only accelerates visual processing in relatively stationary scenes but also maintains robust selectivity to ON/OFF-contrast looming stimuli. Additionally, it can suppress translational motion to a moderate extent. Comparative testing with state-of-the-art based on ON/OFF channels was conducted systematically using a range of visual stimuli, including indoor structured and complex outdoor scenes. The results demonstrated significant time savings in silico while retaining original collision selectivity. Furthermore, the optimized model was implemented in the embedded vision system of a micro-mobile robot, achieving the highest success ratio of collision avoidance at 97.51% while nearly halving the processing time compared with previous models. This highlights a robust and parsimonious collision-sensing mode that effectively addresses real-world challenges.

Tailored Environment-Friendly Reverse Type-I Colloidal Quantum Dots for a Near-Infrared Optical Synapse and Artificial Vision System.

Colloidal quantum dots (QDs) are emerging as potential candidates for constructing near-infrared (NIR) photodetectors (PDs) and artificial optoelectronic synapses due to solution processability and a tunable bandgap. However, most of the current NIR QDs-optoelectronic devices are still fabricated using QDs with incorporated harmful heavy metals of lead (Pb) and mercury (Hg), showing potential health and environment risks. In this work, we tailored eco-friendly reverse type-I ZnSe/InP QDs by copper (Cu) doping and extended the photoresponse from the visible to NIR region. Transient absorption spectroscopy analysis revealed the presence of Cu dopant states in ZnSe/InP:Cu QDs that facilitated the extraction of photogenerated charge carriers, leading to an enhanced photodetection performance. Specifically, under 400 nm illumination, the Cu-doped ZnSe/InP QDs-based PDs presented a broadband photodetection ranging from ultraviolet (UV) to NIR, with a responsivity of 70.5 A W-1 and detectivity of 2.8 × 1011 Jones, surpassing those of the undoped ZnSe/InP QDs-based PDs (49.4 A W-1 and 1.9 × 1011 Jones, respectively). More importantly, the ZnSe/InP:Cu QDs-PDs demonstrated various synapse-like characteristics of short-term plasticity (STP), long-term plasticity (LTP), and learning-forging-relearning under NIR light illumination, which were further used to construct PD array devices for simulating the artificial visual system that is available in prospective optical neuromorphic applications.

Broadband and parallel multiple-order optical spatial differentiation enabled by Bessel vortex modulated metalens

Optical analog image processing technology is expected to provide an effective solution for high-throughput and real-time data processing with low power consumption. In various operations, optical spatial differential operations are essential in edge extraction, data compression, and feature classification. Unfortunately, existing methods can only perform low-order or selectively perform a particular high-order differential operation. Here, we propose and experimentally demonstrate a Bessel vortex modulated metalens composed of a single complex amplitude metasurface, which can perform multiple-order radial differential operations over a wide band by presetting the order of the corresponding Bessel vortex. This architecture further enables angle multiplexing to create multiple information channels that synchronously perform multi-order spatial differential operations, indicating the superiority of the proposed devices in parallel processing. Our approach may find various applications in artificial intelligence, machine vision, autonomous driving, and advanced biomedical imaging.

Self-Powered and Broadband Optical Synapse Device Based on Se-Vacancy Bi2O2Se for Artificial Vision System Application

Self-Powered and Broadband Optical Synapse Device Based on Se-Vacancy Bi₂O₂Se for Artificial Vision System Application

A new occupancy index model based on artificial vision for enhancing beach management

This study proposes a new method to more effectively plan the use of beaches by combining indices and artificial vision systems. The Overcrowding Index (Iocr) measures the number of people on the beach in relation to its surface area, while the Distancing Index (Idis) evaluates the spatial distribution and distance between beachgoers. Both indices are combined to generate an overall index called the Occupancy Index (Iocu). The proposed methodology uses cameras and computer vision algorithms such as YOLOX and ByteTrack to automate the counting of people and measure distances. This allows for continuous monitoring of the quantity (carrying capacity and density) and distribution of beachgoers (degree of social distancing), as well as a functional prototype in which the indices are calculated in real time. It was observed that as density increased, Iocr showed an inverse trend, being close to 0 when approaching maximum density. The calculation of the distance between groups validated that, even with medium densities, close to the shoreline, the reference distance of 2 m was not accomplished, obtaining a very low Idis (0.18). The resulting Iocu was 0.31, validating the appropriate integration of both indices. Overall, the system's effectiveness for accurately monitoring the number of users and their distribution, and calculating the defined indices for beach management, is demonstrated. The proposed approach provides a valuable tool, allowing a more efficient management of beaches according to their actual occupancy and user distribution.

Multifunctional human visual pathway-replicated hardware based on 2D materials

Artificial visual system empowered by 2D materials-based hardware simulates the functionalities of the human visual system, leading the forefront of artificial intelligence vision. However, retina-mimicked hardware that has not yet fully emulated the neural circuits of visual pathways is restricted from realizing more complex and special functions. In this work, we proposed a human visual pathway-replicated hardware that consists of crossbar arrays with split floating gate 2D tungsten diselenide (WSe2) unit devices that simulate the retina and visual cortex, and related connective peripheral circuits that replicate connectomics between the retina and visual cortex. This hardware experimentally displays advanced multi-functions of red–green color-blindness processing, low-power shape recognition, and self-driven motion tracking, promoting the development of machine vision, driverless technology, brain–computer interfaces, and intelligent robotics.

How green can it be? A methodology for calculating green roof retrofit potential in Valencia

Cities are accountable for more than 70% of global CO2 emissions and consume around 65% of the world's energy. In the pathway towards urban decarbonisation, nature-based solutions rise as a promising opportunity. They improve a city's resilience, stabilise temperatures and capture CO2, both mitigating climate change effects and adapting to them. This paper presents a novel methodology to assess the potential effects of green roof retrofit in urban areas, selecting suitable buildings for greening and estimating their decarbonising capacity. The proposed method combines the use of Geographic Information Systems (GIS) and artificial vision algorithms to select the roofs. Once selected, direct carbon sequestration and energetic savings are estimated based on empirical results. This methodology is successfully applied to the L'Illa Perduda neighbourhood of Valencia (Spain) as a case study. The GIS analysis shows that about 50% of the roofs' surface could be greened, directly absorbing around 350 tCO2yr−1 and reducing the energetic emissions of the neighbourhood in about 100 tCO2yr−1 by improving the insulating envelope of the buildings. The artificial vision computation selected 38% of the surface of the neighbourhood residential buildings.

Artificial vision system based on porosity heterojunction for in-sensor processing and dynamic trace extraction

Artificial vision system based on porosity heterojunction for in-sensor processing and dynamic trace extraction

Experimental investigation of buckling and post-buckling behaviour of repaired composite laminates under in-plane shear loading

The performance of Ramped Scarf (RS) and Stepped Scarf (SS) repair schemes for highly loaded composite structures has been extensively studied under compressive, tensile, and flexural loadings. However, there is a lack of research on the buckling and post-buckling behaviour of RS and SS repairs under in-plane shear loading, which is critical for aerostructures. This paper addresses this gap by investigating the buckling and post-buckling performance of these repair schemes. Pristine (P) laminates, RS repairs with a scarf angle of 3°, and SS repairs with an overlap step length of 1/60 were manufactured and tested. The quality of the scarf repairs was inspected using artificial intelligence-based machine vision. Mechanical in-plane shear testing was conducted to evaluate the buckling and post-buckling behaviour of P, RS, and SS laminates. The experimental results indicate that RS repairs demonstrated an average of 35 % higher maximum displacement and 12 % higher failure load compared to pristine laminates. SS repairs showed 19 % higher maximum displacement, and 5 % higher failure load compared to pristine laminates. RS repairs excelled in all key mechanical performance indicators, including buckling load, Hooke's stiffness, maximum displacement, and failure load, compared to SS repairs. The failure modes of P, RS, and SS laminates were similar. However, RS repairs exhibited greater susceptibility to repair patch detachment compared to SS repairs.

Crack Growth Patterns of Aluminum Tubular Specimens Subjected to Cyclic Tensile Loads

This study presents a detailed analysis of a fatigue test campaign in order to identify different crack patterns. It was conducted on 6061-T6 aluminum tubular specimens featuring an internal diameter of 10 mm and different thicknesses (2, 3 and 4 mm). These specimens were subjected to cyclic tensile loads with a load ratio of R = 0.1, utilizing a sinusoidal load function at a frequency of 3 Hz. The investigation examines the crack growth rates, the stress intensity factor, and the final and intermediate fracture zones by applying overloads in some cases. The differences with two-dimensional specimens and the importance of this study for the interpretation of results with biaxial loading states are highlighted. The different states of crack growth detected are analyzed using artificial vision techniques. The differences between the exterior and interior faces of the specimen are revealed, and a series of states prior to the formation of the radial crack front expected in these specimens are identified.

Feline eye-inspired artificial vision for enhanced camouflage breaking under diverse light conditions.

Biologically inspired artificial vision research has led to innovative robotic vision systems with low optical aberration, wide field of view, and compact form factor. However, challenges persist in object detection and recognition against complex backgrounds and varied lighting. Inspired by the feline eye, which features a vertically elongated pupil and tapetum lucidum, this study introduces an artificial vision system designed for superior object detection and recognition in a monocular framework. Using a slit-like elliptical aperture and a patterned metal reflector beneath a hemispherical silicon photodiode array, the system reduces excessive light and enhances photosensitivity. This design achieves clear focus under bright light and enhanced sensitivity in dim conditions. Theoretical and experimental analyses demonstrate the system's ability to filter redundant information and detect camouflaged objects in diverse lighting, representing a substantial advancement in monocular camera technology and the potential of biomimicry in optical innovations.

Spectral Analysis on Color Detection Sharpness of Animal Vision toward Polychromatic Vision System

AbstractSpectral discrimination by animal visions such as human, bird, and butterfly is numerically compared by angular analysis of photo‐response (PR) which will be recorded at photo‐receptors. Hyperspectral imaging system is utilized to simulate various animal vision. Bird vision is acute to discriminate colors among different vegetables due to its evenly spaced and narrow spectral responsivity of photo‐receptors compared to that of human. Butterfly vision is excellent in discriminating red tomato ripening due to the exclusive photo‐receptors detecting only over 600 nm. Even real and fake fruits in the same perceived color for human is discriminated by bird vision. Artificial vision with finely resolved polychromatic artificial cone cells are demonstrated surpassing human vision using visible multispectral camera. This provides insights on designing novel bio‐inspired vision system.

An Ultrasensitive Programmable 2D Photoelectric Synaptic Transistor

AbstractThe burgeoning advancement of information technology has engendered a discernible surge in the examination of neuromorphic devices, notably drawing broader attention to artificial vision systems endowed with sensory recognition capabilities. Current photoelectric synapse devices employed in artificial vision systems are generally well‐suited for well‐illuminated conditions, yet exhibit diminished sensitivity in weak‐light scenarios, resulting in a pronounced deterioration of recognition accuracy. Here, an ultrasensitive photoelectric synaptic transistor based on negative quantum capacitance effect resulted from the 2D semi‐metallic graphene layer that partially enclosed within the gate dielectric layer, which manifests a noteworthy reduction in device control voltage and exhibits perception and storage capabilities for weak light of 39.4 nW cm−2 with detectivity above 1016 cm Hz1/2 W−1 is demonstrated. The voltage amplification effect and the concomitant formation of an equivalent local electrostatic field induced by the negative quantum capacitance effect engenders a robust programmable synaptic plasticity for extremely weak light by modifying the control gate. These results represent the inaugural integration of the negative quantum capacitance effect into optoelectronic devices and furnish a robust hardware foundation for developing vision systems in weak‐light environments.

Humanoid Robot Teleoperation Through 3-D Pose Estimation

The knowledge of how the human motions is performed helps to understand how the human body works. This paper presents a method to estimate the human limbs angles through a kinematic model depicted by Roll-Pitch-Yaw rotationmatrix and the mimic of those angles on a humanoid robot. The advantage of this model is the detailed representation of each joint movement in the coordinate axes (x, y, z). The angles estimation is made with the information provided by algorithms of artificial vision and artificial intelligence. In order to reduce the latency between the human motion capture and robot motions, a fuzzy logic controller is implemented in order to control each robot joint. The final robot limbs angles are compared with the human angles in order to obtain the final error between those measurements. This method shows a similar result on the arms posture regarding previous works.

Optoelectronic synapses with chemical-electric behaviors in gallium nitride semiconductors for biorealistic neuromorphic functionality

Optoelectronic synapses, leveraging the integration of classic photo-electric effect with synaptic plasticity, are emerging as building blocks for artificial vision and photonic neuromorphic computing. However, the fundamental working principles of most optoelectronic synapses mainly rely on physical behaviors while missing chemical-electric synaptic processes critical for mimicking biorealistic neuromorphic functionality. Herein, we report a photoelectrochemical synaptic device based on p-AlGaN/n-GaN semiconductor nanowires to incorporate chemical-electric synaptic behaviors into optoelectronic synapses, demonstrating unparalleled dual-modal plasticity and chemically-regulated neuromorphic functions through the interplay of internal photo-electric and external electrolyte-mediated chemical-electric processes. Electrical modulation by implementing closed or open-circuit enables switching of optoelectronic synaptic operation between short-term and long-term plasticity. Furthermore, inspired by transmembrane receptors that connect extracellular and intracellular events, synaptic responses can also be effectively amplified by applying chemical modifications to nanowire surfaces, which tune external and internal charge behaviors. Notably, under varied external electrolyte environments (ion/molecule species and concentrations), our device successfully mimics chemically-regulated synaptic activities and emulates intricate oxidative stress-induced biological phenomena. Essentially, we demonstrate that through the nanowire photoelectrochemical synapse configuration, optoelectronic synapses can be incorporated with chemical-electric behaviors to bridge the gap between classic optoelectronic synapses and biological synapses, providing a promising platform for multifunctional neuromorphic applications.

AUTORECYCLER: Prototype based on artificial vision to automate the material classification process (Plastic, Glass, Cardboard and Metal)

Environmental protection has gained greater importance over time due to the negative impact and irreversible consequences that have occurred worldwide and stem from pollution. One of the great challenges faced in different parts of the world is the inadequate management and classification of solid waste. In order to contribute to tackling this issue, this paper proposes an automated sorting system based on artificial vision which allows recognition and separation of recyclable materials (Plastic, Glass, Cardboard and Metal) through a webcam connected in real time to the Nvidia® Jetson Nano™ 2 GB programming board, which has a convolutional neural network (CNN) trained for the proper classification of waste. The system had a 95 % accuracy in separating plastic, 96 % in glass and metal, and 94 % in cardboard. With this in mind, we conclude it contributes to the recycling effort, which has an impact on the reduction of environmental pollution worldwide.

Low-Cost Non-Wearable Fall Detection System Implemented on a Single Board Computer for People in Need of Care.

This work aims at proposing an affordable, non-wearable system to detect falls of people in need of care. The proposal uses artificial vision based on deep learning techniques implemented on a Raspberry Pi4 4GB RAM with a High-Definition IR-CUT camera. The CNN architecture classifies detected people into five classes: fallen, crouching, sitting, standing, and lying down. When a fall is detected, the system sends an alert notification to mobile devices through the Telegram instant messaging platform. The system was evaluated considering real daily indoor activities under different conditions: outfit, lightning, and distance from camera. Results show a good trade-off between performance and cost of the system. Obtained performance metrics are: precision of 96.4%, specificity of 96.6%, accuracy of 94.8%, and sensitivity of 93.1%. Regarding privacy concerns, even though this system uses a camera, the video is not recorded or monitored by anyone, and pictures are only sent in case of fall detection. This work can contribute to reducing the fatal consequences of falls in people in need of care by providing them with prompt attention. Such a low-cost solution would be desirable, particularly in developing countries with limited or no medical alert systems and few resources.