Artificial Vision System Research Articles

Biomimetic Flexible High‐Sensitivity Near‐Infrared II Organic Photodetector for Photon Detection and Imaging

AbstractThe flexible near‐infrared (NIR) organic photodetector is an essential prerequisite for the next‐generation visual system for accurate information identification. However, prevailing limitations in NIR organic photodetectors, including high noise, weak responsivity, and low detectivity, directly impede the translation to practical applications. Herein, ultranarrow bandgap acceptors, featuring a 2D highly electron‐donating central core (DTPC), named as YZ and YZ1, are designed and synthesized for NIR organic photodetectors. It is found that the introduction of highly rigid DTPC effectively reduces energetic disorder in active films. Moreover, extending the end conjugation not only broadens the absorption spectrum but also diminishes trap states and improves charge transport. Consequently, the YZ1‐based organic photodetector exhibits an impressive detection range from 300 to 1050 nm with a high responsivity of 0.27 A W−1 at 1000 nm and specific detectivity of 9.24 × 1013 Jones, comparable to commercial silicon photodiodes. Inspired by biological eyes, large‐area parallelly independent photodetector arrays (2000 pixels) for an artificial vision system are fabricated, which enables a clear matter identification in the NIR region. These findings and results simultaneously provide critical insights into molecular design and bridge the gap between material science and device engineering, accelerating the application of NIR organic photodetectors in next‐generation wearable optoelectronics.

Photonic synaptic transistors with new electron trapping layer for high performance and ultra-low power consumption

Photonic synaptic transistors are being investigated for their potential applications in neuromorphic computing and artificial vision systems. Recently, a method for establishing a synaptic effect by preventing the recombination of electron–hole pairs by forming an energy barrier with a double-layer consisting of a channel and a light absorption layer has shown effective results. We report a triple-layer device created by coating a novel electron-trapping layer between the light-absorption layer and the gate-insulating layer. Compared to the conventional double-layer photonic synaptic structure, our triple-layer device significantly reduces the recombination rate, resulting in improved performance in terms of the output photocurrent and memory characteristics. Furthermore, our photonic synaptic transistor possesses excellent synaptic properties, such as paired-pulse facilitation (PPF), short-term potentiation (STP), and long-term potentiation (LTP), and demonstrates a good response to a low operating voltage of − 0.1 mV. The low power consumption experiment shows a very low energy consumption of 0.01375 fJ per spike. These findings suggest a way to improve the performance of future neuromorphic devices and artificial vision systems.

In‐Depth Physical Mechanism Analysis of Polymer Artificial Optoelectronic Synapse with High Endurance and Applications of Visual System and Operant Conditioning

AbstractBeing capable of dealing with both electrical signals and light, artificial optoelectronic synapses are considered to be an important cornerstone of neuromorphic computing. Here, an artificial optoelectronic synapse is reported through a simple solution process using organic poly(3‐hexylthiophene) (P3HT) and a remarkable analog switching characteristic similar to synaptic behavior is observed. The endurance and data retention capability of the P3HT‐based optoelectronic synapse exhibit stable characteristics up to 5000 consecutive cycles and 104 s. Through in‐depth physical mechanism analysis, it is confirmed that the analog switching characteristics of the device are mainly caused by a tunneling mechanism and space charge limited conduction. Furthermore, characterizations such as X‐ray photoelectron spectroscopy and atomic layer deposition prove that the memristive properties of device can be attributed to ion migration. More importantly, the device can co‐modulate the optoelectronic signal and successfully implement a photo‐triggered multi‐signal mode response. Based on this, a 3 × 3 synapse array is developed to demonstrate the potential application of the proposed P3HT‐based optoelectronic synapse in constructing an artificial visual system. Finally, operant conditioning is successfully simulated in the synaptic device. This work provides a reference for the construction of optoelectronic synapses in the neuromorphic visual system.

Use of Artificial Vision during the Lye Treatment of Sevillian-Style Green Olives to Determine the Optimal Time for Terminating the Cooking Process.

This study focuses on characterizing the temporal evolution of the surface affected by industrial treatment with NaOH within the processing tanks during the lye treatment stage of Manzanilla table olives. The lye treatment process is affected by multiple variables, such as ambient temperature, the initial temperature of the olives before lye treatment, the temperature of the NaOH solution, the concentration of the solution, the variety of olives, and their size, which are determinants of the speed of the lye treatment process. Traditionally, an expert, relaying on their subjective judgement, manages the cooking process empirically, leading to variability in the termination timing of the cook. In this study, we introduce a system that, by using an artificial vision system, allows us to know in a deterministic way the percentage of lye treatment achieved at each moment along the cooking process; furthermore, with an interpolator that accumulates values during the lye treatment, it is possible to anticipate the completion of the cooking by indicating the moment when two-thirds, three-fourths, or some other value of the interior surface will be reached with an error of less than 10% relative to the optimal moment. Knowing this moment is crucial for proper processing, as it will affect subsequent stages of the manufacturing process and the quality of the final product.

Self-powered and broadband opto-sensor with bionic visual adaptation function based on multilayer γ-InSe flakes

Visual adaptation that can autonomously adjust the response to light stimuli is a basic function of artificial visual systems for intelligent bionic robots. To improve efficiency and reduce complexity, artificial visual systems with integrated visual adaptation functions based on a single device should be developed to replace traditional approaches that require complex circuitry and algorithms. Here, we have developed a single two-terminal opto-sensor based on multilayer γ-InSe flakes, which successfully emulated the visual adaptation behaviors with a new working mechanism combining the photo-pyroelectric and photo-thermoelectric effect. The device can operate in self-powered mode and exhibit good human-eye-like adaptation behaviors, which include broadband light-sensing image adaptation (from ultraviolet to near-infrared), near-complete photosensitivity recovery (99.6%), and synergetic visual adaptation, encouraging the advancement of intelligent opto-sensors and machine vision systems.

Quantifying the ultraviolet-induced fluorescence intensity in green mould lesions of diverse citrus varieties: Towards automated detection of citrus decay in postharvest

Citrus fungal infections developing during fruit storage and transportation can cause significant economic losses after harvest. The most important is caused by the fungus Penicillium digitatum, which infects the fruit through rind wounds and causes a rot lesion. The symptoms of decay are difficult to notice by the human eye in the initial stages of decay development because the colour of the lesion is very similar to that of the healthy rind. One method to detect this disease early is to illuminate the fruit with ultraviolet (UV) light since the disease causes visible fluorescence. Manual inspection exposes the workers to UV light, which is dangerous for their skin and eyes. An alternative is to use artificial vision systems. But not all varieties show the same level of fluorescence, and even some do not produce this phenomenon, making it challenging to create effective automatic detection systems based on image analysis. This work has studied and determined the fluorescence level of 104 varieties of oranges and mandarins using hyperspectral and colour imaging. The samples were inoculated with spores of the P. digitatum in controlled conditions. Images were captured exposing the fruit under UV light (380 nm) using a colour camera and a hyperspectral imaging system. The fluorescence level of each variety was measured using three colour coordinates and the hyperspectral images. Best correlations between the spectral and the colour-based systems were achieved using the green (G) colour coordinate of the RGB colour space (R2 =0.85). Navel and common oranges emit the most fluorescence, while 16 varieties (mostly blood oranges and other mandarins) have very low or undetectable fluorescence.

Multiscale Simulations on Synaptic Signal Transduction of Energy-Harvesting P(VDF-TrFE)-Based Artificial Retina.

Ferroelectric polymers have drawn a lot of research concerns recently due to their lightness, mechanical flexibility, conformability, and facile processability. Remarkably, these polymers can be used to fabricate biomimetic devices, such as artificial retina or electronic skin, to realize artificial intelligence. The artificial visual system behaves as a photoreceptor, converting incoming light into electric signals. The most widely studied ferroelectric polymer, poly(vinylidene fluoride-trifluoroethylene) [P(VDF-TrFE)], can be used as the building block in this visual system to implement synaptic signal generation. There is a void in computational investigations on the complicated working picture of P(VDF-TrFE)-based artificial retina from a microscopic mechanism to a macroscopic mechanism. Therefore, a multiscale simulation method combining quantum chemistry calculations, first-principles calculations, Monte Carlo simulations, and the Benav model was established to illustrate the whole working principle, involving synaptic signal transduction and consequent communication with neuron cells, of the P(VDF-TrFE)-based artificial retina. This newly developed multiscale method not only can be further applied to other energy-harvesting systems involving synaptic signals but also would be helpful to build microscopic/macroscopic pictures within these energy-harvesting devices.

Optoelectronic Synapses and Photodetectors Based on Organic Semiconductor/Halide Perovskite Heterojunctions: Materials, Devices, and Applications

AbstractBoth photodetectors (PDs) and optoelectronic synaptic devices (OSDs) are optoelectronic devices converting light signals into electrical responses. Optoelectronic devices based on organic semiconductors and halide perovskites have aroused tremendous research interest owing to their exceptional optical/electrical characteristics and low‐cost processability. The heterojunction formed between organic semiconductors and halide perovskites can modify the exciton dissociation/recombination efficiency and modulate the charge‐trapping effect. Consequently, organic semiconductor/halide perovskite heterojunctions can endow PDs and OSDs with high photo responsivity and the ability to simulate synaptic functions respectively, making them appropriate for the development of energy‐efficient artificial visual systems with sensory and recognition functions. This article summarizes the recent advances in this research field. The physical/chemical properties and preparation methods of organic semiconductor/halide perovskite heterojunctions are briefly introduced. Then the development of PDs and OSDs based on organic semiconductor/halide perovskite heterojunctions, as well as their innovative applications, are systematically presented. Finally, some prospective challenges and probable strategies for the future development of optoelectronic devices based on organic semiconductor/halide perovskite heterojunctions are discussed.

CMOS-Compatible Tellurium/Silicon Ultra-Fast Near-Infrared Photodetector.

High-quality photodetectors are always the main way to obtain external information, especially near-infrared sensors play an important role in remote sensing communication. However, due to the limitation of Silicon (Si) wide bandgap and the incompatibility of most near infrared photoelectric materials with traditional integrated circuits, the development of high performance and wide detection spectrum near infrared detectors suitable for miniaturization and integration is still facing many obstacles. Herein, the monolithic integration of large area tellurium optoelectronic functional units is realized by magnetron sputtering technology. Taking advantage of the type II heterojunction constructed by tellurium (Te) and silicon (Si), the photogenerated carriers are effectively separated, which prolongs the carrier lifetime and improves the photoresponse by several orders of magnitude. The tellurium/silicon (Te/Si) heterojunction photodetector demonstrates excellent detectivity and ultra-fast turn-on time. Importantly, an imaging array (20 × 20 pixels) based on the Te/Si heterojunction is demonstrated and high-contrast photoelectric imaging is realized. Because of the high contrast obtained by the Te/Si array, in comparison with the Si arrays, it significantly improve the efficiency and accuracy of the subsequent processing tasks when the electronic pictures are applied to artificial neural network (ANN) to simulate the artificial vision system.

The Mechanism of Orientation Detection Based on Artificial Visual System for Greyscale Images

Human visual system is a crucial component of the nervous system, enabling us to perceive and understand the surrounding world. Advancements in research on the visual system have profound implications for our understanding of both biological and computer vision. Orientation detection, a fundamental process in the visual cortex where neurons respond to linear stimuli in specific orientations, plays a pivotal role in both fields. In this study, we propose a novel orientation detection mechanism for local neurons based on dendrite computation, specifically designed for grayscale images. Our model comprises eight neurons capable of detecting local orientation information, with inter-neuronal interactions facilitated through nonlinear dendrites. Through the extraction of local orientation information, this mechanism effectively derives global orientation information, as confirmed by successful computer simulations. Experimental results demonstrate that our mechanism exhibits remarkable orientation detection capabilities irrespective of variations in size, shape, or position, which aligns with previous physiological research findings. These findings contribute to our understanding of the human visual system and provide valuable insights into both biological and computer vision. The proposed orientation detection mechanism, with its nonlinear dendritic computations, offers a promising approach for improving orientation detection in grayscale images.

Bioinspired In‐Sensor Reservoir Computing for Self‐Adaptive Visual Recognition with Two‐Dimensional Dual‐Mode Phototransistors

AbstractArtificial visual systems that dynamically process spatiotemporal optoelectronic signals under complex real‐life environments bear a wide spectrum of edge applications. Despite significant progress in optoelectronic sensors and neuromorphic computing algorithms, developing visual systems that can adapt to a broad illumination range while retaining high performance, high efficiency, and low training costs remains a challenge. Here, this work reports a bioinspired in‐sensor reservoir computing (RC) for self‐adaptive visual recognition. By leveraging voltage‐tunable photoresponses of the MoS2‐based phototransistor array, the RC system demonstrates both scotopic and photopic adaptation functions and maintains a recognition accuracy of 91%. The horizontal modulation (HM) block enables the reservoir to adapt automatically in real‐time under changing illumination conditions, yielding a 90.64% recognition accuracy (14.21% improvement over conventional RC systems). These results pave the way for the emergence of a reconfigurable in‐sensor RC system with broad applications and enhanced performance for an efficient artificial vision system at the edge.

Machine learning assisted remote forestry health assessment: a comprehensive state of the art review.

Forests are suffering water stress due to climate change; in some parts of the globe, forests are being exposed to the highest temperatures historically recorded. Machine learning techniques combined with robotic platforms and artificial vision systems have been used to provide remote monitoring of the health of the forest, including moisture content, chlorophyll, and nitrogen estimation, forest canopy, and forest degradation, among others. However, artificial intelligence techniques evolve fast associated with the computational resources; data acquisition, and processing change accordingly. This article is aimed at gathering the latest developments in remote monitoring of the health of the forests, with special emphasis on the most important vegetation parameters (structural and morphological), using machine learning techniques. The analysis presented here gathered 108 articles from the last 5 years, and we conclude by showing the newest developments in AI tools that might be used in the near future.

A Motion-Direction-Detecting Model for Gray-Scale Images Based on the Hassenstein–Reichardt Model

The visual system of sighted animals plays a critical role in providing information about the environment, including motion details necessary for survival. Over the past few years, numerous studies have explored the mechanism of motion direction detection in the visual system for binary images, including the Hassenstein–Reichardt model (HRC model) and the HRC-based artificial visual system (AVS). In this paper, we introduced a contrast-response system based on previous research on amacrine cells in the visual system of Drosophila and other species. We combined this system with the HRC-based AVS to construct a motion-direction-detection system for gray-scale images. Our experiments verified the effectiveness of our model in detecting the motion direction in gray-scale images, achieving at least 99% accuracy in all experiments and a remarkable 100% accuracy in several circumstances. Furthermore, we developed two convolutional neural networks (CNNs) for comparison to demonstrate the practicality of our model.

Bioinspired Artificial Visual System Based on 2D WSe2 Synapse Array

AbstractMachine vision systems that capture images for visual inspection and recognition tasks must be able to perceive, memorize, and compute any color scene. To achieve this, most of the current visual systems use circuits and algorithms which may reduce efficiency and increase complexity. Herein, a 2D semiconductor tungsten diselenide (WSe2)‐based phototransistor that successfully demonstrates an artificial vision system integrating the processing capability of visual information sensing memory, is reported. Furthermore, based on a 6 × 6 fabricated retinal perception array, artificial visual information sensing memory and processing system are proposed to perform image recognition tasks, which can avoid the time delay and energy consumption caused by data conversion and movement. On the other hand, highly linear symmetric synaptic plasticity can be achieved based on the modulation of carrier types in WSe2 transistors with different thicknesses, facilitating the high level of training and inference accuracy for artificial neural networks. Last, through training and inference simulations, the feasibility of the hybrid synapses for optical neural networks (ONN) is demonstrated.

The representational hierarchy in human and artificial visual systems in the presence of object-scene regularities.

Human vision is still largely unexplained. Computer vision made impressive progress on this front, but it is still unclear to which extent artificial neural networks approximate human object vision at the behavioral and neural levels. Here, we investigated whether machine object vision mimics the representational hierarchy of human object vision with an experimental design that allows testing within-domain representations for animals and scenes, as well as across-domain representations reflecting their real-world contextual regularities such as animal-scene pairs that often co-occur in the visual environment. We found that DCNNs trained in object recognition acquire representations, in their late processing stage, that closely capture human conceptual judgements about the co-occurrence of animals and their typical scenes. Likewise, the DCNNs representational hierarchy shows surprising similarities with the representational transformations emerging in domain-specific ventrotemporal areas up to domain-general frontoparietal areas. Despite these remarkable similarities, the underlying information processing differs. The ability of neural networks to learn a human-like high-level conceptual representation of object-scene co-occurrence depends upon the amount of object-scene co-occurrence present in the image set thus highlighting the fundamental role of training history. Further, although mid/high-level DCNN layers represent the category division for animals and scenes as observed in VTC, its information content shows reduced domain-specific representational richness. To conclude, by testing within- and between-domain selectivity while manipulating contextual regularities we reveal unknown similarities and differences in the information processing strategies employed by human and artificial visual systems.

Review on metal halide perovskite-based optoelectronic synapses

With the progress of both photonics and electronics, optoelectronic synapses are considered potential candidates to challenge the von Neumann bottleneck and the field of visual bionics in the era of big data. They are also regarded as the basis for integrated artificial neural networks (ANNs) owing to their flexible optoelectronic tunable properties such as high bandwidth, low power consumption, and high-density integration. Over the recent years, following the emergence of metal halide perovskite (MHP) materials possessing fascinating optoelectronic properties, novel MHP-based optoelectronic synaptic devices have been exploited for numerous applications ranging from artificial vision systems (AVSs) to neuromorphic computing. Herein, we briefly review the application prospects and current status of MHP-based optoelectronic synapses, discuss the basic synaptic behaviors capable of being implemented, and assess their feasibility to mimic biological synapses. Then, we focus on the two-terminal optoelectronic synaptic memristors and three-terminal transistor synaptic phototransistors (SPTs), the two essential apparatus structures for optoelectronic synapses, expounding their basic features and operating mechanisms. Finally, we summarize the recent applications of optoelectronic synapses in neuromorphic systems, including neuromorphic computing, high-order learning behaviors, and neuromorphic vision systems, outlining their potential opportunities and future development directions as neuromorphic devices in the field of artificial intelligence (AI).

Neuromorphic Artificial Vision Systems Based on Reconfigurable Ion‐Modulated Memtransistors

Conventional vision systems suffer from lots of data handling between memory and processing units. Inspired by how humans recognize noisy images and the flexible modulation on the timescale of ion dynamics inside an emerging memtransistor, a novel neuromorphic vision system is reported based on the ion‐modulated memtransistors. By controlling the ion‐doping processes under adequate stimuli strengths, both short‐term and long‐term ion dynamics can be utilized to deliver energy‐efficient data processing. When dealing with image reconstructions, the short‐term accumulation effect of the device can help filter noises in a set of received noisy images while enhancing the original pattern information. The increased contrast can help distinguish the actual contents. To demonstrate systematic performances with the reconfiguration of devices, the nonlinear relationship between channel conductance variation and the amplitude of gate pulses into the network‐level simulation is extracted. Also, with the nonvolatile conductance change characteristic, the task of recognizing noisy images is performed to verify the versatility of ion‐modulated memtransistors in the neuromorphic artificial vision systems. An interactive preprint version of the article can be found here: https://doi.org/10.22541/au.167939089.96499861/v1

Tetrachromatic vision-inspired neuromorphic sensors with ultraweak ultraviolet detection

Sensing and recognizing invisible ultraviolet (UV) light is vital for exploiting advanced artificial visual perception system. However, due to the uncertainty of the natural environment, the UV signal is very hard to be detected and perceived. Here, inspired by the tetrachromatic visual system, we report a controllable UV-ultrasensitive neuromorphic vision sensor (NeuVS) that uses organic phototransistors (OPTs) as the working unit to integrate sensing, memory and processing functions. Benefiting from asymmetric molecular structure and unique UV absorption of the active layer, the as fabricated UV-ultrasensitive NeuVS can detect 370 nm UV-light with the illumination intensity as low as 31 nW cm−2, exhibiting one of the best optical figures of merit in UV-sensitive neuromorphic vision sensors. Furthermore, the NeuVS array exbibits good image sensing and memorization capability due to its ultrasensitive optical detection and large density of charge trapping states. In addition, the wavelength-selective response and multi-level optical memory properties are utilized to construct an artificial neural network for extract and identify the invisible UV information. The NeuVS array can perform static and dynamic image recognition from the original color image by filtering red, green and blue noise, and significantly improve the recognition accuracy from 46 to 90%.

Efficient multi-scale representation of visual objects using a biologically plausible spike-latency code and winner-take-all inhibition

Deep neural networks have surpassed human performance in key visual challenges such as object recognition, but require a large amount of energy, computation, and memory. In contrast, spiking neural networks (SNNs) have the potential to improve both the efficiency and biological plausibility of object recognition systems. Here we present a SNN model that uses spike-latency coding and winner-take-all inhibition (WTA-I) to efficiently represent visual stimuli using multi-scale parallel processing. Mimicking neuronal response properties in early visual cortex, images were preprocessed with three different spatial frequency (SF) channels, before they were fed to a layer of spiking neurons whose synaptic weights were updated using spike-timing-dependent-plasticity. We investigate how the quality of the represented objects changes under different SF bands and WTA-I schemes. We demonstrate that a network of 200 spiking neurons tuned to three SFs can efficiently represent objects with as little as 15 spikes per neuron. Studying how core object recognition may be implemented using biologically plausible learning rules in SNNs may not only further our understanding of the brain, but also lead to novel and efficient artificial vision systems.

Multi-modulated optoelectronic memristor based on Ga2O3/MoS2 heterojunction for bionic synapses and artificial visual system

With the development of artificial intelligence, the demand for brain-like intelligent devices capable of breaking the von Neumann bottleneck is growing. In light of the advantages of high speed, low power consumption, and high integration resulting from the combination of memory and underlying perception, the memristor has great potential for implementing bionic synapses and constructing artificial visual system. Herein, a Ga2O3/MoS2 heterojunction-based multi-modulated optoelectronic memristor (MMOM) is proposed and demonstrated. In response to specific electrical signals, the device empowers the emulation of synaptic functions including short/long-term plasticity and shows an adjustable modulation range through changing pulse parameters. Meanwhile, versatile optical signals are also able to evoke synaptic behaviors, for instance, the transition from short-term to long-term memory and the "learning-forgetting-relearning" function. Utilizing the excellent optical response, a 16 × 16 MMOM array is assembled to simulate the perception-memory integrated human visual system. Further, to optimize the static means of modulating bi-terminal visual synapses, a modulatory synapse exploiting varying positive and negative electrical signals is presented and validated. Predictably, a multi-signal engaged heterologous synapse can be constructed for the complex visual system, which greatly enriches the functionality of bionic synapses and offers the possibility to realize a dynamically tunable artificial visual system.