Neuromorphic Vision System Research Articles

A Flexible Artificial Spiking Photoreceptor Enabled by a Single VO2 Mott Memristor for the Spike-Based Electronic Retina.

The neuromorphic vision system that utilizes spikes as information carriers is crucial for the formation of spiking neural networks. Here, we present a bioinspired flexible artificial spiking photoreceptor (ASP), which is realized by using a single VO2 Mott memristor that can simultaneously sense and encode the stimulus light into spikes. The ASP has high spike-encoded photosensitivity and ultrawide photosensing range (405-808 nm) with good endurance (>7 × 107) and high flexibility (bending radius ∼5 mm). Then, we put forward an all-spike electronic retina architecture that comprises one layer of ASPs and one layer of artificial optical nerves (AONs) to process the spike information. Each AON consists of a single Mott memristor connected in series with a neuro-transistor that is a multiple-input floating-gate MOS transistor. Simulation results demonstrate that the all-spike electronic retina can successfully segment images with high Shannon entropy, thus laying the foundation for the development of a spike-based neuromorphic vision system.

Chiral Perovskite Nanowire Optoelectronic Synapse for Full‐Stokes Polarization‐Resolved Perception and Reservoir Computing

AbstractFull‐Stokes polarization‐resolved photonic artificial synapse (PAS) devices with in‐sensor computing capabilities show great potential for multimode vision perception and processing. However, current polarization‐resolved PAS devices are limited to monotonic recognition of linearly polarized light or circularly polarized light. Here, the study develops a chiral perovskite nanowire‐based PAS device as a full‐Stokes polarimeter to achieve polarization‐dependent neuromorphic optoelectronics. The device demonstrates a significantly high circular dichroism of over 400 millidegree and intrinsic linear dichroism, contributing to effectively sensing polarized light. Efficient charge separation at the perovskite NW/MXene heterostructure results in a prolonged carrier lifetime, thereby achieving a high responsivity of 2.3 AW−1, excellent synaptic behavior, and low power consumption of 0.5 pJ per synaptic stimuli. Significantly, the polarization‐modulated reservoir computing is developed based on the full‐Stokes PAS device, which exhibits a low normalized root mean square error of 0.023 in the chaotic system forecasting task. The single‐nanowire PAS device with the capability of sensing full‐Stokes parameters demonstrates great significance in building advanced energy‐efficient polarization‐resolved nanoscale neuromorphic vision systems.

Bioinspired Flexible and Low-Voltage Organic Synaptic Transistors for UV Light-Driven Vision Systems.

Neuromorphic vision systems, particularly those stimulated by ultraviolet (UV) light, hold great potential applications in portable electronics, wearable technology, biological analysis, military surveillance, etc. Organic artificial synaptic devices hold immense potential in this field due to their ease of processing, flexibility, and biocompatibility. In this work, we have fabricated a flexible organic field-effect transistor (OFET) that utilizes chitosan-silver nanoparticles (AgNPs) composite material as the active dielectric material. During UV light illumination, both silver nanoparticles and the pentacene layer generate a large number of charge carriers. The photogenerated carriers lead to a more significant hole accumulation at the pentacene interface, resulting in a current rise. In the absence of light, the trapped electron in the silver nanoparticles persists for a longer duration, preventing the instant recombination with holes. This extended retention of electrons leads to the observed synaptic performance of the transistor. The use of aluminum oxide (Al2O3) as one of the dielectric layers enables the device to operate effectively at low voltage (<1 V). The device mimics various crucial synaptic properties of the brain, including short-term potentiation and long-term potentiation (STP and LTP), paired-pulse facilitation (PPF), spike-duration dependent plasticity (SDDP), spike-number dependent plasticity (SNDP), and spike-rate dependent plasticity (SRDP), etc. This work introduces an approach to develop flexible organic synaptic transistors that operate efficiently at low voltages, paving the way toward high-performance, UV light-driven neuromorphic vision systems.

High Photoresponsivity and Fast Response Speed Ferroelectric Photomemristor for Artificial Visual System Application

AbstractFerroelectric photoelectric devices show promising applications in artificial vision systems. However, this type of devices used in artificial vision systems exhibits low photoresponsivity and this limited cyclability of photocurrents, which hinders the progress of artificial vision systems. In this study, an artificial vision system is constructed with high photoresponsivity, fast photoresponse speed, and high read/write speed based on the Pd/PbZr0.4Ti0.6O3/La0.3Sr0.7MnO3/LaAlO3 ferroelectric photomemristor. The high optical response (18.86 nanoamperes) and read/write speeds (50 nanoseconds) of the device significantly improve the robustness and speed of the neuromorphic visual system while reducing power consumption for recognition. The artificial vision system developed in this study provides excellent real‐time image processing capabilities, including edge detection and classification with 100% accuracy for musical key patterns, and it is expected to offer great potential for the development of high‐performance self‐powered artificial vision systems.

Tunable memory behavior in light stimulated artificial synapse based on ZnO thin film transistors

Abstract Optoelectronic synapses are inevitable for realizing neuromorphic vision systems, which require the integration of image recognition, memory and image processing into a single platform. In this work, we present a three terminal optoelectronic synapse created using zinc oxide (ZnO) thin film transistor. The persistent photoconductivity (PPC) of ZnO thin film is utilized to demonstrate the synaptic behaviour. The change in conductance of the device under UV illumination has been interpreted as the weight change in the synapse. The basic synaptic functions such as sensory memory, short term memory, long term memory, duration time dependent plasticity and paired pulse facilitation have been successfully demonstrated. The device shows a PPF index of 160\%, comparable to other optoelectronic synapses reported in literature. Further, to corroborate the existing theory that PPC is caused by oxygen vacancies, additional characterizations are carried out and the presence of oxygen vacancies is detected in the fabricated ZnO device. Subsequently, pattern recognition of MNIST handwritten dataset has been performed using the conductance tuning curves of the proposed ZnO TFT based synapses in a neural network architecture, thereby demonstrating their feasibility to be used in neuromorphic applications.

Multimodal In‐Sensor Computing Implemented by Easily‐Fabricated Oxide‐Heterojunction Optoelectronic Synapses

AbstractThe advent of big data and the Internet of Things has created urgent demands for in‐sensor computing hardware with multimodal perception that can effectively resolve the inefficiency, high latency, and excessive energy consumption challenges faced by conventional sensory systems. Here, a simple‐structured optoelectronic synaptic device of In2O3·SnO2/Nb:SrTiO3 (ITO/NSTO) heterostructure is proposed, which vividly demonstrates in‐sensor computing and multimodal perception capabilities. First, with the ingenious synaptic responses of the device under both optical and electrical stimuli, a multimodal in‐sensor neuromorphic computing system capable of concurrently perceiving and processing visual and auditory information is constructed. Using this multimodal system to perform a human emotion recognition task, the misjudgment arising from single‐modal cognition can thereby be effectively avoided. Second, utilizing the integrated sensing and processing functions, along with the dynamic memories of the device array, a neuromorphic vision system is implemented for real‐time monitoring of moving vehicles, which displays high recognition efficiency and accuracy. This research not only provides an optoelectronic synaptic device that is low‐cost and easy to mass‐produce, but also paves the way for next‐generation in‐sensor computing that can efficiently process multimodal signals.

Integrated Bionic Human Retina Process and In‐Sensor RC System Based on 2D Retinomorphic Memristor Array

AbstractThe rapid development of machine vision has put forward higher requirements for image perception and visual learning systems. However, limited by the von Neumann bottleneck, traditional artificial vision systems rely on complex optical and circuit designs, and the separate distribution of sensors, memory, and computing modules results in high energy consumption and high latency. Memristor‐based reservoir computing (RC) system with integrated sensing and memory functions provides a solution to effectively improve the computational efficiency of artificial vision networks. Here, a complete artificial visual recognition system based on in‐sensor RC is developed by combining the excellent optoelectronic synaptic properties of 2D WS2 memristor array. Through effectively regulating the conductance of memristors by light or electrical stimulation, image information is converted into high‐dimensional information in the reservoir layer, and the classification can be completed by simple training, significantly reducing the training complexity and cost consumption. With this system, an effective recognition of hand‐written numbers with an accuracy of 88.3% is demonstrated, and a 100% recognition of traffic signals of different colors is achieved. The proposed in‐sensor RC system advances the development and application of artificial vision recognition, paving the way for efficient machine learning and neuromorphic vision systems.

An insect vision-inspired neuromorphic vision systems in low-light obstacle avoidance for intelligent vehicles

An insect vision-inspired neuromorphic vision systems in low-light obstacle avoidance for intelligent vehicles

Visible-light responsive CdS-QDs modified InGaZnO synapse for biologically plausible color-to-gray conversion

The energy-efficient perceptual capabilities of the human visual system have inspired the development of optoelectronic neuromorphic devices to replicate the essential properties. Color vision that is highly important for perception and cognitive behaviors, such as object recognition, esthetics, and communication, has recently been reported based on optoelectronic neuromorphic devices. The responses to different wavelengths of light are mainly dependent on power density, whereas the eye recognizes color images based on luminosity. This Letter presents a quantum dots (QDs) modified InGaZnO (IGZO) synaptic transistor based on an IGZO/CdS-QDs/PMMA heterojunction. These IGZO-based synapses show significant responsivity to 655-nm light through the spectrum broadening by QDs. More importantly, luminance-based color-to-gray conversion can be realized with this device using a spike encoding scheme, transforming color images into grayscale in parallel. Notably, the QDs modified IGZO-based synapses demonstrated a remarkable 34.3% improvement in recognition accuracy on the multicolor MNIST dataset in comparison with the pristine devices. Our results indicate that this visible-light responsive IGZO-based synapse has great potential for endowing the neuromorphic visual system with enhanced color vision and provides a biologically plausible approach for color-to-gray conversion.

A Biodegradable, Stretchable, Healable, and Self-Powered Optoelectronic Synapse Based on Ionic Gelatins for Neuromorphic Vision System.

Optoelectronic synapses have gained increasing attentions as a fundamental building block in the development of neuromorphic visual systems. However, it remains a challenge to integrate multiple functions into a single optoelectronic synapse that can be widely applied in wearable artificial intelligence and implantable neuromorphic vision systems. In this study, a stretchable optoelectronic synapse based on biodegradable ionic gelatin heterojunction is successfully developed. This device exhibits self-powered synaptic plasticity behavior with broad spectral response and excellent elastic properties, yet it degrades rapidly upon disposal. After complete cleavage, the device can be fully repaired within 1min, which is mainly attributed to the non-covalent interactions between different molecular chains. Moreover, the recovery and reprocessing of the ionic gelatins result in optoelectronic properties that are virtually indistinguishable from their original state, showcasing the resilience and durability of ionic gelatins. The combination of biodegradability, stretchability, self-healing, zero-power consumption, ease of large-scale preparation, and low cost makes the work a major step forward in the development of biodegradable and stretchable optoelectronic synapses.

Mimicking the retinal neuron functions by a photoresponsive single transistor with a double gate

To realize a low-cost neuromorphic visual system, employing an artificial neuron capable of mimicking the retinal neuron functions is essential. A photoresponsive single transistor neuron composed of a vertical silicon nanowire is proposed. Similar to retinal neurons, various photoresponsive characteristics of the single transistor neuron can be modulated by light intensity as well as wavelength and have a high responsivity to green light like the human eye. The device is designed with a cylindrical surrounding double-gate structure, enclosed by an independently controlled outer gate and inner gate. The outer gate has the function of selectively inhibiting neuron activity, which can mimic lateral inhibition of amacrine cells to ganglion cells, and the inner gate can be utilized for the adjustment of the firing threshold voltage, which can be used to mimic the regulation of photoresponsivity by horizontal cells for adaptive visual perception. Furthermore, a myelination function that controls the speed of information transmission is obtained according to the inherent asymmetric source/drain structure of a vertical silicon nanowire. This work can enable photoresponsive neuronal function using only a single transistor, providing a promising hardware implementation for building miniaturized neuromorphic vision systems at low cost.

Gate-Tunable Positive and Negative Photoconductance in Near-Infrared Organic Heterostructures for In-Sensor Computing.

The rapid growth of sensor data in the artificial intelligence often causes significant reductions in processing speed and power efficiency. Addressing this challenge, in-sensor computing is introduced as an advanced sensor architecture that simultaneously senses, memorizes, and processes images at the sensor level. However, this is rarely reported for organic semiconductors that possess inherent flexibility and tunable bandgap. Herein, an organic heterostructure that exhibits a robust photoresponse to near-infrared (NIR) light is introduced, making it ideal for in-sensor computing applications. This heterostructure, consisting of partially overlapping p-type and n-type organic thin films, is compatible with conventional photolithography techniques, allowing for high integration density of up to 520 devices cm-2 with a 5µm channel length. Importantly, by modulating gate voltage, both positive and negative photoresponses to NIR light (1050nm) are attained, which establishes a linear correlation between responsivity and gate voltage and consequently enables real-time matrix multiplication within the sensor. As a result, this organic heterostructure facilitates efficient and precise NIR in-sensor computing, including image processing and nondestructive reading and classification, achieving a recognition accuracy of 97.06%. This work serves as a foundation for the development of reconfigurable and multifunctional NIR neuromorphic vision systems.

Multi‐Wavelength Optoelectronic Synaptic Transistors Based on Transition Metal Telluride‐Sulfide Heterostructures

AbstractNeuromorphic visual systems based on optogenetic techniques have colossal potential for in‐memory computing with prospects of developing artificial intelligence vision systems. However, conventional transistor architectures face formidable challenges in efficient signal processing owing to limitations in the intrinsic properties of active channel materials. In this work, a novel transition metal telluride‐sulfide hybrid heterojunction‐based optoelectronic synaptic phototransistor is proposed, in which UV–vis responsive zinc oxide encapsulated few‐layer tungsten disulfide channel is decorated with near‐infrared sensitive 0D cobalt ditelluride (CoTe2) nanocrystals (NCs), eliciting the ability to sense, store, and process optical signals across a broad range of the electromagnetic spectrum. This meticulously designed three‐layered heterostructure, based on their interfacial band alignments, enables high photoresponsivity up to ≈2.6 × 103 A W−1 at a back‐gate bias of 20 V, leading to the brain‐inspired synaptic applications with an average power consumption as low as 75 pJ for each training process. The device exhibits excitatory postsynaptic current, paired‐pulse facilitation with an index above 150%, as well as light‐modulated synaptic plasticity by mimicking biological synapses, which mainly originate from trapped holes in Co‐vacancy mediated surface defect states of CoTe2 NCs. Hence, this 2D material‐based hybrid phototransistor appears to be a promising candidate for energy‐efficient next‐generation brain‐inspired neuromorphic vision systems.

Robust Self‐powered Optoelectronic Synapses Based on Epitaxial InSe/GaN Heterojunction with Interfacial Charge‐Trapping Layer

AbstractNeuromorphic devices that parallelize perception, preprocessing, and computation functions are expected to play a significant role in future non‐von Neumann architecture computers. Herein, a new retina‐inspired broadband self‐powered optoelectronic synaptic device based on 2D/3D heterojunction of epitaxial InSe on GaN(0001) is reported. Few‐layer n‐type InSe is grown on p‐type GaN by physical vapor deposition in an ultra‐high vacuum (UHV) environment. The devices are fabricated using a shadow mask assisted UHV electrode deposition technique. High‐resolution transmission electron microscopy images reveal that an atomically thin amorphous layer, which induces highly efficient charge trapping, is formed at the InSe/GaN interface. The photoresponse spans from visible to near‐infrared, and the response time is prolonged to 103 ms owing to the deep trapping levels. Thus, synaptic functions, including excitatory postsynaptic current, paired‐pulse facilitation with a high index of up to 170%, short‐term plasticity, and high‐pass filtering characteristics, are realized. Additionally, the synapses demonstrated the merit of realizing image sharpening and arithmetic operations on the same device under infrared and visible light illumination. This study provides a new platform of 2D/3D heterostructures for robust optoelectronic synapses that may find applications in post‐Moore era neuromorphic vision systems.

Strain-insensitive viscoelastic perovskite film for intrinsically stretchable neuromorphic vision-adaptive transistors

Stretchable neuromorphic optoelectronics present tantalizing opportunities for intelligent vision applications that necessitate high spatial resolution and multimodal interaction. Existing neuromorphic devices are either stretchable but not reconcilable with multifunctionality, or discrete but with low-end neurological function and limited flexibility. Herein, we propose a defect-tunable viscoelastic perovskite film that is assembled into strain-insensitive quasi-continuous microsphere morphologies for intrinsically stretchable neuromorphic vision-adaptive transistors. The resulting device achieves trichromatic photoadaptation and a rapid adaptive speed (<150 s) beyond human eyes (3 ~ 30 min) even under 100% mechanical strain. When acted as an artificial synapse, the device can operate at an ultra-low energy consumption (15 aJ) (far below the human brain of 1 ~ 10 fJ) with a high paired-pulse facilitation index of 270% (one of the best figures of merit in stretchable synaptic phototransistors). Furthermore, adaptive optical imaging is achieved by the strain-insensitive perovskite films, accelerating the implementation of next-generation neuromorphic vision systems.

Photonic Synaptic Transistor with Memory Mode Switching for Neuromorphic Visual System

AbstractThe human retina is able to extract key feature information from a large amount of redundant visual information, which is the basis for efficient information processing in the human visual system. However, current retina‐inspired photonic synaptic devices lack fast noise filtering capabilities, limiting the speed of image preprocessing in neuromorphic visual systems. Here, a photonic synaptic transistor (PST) based on graphene/organic heterojunction that exhibits high photosensitivity and optically tunable synaptic characteristics from visible to near‐infrared (488–1310 nm) is demonstrated. The PST enables light‐intensity‐controlled memory‐free and long‐memory mode switching, allowing to achieve fast image noise filtering in a PST‐based vision sensor (processing times as low as 30 ms). In addition, image recognition in an artificial neural network connected by the PST, and the efficiency and accuracy of image recognition can be significantly improved by performing image noise filtering at the front‐end is demonstrated. This work provides the potential to improve the information processing speed of bio‐inspired neuromorphic visual systems and contribute to the development of machine vision applications.

Geometric Shape Recognition with an Ultra-High Density Perovskite Nanowire Array-Based Artificial Vision System.

Artificial vision systems (AVS) have potential applications in visual prosthetics and artificially intelligent robotics, and they require a preprocessor and a processor to mimic human vision. Halide perovskite (HP) is a promising preprocessor and processor due to its excellent photoresponse, ubiquitous charge migration pathways, and innate hysteresis. However, the material instability associated with HP thin films hinders their utilization in physical AVSs. Herein, we have developed ultrahigh-density arrays of robust HP nanowires (NWs) rooted in a porous alumina membrane (PAM) as the active layer for an AVS. The NW devices exhibit gradual photocurrent change, responding to changes in light pulse duration, intensity, and number, and allow contrast enhancement of visual inputs with a device lifetime of over 5 months. The NW-based processor possesses temporally stable conductance states with retention >105 s and jitter <10%. The physical AVS demonstrated 100% accuracy in recognizing different shapes, establishing HP as a reliable material for neuromorphic vision systems.

A Fourier neuromorphic visual system based on InGaZnO synaptic transistor

The hierarchical structure of the biological visual system enables multilevel features of sensory stimuli to be pre-extracted before being transmitted to the nerve center, rendering the remarkable ability to perceive, filter, categorize, and identify targets in complex environments. However, it is a challenge to resemble such extraction capability with respect to spatial features in a neuromorphic visual system. In this Letter, we propose an indium-gallium-zinc-oxide synaptic transistor-based Fourier neuromorphic visual system for image style classifying. The images are transformed into the frequency domain through an optic Fourier system, greatly reducing energy and time dissipation in comparison with numerical computation. Then, the transformed information is coded into spike trains, which are nonlinearly filtered by synaptic transistors. The energy consumption for this filtering process is estimated to be ∼1.28 nJ/pixel. The features of drawing style could be enhanced through the filtering process, which facilitates the followed pattern recognition. The recognition accuracy in classifying stylized images is significantly improved to 92% through such Fourier transform and filtering process. This work would be of profound implications for advancing neuromorphic visual system with Fourier optics enhanced feature extraction capabilities.

Multisensory Ferroelectric Semiconductor Synapse for Neuromorphic Computing

AbstractIntegrated multifunctionality in visual information processing is crucial in the artificial intelligence era. Compared to the parallel human vision system, current bionic vision devices exhibit a complex structure with single functionality, challenging intelligent processing and integration. Here, a multisensory artificial synapse with a crossbar structure comprising graphene/α‐In2Se3/graphene layers is demonstrated, merging sensing, memory, and computing while mimicking various synaptic properties. The Schottky barrier height is modulated by the polarization of ferroelectric semiconductor α‐In2Se3, enabling reconfigurable device conductance and photoresponsivity. This conductance emulates synaptic short‐term and long‐term plasticity through electrical pulse modulation, boasting a rapid 40 ns programming speed. The device also exhibits linearly regulated photoresponsivity under illumination, with synaptic plasticity from optical pulses. The fusion of electronic and optoelectronic devices enables both image front‐end processing and advanced post‐processing. In‐sensor front‐end processing enhances subsequent processing efficiency, with pattern recognition accuracy reaching 97%. This design fosters the advancement of multisensory and highly integrated neuromorphic vision systems.

Optoelectronic stimuli-driven switchable memristors with multilevel resistance states for neuromorphic vision sensors

The developed optoelectronic synaptic devices with resistive switching enable efficient, fault-resistant image recognition. The PMMA/ZnO/2IF memristor array advances neuromorphic vision systems, offering high accuracy and low power consumption.