Artificial Perception Systems Research Articles

Biomimic and bioinspired soft neuromorphic tactile sensory system

The progress in flexible and neuromorphic electronics technologies has facilitated the development of artificial perception systems. By closely emulating biological functions, these systems are at the forefront of revolutionizing intelligent robotics and refining the dynamics of human–machine interactions. Among these, tactile sensory neuromorphic technologies stand out for their ability to replicate the intricate architecture and processing mechanisms of the brain. This replication not only facilitates remarkable computational efficiency but also equips devices with efficient real-time data-processing capability, which is a cornerstone in artificial intelligence evolution and human–machine interface enhancement. Herein, we highlight recent advancements in neuromorphic systems designed to mimic the functionalities of the human tactile sensory system, a critical component of somatosensory functions. After discussing the tactile sensors which biomimic the mechanoreceptors, insights are provided to integrate artificial synapses and neural networks for advanced information recognition emphasizing the efficiency and sophistication of integrated system. It showcases the evolution of tactile recognition biomimicry, extending beyond replicating the physical properties of human skin to biomimicking tactile sensations and efferent/afferent nerve functions. These developments demonstrate significant potential for creating sensitive, adaptive, plastic, and memory-capable devices for human-centric applications. Moreover, this review addresses the impact of skin-related diseases on tactile perception and the research toward developing artificial skin to mimic sensory and motor functions, aiming to restore tactile reception for perceptual challenged individuals. It concludes with an overview of state-of-the-art biomimetic artificial tactile systems based on the manufacturing–structure–property–performance relationships, from devices mimicking mechanoreceptor functions to integrated systems, underscoring the promising future of artificial tactile sensing and neuromorphic device innovation.

Memory-electroluminescence for multiple action-potentials combination in bio-inspired afferent nerves

The development of optoelectronics mimicking the functions of the biological nervous system is important to artificial intelligence. This work demonstrates an optoelectronic, artificial, afferent-nerve strategy based on memory-electroluminescence spikes, which can realize multiple action-potentials combination through a single optical channel. The memory-electroluminescence spikes have diverse morphologies due to their history-dependent characteristics and can be used to encode distributed sensor signals. As the key to successful functioning of the optoelectronic, artificial afferent nerve, a driving mode for light-emitting diodes, namely, the non-carrier injection mode, is proposed, allowing it to drive nanoscale light-emitting diodes to generate a memory-electroluminescence spikes that has multiple sub-peaks. Moreover, multiplexing of the spikes can be obtained by using optical signals with different wavelengths, allowing for a large signal bandwidth, and the multiple action-potentials transmission process in afferent nerves can be demonstrated. Finally, sensor-position recognition with the bio-inspired afferent nerve is developed and shown to have a high recognition accuracy of 98.88%. This work demonstrates a strategy for mimicking biological afferent nerves and offers insights into the construction of artificial perception systems.

An Artificial Olfactory Chemical‐Resistant Synapse for Training‐Free Gas Recognition

AbstractTransferring the concept of chemical‐driven responses into artificial intelligence technology holds the key to mimicking olfactory for neuromorphic computing of chemical recognition. Currently, artificial olfactory systems are designed based on chemical sensor arrays. Time‐dependent responses of the sensor arrays are processed by artificial neural networks for recognition. However, the sensors generate instantly volatile responses, and algorithms for the processing of the time‐dependent responses have not been involved. The recognition accuracy and speed are severely impeded. A sensor array can only achieve an accuracy of 90% after at least 5 training epochs. Herein an artificial olfactory chemical‐resistant synapse consisting of 3D hierarchical WO3@WO3 nanofibers are demonstrated. The nanofibers exhibit persistent resistance responses through chemical exposures due to the strong chemisorption of water molecules. Typical synaptic behaviors including paired‐pulse facilitation, long‐term −1 short‐term memory, and learning experience have been achieved. Next, a recurrent neural network that is committed to processing the time‐dependent data is used to identify gas‐phase chemicals of 3‐hydroxy‐2‐butanone, triethylamine, and trimethylamine. Training‐free gas recognition has been realized by a WO3@WO3 nanofiber synapse only, in which the accuracy is above 90% at the first epoch. The results have great potential to satisfy stringent performance requirements on artificial perception systems.

Artificial visual‐tactile perception array for enhanced memory and neuromorphic computations

AbstractThe emulation of human multisensory functions to construct artificial perception systems is an intriguing challenge for developing humanoid robotics and cross‐modal human–machine interfaces. Inspired by human multisensory signal generation and neuroplasticity‐based signal processing, here, an artificial perceptual neuro array with visual‐tactile sensing, processing, learning, and memory is demonstrated. The neuromorphic bimodal perception array compactly combines an artificial photoelectric synapse network and an integrated mechanoluminescent layer, endowing individual and synergistic plastic modulation of optical and mechanical information, including short‐term memory, long‐term memory, paired pulse facilitation, and “learning‐experience” behavior. Sequential or superimposed visual and tactile stimuli inputs can efficiently simulate the associative learning process of “Pavlov's dog”. The fusion of visual and tactile modulation enables enhanced memory of the stimulation image during the learning process. A machine‐learning algorithm is coupled with an artificial neural network for pattern recognition, achieving a recognition accuracy of 70% for bimodal training, which is higher than that obtained by unimodal training. In addition, the artificial perceptual neuron has a low energy consumption of ∼20 pJ. With its mechanical compliance and simple architecture, the neuromorphic bimodal perception array has promising applications in large‐scale cross‐modal interactions and high‐throughput intelligent perceptions.image

Micro‐Pore Composed Soft Pressure Sensor with Ether‐Surfactant via High‐Sensitivity in Wide Range for Robotic Machine Interface Application

AbstractRobotic machine interfaces with sensor devices in robotics applications can facilitate the proper handling of objects. The wide‐range responses of the sensors are important for achieving statement of recognition in tactile sensing systems. Thus, this study proposes a novel micro‐pore‐induced soft pressure sensor via wide‐range response by synthesizing functional composite materials with ether‐surfactant and conductive materials. The aim is to produce soft pressure sensors that are highly responsive and exhibit a wide sensitivity range for the detection of an applied pressure. The sensor comprises a multiwalled‐carbon nanotube, an ether‐based surfactant, and polymeric soft materials, which exhibit high‐speed response characteristics because of the microporous structure of the sensing layer. Moreover, a novel wearable robotic machine interface for facilitating object manipulation is achieved. Furthermore, the findings of this study in terms of the performance of the robotic‐tactile sensing sensor confirm its suitability for the machine interface of an artificial perception system. Furthermore, a real‐time tactile sensing system using the manufactured sensor in the form of wearable e‐skins is experimentally demonstrated.

Brain-like optoelectronic artificial synapses with ultralow energy consumption based on MXene floating-gates for emotion recognition

In the new generation of brain-like optoelectronic visual signal processing and artificial perception systems, floating-gate artificial synaptic devices based on two-dimensional materials represent a feasible route.

An artificial neuromorphic somatosensory system with spatio-temporal tactile perception and feedback functions

The advancement in flexible electronics and neuromorphic electronics has opened up opportunities to construct artificial perception systems to emulate biological functions which are of great importance for intelligent robotics and human-machine interactions. However, artificial systems that can mimic the somatosensory feedback functions have not been demonstrated yet despite the great achievement in this area. In this work, inspired by human somatosensory feedback pathways, an artificial somatosensory system with both perception and feedback functions was designed and constructed by integrating the flexible tactile sensors, synaptic transistor, artificial muscle, and the coupling circuit. Also, benefiting from the synaptic characteristics of the designed artificial synapse, the system shows spatio-temporal information-processing ability, which can further enhance the efficiency of the system. This research outcome has a potential contribution to the development of sensor technology from signal sensing to perception and cognition, which can provide a special paradigm for the next generation of bionic tactile perception systems towards e-skin, neurorobotics, and advanced bio-robots.

Self-powered perception system based on triboelectric nanogenerator and artificial neuron for fast-speed multilevel feature recognition

Inspired by information processing in biological systems, the data processing efficiency of sensor systems combined with artificial neuromorphic devices has been significantly improved. However, the current reports on artificial perception systems are mainly based on artificial synapses, while little attention has been taken on the artificial neuron-based perception systems. Here, we propose a self-powered sensory neuron (SPSN) composed of a TaOx-based device-level artificial neuron and a high-sensitivity triboelectric nanogenerator (TENG) for self-powered artificial tactile sensing system. The SPSN with volatile switching and a high on/off ratio of 105 can mimic the leaky-integrate-and-fire (LIF) neuron model without additional circuitry and reset operations, while multiple stimulus inputs can be integrated to extract characteristic events due to its corresponding firing times to different stimuli. Moreover, we successfully constructed a 64´64 neuron-based artificial sensing array system and extracted the pressing trajectories and textures by the dynamic threshold characteristics of sensory neurons. Compared with the synapse-based artificial perception system, the neuron-based artificial perception system provides more feature extraction layers and faster data processing speed. Our results provide an effective strategy for building next-generation neuromorphic perception networks, intelligent human-computer interaction and high-speed feature recognition.

A Fully Solution-Printed Photosynaptic Transistor Array with Ultralow Energy Consumption for Artificial-Vision Neural Networks.

Photosynaptic organic field-effect transistors (OFETs) represent a viable pathway to develop bionic optoelectronics. However, the high operating voltage and current of traditional photosynaptic OFETs lead to huge energy consumption greater than that of the real biological synapses, hindering their further development in new-generation visual prosthetics and artificial perception systems. Here, a fully solution-printed photosynaptic OFET (FSP-OFET) with substantial energy consumption reduction is reported, where a source Schottky barrier is introduced to regulate charge-carrier injection, and which operates with a fundamentally different mechanism from traditional devices. The FSP-OFET not only significantly lowers the working voltage and current but also provides extraordinary neuromorphic light-perception capabilities. Consequently, the FSP-OFET successfully emulates visual nervous responses to external light stimuli with ultralow energy consumption of 0.07-34 fJ per spike in short-term plasticity and 0.41-19.87 fJ per spike in long-term plasticity, both approaching the energy efficiency of biological synapses (1-100 fJ). Moreover, an artificial optic-neural network made from an 8 × 8 FSP-OFET array on a flexible substrate shows excellent image recognition and reinforcement abilities at a low energy cost. The designed FSP-OFET offers an opportunity to realize photonic neuromorphic functionality with extremely low energy consumption dissipation.

Optoelectronic neuromorphic devices and their applications

Conventional computers based on the von Neumann architecture are inefficient in parallel computing and self-adaptive learning, and therefore cannot meet the rapid development of information technology that needs efficient and high-speed computing. Owing to the unique advantages such as high parallelism and ultralow power consumption, bioinspired neuromorphic computing can have the capability of breaking through the bottlenecks of conventional computers and is now considered as an ideal option to realize the next-generation artificial intelligence. As the hardware carriers that allow the implementing of neuromorphic computing, neuromorphic devices are very critical in building neuromorphic chips. Meanwhile, the development of human visual systems and optogenetics also provides a new insight into how to study neuromorphic devices. The emerging optoelectronic neuromorphic devices feature the unique advantages of photonics and electronics, showing great potential in the neuromorphic computing field and attracting more and more attention of the scientists. In view of these, the main purpose of this review is to disclose the recent research advances in optoelectronic neuromorphic devices and the prospects of their practical applications. We first review the artificial optoelectronic synapses and neurons, including device structural features, working mechanisms, and neuromorphic simulation functions. Then, we introduce the applications of optoelectronic neuromorphic devices particularly suitable for the fields including artificial vision systems, artificial perception systems, and neuromorphic computing. Finally, we summarize the challenges to the optoelectronic neuromorphic devices, which we are facing now, and present some perspectives about their development directions in the future.

Flexible memristive spiking neuron for neuromorphic sensing and computing

Inspired by the working modes of the human brain, the spiking neuron plays an important role as the basic computing unit of artificial perception systems and neuromorphic computing systems. However, the neuron circuit based on complementary metal-oxide-semiconductor technology has a complex structure, high power consumption, and limited flexibility. These features are not conducive to the large-scale integration and the application of flexible sensing systems compatible with the human body. The flexible memristor prepared in this work shows stable threshold switching characteristics and excellent mechanical bending characteristics with bending radius up to 1.5 mm and bending times up to 10<sup>4</sup>. The compact neuron circuit based on this device shows the key features of the neuron, such as threshold-driven spiking, all-or-nothing, refractory period, and strength-modulated frequency response. The frequency-input voltage relationship of the neuron shows the similarity of the rectified linear unit, which can be used to simulate the function of rectified linear unit in spiking neural networks. In addition, based on the electron transport mechanism, a core-shell model is introduced to analyze the working mechanism of the flexible memristor and explain the output characteristics of the neuron. In this model, the shell region consisting of Nb<sub>2</sub>O<sub>5–<i>x</i></sub> is subjected to ohmic conduction, while the core region consisting of NbO<sub>2</sub> is dominated by Poole-Frenkel conduction. These two mechanisms, combined with Newton’s law of cooling, dominate the threshold switching behavior of flexible memristor device. Furthermore, the threshold switching characteristic of the memristor is simulated, verifying the rationality of the working mechanism of the flexible memristor. Considering the fact that the threshold voltage decreases with temperature increasing, a correction term is added to the temperature of the shell region. Subsequently, the output characteristics of the neuron regulated by the input voltage are simulated. The simulation results show that the frequency increases but the threshold voltage decreases with the input voltage increasing, which is consistent with the experimental result. The introduction of the correction term confirms the influence of the thermal accumulation effect of the flexible substrate on neuron output characteristics. Finally, we build a spiking neural network based on memristive spiking neurons to implement handwriting recognition, achieving a 95.6% recognition rate, which is comparable to the ideal result of the artificial neural network (96%). This result shows the potential application of the memristive spiking neurons in neuromorphic computing. In this paper, the study of flexible neurons can guide the design of neuromorphic sensing and computing systems.

Self-Powered Synaptic Transistor for Artificial Perception

Artificial perception system is expected to perceive and integrate a large amount of sensory data to dynamically train inspired neural networks. It is expected to become a promising candidate for the realization of new imitating neural networks. But all reported artificial perception usually requires additional voltage to operate the tactile sensor, and most of the research on tactile sensing lacks learning and memory functions. Here, we propose a self-powered artificial perception system, which is combined with triboelectric nanogenerators (TENG) and synaptic transistors to realize a unique self-powered artificial synapse system. It successfully demonstrates the typical characteristics of sensory memory, such as excitatory postsynaptic current and paired impulse promotion. The impulse signal response under different pressure senses is created. Finally, Ebbinghaus memory is also simulated by the system. The artificial nervous system is expected to become a promising technology for neural perception and artificial skin systems in the future.

Flexible Artificial Sensory Systems Based on Neuromorphic Devices.

Emerging flexible artificial sensory systems using neuromorphic electronics have been considered as a promising solution for processing massive data with low power consumption. The construction of artificial sensory systems with synaptic devices and sensing elements to mimic complicated sensing and processing in biological systems is a prerequisite for the realization. To realize high-efficiency neuromorphic sensory systems, the development of artificial flexible synapses with low power consumption and high-density integration is essential. Furthermore, the realization of efficient coupling between the sensing element and the synaptic device is crucial. This Review presents recent progress in the area of neuromorphic electronics for flexible artificial sensory systems. We focus on both the recent advances of artificial synapses, including device structures, mechanisms, and functions, and the design of intelligent, flexible perception systems based on synaptic devices. Additionally, key challenges and opportunities related to flexible artificial perception systems are examined, and potential solutions and suggestions are provided.

Bioinspired Flexible, Dual‐Modulation Synaptic Transistors toward Artificial Visual Memory Systems

AbstractA prominent challenge for artificial synaptic devices toward artificial perception systems is hardware redundancy, which demands neuromorphic devices that integrate both sensing and processing functions. Inspired by the biological visual and nervous systems, a novel flexible, dual‐modulation synaptic field‐effect transistor (SFET) is demonstrated in this work. The flexible SFET is constructed with zinc oxide nanowires and sodium alginate, which acts as the semiconductor layer and the gate dielectric, respectively. An excitatory postsynaptic current in this artificial synapse can be triggered by both electrical and ultraviolet stimuli as presynaptic spikes as a result of the electric double layer effect and the photoelectric effect. More importantly, through the co‐modulation of light and electric stimuli, the memory level of the artificial synapses can be tuned based on the transformation between short‐term plasticity and long‐term plasticity initiated by the gate voltage. Different voltages can modulate the memory retention levels of the optical inputs similar to the function of the optic nerve system. The underlying mechanisms for the SFET are investigated using Fourier transform infrared spectroscopy, photoluminescence, and X‐ray photoelectron spectroscopy. Overall, the devices provide a novel idea to mimic visual memory, showing a promising strategy for future electronic eyes.

Structure-Defined DNA-Carbon Nanotube Hybrids and Their Applications

In this paper, I present first our current understanding of structure-defined DNA-carbon nanotube hybrids, then a description of the DNA sequence selection problem, and finally utility of DNA-carbon nanotube hybrids in molecular sensing. An artificial perception system, i.e. molecular perceptron, is proposed to take full advantage of the structure diversity of DNA-carbon nanotube hybrids.

A Dual-Organic-Transistor-Based Tactile-Perception System with Signal-Processing Functionality.

Organic-device-based tactile-perception systems can open up new opportunities for the next generation of intelligent products. To meet the critical requirements of artificial perception systems, the efficient construction of organic smart elements with integrated sensing and signal processing functionalities is highly desired, but remains a challenge. This study presents a dual-organic-transistor-based tactile-perception element (DOT-TPE) with biomimetic functionality by the construction of organic synaptic transistors with integrated sensing transistors. The unique geometry of the DOT-TPE permits instantaneous sensing of pressure stimuli and synapse-like processing of an electric signal in a single element. More importantly, these organic-transistor-based tactile-perception elements can be built into arrays to serve as bionic tactile-perception systems. The combined biomimetic functionality of tactile-perception systems, together with their promising features of flexibility and large-area fabrication, makes this work represent a step forward toward novel e-skin devices for artificial intelligence.

Introduction to Stochastic Computing using a Remote Lab with Reconfigurable Logic

This short paper introduces the basic concepts of Stochastic Computing (SC), and presents additions to a remote lab with reconfigurable logic to allow testing SC circuits. Recently, SC has been revisited and evaluated as a possible way of performing approximate probabilistic computations for artificial perception systems. New modules allow the generation of pseudo-random numbers, given a seed key and using linear feedback shift registers, but also having true random number generation using ring oscillators and embedded PLLs. Stochastic computing allows a tradeoff between resource usage and precision, allowing very simple circuits to perform computations, at the expense of a longer integration time to have reasonable results. We provide the basic stochastic computing modules, so that any user can use them to build a stochastic computing circuit and go beyond software simulations, providing a remote hardware device to test real circuits at high clock speeds.

Microsoft kinect-based artificial perception system for control of functional electrical stimulation assisted grasping.

We present a computer vision algorithm that incorporates a heuristic model which mimics a biological control system for the estimation of control signals used in functional electrical stimulation (FES) assisted grasping. The developed processing software acquires the data from Microsoft Kinect camera and implements real-time hand tracking and object analysis. This information can be used to identify temporal synchrony and spatial synergies modalities for FES control. Therefore, the algorithm acts as artificial perception which mimics human visual perception by identifying the position and shape of the object with respect to the position of the hand in real time during the planning phase of the grasp. This artificial perception used within the heuristically developed model allows selection of the appropriate grasp and prehension. The experiments demonstrate that correct grasp modality was selected in more than 90% of tested scenarios/objects. The system is portable, and the components are low in cost and robust; hence, it can be used for the FES in clinical or even home environment. The main application of the system is envisioned for functional electrical therapy, that is, intensive exercise assisted with FES.

Engineering of holonic multi agent intelligent forest fire monitoring system

In this paper we describe holonic multi agent architecture of iForestFire Intelligent Forest Fire Monitoring System through phases and activities of the holonic software engineering process. iForestFire is a web-based information system used in all aspects of forest fire management. Its central part is an artificial perception system, observer network, whose aim is early detection of forest fires. Although the agents that the system consists of are inseparably connected, they can be clustered in several independent systems that can be used separately. Independent holons are built from agents that are part of the observer network, as well as other agents with auxiliary functionalities.

Novel Neural Network Models for Computing HomotheticInvariances: An Image Algebra Notation

In this paper we propose a theoretical approach to invariant perception. Invariant perception is an important aspect in both natural and artificial perception systems, and it remains an important unsolved problem in heuristically based pattern recognition. Our approach is based on a general theory of neural networks and studies of invariant perception by the cortex. The neural structures that we propose uphold both the architecture and functionality of the cortex as currently understood. The formulation of the proposed neural structures is in the language of image algebra, a mathematical environment for expressing image processing algorithms. Thus, an additional benefit of our study is the implication that image algebra provides an excellent environment for expressing and developing artificial perception systems. The focus of our study is on invariances that are expressible in terms of affine transformations, specifically, homothetic transformations. Our discussion will include both one-dimensional and two-dimensional signal patterns. The main contribution of this paper is the formulation of several novel morphological neural networks that compute homothetic auditory and visual invariances. With respect to the latter, we employ the theory and trends of currently popular artificial vision systems.