Bio-inspired Sensor Research Articles

A Selective-Response Bioinspired Strain Sensor Using Viscoelastic Material as Middle Layer.

Flexible strain sensors have an irreplaceable role in critical and emerging fields, such as electronic skins, flexible robots, and prosthetics. Although numerous efforts have been made to improve sensor sensitivity to meet specific application scenarios, the signal-to-noise ratio (SNR) is an extremely critical and non-negligible indicator, which takes into account higher sensitivity, meaning that they can also detect the noise signals with high sensitivity. Coincidentally, scorpions with ultrasensitive vibration sensilla also face such a dilemma. Here, it is found that the scorpion ingeniously uses the viscoelastic material in front of its slit sensilla to realize efficient preprocessing of the signal. Its mechanism is that the loss factor of materials changes with frequency, affecting energy storage and transmission. Inspired by this ingenious strategy, a bioinspired strain sensor insensitive to a low strain rate was designed using a two-step template transfer method. As a result, its relative change in resistance reached 110% under the same strain (0.3197%) but with different strain rates (0.1 Hz and ∼20 Hz). The noncontact vibration experiments also show different responses to low-frequency vibration and high-frequency impact. Moreover, it can also be used as a typical flexible strain sensor. Under the tensile state, it has a gauge factor (GF) as high as 4596 upon 0.6% strain, and the response time is 140 ms. Therefore, it is expected that this strain sensor will be used in many important ultraprecision measurement fields, especially when the measured signal is small.

An ultra-high-speed hardware accelerator for image reconstruction and stereo rectification on event-based camera

Event-based cameras are novel bio-inspired vision sensors, which sense brightness changes rather than the actual intensity level. In contrast to conventional cameras, such cameras capture new information about the scene in the form of sparse events at a very short latency. Recently, some researchers have studied on efficient event-based camera reconstruction approaches to obtain high-quality images. These efforts make performing stereo vision based on event-cameras being possible. However, traditional stereo architecture cannot process such high-speed image streams generated by the event-based camera in real-time (in the order of μs), which calls for new approaches.In this paper, we provide a set of practical solutions on visual image reconstruction and stereo rectification for spike camera, which serves two important procedures of stereo vision. We first provide FPGA-accelerated hardware architecture to achieve ultra-high-speed visual image reconstruction with full texture of natural scenes from spike data. Then, an FPGA-accelerated ultra-high-speed stereo rectification architecture is proposed to rectify the reconstructed images generated for stereo vision. In this architecture, a fully pipelined calculation module is designed to process the complex coordinate transformation operations, which puts in pipelined registers to maximize the clock frequency. To further improve the throughput, we propose a parallel processing architecture that uses multiple processing elements (PEs) to process multiple pixels per cycle. In addition, we design a memory management unit (MMU) to optimize the memory usage of the hardware resource. The whole architecture is effectively implemented on a Xilinx Zynq7100 FPGA chip. We evaluate the proposed architecture with different settings. The experiments show that our architecture can process the spike streams in real-time manner.

Bio-inspired robotics enabled schemes in blockchain-fog-cloud assisted IoMT environment

Due to emerging developments in sports games, the usage of bio-ankle sensors has been growing progressively. Whereas, Internet of Medical Things (IoMT) is an emerging network that boosts bio-inspired sensors’ performances onto the fog-cloud network. However, a sequence of processes is required to complete the healthcare process for one sportsman. Therefore, workflow-enabled bio-inspired sensors tasks scheduled in IoMT postures different challenges. For instance, cost-efficient scheduling, security, and data validation in distributed hospitals to share their data. In this paper, we devise bio-inspired robotics-enabled schemes in the blockchain-fog-cloud-assisted IoMT environment. The goal is to minimize execution cost and blockchain of applications. Based on the proposed system, the study devises bio-inspired robotics function blockchain task scheduling (BIR-FBTS) schemes, determining the optimal assignment of tasks to the available nodes. The simulation results show that the proposed methods minimized 50% of the service cost and 40% of mined cost in the system compared to all existing bio-inspired healthcare systems.

Fusion-Based Feature Attention Gate Component for Vehicle Detection Based on Event Camera

In the field of autonomous vehicles, various heterogeneous sensors, such as LiDAR, Radar, camera, etc, are combined to improve the vehicle ability of sensing accuracy and robustness. Multi-modal perception and learning has been proved to be an effective method to help vehicle understand the nature of complex environments. Event camera is a bio-inspired vision sensor that captures dynamic changes in the scene and filters out redundant information with high temporal resolution and high dynamic range. These characteristics of the event camera make it have a certain application potential in the field of autonomous vehicles. In this paper, we introduce a fully convolutional neural network with feature attention gate component (FAGC) for vehicle detection by combining frame-based and event-based vision. Both grayscale features and event features are fed into the feature attention gate component (FAGC) to generate the pixel-level attention feature coefficients to improve the feature discrimination ability of the network. Moreover, we explore the influence of different fusion strategies on the detection capability of the network. Experimental results demonstrate that our fusion method achieves the best detection accuracy and exceeds the accuracy of the method that only takes single-mode signal as input.

DA4Event: Towards Bridging the Sim-to-Real Gap for Event Cameras Using Domain Adaptation

Event cameras are novel bio-inspired sensors, which asynchronously capture pixel-level intensity changes in the form of “events”. The innovative way they acquire data presents several advantages over standard devices, especially in poor lighting and high-speed motion conditions. However, the novelty of these sensors results in the lack of a large amount of training data capable of fully unlocking their potential. The most common approach implemented by researchers to address this issue is to leverage simulated event data. Yet, this approach comes with an open research question: how well simulated data generalize to real data? To answer this, we propose to exploit, in the event-based context, recent Domain Adaptation (DA) advances in traditional computer vision, showing that DA techniques applied to event data help reduce the sim-to-real gap. To this purpose, we propose a novel architecture, which we call Multi-View DA4E (MV-DA4E), that better exploits the peculiarities of frame-based event representations while also promoting domain invariant characteristics in features. Through extensive experiments, we prove the effectiveness of DA methods and MV-DA4E on N-Caltech101. Moreover, we validate their soundness in a real-world scenario through a cross-domain analysis on the popular RGB-D Object Dataset (ROD), which we extended to the event modality (RGB-E).

Bio-inspired uniform flow microfluidic sensor platform for multi-analyte sensing: a simulation-based outflow and injection study

Bio-inspired uniform flow microfluidic sensor platform for multi-analyte sensing: a simulation-based outflow and injection study

Spiking Neural Network-Based Near-Sensor Computing for Damage Detection in Structural Health Monitoring

In this work, we present an innovative approach for damage detection of infrastructures on-edge devices, exploiting a brain-inspired algorithm. The proposed solution exploits recurrent spiking neural networks (LSNNs), which are emerging for their theoretical energy efficiency and compactness, to recognise damage conditions by processing data from low-cost accelerometers (MEMS) directly on the sensor node. We focus on designing an efficient coding of MEMS data to optimise SNN execution on a low-power microcontroller. We characterised and profiled LSNN performance and energy consumption on a hardware prototype sensor node equipped with an STM32 embedded microcontroller and a digital MEMS accelerometer. We used a hardware-in-the-loop environment with virtual sensors generating data on an SPI interface connected to the physical microcontroller to evaluate the system with a data stream from a real viaduct. We exploited this environment also to study the impact of different on-sensor encoding techniques, mimicking a bio-inspired sensor able to generate events instead of accelerations. Obtained results show that the proposed optimised embedded LSNN (eLSNN), when using a spike-based input encoding technique, achieves 54% lower execution time with respect to a naive LSNN algorithm implementation present in the state-of-the-art. The optimised eLSNN requires around 47 kCycles, which is comparable with the data transfer cost from the SPI interface. However, the spike-based encoding technique requires considerably larger input vectors to get the same classification accuracy, resulting in a longer pre-processing and sensor access time. Overall the event-based encoding techniques leads to a longer execution time (1.49×) but similar energy consumption. Moving this coding on the sensor can remove this limitation leading to an overall more energy-efficient monitoring system.

EvAn: Neuromorphic Event-Based Sparse Anomaly Detection.

Event-based cameras are bio-inspired novel sensors that asynchronously record changes in illumination in the form of events. This principle results in significant advantages over conventional cameras, such as low power utilization, high dynamic range, and no motion blur. Moreover, by design, such cameras encode only the relative motion between the scene and the sensor and not the static background to yield a very sparse data structure. In this paper, we leverage these advantages of an event camera toward a critical vision application—video anomaly detection. We propose an anomaly detection solution in the event domain with a conditional Generative Adversarial Network (cGAN) made up of sparse submanifold convolution layers. Video analytics tasks such as anomaly detection depend on the motion history at each pixel. To enable this, we also put forward a generic unsupervised deep learning solution to learn a novel memory surface known as Deep Learning (DL) memory surface. DL memory surface encodes the temporal information readily available from these sensors while retaining the sparsity of event data. Since there is no existing dataset for anomaly detection in the event domain, we also provide an anomaly detection event dataset with a set of anomalies. We empirically validate our anomaly detection architecture, composed of sparse convolutional layers, on this proposed and online dataset. Careful analysis of the anomaly detection network reveals that the presented method results in a massive reduction in computational complexity with good performance compared to previous state-of-the-art conventional frame-based anomaly detection networks.

A bio-inspired optical directional microphone with cavity-coupled diaphragms.

A bio-inspired acoustic sensor for sound source localization is presented, mimicking the internally coupled ears found in many terrestrial vertebrates and insects. It consists of two aluminum diaphragms coupled by a U-shaped cavity and detected by a low-coherence fiber optic interferometer system. A large-scale prototype with a center-to-center separation of 1″ is fabricated and experimentally demonstrated to amplify the interaural phase difference by a factor of 2 to 4 for a wide frequency range (0.5-2 kHz), which agrees well with simulation. This work presents a mechanism of using cavity-coupled diaphragms to develop acoustic sensors for sound source localization.

Event Encryption for Neuromorphic Vision Sensors: Framework, Algorithm, and Evaluation.

Nowadays, our lives have benefited from various vision-based applications, such as video surveillance, human identification and aided driving. Unauthorized access to the vision-related data greatly threatens users’ privacy, and many encryption schemes have been proposed to secure images and videos in those conventional scenarios. Neuromorphic vision sensor (NVS) is a brand new kind of bio-inspired sensor that can generate a stream of impulse-like events rather than synchronized image frames, which reduces the sensor’s latency and broadens the applications in surveillance and identification. However, the privacy issue related to NVS remains a significant challenge. For example, some image reconstruction and human identification approaches may expose privacy-related information from NVS events. This work is the first to investigate the privacy of NVS. We firstly analyze the possible security attacks to NVS, including grayscale image reconstruction and privacy-related classification. We then propose a dedicated encryption framework for NVS, which incorporates a 2D chaotic mapping to scramble the positions of events and flip their polarities. In addition, an updating score has been designed for controlling the frequency of execution, which supports efficient encryption on different platforms. Finally, extensive experiments have demonstrated that the proposed encryption framework can effectively protect NVS events against grayscale image reconstruction and human identification, and meanwhile, achieve high efficiency on various platforms including resource-constrained devices.

Deep Event Stereo Leveraged by Event-to-Image Translation

Depth estimation in real-world applications requires precise responses to fast motion and challenging lighting conditions. Event cameras use bio-inspired event-driven sensors that provide instantaneous and asynchronous information of pixel-level log intensity changes, which makes them suitable for depth estimation in such challenging conditions. However, as the event cameras primarily provide asynchronous and spatially sparse event data, it is hard to provide accurate dense disparity map in stereo event camera setups - especially in estimating disparities on local structures or edges. In this study, we develop a novel deep event stereo network that reconstructs spatial intensity image features from embedded event streams and leverages the event features using the reconstructed image features to compute dense disparity maps. To this end, we propose a novel event-to-image translation network with a cross-semantic attention mechanism that calculates the global semantic context of the event features for the intensity image reconstruction. In addition, a feature aggregation module is developed for accurate disparity estimation, which modulates the event features with the reconstructed image features by a stacked dilated spatially-adaptive denormalization mechanism. Experimental results reveal that our method can outperform the state-of-the-art methods by significant margins both in quantitative and qualitative measures.

Residual Interfacial Deformation in Flexible Copper Clad Laminate Occurring During Roll-to-Roll Composite Film Manufacturing

Flexible copper clad laminate has been used to fabricate flexible electronics such as bio-inspired tactile sensor, microfluidics-based sensor, and flexible printed circuits board. Herein, the effects of thermal property difference between the component layers on the residual interfacial deformation in a flexible copper clad laminate occurring during roll-to-roll composite film manufacturing were analyzed. The thermal behaviors of polyimide film and copper meshes in the laminate were examined, and it was found that differences of the coefficient of thermal expansion and elastic modulus between the two components could cause the residual deformation in the laminate. The three factors affecting the residual strain due to the thermal characteristic difference, i.e., temperature and pressure of the lamination roll and web tension, were selected in the controllable operating conditions in a roll-to-roll lamination process. Mathematical models to estimate the elastic and thermal residual strains according to the factors were developed and experimentally verified. Finally, the effects of the three factors on the residual strain in the laminate were determined using the developed model. In this study, we demonstrate that, in the lamination process, the thermal strain of the laminate component materials after thermal cycling generates a significant residual deformation, which causes defects in the flexible copper clad laminate. The developed model can be used to accurately estimate the residual elastic and thermal residual strains occurring during the roll-to-roll manufacturing according to the process conditions.

Bio-inspired butterfly core-shaped photonic crystal fiber-based refractive index sensor

Light controllability, design flexibility, and non-linearity features of photonic crystal fiber (PCF) based surface plasmon resonance (SPR) sensor enable high sensitivity in the field of biosensing. Here, bio-inspired butterfly-core shaped microstructure fiber-based plasmonic sensor is proposed where circular air-holes are arranged to enhance the sensing performance. Butterfly shaped core is designed to confine the incident light into the core by preventing light scattering through the cladding and helps to excite surface electron of plasmonic metal layer. Chemically stable plasmonic material gold is used to produce the SPR phenomenon. The analyte detection layer and the plasmon layer are located externally on the PCF surface to make the detection process more feasible. The sensor performance is studied based on the finite element method (FEM), and the structural parameters are tuned to obtain maximum sensor performance. This modified core-based sensor exhibits the maximum wavelength sensitivity (WS) of 56,000 nm/RIU and the amplitude sensitivity (AS) of 1,584 RIU-1 for the x-polarized mode. It also shows an improved sensor resolution (SR) of 1.8 ×10−6 RIU, along with a decent figure of merit (FOM) of 691 RIU-1. Moreover, this sensor can detect analyte refractive indexes (RI) within a broad RI range of 1.33 to 1.42 in the visible to near-infrared wavelength range (450–2100 nm). Finally, the proposed sensor may have possible application to detect organic chemicals, food quality, and diseases with high accuracy due to outstanding sensitivity and linearity.

Bioinspired 3D Printable, Self-Healable, and Stretchable Hydrogels with Multiple Conductivities for Skin-like Wearable Strain Sensors

Bioinspired hydrogels have promising prospects in applications such as wearable devices, human health monitoring equipment, and soft robots due to their multifunctional sensing properties resembling natural skin. However, the preparation of intelligent hydrogels that provide feedback on multiple electronic signals simultaneously, such as human skin receptors, when stimulated by external contact pressure remains a substantial challenge. In this study, we designed a bioinspired hydrogel with multiple conductive capabilities by incorporating carbon nanotubes into a chelate of calcium ions with polyacrylic acid and sodium alginate. The bioinspired hydrogel consolidates self-healing ability, stretchability, 3D printability, and multiple conductivities. It can be fabricated as an integrated strain sensor with simultaneous piezoresistive and piezocapacitive performances, exhibiting sensitive (gauge factor of 6.29 in resistance mode and 1.25 kPa-1 in capacitance mode) responses to subtle pressure changes in the human body, such as finger flexion, knee flexion, and respiration. Furthermore, the bioinspired strain sensor sensitively and discriminatively recognizes the signatures written on it. Hence, we expect our ideas to provide inspiration for studies exploring the use of advanced hydrogels in multifunctional skin-like smart wearable devices.

Low-Latency and Scene-Robust Optical Flow Stream and Angular Velocity Estimation

Event cameras are bio-inspired sensors that capture intensity changes of pixels individually, and generate asynchronous and independent “events”. Due to the fundamental difference from the conventional cameras, most research on event cameras builds a global event frame by grouping events according to their timestamps or their number to employ traditional computer vision algorithms. However, in order to take advantage of event cameras, it makes sense to generate asynchronous output on an event-by-event basis. In this paper, we propose an optical flow estimation algorithm with low latency and robustness to various scenes to utilize the advantage of the event camera by enhancing the existing optical flow algorithm. Furthermore, we estimate angular velocity with low latency using the proposed optical flow stream. For the validation of algorithms, we evaluate the accuracy and latency of optical flow with publicly available datasets. Moreover, we assess the performance of the proposed angular velocity estimation in comparison to the existing algorithms. Both validations suggest that our asynchronous optical flow shows comparable accuracy to the existing algorithms and the latency is reduced by half compared to the existing block matching algorithm on average. Also, our angular velocity estimation is superior to the existing algorithms in terms of accuracy and robustness while showing low latency within 15 ms consistently.

Globally-Optimal Contrast Maximisation for Event Cameras.

Event cameras are bio-inspired sensors that perform well in challenging illumination conditions and have high temporal resolution. However, their concept is fundamentally different from traditional frame-based cameras. The pixels of an event camera operate independently and asynchronously. They measure changes of the logarithmic brightness and return them in the highly discretised form of time-stamped events indicating a relative change of a certain quantity since the last event. New models and algorithms are needed to process this kind of measurements. The present work looks at several motion estimation problems with event cameras. The flow of the events is modelled by a general homographic warping in a space-time volume, and the objective is formulated as a maximisation of contrast within the image of warped events. Our core contribution consists of deriving globally optimal solutions to these generally non-convex problems, which removes the dependency on a good initial guess plaguing existing methods. Our methods rely on branch-and-bound optimisation and employ novel and efficient, recursive upper and lower bounds derived for six different contrast estimation functions. The practical validity of our approach is demonstrated by a successful application to three different event camera motion estimation problems.

A Triboelectric-Based Artificial Whisker for Reactive Obstacle Avoidance and Local Mapping.

Since designing efficient tactile sensors for autonomous robots is still a challenge, this paper proposes a perceptual system based on a bioinspired triboelectric whisker sensor (TWS) that is aimed at reactive obstacle avoidance and local mapping in unknown environments. The proposed TWS is based on a triboelectric nanogenerator (TENG) and mimics the structure of rat whisker follicles. It operates to generate an output voltage via triboelectrification and electrostatic induction between the PTFE pellet and copper films (0.3 mm thickness), where a forced whisker shaft displaces a PTFE pellet (10 mm diameter). With the help of a biologically inspired structural design, the artificial whisker sensor can sense the contact position and approximate the external stimulation area, particularly in a dark environment. To highlight this sensor's applicability and scalability, we demonstrate different functions, such as controlling LED lights, reactive obstacle avoidance, and local mapping of autonomous surface vehicles. The results show that the proposed TWS can be used as a tactile sensor for reactive obstacle avoidance and local mapping in robotics.

Bioinspired, Superhydrophobic, and Paper-Based Strain Sensors for Wearable and Underwater Applications.

There is currently a growing demand for flexible strain sensors with high performance and water repellency for various applications such as human motion monitoring, sweat or humidity detection, and certain underwater tests. Among these strain sensors, paper-based ones have attracted increasing attention because they coincide with the future development trend of environment-friendly electronic products. However, paper-based electronics are easy to fail when they encounter water and are thus unable to be applied to humid or underwater circumstances. Herein, based on a strategy of coupling bionics inspired by lotus leaf and scorpion, which exhibit superhydrophobic characteristics and ultrasensitive vibration-sensing capacity, respectively, a paper-based strain sensor with high sensitivity and water repellency is successfully fabricated. As a result, the strain sensor exhibits a gauge factor of 263.34, a high strain resolution (0.098%), a fast response time (78 ms), excellent stability over 12,000 cycles, and a water contact angle of 164°. Owing to the bioinspired structures and function mechanisms, the paper-based strain sensor is suitable to not only serve as regular wearable electronics to monitor human motions in real-time but also to detect subtle underwater vibrations, demonstrating its great potential for numerous applications like wearable electronics, water environmental protection, and underwater robots.

Materials, Actuators, and Sensors for Soft Bioinspired Robots.

Biological systems can perform complex tasks with high compliance levels. This makes them a great source of inspiration for soft robotics. Indeed, the union of these fields has brought about bioinspired soft robotics, with hundreds of publications on novel research each year. This review aims to survey fundamental advances in bioinspired soft actuators and sensors with a focus on the progress between 2017 and 2020, providing a primer for the materials used in theirdesign.

Face Pose Alignment with Event Cameras.

Event camera (EC) emerges as a bio-inspired sensor which can be an alternative or complementary vision modality with the benefits of energy efficiency, high dynamic range, and high temporal resolution coupled with activity dependent sparse sensing. In this study we investigate with ECs the problem of face pose alignment, which is an essential pre-processing stage for facial processing pipelines. EC-based alignment can unlock all these benefits in facial applications, especially where motion and dynamics carry the most relevant information due to the temporal change event sensing. We specifically aim at efficient processing by developing a coarse alignment method to handle large pose variations in facial applications. For this purpose, we have prepared by multiple human annotations a dataset of extreme head rotations with varying motion intensity. We propose a motion detection based alignment approach in order to generate activity dependent pose-events that prevents unnecessary computations in the absence of pose change. The alignment is realized by cascaded regression of extremely randomized trees. Since EC sensors perform temporal differentiation, we characterize the performance of the alignment in terms of different levels of head movement speeds and face localization uncertainty ranges as well as face resolution and predictor complexity. Our method obtained 2.7% alignment failure on average, whereas annotator disagreement was 1%. The promising coarse alignment performance on EC sensor data together with a comprehensive analysis demonstrate the potential of ECs in facial applications.