Multimodal Neuromorphic Sensory-Processing System With Memristor Circuits for Smart Home Applications

Abstract

To enable smart home applications, embedding home monitoring systems with different sensors can be a feasible remedy to capture the multimodal sensory information from daily life. However, the previously developed monitoring systems have high implementation cost, large power consumption, and complicated device configuration, which are not conducive to achieving carbon neutrality and carbon peak goals. Here, we demonstrate a multimodal neuromorphic sensory-processing system with memristor circuits for smart homes, offering a more environmentally friendly approach with low cost and easily deployable hardware. We used three components to facilitate the multimodal neuromorphic sensory-processing system design. First, a memristor crossbar array is fabricated using low-cost, reliable, and eco-friendly 2-D materials; enabling a parallel connection to reduce the system complexity and the deep-learning computational cost. Second, a neuromorphic multimodal sensory-processing module is designed for collecting and processing multiple sensory cues efficiently; to establish accurate depiction of the environment and realize timely response. Third, the biomimetic hierarchical learning module is realized for smart home monitoring and relative applications (e.g., indoor human behavior recognition). The developed multimodal neuromorphic sensory-processing system effectively combines the nanotechnology, intelligent sensing, and deep learning; indicating an advancement in smart home applications.