Intelligent Perception System Research Articles

Robotic Assembly – Mobile Platform for Construction Field

"Construction sites often involve the manual transportation of heavy objects, posing risks to workers' safety and efficiency. In this paper, it was proposed a design, development, and evaluation of a service robot specifically created for carrying heavy objects in the construction field. The robot is equipped with robust locomotion capabilities, intelligent perception systems, and efficient control mechanisms to navigate through complex environments and handle various types of loads. It was presented the technical design of the robot, including its hardware components, sensing modalities, and software architecture. Furthermore, it was discussed the experimental validation of the robot's performance in real-world construction scenarios, highlighting its effectiveness in improving productivity, reducing labor costs, and enhancing workplace safety. Through this research, the aim is to demonstrate the potential of service robots as valuable assets in the construction industry, paving the way for future advancements in robotic assistance technologies. Keywords: human-robot collaboration, manipulation, autonomous systems, object transportation, mechanics."

All-in-One 2D Molecular Crystal Optoelectronic Synapse for Polarization-Sensitive Neuromorphic Visual System.

Neuromorphic visual systems (NVSs) hold the potential to not only preserve but also enhance human visual capabilities. One such augmentation lies in harnessing polarization information from light reflected or scattered off surfaces like bees, which can disclose unique characteristics imperceptible to the human eyes. While creating polarization-sensitive optoelectronic synapses presents an intriguing avenue for equipping NVS with this capability, integrating functions like polarization sensitivity, photodetection, and synaptic operations into a singular device has proven challenging. This integration typically necessitates distinct functional components for each performance metric, leading to intricate fabrication processes and constraining overall performance. Herein, a pioneering linear polarized light sensitive synaptic organic phototransistor (OPT) based on 2D molecular crystals (2DMCs) with highly integrated, all-in-one functionality, is demonstrated. By leveraging the superior crystallinity and molecular thinness of 2DMC, the synaptic OPT exhibits comprehensive superior performance, including a linear dichroic ratio up to 3.85, a high responsivity of 1.47 × 104 A W-1, and the adept emulation of biological synapse functions. A sophisticated application in noncontact fingerprint detection achieves a 99.8% recognition accuracy, further highlights its potential. The all-in-one 2DMC optoelectronic synapse for polarization-sensitive NVS marks a new era for intelligent perception systems.

Skin-inspired multimodal tactile sensor aiming at smart space extravehicular multi-finger operations

Tactile sensing of skin shapes the interactions between hand and the surrounding world, owing to the remarkable natural sensory system. But for astronauts, tactile feedback cannot be obtained biologically due to the very thick protective gloves, which seriously hinders the flexibility of the extravehicular activity. In this work, inspired by human skin, we develop a biomimetic multi-layer tactile sensor (BMLTS) that can work like the fast adaptive and slow adaptive receptors of fingertip to achieve the multimodal tactile perception based on the fusion of thermosensitive, piezoelectric, triboelectric and piezoresistive materials. Based on the BMLTS, the tactile perception for temperature, surface roughness discrimination and smart object grasping is successfully achieved, which fully simulates the basic tactile perceptions in typical extravehicular operations. Furthermore, by the combination of BMLTS and deep learning, a biomimetic intelligent perception system (BIPS) that can make decisions to the tactile signal just like human brain is constructed, which can provide real-time tactile information for astronauts. Intelligent real-time object material identification, writing and recording are conducted using BIPS, reaching the recognition accuracy rate higher than 95%. This work lays the foundation for the systematization and integration of tactile sensors, and paves the way for the development of dexterous tactile perception for smart extravehicular activities of astronauts.

Remote intelligent perception system for multi-object detection.

During the last few years, a heightened interest has been shown in classifying scene images depicting diverse robotic environments. The surge in interest can be attributed to significant improvements in visual sensor technology, which has enhanced image analysis capabilities. Advances in vision technology have a major impact on the areas of multiple object detection and scene understanding. These tasks are an integral part of a variety of technologies, including integrating scenes in augmented reality, facilitating robot navigation, enabling autonomous driving systems, and improving applications in tourist information. Despite significant strides in visual interpretation, numerous challenges persist, encompassing semantic understanding, occlusion, orientation, insufficient availability of labeled data, uneven illumination including shadows and lighting, variation in direction, and object size and changing background. To overcome these challenges, we proposed an innovative scene recognition framework, which proved to be highly effective and yielded remarkable results. First, we perform preprocessing using kernel convolution on scene data. Second, we perform semantic segmentation using UNet segmentation. Then, we extract features from these segmented data using discrete wavelet transform (DWT), Sobel and Laplacian, and textual (local binary pattern analysis). To recognize the object, we have used deep belief network and then find the object-to-object relation. Finally, AlexNet is used to assign the relevant labels to the scene based on recognized objects in the image. The performance of the proposed system was validated using three standard datasets: PASCALVOC-12, Cityscapes, and Caltech 101. The accuracy attained on the PASCALVOC-12 dataset exceeds 96% while achieving a rate of 95.90% on the Cityscapes dataset. Furthermore, the model demonstrates a commendable accuracy of 92.2% on the Caltech 101 dataset. This model showcases noteworthy advancements beyond the capabilities of current models.

Evaluating the Feasibility of Intelligent Blind Road Junction V2I Deployments

Cellular Vehicle-to-Everything (C-V2X) communications is a technology that enables intelligent vehicles to exchange information and thus coordinate with other vehicles, road users, and infrastructure. However, despite advancements in cellular technology for V2X applications, significant challenges remain regarding the ability of the system to meet stringent Quality-of-Service (QoS) requirements when deployed at scale. Thus, smaller-scale V2X use case deployments may embody a necessary stepping stone to address these challenges. This work assesses network architectures for an Intelligent Perception System (IPS) blind road junction or blind corner scenarios. Measurements were collected using a private 5G NR network with Sub-6GHz and mmWave connectivity, evaluating the feasibility and trade-offs of IPS network configurations. The results demonstrate the feasibility of the IPS as a V2X application, with implementation considerations based on deployment and maintenance costs. If computation resources are co-located with the sensors, sufficient performance is achieved. However, if the computational burden is instead placed upon the intelligent vehicle, it is questionable as to whether an IPS is achievable or not. Much depends on image quality, latency, and system performance requirements.

Intelligent Analysis of E-commerce Reviews based on Complex-Valued Spatio-temporal Graph Convolutional Neural Network Optimized Tyrannosaurus Optimization Algorithm

Intelligent analysis of E-commerce reviews based on multimodal social media big data using applies the benefit of innovative artificial intelligence method, especially deep learning algorithms, to study and realize web-based product consider and social network data connected to e-commerce. In this manuscript, Intelligent Analysis of E-commerce reviews based on Complex-Valued Spatio-temporal Graph Convolutional Neural Network Optimized Tyrannosaurus Optimization Algorithm (IAER-CVSGCNN-TOA)is proposed for Intelligent Analysis of E-commerce Reviews. The input data are collected from E-commerce big Dataset. Then data are pre-processing utilizing federated neural collaborative filtering (FNCF) to remove the noise and clean the data. The pre-processed data is provided to the CVSGCNN is used for intelligent analysis of e-commerce reviews. In general, CVSGCNN does no express adapting optimization approaches to determine optimal parameters to ensure accurate prediction. Hence, proposed to utilize the Tyrannosaurus Optimization Algorithm enhancement CVSGCNN for intelligent analysis of e-commerce reviews. The proposed IAER-CVSGCNN-TOA method is implemented on python. Then performance of proposed technique is analyzed with other existing techniques. The proposed technique attains 30.26%, 28.70% and 46.25% higher accuracy, 20.53%, 32.60%, and 53.43% higher precision, 25.23%, 23.15%, and 22.33%higher recall, 25.75%, 44.30% and 53.70%higher specificity comparing with the existing methods such as a Retracted: Online Troll Reviewer Detection Utilizing Deep Learning Methods (IAER-CNN), Developing an Intelligent System and DL Algorithms for Sentiment Analysis of E-Commerce Product Reviews(IAER-LSTM), Intelligent Perception System of Big Data Decision in Cross-Border e-Commerce Depend on Data Fusion(IAER-DFN) respectively.

Recent Progress in Wearable Near-Sensor and In-Sensor Intelligent Perception Systems.

As the Internet of Things (IoT) becomes more widespread, wearable smart systems will begin to be used in a variety of applications in people's daily lives, not only requiring the devices to have excellent flexibility and biocompatibility, but also taking into account redundant data and communication delays due to the use of a large number of sensors. Fortunately, the emerging paradigms of near-sensor and in-sensor computing, together with the proposal of flexible neuromorphic devices, provides a viable solution for the application of intelligent low-power wearable devices. Therefore, wearable smart systems based on new computing paradigms are of great research value. This review discusses the research status of a flexible five-sense sensing system based on near-sensor and in-sensor architectures, considering material design, structural design and circuit design. Furthermore, we summarize challenging problems that need to be solved and provide an outlook on the potential applications of intelligent wearable devices.

Research on intelligent perception and decision system for special welding mechanical arm of float plate in crude oil storage tank

Abstract In the rapid development of the petrochemical industry in the context of the increasing number of large tanks, large tanks in the production and installation process encountered the first problem the pontoon plate welding deformation and weld seam is too large and other defects, through the construction of the ship plate welding special robotic arm of the intelligent perception and decision-making system to solve such defects. Summarize the Structure of crude oil storage tanks and welding process characteristics based on the ultrasonic distance measurement in the distance sensor to obtain the characteristic data set, combined with the distinct data and computer programming language to build the intelligent perception and decision-making system of the unique robotic arm for ship plate welding. They set the relevant parameter values based on simulation experiments on the welding of a particular robot arm perception and decision-making system for the application of example analysis. Compared to the traditional LSTM algorithm, the YOLO-v5 network-driven weld target detection algorithm has a 5.5% higher accuracy and a 9.1% higher success rate in time robustness evaluation. In addition, the welding decision model based on the DDPG algorithm possesses excellent computational accuracy, and the average error of each parameter in the welding decision of the ship plate is controlled within 5%. This study greatly improves the welding quality of the pontoon plate of crude oil storage tanks and provides solid technical support for the successful completion of the project.

Research on the Three-Machines Perception System and Information Fusion Technology for Intelligent Work Faces.

The foundation of intelligent collaborative control of a shearer, scraper conveyor, and hydraulic support (three-machines) is to achieve the precise perception of the status of the three-machines and the full integration of information between the equipment. In order to solve the problems of information isolation and non-flow, independence between equipment, and weak cooperation of three-machines due to an insufficient fusion of perception data, a fusion method of the equipment's state perception system on the intelligent working surface was proposed. Firstly, an intelligent perception system for the state of the three-machines in the working face was established based on fiber optic sensing technology and inertial navigation technology. Then, the datum coordinate system is created on the working surface to uniformly describe the status of the three-machines and the spatial position relationship between the three-machines is established using a scraper conveyor as a bridge so that the three-machines become a mutually restricted and collaborative equipment system. Finally, an indoor test was carried out to verify the relational model of the spatial position of the three-machines. The results indicate that the intelligent working face three-machines perception system based on fiber optic sensing technology and inertial navigation technology can achieve the fusion of monitoring data and unified expression of equipment status. The research results provide an important reference for building an intelligent perception, intelligent decision-making, and automatic execution system for coal mines.

Research on Emotion Recognition Method of Flight Training Based on Multimodal Fusion

The emotional activities of the human body are mainly regulated by the autonomic nervous system, the central nervous system, and the advanced cognition of the human brain. This paper proposes an emotional state recognition method in pilot training tasks based on multimodal information fusion. A set of emotion perception recognition systems, a two-dimensional valence-arousal emotional model, and a multimodal information intelligent perception model were established based on the human-computer interaction mode during flight training; an intelligent perception system was designed to collect and intelligently perceive four kinds of peripheral physiological signals of pilots in real-time. And based on traditional machine learning models, a binary tree support vector machine was designed to optimize and improve the multimodal information co-integration decision model, which increased the accuracy of emotional state recognition in flight training by 37.58% on average. The experimental result showed that it realizes accurate monitoring and identification of real-time emotional state, helps to improve the effect of flight training and flight safety, maintains the efficiency of operation, and has important research significance and application prospects in the field of pilot training.

Advances in flexible sensors for intelligent perception system enhanced by artificial intelligence

AbstractIntelligent perception means that with the assistance of artificial intelligence (AI)‐motivated brain, flexible sensors achieve the ability of memory, learning, judgment, and reasoning about external information like the human brain. Due to the superiority of machine learning (ML) algorithms in data processing and intelligent recognition, intelligent perception systems possess the ability to match or even surpass human perception systems. However, the built‐in flexible sensors in these systems need to work on dynamic and irregular surfaces, inevitably affecting the precision and fidelity of the acquired data. In recent years, the strategy of introducing the developed functional materials and innovative structures into flexible sensors has made some progress toward the above challenges, and with the blessing of ML algorithms, accurate perception and reasoning in various scenarios have been achieved. Here, the most representative functional materials and innovative structures for constructing flexible sensors are comprehensively reviewed, the research progress of intelligent perception systems based on flexible sensors and ML algorithms is further summarized, and the intersection of the two is expected to unlock new opportunities for next‐stage AI development.image

Design and Simulation Verification of Intelligent Perception and Adaptive Control System for Control Rod Drive Mechanism

Design and Simulation Verification of Intelligent Perception and Adaptive Control System for Control Rod Drive Mechanism

Preface

The 2022 3rd International Conference on Internet of Things, Artificial Intelligence and Mechanical Automation (IoTAIMA 2022) was held virtually on July 22-24, 2022 in Shanghai, China. The conference provided a communication forum for the international experts and researchers in the fields of Internet of Things, Artificial Intelligence and Mechanical Automation to share research achievements, explore solutions to research challenges, encourage international collaborations, and boost development of science and technology.In the conference, we were greatly honored to have Prof. Ke Zhang from Shanghai Institute of Technology, China to serve as our Conference Chair. 120 individuals around the world took part in the conference. Divided into three parts, the conference agenda included keynote speeches, oral presentations and online Q&A discussion. Primarily, the keynote speakers were each allocated 30-45 minutes to address their speeches. Then in the next part, oral presentations, the excellent papers we had selected were presented by their authors one by one.Four distinguished professors were invited to hold keynote speeches during the conference. Among them, Prof. Xiaodong Liu from Dalian University of Technology performed a thought-provoking speech on Knowledge Discovery and Semantic Learning in the Framework of AFS Theory. Under the background that AFS theory gains increasing interest from various research fields, this survey aimed to report the current developments of theoretical research and practical advances on AFS theory. Firstly some notions and foundations of AFS theory with an illustrative example were reviewed and then various extensions of AFS theory for knowledge discovery were analyzed. And then Prof. Yudong Zhang from University of Leicester addressed a speech: Artificial Intelligence Theories and Methods for COVID-19 Recognition. In this research, CT scan, a medical imaging technique used in radiology to get detailed images of the body noninvasively for diagnostic purposes, was applied to discuss the recent AI theories and methods in chest CT-based COVID-19 diagnosis. Two other chest-related diseases: secondary pulmonary tuberculosis and community-acquired pneumonia, were covered in the study. Their insightful speeches had triggered heated discussion. And every participant praised this conference for disseminating useful and insightful knowledge.We are glad to share with you that we received lots of submissions from the conference and we selected a bunch of high-quality papers and compiled them into the proceedings after rigorously reviewing them. These papers feature but are not limited to the following topics: Radio Frequency Identification, New Sensing Technology, Intelligent Perception Systems, Unmanned System Control Technology, Automation etc. All the papers have been checked through rigorous review and processes to meet the requirements of publication.We would like to acknowledge all of those who supported IoTAIMA 2022 and made it a great success. Particularly, we would like to thank the Journal of Physics: Conference Series, for all the efforts of its colleagues in publishing this paper volume. We sincerely hope that the IoTAIMA 2022 turned out to be a forum for excellent discussions that enable new ideas to come about, promoting collaborative research.Committee of IoTAIMA 2022List of Committee member is available in this pdf.

A Review of Art and Real World Applications of Intelligent Perception Systems

Sensory data and AI/ML techniques are crucial to several robotics applications, which is why perception in robots is a hot topic. Some of these applications include: object recognition, scene understanding, environment representation, activity identification, semantic location classification, object modeling, and pedestrian/human detection. Robotic perception, as used in this article, is the collection of machine learning (ML) techniques and methods that allow robots to process sensory data and form conclusions and perform actions accordingly. It is clear that recent development in the field of ML, mostly deep learning methodologies, have led to improvements in robotic perception systems, which in turn make it possible to realize applications and activities that were previously unimaginable. These recent advancements in complex robotic tasks, human-robot interaction, decision-making, and intelligent thought are in part due to the fast development and widespread usage of ML algorithms. This article provides a survey of real-world and state of the art applications of intelligent perception systems in robots.

Monocular 3D Object Detection With Sequential Feature Association and Depth Hint Augmentation

Monocular 3D object detection, with the aim of predicting the geometric properties of on-road objects, is a promising research topic for the intelligent perception systems of autonomous driving. Most state-of-the-art methods follow a keypoint-based paradigm, where the keypoints of objects are predicted and employed as the basis for regressing the other geometric properties. In this work, a unified network named as FADNet is presented to address the task of monocular 3D object detection. In contrast to previous keypoint-based methods, we propose to divide the output modalities into different groups according to the estimation difficulty of object properties. Different groups are treated differently and sequentially associated by a convolutional Gated Recurrent Unit. Another contribution of this work is the strategy of depth hint augmentation. To provide characterized depth patterns as hints for depth estimation, a dedicated depth hint module is designed to generate row-wise features named as depth hints, which are explicitly supervised in a bin-wise manner. The contributions of this work are validated by conducting experiments and ablation study on the KITTI benchmark. Without utilizing depth priors, post optimization, or other refinement modules, our network performs competitively against state-of-the-art methods while maintaining a decent running speed.

A Training-Free Multipath Enhancement (TFME-RTI) Method for Device-Free Multi-Target Localization

Many intelligent perception systems are progressively approaching people’s daily lives. Device-free localization (DFL), as a technology that can achieve positioning without the target being equipped with any equipment, has broad development prospects. RFID-based radio tomographic imaging (RTI) method has attracted much attention because of its low cost, easy deployment, and strong real-time performance. However, RTI, as a real-time imaging method that uses the line-of-sight path information and analyzes the received signal strength behavior, cannot effectively resist multipath interference. As the number of targets increases, the positioning performance using RTI method will be hit. In response to this problem, this paper proves the feasibility of applying static reflection multipath to device-free localization, and proposes a new training-free multipath enhancement (TFME-RTI) method using known spatial information. By analyzing the static multipath superposition model and target interference characteristics, a transformation model is established to take advantage of the multipath effect to enhance the positioning performance. Experiments demonstrate that the TFME-RTI method has better positioning performance and robustness in two typical indoor scenarios.

Artificial Multisensory Neurons with Fused Haptic and Temperature Perception for Multimodal In‐Sensor Computing

The human receives and transmits various information from the outside world through different sensory systems. The sensory neurons integrate various sensory inputs into a synthetical perception to monitor complex environments, and this fundamentally determines the way how we perceive the world. Developing multifunctional artificial sensory elements that can integrate multisensory perception plays a vital role in future intelligent perception systems, whereas prior spiking neurons reported can only handle single‐mode physical signals. Herein, a bioinspired haptic‐temperature fusion spiking neuron based upon a serial connection of piezoresistive sensor and VO2 volatile memristor is presented. The artificial sensory neuron is capable of detecting and encoding pressure and temperature inputs based on the voltage dividing effect and the intrinsic thermal sensitivity of metal–insulator transition in VO2. Recognition of Braille characters is achieved through multiple piezoresistive sensors, taking advantage of the spatial integration capabilities of such spiking neurons. Notably, the traditionally separate haptic and temperature signals can be fused physically in the sensory neuron when synchronizing the two sensory cues, which is able to recognize multimodal haptic/temperature patterns. The artificial multisensory neuron thus provides a promising approach toward e‐skin, neurorobotics, and human–machine interaction technologies. A preprint version of the article can be found at: https://www.authorea.com/doi/full/10.22541/au.164668806.60849882.

Research on Efficacy Evaluation of Large-Scale Networked Intelligent Perception System

Under the smart engineering system (SES), there is a huge demand for evaluating the efficacy of a large-scale networked intelligent perception system (IPS). Considering the large-scale, distributed, and networked system characteristics and perception task demands, this paper proposes a conceptual system for IPS efficacy evaluation and, on this basis, designs the architecture of the efficacy evaluation system. A networked IPS model is constructed based on domain ontology, an index system is quickly established for efficacy evaluation, the evaluation methods are assembled automatically, and adaptive real-time organization strategies are generated for networked perception based on efficacy estimate. After exploring these key technologies, a prototype system is created for the service-oriented integrated efficacy evaluation platform and used to verify and integrate research results. The research provides support for the efficacy evaluation theories and methods of large-scale networked IPS.

3D shape sensing and deep learning-based segmentation of strawberries

Automation and robotisation of the agricultural sector are seen as a viable solution to socio-economic challenges faced by this industry. This technology often relies on intelligent perception systems providing information about crops, plants and the entire environment. The challenges faced by traditional 2D vision systems can be addressed by modern 3D vision systems which enable straightforward localisation of objects, size and shape estimation, or handling of occlusions. So far, the use of 3D sensing was mainly limited to indoor or structured environments. In this paper, we evaluate modern sensing technologies including stereo and time-of-flight cameras for 3D perception of shape in agriculture and study their usability for segmenting out soft fruit from background based on their shape. To that end, we propose a novel 3D deep neural network which exploits the organised nature of information originating from the camera-based 3D sensors. We demonstrate the superior performance and efficiency of the proposed architecture compared to the state-of-the-art 3D networks. Through a simulated study, we also show the potential of the 3D sensing paradigm for object segmentation in agriculture and provide insights and analysis of what shape quality is needed and expected for further analysis of crops. The results of this work should encourage researchers and companies to develop more accurate and robust 3D sensing technologies to assure their wider adoption in practical agricultural applications.

Intelligent Perception System of Big Data Decision in Cross‐Border e‐Commerce Based on Data Fusion

There is a gap between supply and demand in cross‐border e‐trading platforms. The so‐called supply and demand gap refers to the existing good companies that cannot meet the needs of buyers. Cross‐border e‐commerce, the buyer’s purchase demand is often included in the buyer’s behavior, such as searching for goods by pressing buttons, clicking price, category and other means, and on‐site delivery time. It is important for CBEC to analyze the buyer’s needs to protect the procurement and provide the seller with reference to supply, so as to solve the supply and demand gap between the buyer and the seller. This paper mainly studies the CBEC big data decision intelligent perception system based on data fusion. It is the same as the innovation process of better application of data fusion to the e‐commerce business model. At the same time, the key technologies of data extraction, conversion, data warehouse, and front‐end display in the system construction are analyzed and designed. This paper mainly uses data fusion algorithm, data fusion network model, quaternion method, big data decision intelligent sensing system framework design experiment, and CBEC user experiment to study the CBEC big data decision intelligent sensing system based on data fusion, and also with the development of CBEC, I hope more people can participate in CBEC practice. The results show that with the increase of data volume of the CBEC platform from 2016 to 2020, the construction of the CBEC platform business of big data production intelligent perception system considers the situation of users from seven aspects. With the increase of consumption, domestic consumers are increasingly pursuing a better life, cross‐border import e‐commerce ushered in a great development era.