Real-time Recognition Research Articles

Improving Data Augmentation for YOLOv5 Using Enhanced Segment Anything Model

As one of the state-of-the-art object detection algorithms, YOLOv5 relies heavily on the quality of the training dataset. In order to improve the detection accuracy and performance of YOLOv5 and to reduce its false positive and false negative rates, we propose to improve the Segment Anything Model (SAM) used for data augmentation. The feature maps and mask predictions generated by the SAM are used as auxiliary inputs for the Mask-to-Mask (M2M) module. The experimental results show that after processing the dataset with the improved Segment Anything Model, the detection performance of YOLOv5 is improved with 99.9% precision and 99.1% recall. The improved YOLOv5 model has a higher license plate recognition accuracy than the original detection model under strong snowfall conditions, and the incidence of false-negative and false-positive is greatly reduced. The enhanced model can meet the requirement of accurate real-time recognition of license plates under strong snowfall weather conditions.

Thin-film transistor for temporal self-adaptive reservoir computing with closed-loop architecture.

Reservoir computing is a powerful neural network-based computing paradigm for spatiotemporal signal processing. Recently, physical reservoirs have been explored based on various electronic devices with outstanding efficiency. However, the inflexible temporal dynamics of these reservoirs have posed fundamental restrictions in processing spatiotemporal signals with various timescales. Here, we fabricated thin-film transistors with controllable temporal dynamics, which can be easily tuned with electrical operation signals and showed excellent cycle-to-cycle uniformity. Based on this, we constructed a temporal adaptive reservoir capable of extracting temporal information of multiple timescales, thereby achieving improved accuracy in the human-activity-recognition task. Moreover, by leveraging the former computing output to modify the hyperparameters, we constructed a closed-loop architecture that equips the reservoir computing system with temporal self-adaptability according to the current input. The adaptability is demonstrated by accurate real-time recognition of objects moving at diverse speed levels. This work provides an approach for reservoir computing systems to achieve real-time processing of spatiotemporal signals with compound temporal characteristics.

Real-time, noise and drift resilient formaldehyde sensing at room temperature with aerogel filaments

Formaldehyde, a known human carcinogen, is a common indoor air pollutant. However, its real-time and selective recognition from interfering gases remains challenging, especially for low-power sensors suffering from noise and baseline drift. We report a fully 3D-printed quantum dot/graphene-based aerogel sensor for highly sensitive and real-time recognition of formaldehyde at room temperature. By optimizing the morphology and doping of printed structures, we achieve a record-high and stable response of 15.23% for 1 part per million formaldehyde and an ultralow detection limit of 8.02 parts per billion consuming only ∼130-microwatt power. On the basis of measured dynamic response snapshots, we also develop intelligent computational algorithms for robust and accurate detection in real time despite simulated substantial noise and baseline drift, hitherto unachievable for room temperature sensors. Our framework in combining materials engineering, structural design, and computational algorithm to capture dynamic response offers unprecedented real-time identification capabilities of formaldehyde and other volatile organic compounds at room temperature.

Research and Development of a Traffic Sign Recognition Module in Vietnam

Automatic traffic sign recognition is essential in researching and developing driver assistance systems and autonomous vehicles. This paper presents the research and development of an automated traffic sign recognition module in Vietnam. The recognition model is developed based on the deep learning model YOLOv5 and incorporates architectural modifications to reduce computational complexity, increase inference speed, and meet real-time requirements for embedded system applications. The model is trained using a custom dataset collected by the research team from real-world street environments in Vietnam, encompassing diverse locations, times, and weather conditions. The trained recognition model is deployed on the Jetson embedded system, yielding high-quality recognition results and meeting real-time recognition needs.

An Adam based CNN and LSTM approach for sign language recognition in real time for deaf people

Hand gestures and sign language are crucial modes of communication for deaf individuals. Since most people can't understand sign language, it's hard for a mute and an average person to talk to each other. Because of technological progress, computer vision and deep learning can now be used to count. This paper shows two ways to use deep knowledge to recognize sign language. These methods help regular people understand sign language and improve their communication. Based on American sign language (ASL), two separate datasets have been constructed; the first has 26 signs, and the other contains three significant symbols with the crucial sequence of frames or videos for regular communication. This study looks at three different models: the improved ResNet-based convolutional neural network (CNN), the long short-term memory (LSTM), and the gated recurrent unit (GRU). The first dataset is used to fit and assess the CNN model. With the adaptive moment estimation (Adam) optimizer, CNN obtains an accuracy of 89.07%. In contrast, the second dataset is given to LSTM and GRU and a comparison has been conducted. LSTM does better than GRU in all classes. LSTM has a 94.3% accuracy, while GRU only manages 79.3%. Our preliminary models' real-time performance is also highlighted.

Real-time recognition of Indonesian sign language using recurrent neural network

Hand gestures serve as a vital means of communication for deaf individuals. They often face communication challenges in their daily interactions due to the language barrier. This underscores the necessity of sign language interpreters. However, prevailing methods primarily rely on the Indonesian Sign Language System (SIBI), despite the widespread use of Indonesian Sign Language (BISINDO) for communication. Additionally, the effectiveness of these methods hinges greatly on the accuracy of feature extraction. To address this limitation, this study introduces a Recurrent Neural Network (RNN) approach for BISINDO interpretation. Data acquisition involved the use of a webcam to capture video data, subsequently transformed into frames andarrays. Collected from three respondents, the dataset comprises 3,240 videos and 97,200 array data points, encompassing letters and numbers. Among the tested parameters, training results indicate that utilizing the Adam optimizer with a learning rate of 0.0001 and 500 epochs yields the highest accuracy and minimal loss compared to other configurations. Subsequently, this modelunderwent real-time testing, conducted five times for 36 classes, achieving an accuracy of 81.67%. It is important to note that errors may arise due to similarities within hand signal language, particularly involving characters such as I, J, D, P, M, and N.

Survey Paper - Sign Language Detection and Conversion using Deep Learning

Abstract: Communication for the deaf and mute relies on intricate gestures and visual cues, presenting a challenge for accurate interpretation. This project employs advanced deep learning techniques, utilizing SVM and K-means Clustering algorithms for precise real-time recognition of these expressions from images. Our aim is to bridge the communication gap, empowering the hearing-impaired to engage meaningfully and be a vital part of human interaction. Through this innovative approach, we create an inclusive society where everyone can express themselves effectively, regardless of their hearing abilities.

Applying optimized YOLOv8 for heritage conservation: enhanced object detection in Jiangnan traditional private gardens

This study aims to promote the protection and inheritance of cultural heritage in private gardens in the Jiangnan area of China. By establishing a precise visual labeling system and accelerating the construction of a database for private garden features, we deepen the understanding of garden design philosophy. To this end, we propose an improved Jiangnan private garden recognition model based on You Only Look Once (YOLO) v8. This model is particularly suitable for processing garden environments with characteristics such as single or complex structures, rich depth of field, and cluttered targets, effectively enhancing the accuracy and efficiency of object recognition. This design integrates the Diverse Branch Block (DBB), Bidirectional Feature Pyramid Network (BiFPN), and Dynamic Head modules (DyHead) to optimize model accuracy, feature fusion, and object detection representational capability, respectively. The enhancements elevated the model's accuracy by 8.7%, achieving a mean average precision (mAP@0.5) value of 57.1%. A specialized dataset, comprising 4890 images and encapsulating various angles and lighting conditions of Jiangnan private gardens, was constructed to realize this. Following manual annotation and the application of diverse data augmentation strategies, the dataset bolsters the generalization and robustness of the model. Experimental outcomes reveal that, compared to its predecessor, the improved model has witnessed increments of 15.16%, 3.25%, and 11.88% in precision, mAP0.5, and mAP0.5:0.95 metrics, respectively, demonstrating exemplary performance in the accuracy and real-time recognition of garden target elements. This research not only furnishes robust technical support for the digitization and intelligent research of Jiangnan private gardens but also provides a potent methodological reference for object detection and classification research in analogous domains.

Liquid Metal Composites-Enabled Real-Time Hand Gesture Recognizer with Superior Recognition Speed and Accuracy.

Prosthetic hands play a vital role in restoring forearm functionality for patients who have suffered hand loss or deformity. The hand gesture intention recognition system serves as a critical component within the prosthetic hand system. However, accurately and swiftly identifying hand gesture intentions remains a challenge in existing approaches. Here, a real-time motion intention recognition system utilizing liquid metal composite sensor bracelets is proposed. The sensor bracelet detects pressure signals generated by forearm muscle movements to recognize hand gesture intent. Leveraging the remarkable pressure sensitivity of liquid metal composites and the efficient classifier based on the optimized recognition algorithm, this system achieves an average offline and real-time recognition accuracy of 98.2% and 92.04%, respectively, with an average recognition speed of 0.364 s. Thus, this wearable system shows advantages in superior recognition speed and accuracy. Furthermore, this system finds applications in master-slave control of prosthetic hands in unmanned scenarios, such as electrically powered operations, space exploration, and telemedicine. The proposed system promises significant advances in next-generation intent-controlled prosthetic hands and robots.

Features selection for recognition of severe slugging in a long pipeline with an S-shaped riser by decision tree

In deep-sea oil and gas, S-shaped riser is the key equipment to transport the multiphase working substance produced underwater to the surface platform. The occurrence of severe slugging will seriously threaten the safe production of offshore oil and gas fields. Therefore, accurate and rapid recognition of hazardous flow patterns such as severe slugging is an urgent need to ensure the safety of multiphase flow in deep-sea long-distance pipelines. This paper has built a set of long S-shaped riser with industrial parameters, including 1687 m horizontal and downward inclined pipelines and 16.7 m S-shaped riser. The gas-liquid two-phase flow patterns, including severe slugging, oscillating flow and stable flow, are studied in this system. The pressure and differential pressure features of different signals are extracted from different positions along the pipeline, and a more comprehensive and reasonable evaluation criterion of signal performance is established. Making use of the advantages of visualization of the structure of decision tree, a new method to evaluate the recognition effect of features is proposed based on its classification principle. The original signal is decomposed into components of different scales through wavelet multi-scale analysis, and then the statistical features with good flow pattern differentiation ability are calculated. It is found that the mean of wavelet approximate component has the best recognition effect among all features. On the basis of feature correlation analysis, 7 features with high recognition rate applicable to 22 signals along the pipeline are optimally selected from 13 original features. For the water differential pressure signal DP13, the recognition rate obtained by using its 4 optimal features is 91.4 %, and the error between the recognition rate obtained by using all features is only 0.2 %. The overall recognition rate of the signal is the highest and the practicability is the best. When the sample duration is 6.2s, the recognition rate using the optimal feature reaches 90.8 %, thus ensuring the real-time recognition of the flow pattern while obtaining a high recognition rate.

Surviving Sepsis Campaign Research Priorities 2023.

To identify research priorities in the management, epidemiology, outcome, and pathophysiology of sepsis and septic shock. Shortly after publication of the most recent Surviving Sepsis Campaign Guidelines, the Surviving Sepsis Research Committee, a multiprofessional group of 16 international experts representing the European Society of Intensive Care Medicine and the Society of Critical Care Medicine, convened virtually and iteratively developed the article and recommendations, which represents an update from the 2018 Surviving Sepsis Campaign Research Priorities. Each task force member submitted five research questions on any sepsis-related subject. Committee members then independently ranked their top three priorities from the list generated. The highest rated clinical and basic science questions were developed into the current article. A total of 81 questions were submitted. After merging similar questions, there were 34 clinical and ten basic science research questions submitted for voting. The five top clinical priorities were as follows: 1) what is the best strategy for screening and identification of patients with sepsis, and can predictive modeling assist in real-time recognition of sepsis? 2) what causes organ injury and dysfunction in sepsis, how should it be defined, and how can it be detected? 3) how should fluid resuscitation be individualized initially and beyond? 4) what is the best vasopressor approach for treating the different phases of septic shock? and 5) can a personalized/precision medicine approach identify optimal therapies to improve patient outcomes? The five top basic science priorities were as follows: 1) How can we improve animal models so that they more closely resemble sepsis in humans? 2) What outcome variables maximize correlations between human sepsis and animal models and are therefore most appropriate to use in both? 3) How does sepsis affect the brain, and how do sepsis-induced brain alterations contribute to organ dysfunction? How does sepsis affect interactions between neural, endocrine, and immune systems? 4) How does the microbiome affect sepsis pathobiology? 5) How do genetics and epigenetics influence the development of sepsis, the course of sepsis and the response to treatments for sepsis? Knowledge advances in multiple clinical domains have been incorporated in progressive iterations of the Surviving Sepsis Campaign guidelines, allowing for evidence-based recommendations for short- and long-term management of sepsis. However, the strength of existing evidence is modest with significant knowledge gaps and mortality from sepsis remains high. The priorities identified represent a roadmap for research in sepsis and septic shock.

Online human motion analysis in industrial context: A review

Human motion analysis plays a crucial role in industry 4.0 and, more recently, in industry 5.0 where human-centered applications are becoming increasingly important, demonstrating its potential for enhancing safety, ergonomics and productivity. Considering this opportunity, an increasing number of studies are proposing works on the analysis of human motion in an industrial context, taking advantage of the rise of artificial intelligence technologies and sensor technologies. The objective of this work is to provide a review of recent studies exploring these technologies in the analysis of human movement while specifically considering industrial context. First, a taxonomy of key human motion analysis applications is proposed, presenting statistical insights to reveal trends and highlighting lacks in current research. Furthermore, this work identifies benchmark datasets acquired in various industrial case studies and associated sensors. Many recommendations for selecting optimal sensors and valuable benchmarks are proposed. Then, the paper outlines the current trend of utilizing hybrid deep learning methodologies in human movement analysis while underscoring the performance and limitations of these proposed methods, considering industrial constraints such as real-time recognition and frugality. Finally, challenges and future works are highlighted, focusing on the opportunities to address problems related to the complex industrial environment in order to achieve reliable performances.

Artificial intelligence based abnormal detection system and method for wind power equipment

The existing traditional monitoring methods are not ideal for identifying abnormal states in remote real-time monitoring of power system equipment due to adverse weather conditions such as haze and snow. The traditional monitoring methods for wind power equipment mainly rely on manual inspections and regular maintenance, which have problems such as high labor costs, low monitoring efficiency, and low monitoring accuracy. In order to improve the effectiveness of anomaly recognition for power system equipment, this paper proposes a remote real-time monitoring anomaly recognition method for power system equipment based on artificial intelligence technology. Firstly, this article uses pan tilt cameras to capture real-time images of power system equipment and transmit them to a remote monitoring center. The remote monitoring center captures real-time images of the equipment in the video stream and performs real-time image preprocessing. Then, the variational mode decomposition method is used to extract the abnormal state features of power system equipment in the image, and this feature is input into the support vector machine as a training sample to achieve real-time recognition of abnormal state of power system equipment. This article was also conducting fault diagnosis for four sets of wind turbines in the experimental section. According to the experimental results, it can be concluded that the rotational speed of wind turbine 1 increased from around 12 rpm to around 30 rpm during the day. During the day, the temperature of wind turbine 2 increased from around 25 °C to around 40 °C, while wind turbine 3 remained in a stopped state and the current speed decreased to 0. Wind turbine 4 also decreased from around 10 rpm to 0 during the day. From this, it can be concluded that the wind power equipment anomaly detection system based on artificial intelligence can clearly determine the abnormal situation of wind turbines. The wind power equipment anomaly detection system based on artificial intelligence can timely and accurately identify the abnormal situation of WPE, and can provide a new wind power equipment anomaly detection method based on artificial intelligence for the wind power industry. This can provide more accurate and reliable support for the operation and maintenance of wind power equipment, and also provide practice and exploration for the application of artificial intelligence in the industrial field.

Real-Time Recognition of Molten Pools Based on Improved DeepLabV3+ in Keyhole Tungsten Inert Gas Welding Applications

During the Keyhole Tungsten Inert Gas (K-TIG) welding process, a significant amount of information related to the weld quality can be obtained from the weld pool and the keyhole of the topside molten pool image, which provides a vital basis for the control of welding quality. However, the topside molten pool image has the unstable characteristic of strong arc light, which leads to difficulty in contour extraction. The existing image segmentation algorithms cannot satisfy the requirements for accuracy, timing, and robustness. Aiming at these problems, a real-time recognition method, based on improved DeepLabV3+, for identifying the molten pool more accurately and effectively was proposed in this paper. First, MobileNetV2 was selected as the feature extraction network with which to improve detection efficiency. Then, the atrous rates of atrous convolution layers were optimized to reduce the receptive field and balance the sensitivity of the model to molten pools of different scales. Finally, the convolutional block attention module (CBAM) was introduced to improve the segmentation accuracy of the model. The experimental results verified that the proposed model had a fast segmentation speed and higher segmentation accuracy, with an average intersection ratio of 89.89% and an inference speed of 103 frames per second. Furthermore, the trained model was deployed in a real-time system and achieved a real-time performance of up to 28 frames per second, thus meeting the real-time and accuracy requirements of the K-TIG molten pool monitoring system.

Lightweight Neural Network for Sleep Posture Classification Using Pressure Sensing Mat at Various Sensor Densities.

Recently, pressure-sensing mats have been widely used to capture static and dynamic pressure over sleep for posture recognition. Both a full-size mat with a low-density sensing array for figuring out the structure of the whole body and a miniature scale mat with a high-density sensing array for identifying the local characteristics around the chest have been investigated. However, both of the mat systems may face the challenge in the trade-off between the computational complexity (involving the size, density, etc. of the mat) and the performance of sleep posture recognition, where high performance may requires overcomplex computation and result in time latency in real-time sleep posture monitoring. In this paper, a lightweight neural network named ConcatNet, is proposed to realize sleep postures (supine, left, right, and prone) recognition in real time while maintaining a favorable performance. In ConcatNet, the inception module is proposed to extract the image features under multiple receptive fields, while the multi-layer feature fusion module is utilized to fuse deep and shallow features to enhance the model performance. To further improve the efficiency of the model, the depthwise convolution is adopoted. ConcatNet models in 3 different scales (ConcatNet-S, ConcatNet-M, and ConcatNet-L) are built to explore the impact of the sensor density on sleep posture recognition performance. Experimental results exhibit that ConcatNet-M corresponding to medium sensor density ( 16×16 ) achieved the best comprehensive performance, with short-term data cross-validation accuracy at 95.56% and overnight data testing accuracy at 94.68%. The model size is 7.91KB, FLOPs is 56.47K, and the inference time is only 0.38ms, which shows an outstanding performance of real-time sleep posture recognition with minimum consumption, indicating the potential to be deployed in mobile devices.

A Review of AIoT-based Human Activity Recognition: From Application to Technique.

This scoping review paper redefines the Artificial Intelligence-based Internet of Things (AIoT) driven Human Activity Recognition (HAR) field by systematically extrapolating from various application domains to deduce potential techniques and algorithms. We distill a general model with adaptive learning and optimization mechanisms by conducting a detailed analysis of human activity types and utilizing contact or non-contact devices. It presents various system integration mathematical paradigms driven by multimodal data fusion, covering predictions of complex behaviors and redefining valuable methods, devices, and systems for HAR. Additionally, this paper establishes benchmarks for behavior recognition across different application requirements, from simple localized actions to group activities. It summarizes open research directions, including data diversity and volume, computational limitations, interoperability, real-time recognition, data security, and privacy concerns. Finally, we aim to serve as a comprehensive and foundational resource for researchers delving into the complex and burgeoning realm of AIoT-enhanced HAR, providing insights and guidance for future innovations and developments.

Real-Time Recognition and Localization of Kiwifruit Based on Improved YOLOv5s Algorithm

Real-Time Recognition and Localization of Kiwifruit Based on Improved YOLOv5s Algorithm

Real-Time Recognition and Tracking in Urban Spaces Through Deep Learning: A Case Study

Real-Time Recognition and Tracking in Urban Spaces Through Deep Learning: A Case Study

An Improved Extreme Learning Machine (ELM) Algorithm for Intent Recognition of Transfemoral Amputees With Powered Knee Prosthesis.

To overcome the challenges posed by the complex structure and large parameter requirements of existing classification models, the authors propose an improved extreme learning machine (ELM) classifier for human locomotion intent recognition in this study, resulting in enhanced classification accuracy. The structure of the ELM algorithm is enhanced using the logistic regression (LR) algorithm, significantly reducing the number of hidden layer nodes. Hence, this algorithm can be adopted for real-time human locomotion intent recognition on portable devices with only 234 parameters to store. Additionally, a hybrid grey wolf optimization and slime mould algorithm (GWO-SMA) is proposed to optimize the hidden layer bias of the improved ELM classifier. Numerical results demonstrate that the proposed model successfully recognizes nine daily motion modes including low-, mid-, and fast-speed level ground walking, ramp ascent/descent, sit/stand, and stair ascent/descent. Specifically, it achieves 96.75% accuracy with 5-fold cross-validation while maintaining a real-time prediction time of only 2 ms. These promising findings highlight the potential of onboard real-time recognition of continuous locomotion modes based on our model for the high-level control of powered knee prostheses.

Traditional technique application v/s YOLOv5 model for marine underwater objects detection by computer vision method

The paper compares the effectiveness of two methods application for detecting and recognizing micro-objects in an aqua environment on the example of plankton using neural networks and various technologies and developed in different programming languages. At first the traditional detection method was investigated and applied based on the extraction of Gabor and multilayer perceptron features, realized in MATrixLABoratory (MATLAB) language. Secondly, YOLOv5 (an acronym for ‘You only look once’), as a single-stage neural network was used which was implemented in Python language. The results of the work of these methods for the plankton detection are presented. Accuracy and completeness metrics are calculated to determine best of two methods. Images with the recognition result, programmatically calculated performance metrics were obtained after using the detection methods. The study was conducted on the effectiveness of method applications for realtime recognition using short video images. In conclusion it is stated that the YOLOv5 model has shown significant superiority over the traditional method in the task of detecting marine objects, in particular plankton. Its accuracy was 30% higher; the completeness of object detection was 27% higher.