Deep Learning-based Recognition Methods Research Articles

Research on the Recognition Method of Dial Scales for Arrester Pointer Instruments Based on Deep Learning

To address the recognition challenges faced by arrester pointer instruments’ dial scales in various scenarios, this paper introduces a deep learning-based recognition method for pointer instrument scales. An attention module is integrated into the YOLOv5 network architecture, enhancing the accuracy and robustness of the model. After correcting the dial, dial recognition is conducted with OpenCV to achieve precise identification of the instrument scales. The proposed method was tested using images of arrester pointer instruments against diverse backgrounds. The experimental results demonstrate that the method processes instrument data images in an average time of 0.662 s and achieves a successful recognition rate of 96% with an average error of 0.923%. This method provides a rapid and efficient approach for recognizing instrument scales and offers a novel solution for identifying similar types of instruments.

Detection of Pig Movement and Aggression Using Deep Learning Approaches.

Motion and aggressive behaviors in pigs provide important information for the study of social hierarchies in pigs and can be used as a selection indicator for pig health and aggression parameters. However, relying only on visual observation or surveillance video to record the number of aggressive acts is time-consuming, labor-intensive, and lasts for only a short period of time. Manual observation is too short compared to the growth cycle of pigs, and complete recording is impractical in large farms. In addition, due to the complex process of assessing the intensity of pig aggression, manual recording is highly influenced by human subjective vision. In order to efficiently record pig motion and aggressive behaviors as parameters for breeding selection and behavioral studies, the videos and pictures were collected from typical commercial farms, with each unit including 8~20 pigs in 7~25 m2 space; they were bred in stable social groups and a video was set up to record the whole day's activities. We proposed a deep learning-based recognition method for detecting and recognizing the movement and aggressive behaviors of pigs by recording and annotating head-to-head tapping, head-to-body tapping, neck biting, body biting, and ear biting during fighting. The method uses an improved EMA-YOLOv8 model and a target tracking algorithm to assign a unique digital identity code to each pig, while efficiently recognizing and recording pig motion and aggressive behaviors and tracking them, thus providing statistics on the speed and duration of pig motion. On the test dataset, the average precision of the model was 96.4%, indicating that the model has high accuracy in detecting a pig's identity and its fighting behaviors. The model detection results were highly correlated with the manual recording results (R2 of 0.9804 and 0.9856, respectively), indicating that the method has high accuracy and effectiveness. In summary, the method realized the detection and identification of motion duration and aggressive behavior of pigs under natural conditions, and provided reliable data and technical support for the study of the social hierarchy of pigs and the selection of pig health and aggression phenotypes.

Tomato Leaf Disease Identification Method Based on Improved YOLOX

In tomato leaf disease identification tasks, the high cost and consumption of deep learning-based recognition methods affect their deployment and application on embedded devices. In this study, an improved YOLOX-based tomato leaf disease identification method is designed. To address the issue of positive and negative sample imbalance, the sample adaptive cross-entropy loss function (LBCE−β) is proposed as a confidence loss, and MobileNetV3 is employed instead of the YOLOX backbone for lightweight model feature extraction. By introducing CBAM (Convolutional Block Attention Module) between the YOLOX backbone and neck network, the model’s feature extraction performance is increased. CycleGAN is used to enhance the data of tomato disease leaf samples in the PlantVillage dataset, solving the issue of an imbalanced sample number. After data enhancement, simulation experiments and field tests revealed that the YOLOX’s accuracy improved by 1.27%, providing better detection of tomato leaf disease samples in complex environments. Compared with the original model, the improved YOLOX model occupies 35.34% less memory, model detection speed increases by 50.20%, and detection accuracy improves by 1.46%. The enhanced network model is quantized by TensorRT and works at 11.1 FPS on the Jetson Nano embedded device. This method can provide an efficient solution for the tomato leaf disease identification system.

Dynamic Dataset Augmentation for Deep Learning-based Oracle Bone Inscriptions Recognition

Oracle bone inscriptions (OBIs) are a kind of hieroglyph, used about 3,600 years ago for divination and the recording of events. The characters on these OBIs are of great interest because they are precursors to the modern Chinese characters widely used across Asia today. However, as the OBIs were only discovered in 1899, there are currently few documents to describe them. Hence, recognizing and unlocking the meaning of OBIs helps to understand the ancient history of China, the evaluation of Chinese characters, and more. Currently, deep learning has made great progress and brought about a revolution in the research field of recognition, and shows good potential to meet the challenges of OBIs recognition. Due to the scarcity of resources, many OBIs contain only a small number of instances, which causes dataset imbalance and limits the accuracy of recognition. This paper attempts to provide a suite of OBIs recognition methods comprising an original OBIs dataset creation, dynamic dataset augmentation, and a novel deep learning-based recognition method. To this end, we create an original OBIs dataset and propose a modified Generative Adversarial Network for augmenting the original OBIs dataset. The augmented data is then dynamically selected for training the deep learning model considering the data imbalance problem. A novel model called C-A Net is proposed for OBIs recognition. The results of evaluation experiments show that the dynamical dataset augmentation can effectively locate a suitable training dataset for the deep learning model and solve the problem of imbalanced OBIs distribution. In addition, the recognition accuracy of C-A Net is 91.10%, which is higher than that of eight state-of-the-art models, and thus effectively suppresses the occurrence of overfitting. We also present an original OBIs dataset named OBI125, which is currently the only rubbing-type OBIs dataset that is open to the public. The code is available at http://www.ihpc.se.ritsumei.ac.jp/obidataset.html.

Presentation Attacks in Palmprint Recognition Systems

Background: A presentation attack places the printed image or displayed video at the front of the sensor to deceive the biometric recognition system. Usually, presentation attackers steal a genuine user’s biometric image and use it for presentation attack. In recent years, reconstruction attack and adversarial attack can generate high-quality fake images, and have high attack success rates. However, their attack rates degrade remarkably after image shooting. Methods: In order to comprehensively analyze the threat of presentation attack to palmprint recognition system, this paper makes six palmprint presentation attack datasets. The datasets were tested on texture coding-based recognition methods and deep learning-based recognition methods. Results and conclusion: The experimental results show that the presentation attack caused by the leakage of the original image has a high success rate and a great threat; while the success rates of reconstruction attack and adversarial attack decrease significantly.

Deep learning and RGB-D based human action, human–human and human–object interaction recognition: A survey

Human activity recognition is one of the most studied topics in the field of computer vision. In recent years, with the availability of RGB-D sensors and powerful deep learning techniques, research on human activity recognition has gained momentum. From simple human atomic actions, the research has advanced towards recognizing more complex human activities using RGB-D data. This paper presents a comprehensive survey of the advanced deep learning based recognition methods and categorizes them in human atomic action, human–human interaction, human–object interaction. The reviewed methods are further classified based on the individual modality used for recognition i.e. RGB based, depth based, skeleton based, and hybrid. We also review and categorize recent challenging RGB-D datasets for the same. In addition, the paper also briefly reviews RGB-D datasets and methods for online activity recognition. The paper concludes with a discussion on limitations, challenges, and recent trends for promising future directions.

PlantNet: transfer learning-based fine-grained network for high-throughput plants recognition

In high-throughput phenotyping, recognizing individual plant categories is a vital support process for plant breeding. However, different plant categories have different fine-grained characteristics, i.e., intra-class variation and inter-class similarity, making the process challenging. Existing deep learning-based recognition methods fail to effectively address this recognition task under challenging requirements, leading to technical difficulties such as low accuracy and lack of generalization robustness. To address these requirements, this paper proposes PlantNet, a fine-grained network for plant recognition based on transfer learning and a bilinear convolutional neural network, which achieves high recognition accuracy in high-throughput phenotyping requirements. The network operates as follows. First, two deep feature extractors are constructed using transfer learning. The outer product of the different spatial locations corresponding to the two features is then calculated, and the bilinear convergence is computed for the different spatial locations. Finally, the fused bilinear vectors are normalized via maximum expectation to generate the network output. Experiments on a publicly available Arabidopsis dataset show that the proposed bilinear model performed better than related state-of-the-art methods. The interclass recognition accuracy of the four different species of Arabidopsis Sf-2, Cvi, Landsberg and Columbia are found to be 98.48%, 96.53%, 96.79% and 97.33%, respectively, with an average accuracy of 97.25%. Thus, the network has good generalization ability and robust performance, satisfying the needs of fine-grained plant recognition in agricultural production.

LiDAR Point Cloud Recognition and Visualization with Deep Learning for Overhead Contact Inspection.

As overhead contact (OC) is an essential part of power supply systems in high-speed railways, it is necessary to regularly inspect and repair abnormal OC components. Relative to manual inspection, applying LiDAR (light detection and ranging) to OC inspection can improve efficiency, accuracy, and safety, but it faces challenges to efficiently and effectively segment LiDAR point cloud data and identify catenary components. Recent deep learning-based recognition methods are rarely employed to recognize OC components, because they have high computational complexity, while their accuracy needs to be improved. To track these problems, we first propose a lightweight model, RobotNet, with depthwise and pointwise convolutions and an attention module to recognize the point cloud. Second, we optimize RobotNet to accelerate its recognition speed on embedded devices using an existing compilation tool. Third, we design software to facilitate the visualization of point cloud data. Our software can not only display a large amount of point cloud data, but also visualize the details of OC components. Extensive experiments demonstrate that RobotNet recognizes OC components more accurately and efficiently than others. The inference speed of the optimized RobotNet increases by an order of magnitude. RobotNet has lower computational complexity than other studies. The visualization results also show that our recognition method is effective.

One-shot learning gesture recognition based on joint training of 3D ResNet and memory module

As a research hotspot in the field of human-machine interaction, a great progress of hand gesture recognition has been achieved with the development of deep learning of neural networks. However, in the deep learning based recognition methods, it is necessary to rely heavily on large-scale labeled dataset which is very hard to build in practical applications. In order to achieve a well performance under some strict constraint of few sample data, one-shot learning gesture recognition is studied and a joint deep training method by combination of 3D ResNet with a memory module is presented in this paper. In our scheme a combinatorial optimization of feature extraction by 3D ResNet with memory capacity of rare event by memory module is carried out with an effective strategy of optimal decision and two relative performance indices. In order to implement one-shot learning gesture recognition, the memory module is employed to remember the features extracted by well-trained 3D ResNet and the classification decision is performed by the nearest neighbor algorithm with cosine similarity measure. In view of real-world applications about human-machine interaction technology, its ability to deal with negative samples plays a significant role thus a mechanism based on the threshold of cosine similarity is built to realize effective classification and rejection respectively. In order to validate and evaluate the performance of our proposed method, a special hand gesture dataset containing 3045 gesture videos is built and a series of experiment results on our collected dataset and public datasets demonstrate the feasibility and effectiveness of our method.

A deep learning-based recognition method for degradation monitoring of ball screw with multi-sensor data fusion

In this paper, a novel intelligent ball screw degradation recognition method based on deep belief networks (DBN) and multi-sensor data fusion is proposed. First, the derived method calculates frequency spectrums of raw signals, and the fused frequency spectrums are calculated by the multi-sensor data fusion. Then, a deep learning-based recognition model that can estimate the degradation condition of ball screw automatically is established with the fused dataset. The effectiveness of the proposed method is validated using dataset collected from the degradation test of ball screw. The dataset contains massive samples involving 7 degradation stages under 9 working conditions by 3 accelerometers. The classification results indicate that the proposed DBN-based method is able to mine intrinsic characteristics from the fused frequency spectrums adaptively and obtain a superior recognition accuracy. Finally, two comparative studies are performed to show the advantage of the proposed DBN-based method in ball screw degradation condition recognition.

Epithelium-Stroma Classification via Convolutional Neural Networks and Unsupervised Domain Adaptation in Histopathological Images.

Epithelium-stroma classification is a necessary preprocessing step in histopathological image analysis. Current deep learning based recognition methods for histology data require collection of large volumes of labeled data in order to train a new neural network when there are changes to the image acquisition procedure. However, it is extremely expensive for pathologists to manually label sufficient volumes of data for each pathology study in a professional manner, which results in limitations in real-world applications. A very simple but effective deep learning method, that introduces the concept of unsupervised domain adaptation to a simple convolutional neural network (CNN), has been proposed in this paper. Inspired by transfer learning, our paper assumes that the training data and testing data follow different distributions, and there is an adaptation operation to more accurately estimate the kernels in CNN in feature extraction, in order to enhance performance by transferring knowledge from labeled data in source domain to unlabeled data in target domain. The model has been evaluated using three independent public epithelium-stroma datasets by cross-dataset validations. The experimental results demonstrate that for epithelium-stroma classification, the proposed framework outperforms the state-of-the-art deep neural network model, and it also achieves better performance than other existing deep domain adaptation methods. The proposed model can be considered to be a better option for real-world applications in histopathological image analysis, since there is no longer a requirement for large-scale labeled data in each specified domain.