Recognition Results Research Articles

Modeling Emotion Recognition System from Facial Images Using Convolutional Neural Networks

Emotion classification is the process of identifying human emotions. Implementing technology to help people with emotional classification is considered a relatively popular research field. Until now, most of the work has been done to automate the recognition of facial cues (e.g., expressions) from several modalities (e.g., image, video, audio, and text). Deep learning architecture such as Convolutional Neural Networks (CNN) demonstrates promising results for emotion recognition. The research aims to build a CNN model while improving accuracy and performance. Two models are proposed in the research with some hyperparameter tuning followed by two datasets and other existing architecture that will be used and compared with the proposed architecture. The two datasets used are Facial Expression Recognition 2013 (FER2013) and Extended Cohn-Kanade (CK+), both of which are commonly used datasets in FER. In addition, the proposed model is compared with the previous model using the same setting and dataset. The result shows that the proposed models with the CK+ dataset gain higher accuracy, while some models with the FER2013 dataset have lower accuracy compared to previous research. The model trained with the FER2013 dataset has lower accuracy because of overfitting. Meanwhile, the model trained with CK+ has no overfitting problem. The research mainly explores the CNN model due to limited resources and time.

Study on in-situ measurement and analysis method of rock pores based on borehole camera technology

In-situ measurement and analysis of pores are helpful to understand the properties of rocks. However, continuous quantitative analysis of pores in whole borehole is still difficult. Borehole camera technology can capture the borehole wall image of the whole well section by going deep into the borehole, which provides technical conditions for the measurement and analysis of pores. In this paper, a continuous measurement method of rock pores based on borehole camera technology is introduced. According to the characteristics of the optical image of borehole wall, a method of pore recognition and analysis is proposed. Specifically, the influence of redundant information on pore recognition is eliminated, and then the pores are accurately recognized by binarization and morphological operation. Based on the results of pore recognition and the coordinate information provided by borehole wall image, the calculation methods of surface porosity and line porosity are proposed, and the statistical analysis of pore distribution is realized according to the calculation results. In addition, the morphological characteristics of pores are also discussed. This method realizes the accurate recognition and quantitative calculation of pore structure based on borehole wall image, and provides a new method for continuous analysis of pore structure in the whole well.

Adaptive hierarchical energy management strategy for fuel cell mobile robot hybrid power system based on working condition recognition

Energy management of hybrid power is critical to maintain the economical and efficient operation of fuel cell mobile robots. To improve the energy distribution between the proton exchange membrane fuel cell (PEMFC) and battery under different working conditions, this paper proposes an adaptive hierarchical energy management strategy (AHEMS) based on the recognition and management levels. Firstly, the recognition level realizes the identification of different working conditions based on the machine learning (ML) methods including the K-means and KNN. Secondly, the fuel cell hydrogen consumption and efficiency are both optimized by adaptive multi-objective particle swarm optimization (AMOPSO) at the management level. Specifically, an adaptive flight parameter strategy based on the particle dispersity (PD) information is proposed to balance the convergence and diversity of Pareto solutions. Besides, to overcome the parameter uncertainty caused by different working states and improve the system performance, an interval optimization scheme is proposed based on the Pareto solutions. Finally, the fuzzy decision combined with the recognition results and state of charge (SOC) of the battery is performed to find the most appropriate power distribution of the PEMFC and battery. The proposed AHEMS algorithm is compared with different algorithms in the numerical simulation and hardware-in-loop (HIL) experiments. These results demonstrate that the hybrid power system with the proposed optimization scheme performs better than the base model and classical optimization algorithms in terms of the hydrogen consumption and efficiency indexes, revealing the success of this AHEMS approach in solving the energy distribution problem in different working conditions.

Automatic location and recognition of horse freezing brand using rotational YOLOv5 deep learning network

Individual livestock identification is of great importance to precision livestock farming. Liquid nitrogen freezing labeled horse brand is an effective way for livestock individual identification. Along with various technological developments, deep-learning-based methods have been applied in such individual marking recognition. In this research, a deep learning method for oriented horse brand location and recognition was proposed. Firstly, Rotational YOLOv5 (R-YOLOv5) was adopted to locate the oriented horse brand, then the cropped images of the brand area were trained by YOLOv5 for number recognition. In the first step, unlike classical detection methods, R-YOLOv5 introduced the orientation into the YOLO framework by integrating Circle Smooth Label (CSL). Besides, Coordinate Attention (CA) was added to raise the attention to positional information in the network. These improvements enhanced the accuracy of detecting oriented brands. In the second step, number recognition was considered as a target detection task because of the requirement of accurate recognition. Finally, the whole brand number was obtained according to the sequences of each detection box position. The experiment results showed that R-YOLOv5 outperformed other rotating target detection algorithms, and the AP (Average Accuracy) was 95.6 %, the FLOPs were 17.4 G, the detection speed was 14.3 fps. As for the results of number recognition, the mAP (mean Average Accuracy) was 95.77 %, the weight size was 13.71 MB, and the detection speed was 68.6 fps. The two-step method can accurately identify brand numbers with complex backgrounds. It also provides a stable and lightweight method for livestock individual identification.

Identifying Truck Activity from Global Positioning System Data Based on Satellite Imagery

Road freight transportation plays a crucial role in fostering economic growth and enhancing logistics efficiency. Existing research primarily focuses on analyzing freight truck activity characteristics based on global positioning system (GPS) data of freight truck trajectories. Those studies proposed a variety of identification methods from different perspectives, but they mainly rely on rule-based methods and subjectively defined key threshold parameters, ignoring the complexity of truck freight activities and relying on long sequences of trajectory data. To bridge this gap, we propose the satellite image augmented truck activity recognition method, a structured framework that utilizes satellite imagery to identify truck activities from GPS data. Firstly, we categorize truck activities into three types: loading/unloading, resting, and driving. We extract geographic entities from the area of interest data to create the satellite image dataset of truck activity. Secondly, we evaluate the recognition accuracy of benchmark image classification models on our dataset, and ResNet50 achieves 98.04% accuracy on the test set. Thirdly, we apply the image classification model to identify truck activity from GPS trajectory data and use the recognition results of neighboring points to improve the identification results. The framework delineated in this paper entails the extraction of truck activities from GPS data at the satellite image level. Our approach holds promise in facilitating logistics operation fleets and traffic management authorities to maintain near real-time surveillance of the truck status, thereby fostering enhancements in both the efficiency and safety of freight transport.

Multi-Source Data-Driven Extraction of Urban Residential Space: A Case Study of the Guangdong–Hong Kong–Macao Greater Bay Area Urban Agglomeration

The accurate extraction of urban residential space (URS) is of great significance for recognizing the spatial structure of urban function, understanding the complex urban operating system, and scientific allocation and management of urban resources. The traditional URS identification process is generally conducted through statistical analysis or a manual field survey. Currently, there are also superpixel segmentation and wavelet transform (WT) processes to extract urban spatial information, but these methods have shortcomings in extraction efficiency and accuracy. The superpixel wavelet fusion (SWF) method proposed in this paper is a convenient method to extract URS by integrating multi-source data such as Point of Interest (POI) data, Nighttime Light (NTL) data, LandScan (LDS) data, and High-resolution Image (HRI) data. This method fully considers the distribution law of image information in HRI and imparts the spatial information of URS into the WT so as to obtain the recognition results of URS based on multi-source data fusion under the perception of spatial structure. The steps of this study are as follows: Firstly, the SLIC algorithm is used to segment HRI in the Guangdong–Hong Kong–Macao Greater Bay Area (GBA) urban agglomeration. Then, the discrete cosine wavelet transform (DCWT) is applied to POI–NTL, POI–LDS, and POI–NTL–LDS data sets, and the SWF is carried out based on different superpixel scale perspectives. Finally, the OSTU adaptive threshold algorithm is used to extract URS. The results show that the extraction accuracy of the NLT–POI data set is 81.52%, that of the LDS–POI data set is 77.70%, and that of the NLT–LDS–POI data set is 90.40%. The method proposed in this paper not only improves the accuracy of the extraction of URS, but also has good practical value for the optimal layout of residential space and regional planning of urban agglomerations.

Electroencephalography Emotion Recognition Based on Rhythm Information Entropy Extraction

Electroencephalography (EEG) is a physiological signal directly generated by the central nervous system. Brain rhythm is closely related to a person’s emotional state and is widely used for EEG emotion recognition. In previous studies, the rhythm specificity between different brain channels was seldom explored. In this paper, the rhythm specificity of brain channels is studied to improve the accuracy of EEG emotion recognition. Variational mode decomposition is used to decompose rhythm signals and enhance features, and two kinds of information entropy, i.e., differential entropy (DE) and dispersion entropy (DispEn) are extracted. The rhythm being used to get the best result of single channel emotion recognition is selected as the representative rhythm, and the remove one method is employed to obtain rhythm information entropy feature. In the experiment, the DEAP database was used for EEG emotion recognition in valence-arousal space. The results showed that the best result of rhythm DE feature classification in the valence dimension is 77.04%, and the best result of rhythm DispEn feature classification in the arousal dimension is 79.25%.

High-Precision Automated Soybean Phenotypic Feature Extraction Based on Deep Learning and Computer Vision.

The automated collection of plant phenotypic information has become a trend in breeding and smart agriculture. Four YOLOv8-based models were used to segment mature soybean plants placed in a simple background in a laboratory environment, identify pods, distinguish the number of soybeans in each pod, and obtain soybean phenotypes. The YOLOv8-Repvit model yielded the most optimal recognition results, with an R2 coefficient value of 0.96 for both pods and beans, and the RMSE values were 2.89 and 6.90, respectively. Moreover, a novel algorithm was devised to efficiently differentiate between the main stem and branches of soybean plants, called the midpoint coordinate algorithm (MCA). This was accomplished by linking the white pixels representing the stems in each column of the binary image to draw curves that represent the plant structure. The proposed method reduces computational time and spatial complexity in comparison to the A* algorithm, thereby providing an efficient and accurate approach for measuring the phenotypic characteristics of soybean plants. This research lays a technical foundation for obtaining the phenotypic data of densely overlapped and partitioned mature soybean plants under field conditions at harvest.

Improved vision-only localization method for mobile robots in indoor environments

To solve the problem of mobile robots needing to adjust their pose for accurate operation after reaching the target point in the indoor environment, a localization method based on scene modeling and recognition has been designed. Firstly, the offline scene model is created by both handcrafted feature and semantic feature. Then, the scene recognition and location calculation are performed online based on the offline scene model. To improve the accuracy of recognition and location calculation, this paper proposes a method that integrates both semantic features matching and handcrafted features matching. Based on the results of scene recognition, the accurate location is obtained through metric calculation with 3D information. The experimental results show that the accuracy of scene recognition is over 90%, and the average localization error is less than 1 meter. Experimental results demonstrate that the localization has a better performance after using the proposed improved method.

GRNet: a graph reasoning network for enhanced multi-modal learning in scene text recognition

Abstract Recent advancements in scene text recognition have predominantly focused on leveraging textual semantics. However, an over-reliance on linguistic priors can impede a model’s ability to handle irregular text scenes, including non-standard word usage, occlusions, severe distortions, or stretching. The key challenges lie in effectively localizing occlusions, perceiving multi-scale text, and inferring text based on scene context. To address these challenges and enhance visual capabilities, we introduce the Graph Reasoning Model (GRM). The GRM employs a novel feature fusion method to align spatial context information across different scales, beginning with a feature aggregation stage that extracts rich spatial contextual information from various feature maps. Visual reasoning representations are then obtained through graph convolution. We integrate the GRM module with a language model to form a two-stream architecture called GRNet. This architecture combines pure visual predictions with joint visual-linguistic predictions to produce the final recognition results. Additionally, we propose a dynamic iteration refinement for the language model to prevent over-correction of prediction results, ensuring a balanced contribution from both visual and linguistic cues. Extensive experiments demonstrate that GRNet achieves state-of-the-art average recognition accuracy across six mainstream benchmarks. These results highlight the efficacy of our multi-modal approach in scene text recognition, particularly in challenging scenarios where visual reasoning plays a crucial role.

Semi-Supervised Remote Sensing Building Change Detection with Joint Perturbation and Feature Complementation

The timely updating of the spatial distribution of buildings is essential to understanding a city’s development. Deep learning methods have remarkable benefits in quickly and accurately recognizing these changes. Current semi-supervised change detection (SSCD) methods have effectively reduced the reliance on labeled data. However, these methods primarily focus on utilizing unlabeled data through various training strategies, neglecting the impact of pseudo-changes and learning bias in models. When dealing with limited labeled data, abundant low-quality pseudo-labels generated by poorly performing models can hinder effective performance improvement, leading to the incomplete recognition results of changes to buildings. To address this issue, we propose a feature multi-scale information interaction and complementation semi-supervised method based on consistency regularization (MSFG-SemiCD), which includes a multi-scale feature fusion-guided change detection network (MSFGNet) and a semi-supervised update method. Among them, the network facilitates the generation of multi-scale change features, integrates features, and captures multi-scale change targets through the temporal difference guidance module, the full-scale feature fusion module, and the depth feature guidance fusion module. Moreover, this enables the fusion and complementation of information between features, resulting in more complete change features. The semi-supervised update method employs a weak-to-strong consistency framework to achieve model parameter updates while maintaining perturbation invariance of unlabeled data at both input and encoder output features. Experimental results on the WHU-CD and LEVIR-CD datasets confirm the efficacy of the proposed method. There is a notable improvement in performance at both the 1% and 5% levels. The IOU in the WHU-CD dataset increased by 5.72% and 6.84%, respectively, while in the LEVIR-CD dataset, it improved by 18.44% and 5.52%, respectively.

Terrain Recognition Using Deep Learning

This paper presents the method for a vision-based terrain recognition system that uses a convolutional neural network (CNN). The dataset includes images of various terrains under different conditions. The pretrained CNN is used for extracting features from pre-processed images, which are then used to classify the terrain types. The SoftMax layer of the architecture provides the result of terrain recognition. In the present work, experiments are performed with two pre-trained models. The system provides 80% accuracy using Tensorflow and ensemble learning Keywords— CNN; ; Tensorflow; terrain recognition; deep learning

Human Muscle sEMG Signal and Gesture Recognition Technology Based on Multi-Stream Feature Fusion Network

Surface electromyography signals have significant value in gesture recognition due to their ability to reflect muscle activity in real time. However, existing gesture recognition technologies have not fully utilized surface electromyography signals, resulting in unsatisfactory recognition results. To this end, firstly, a Butterworth filter was adopted to remove high-frequency noise from the signal. A combined method of moving translation threshold was introduced to extract effective signals. Then, a gesture recognition model based on multi-stream feature fusion network was constructed. Feature extraction and fusion were carried out through multiple parallel feature extraction paths, combined with convolutional neural networks and residual attention mechanisms. Compared to popular methods of the same type, this new recognition method had the highest recognition accuracy of 92.1% and the lowest recognition error of 5%. Its recognition time for a single-gesture image was as short as 4s, with a maximum Kappa coefficient of 0.92. Therefore, this method combining multi-stream feature fusion networks can effectively improve the recognition accuracy and robustness of gestures and has high practical value.

Accurate recognition of micromorphology images of epoxy coatings for deep-sea environments based on a deep learning super-resolution method

Crack initiation and extension occur in organic coatings during service in deep-sea environments. However, when extracting detailed crack information from SEM images of epoxy mica coatings at different time periods in a simulated deep-sea fluid-hydraulic environment, uninteresting background regions are treated equally, resulting in unnecessary computational redundancy. To address the relatively blurred edges and unclear textures of SEM images, a crack image super-resolution network based on global mixed attention (GMA-net) is proposed for application to SEM images of organic coatings. The recognition results of the images processed with GMA-net were compared with those of the original images and the images processed with bicubic method, respectively. The results show that this method not only refrains from destroying the clarity of the original images but also greatly outperforms bicubic method in terms of precision, recall, mAP50 and mPA50-95, which are improved by approximately 23.1%, 32.4%, 36.4% and 26.7%, respectively. This method effectively highlights the details and improves the recognition accuracy of the edge texture with the aim of providing a good basis for subsequent recognition and even lifetime prediction studies.

An outlier identification method based on parallel clustering and the Adaboost algorithm for power multi-source heterogeneous data

Abstract Influenced by the diversity of data scheduling and the multi-level differentiation of data distribution, the multi-source heterogeneous data of electric power have serious deviations in space and time scheduling, which makes it difficult to identify outliers. Therefore, a method for identifying outliers of multi-source heterogeneous data of electric power based on parallel clustering and the Adaboost algorithm is proposed. A sampling node model of heterogeneous data is constructed, and random numbers that obey Gaussian distribution are generated according to the mean value of data, and their mean square loss values are optimized to form an objective function of abnormal values of multi-source heterogeneous data of electric power. According to parallel clustering and the Adaboost algorithm, a unified format is created to obtain the load control results of abnormal values of multi-source heterogeneous data of electric power, so as to realize the identification of abnormal values of multi-source heterogeneous data of electric power. The experimental results show that at the time point of 18 s, the data identification quality of this method has reached 80, and with the increase of time, its data identification quality remains stable. It is demonstrated that the method has good recognition results.

Logging Identification Method for Reservoir Facies in Fractured-Vuggy Dolomite Reservoirs Based on AI: A Case Study of Ediacaran Dengying Formation, Sichuan Basin, China

As an important target for deep to ultra-deep carbonate oil and gas exploration, Fractured-Vuggy dolomite reservoirs have strong heterogeneity. Accurate characterization of reservoir facies is crucial for their exploration and exploitation. Three methods, including the unsupervised intelligent clustering method of improved Fuzzy C-means clustering Algorithm Based on Density Sensitive Distance and Fuzzy Partrition (FCM-DSDFP), the fusion method of Principal Components Analysis (PCA) dimensionality reduction and noise reduction, and the principle of clustering feature analysis are applied to identify reservoir facies based on logging data. Based on the PCA method, the logging response characteristics of the reservoir facies are excavated, and the fusion characterization data of dimensionality reduction and noise reduction are extracted. The FCM-DSDFP unsupervised intelligent clustering method, a model that approximates the subsurface conditions is established, and the reliability of the model is tested according to the elbow rule and silhouette coefficient. Combining drilling core observation, Fractured-Vuggy type, partially cemented Fractured-Vuggy type, Pore-Vuggy type, Pore Type I, Pore Type II, Tight Type I, and Tight Type II are divided in the Dengying Formation 4th Member. Fractured-Vuggy type, partially cemented Fractured-Vuggy type, Pore-Vuggy Type I, Pore-Vuggy Type II, Pore Type I, Pore Type II, and Tight Type are divided in the Dengying Formation 2nd Member, respectively. Two methods were applied to verify the reservoir facies identification results based on intelligent algorithms. The first method is to compare the identification results with the reservoir facies types identified by core observations (Well PT103 and PS13). The second method is to verify the recognition results of intelligent algorithms by utilizing the relationship between reservoir facies types and bitumen. The test results show that the accuracy of the reservoir level identification is higher than 0.9, and the applicability is better than the commonly used algorithms such as FCM and K-means.

A reference learning network for fault diagnosis of rotating machinery under strong noise

The strong noise often masks the fault characteristics of equipment, which reduces the accuracy of fault diagnosis and even leads to the inability of intelligent fault diagnosis algorithms to be applied in industrial environments. This has always been a challenge in the field of mechanical fault diagnosis. As known that equipment failure results from the continuous degradation of the equipment’s state, with the failure state evolving from the healthy state. Considering that both healthy signals and fault signals contain similar noise, this paper proposes a Reference Learning Network (RLNet) model. The model aims to enhance the distinguishing features between healthy and faulty samples through reference units, thereby eliminating the influence of noise on feature distribution. Firstly, the impact of variable speed on the model’s robustness is mitigated using the computed order tracking method. Then, the difference features between healthy samples and a class of fault samples are extracted through the binary classification reference learning unit (RLU). Next, the extracted differential features are used to train the state classifier. Finally, membership weights are employed to effectively combine the feature recognition results, reducing the influence of fault features from mismatched RLUs. The robustness and superiority of the proposed method were verified by comparing it with five other intelligent fault diagnosis methods on the gear and bearing datasets. RLNet is of great significance for the engineering application of intelligent fault diagnosis methods in industrial noise environments.

Research on road crack segmentation based on deep convolution and transformer with multi-branch feature fusion

Abstract Efficient and precise identification of road pavement cracks contributes to better evaluation of road conditions. In practical road maintenance and safety assessment, traditional manual crack detection methods are time-consuming, physically demanding, and highly subjective. In addition, crack recognition based on image processing techniques lacks robustness. In this paper, a multi-branch feature fusion road crack segmentation network model (DTPC) based on deep convolution and transformer modules is proposed. The model is used for pixel-level segmentation of road crack images, which is a good solution to the existing needs and helps to repair dangerous cracks promptly in the follow-up work to prevent serious disasters due to crack breakage. Firstly, combine deep convolution with transformer modules to achieve precise local extraction and global contextual feature extraction. Secondly, a dual-channel attention mechanism is em-ployed to help the model better address information loss and positional offset issues. Finally, three-branch outputs are fused to obtain prediction maps that intuitively determine recognition results. The proposed model is tested for accuracy using a dedicated road pavement crack dataset. Results show that compared to mainstream models such as SegFormer, HRNet, PSPNet, and FCN, the DTPC model achieves the highest MIoU score (86.72%) and F1 score (92.49%).

Hand Trajectory Recognition by Radar with a Finite-State Machine and a Bi-LSTM

Gesture plays an important role in human–machine interaction. However, the insufficient accuracy and high complexity of gesture recognition have blocked its widespread application. A gesture recognition method that combines state machine and bidirectional long short-term memory (Bi-LSTM) fusion neural network is proposed to improve the accuracy and efficiency. Firstly, gestures with large movements are categorized into simple trajectory gestures and complex trajectory gestures in advance. Afterwards, different recognition methods are applied for the two categories of gestures, and the final result of gesture recognition is obtained by combining the outputs of the two methods. The specific method used is a state machine that recognizes six simple trajectory gestures and a bidirectional LSTM fusion neural network that recognizes four complex trajectory gestures. Finally, the experimental results show that the proposed simple trajectory gesture recognition method has an average accuracy of 99.58%, and the bidirectional LSTM fusion neural network has an average accuracy of 99.47%, which can efficiently and accurately recognize 10 gestures with large movements. In addition, by collecting more gesture data from untrained participants, it was verified that the proposed neural network has good generalization performance and can adapt to the various operating habits of different users.

PointSurFace: Discriminative point cloud surface feature extraction for 3D face recognition

Due to the geometric information in the 3D face data, the 3D face recognition methods exhibit better robustness against the physical attacks compared to the 2D recognition methods. In this paper, we propose PointSurFace, a 3D face point cloud recognition method integrated with a specially designed facial surface feature extraction (SurF) module. In PointSurFace, SurF comprehensively combines the coordinate positions, normal vectors, and angle relationships, etc., of each patch in the 3D face point cloud, to explore the explicit local geometric information of 3D faces and generate facial surface features. Subsequently, the surface features are fed into an encoder consisting of four set abstraction (SA) modules and three Inverted Residual MLP (InvResMLP) modules to extract more discriminative 3D facial features, where InvResMLP could suppress the overfitting and reduce information loss in the model training. In addition, PointSurFace utilizes channel fusion operations to integrate position information into the extracted 3D facial features, further enhancing the model performance. Experimental results in 3D facial recognition and verification across multiple datasets demonstrate that PointSurFace achieves the best 3D face recognition performance to date. For example, on Lock3DFace, it achieves the state-of-the-art recognition and verification accuracy of 90.03% and 80.01%, respectively, surpassing the previous methods by 2.02% and 3.19%. The ablation studies evaluate each module of PointSurFace, suggesting that the proposed surface feature extraction module significantly enhances the model performance.