Automatic Recognition Research Articles

MoSFPAD: An end-to-end ensemble of MobileNet and Support Vector Classifier for fingerprint presentation attack detection

Automatic fingerprint recognition systems are the most extensively used systems for person authentication although they are vulnerable to Presentation attacks. Artificial artifacts created with the help of various materials are used to deceive these systems causing a threat to the security of fingerprint-based applications. This paper proposes a novel end-to-end model to detect fingerprint Presentation attacks. The proposed model incorporates MobileNet as a feature extractor and a Support Vector Classifier as a classifier to detect presentation attacks in cross-material and cross-sensor paradigms. The feature extractor’s parameters are learned with the loss generated by the support vector classifier. The proposed model eliminates the need for intermediary data preparation procedures, unlike other static hybrid architectures. The performance of the proposed model has been validated on benchmark LivDet 2011, 2013, 2015, 2017, and 2019 databases, and overall accuracy of 98.64%, 99.50%, 97.23%, 95.06%, and 95.20% are achieved on these databases, respectively. The performance of the proposed model is compared with state-of-the-art methods and is able to reduce the average classification error of 3.63%, 1.86%, 1.83%, 0.05%, 0.93% on LivDet 2011, 2013, 2015, 2017, and 2019 databases, respectively for same and cross material protocols in intra-sensor paradigm. The proposed method also reduced the average classification error of 1.59%, 1.41%, and 2.29% for LivDet 2011, 2013, and 2017 databases, respectively for the cross-sensor paradigm. It is evident from the results that the proposed method outperforms state-of-the-art methods in intra-sensor as well as cross-sensor paradigms in terms of average classification error.

Towards Integrating Automatic Emotion Recognition in Education: A Deep Learning Model Based on 5 EEG Channels

In a technologically advanced world, artificial intelligence has impacted all fields of activity. The augmentation of online learning by means of emotion recognition systems raises new challenges in terms of obtaining high-performance systems and in interpreting the results. The paper aims to investigate the usage of automated emotion recognition in learning and to develop a deep learning model based on physiological data to recognize emotions often encountered in classrooms. So, an 1D-CNN model based on physiological data is used to recognize seven emotions: boredom, confusion, frustration, curiosity, excitement, concentration, and anxiety. These emotions are described according to the PAD model and the 5 EEG signals, FP1, AF3, F7, T7, FP2, are taken from the DEAP dataset to train and to test the convolutional neural network model. The high accuracy we obtained (i.e. boredom—99.64%, confusion—99.70%, frustration—99.66%, curiosity—99.80%, excitement—99.91%, concentration—99.70%, anxiety—99.21%) proves that the use of signals obtained via only five channels is sufficient to recognize the presence of emotions. Furthermore, an improved method of analysis based on LIME is proposed and used to obtain reliable explanations for the predictions of our model.

Comparative analysis of artificial intelligence and expert assessments in detecting neonatal procedural pain

Assessing pain in newborns in the NICU is crucial due to their frequent exposure to painful stimuli, yet it's challenging due to the subjective nature of current methods. This study aimed to evaluate the effectiveness of an AI system designed for automatic facial recognition by comparing its performance with the expert opinion of health care provider. This is a secondary analysis from an eye-tracking study, assessing neonatal pain evaluations by healthcare professionals. The performance of AI software, FaceReader 9, was compared to experts' evaluations using a visual-analog scale, focusing on identifying specific facial action units associated with different pain levels. The study found significant differences in AI-generated metrics—arousal and valence—across three stimulus types: non-noxious thermal, short-noxious, and prolonged-noxious, with p-values below 0.001. A strong correlation (r = 0.84, p ≤ .001) was observed between AI metrics and expert ratings. Eleven facial action units were identified as relevant to describe neonatal pain. The findings highlight the AI system's potential in accurately detecting and analyzing newborn facial expressions in response to varying pain intensities, demonstrating a significant correlation with healthcare professionals' assessments. This suggests that AI technology could enhance objective pain assessment in neonates.

Automatic classification of bird species related to power line faults using deep convolution features and ECOC‐SVM model

AbstractBird‐related outages greatly threaten the safety of overhead transmission and distribution lines, while electrocution and collisions of birds with power lines, especially endangered species, are significant environmental concerns. Automatic bird recognition can be helpful to mitigate this contradiction. This paper proposes a method for automatic classification of bird species related to power line faults combining deep convolution features with error‐correcting output codes support vector machine (ECOC‐SVM). An image dataset of about 20 high‐risk and 20 low‐risk bird species was constructed, and the feed‐forward denoising convolutional neural network was used for image preprocessing. The deep convolution features of bird images were extracted by DarkNet‐53, and taken as inputs of the ECOC‐SVM for model training and bird species classification. The gradient‐weighted class activation mapping was used for visual explanations of the model decision region. The experimental results indicate that the average accuracy of the proposed method can reach 94.39%, and its performance was better than other models using different feature extraction networks and classification algorithms.

Barcode-less Fruits Classification Using Deep Learning

<p>Barcode-less fruit recognition technology has revolutionized the checkout process by eliminating manual barcode scanning. This technology automatically identifies and adds fruit items to the purchase list, significantly reducing waiting times at the cash register. Faster checkouts enhance customer convenience and optimize operational efficiency for retailers. Adding barcode to fruits require using adhesives on the fruit surface that may cause health hazards. Leveraging deep learning techniques for barcode-less fruit recognition brings valuable advantages to industries, including advanced automation, enhanced accuracy, and increased efficiency. These benefits translate into improved productivity, cost reduction, and superior quality control. This study introduces a Convolutional Neural Network (CNN) designed explicitly for automatic fruit recognition, even in challenging real-world scenarios. The proposed method assists fruit sellers in accurately identifying and distinguishing between different types of fruit that may exhibit similarities. A dataset that includes 44,406 images of different fruit types is used to train and test our technique. Employing a CNN, the developed model achieves an impressive classification accuracy of 97.4% during the training phase and 88.6% during the testing phase respectively, showcasing its effectiveness in precise fruit recognition.</p>

276. AI NAVIGATION SURGERY FOR ESOPHAGEAL CANCER

Abstract With recent technological innovations, artificial intelligence (AI) has begun to be used not only in endoscopic diagnosis and pathology diagnosis but also in the field of surgery. AI enables objective and consistent recognition and evaluation, making it suitable for surgical assistance in surgical operations as well as for the evaluation of surgical techniques and learning curves. First, an AI-assisted surgical procedure recognition system for robotic esophagectomy was developed. By focusing on surgical procedures, the system automatically recognizes surgical procedures with 84% accuracy using AI. In particular, more than 90% of accuracy was obtained in the phase for Left RLN lymph node dissection and Post-dissection to completion of surgery. In addition, as intraoperative navigation, a system for automatic recognition of critical organs such as the recurrent laryngeal nerve, trachea, and subclavian artery was developed, and its recognition accuracy is gradually improving, enabling organ recognition in real time. Currently, with data from more than 100 cases, the system is able to predict Intersection over Union (IoU) with an accuracy of 0.3-0.4. Similar developments are underway in other areas, such as automatic recognition of the surgical process in robotic distal gastrectomy, and a surgical difficulty prediction system is being built using this technology. This system is expected to make it possible to automatically determine the degree of difficulty of surgery during the operation, leading to the decision to change the surgical team during the operation and the prediction of the operation completion time. These technologies can be useful not only for intraoperative navigation, but also surgical education and postoperative care. Using them, young surgeons have the opportunity to dive into authentic scenarios and acquire practical experience safely, bypassing the dangers inherent in real-life surgeries. Through the use of AI-powered interactive simulations, they can deeply grasp surgical methods, establishing the groundwork of crucial practical abilities needed for performing successful operations. In the future, we intend to verify the external validity of these systems using data from multiple facilities, and further work on the development of a surgical support system that transcends fields, with the aim of obtaining regulatory approval as a medical device. In addition to the potential clinical applications of the systems under development, we would like to discuss the current status and future of AI-based surgical support systems in the surgical field.

Morphological dictionary learning based sparse classification for small electric motor state recognition under unbalanced samples

Accurately recognizing sound states in the production line of small electric motors is of great importance for manufacturers to carry out quick repairs and ensure high quality deliveries. Since the number of normal samples is much larger than the number of abnormal samples in practice, resulting in unbalanced data, which poses huge challenges to traditional detection methods. To overcome these difficulties, this study presents a morphological dictionary learning-based sparse classification (MDL-SC) combined with audio data augmentation method for small electric motor state recognition under unbalanced samples. Firstly, audio data augmentation methods such as adding background noise, pitch shifting, time stretching and combined augmentation are investigated for augmenting the number and diversity of samples. Secondly, morphological dictionary learning is proposed for characterizing transient sounds of small electric motors and enhancing the discriminative feature learning capability of the dictionary. Finally, the minimum reconstruction error strategy is relied upon to establish automatic recognition of small electric motor states. Three small motor datasets with unbalanced ratios are established in the experiments to verify the effectiveness of the proposed MDL-SC, which has higher recognition accuracy under unbalanced conditions compared with traditional dictionary learning based sparse classification (DL-SC), k-nearest neighbors, support vector machines and convolutional neural networks. This study can provide some theoretical implications for the later development of online detection of small electric motors or other types of electric motors.

Computer vision in the food industry: Accurate, real-time, and automatic food recognition with pretrained MobileNetV2

In contemporary society, the application of artificial intelligence for automatic food recognition offers substantial potential for nutrition tracking, reducing food waste, and enhancing productivity in food production and consumption scenarios. Cutting-edge technologies such as Computer Vision and Deep Learning are highly beneficial, enabling machines to learn automatically, thereby facilitating automatic visual recognition. Despite some research in this field, the challenge of achieving accurate automatic food recognition quickly remains a significant research gap. Some models have been developed and implemented, but maintaining high performance swiftly, with low computational cost and low access to expensive hardware accelerators, is still an area that needs further exploration and improvement. This study employs the pre-trained MobileNetV2 model, which is cost-efficient and fast, for automatic food recognition on the public Food11 dataset, comprising 16643 images. It also utilizes various tools and techniques such as dataset understanding, transfer learning, data augmentation, regularization, dynamic learning rate, hyperparameter tuning, and consideration of images in different sizes to cut time and enhance the model’s performance and robustness. These techniques aid in choosing appropriate metrics, achieving better performance, avoiding overfitting and accuracy fluctuations, speeding up the model, and increasing the generalization of findings, making the study and its results applicable to real-world applications. Despite MobileNetV2′s simpler structure and fewer trainable parameters compared to some deep-dense models in the deep learning area, it quickly achieved approximately 92.97 % accuracy. This underscores the model’s potential for implementation in real-world applications, which is the main intention of this study.

Automatic recognition system for concrete cracks with support vector machine based on crack features

Cracks are a common problem in concrete surfaces. With the continuous optimization of machine vision-based inspection systems, effective crack detection and recognition is the core of the entire system. In this study, support vector machine (SVM) was used to distinguish cracks from other regions. To complete the recognition system of the SVM, a framework consisting of an image processing and recognition model was proposed. An algorithm combining the Prewitt operator with the Otsu threshold was proposed for image segmentation. The binary image processed by the new algorithm combined with mathematical morphology can result in a more complete crack zone and fewer interference regions. After the initial parameter extraction, most of the impurity areas were screened by preliminary discrimination, removing 99% of the impurities. This processing step ensured the balance and effectiveness of the samples. To establish an automatic identification model based on SVM with a radial basis function, compactness, occupancy rate, and length–width ratio were selected as input parameters after comparing these three features with all six features of the crack. The recognition accuracy of this system reaches 97.14%, demonstrating that the proposed method is effective and satisfies practical requirements.

Paper-Recorded ECG Digitization Method with Automatic Reference Voltage Selection for Telemonitoring and Diagnosis.

In electrocardiograms (ECGs), multiple forms of encryption and preservation formats create difficulties for data sharing and retrospective disease analysis. Additionally, photography and storage using mobile devices are convenient, but the images acquired contain different noise interferences. To address this problem, a suite of novel methodologies was proposed for converting paper-recorded ECGs into digital data. Firstly, this study ingeniously removed gridlines by utilizing the Hue Saturation Value (HSV) spatial properties of ECGs. Moreover, this study introduced an innovative adaptive local thresholding method with high robustness for foreground-background separation. Subsequently, an algorithm for the automatic recognition of calibration square waves was proposed to ensure consistency in amplitude, rather than solely in shape, for digital signals. The original signal reconstruction algorithm was validated with the MIT-BIH and PTB databases by comparing the difference between the reconstructed and the original signals. Moreover, the mean of the Pearson correlation coefficient was 0.97 and 0.98, respectively, while the mean absolute errors were 0.324 and 0.241, respectively. The method proposed in this study converts paper-recorded ECGs into a digital format, enabling direct analysis using software. Automated techniques for acquiring and restoring ECG reference voltages enhance the reconstruction accuracy. This innovative approach facilitates data storage, medical communication, and remote ECG analysis, and minimizes errors in remote diagnosis.

Unveiling the human-like similarities of automatic facial expression recognition: An empirical exploration through explainable ai

AbstractFacial expression recognition is vital for human behavior analysis, and deep learning has enabled models that can outperform humans. However, it is unclear how closely they mimic human processing. This study aims to explore the similarity between deep neural networks and human perception by comparing twelve different networks, including both general object classifiers and FER-specific models. We employ an innovative global explainable AI method to generate heatmaps, revealing crucial facial regions for the twelve networks trained on six facial expressions. We assess these results both quantitatively and qualitatively, comparing them to ground truth masks based on Friesen and Ekman’s description and among them. We use Intersection over Union (IoU) and normalized correlation coefficients for comparisons. We generate 72 heatmaps to highlight critical regions for each expression and architecture. Qualitatively, models with pre-trained weights show more similarity in heatmaps compared to those without pre-training. Specifically, eye and nose areas influence certain facial expressions, while the mouth is consistently important across all models and expressions. Quantitatively, we find low average IoU values (avg. 0.2702) across all expressions and architectures. The best-performing architecture averages 0.3269, while the worst-performing one averages 0.2066. Dendrograms, built with the normalized correlation coefficient, reveal two main clusters for most expressions: models with pre-training and models without pre-training. Findings suggest limited alignment between human and AI facial expression recognition, with network architectures influencing the similarity, as similar architectures prioritize similar facial regions.

A facial expression recognition network based on attention double branch enhanced fusion.

The facial expression reflects a person's emotion, cognition, and even physiological or mental state to a large extent. It has important application value in medical treatment, business, criminal investigation, education, and human-computer interaction. Automatic facial expression recognition technology has become an important research topic in computer vision. To solve the problems of insufficient feature extraction, loss of local key information, and low accuracy in facial expression recognition, this article proposes a facial expression recognition network based on attention double branch enhanced fusion. Two parallel branches are used to capture global enhancement features and local attention semantics respectively, and the fusion and complementarity of global and local information is realized through decision-level fusion. The experimental results show that the features extracted by the network are made more complete by fusing and enhancing the global and local features. The proposed method achieves 89.41% and 88.84% expression recognition accuracy on the natural scene face expression datasets RAF-DB and FERPlus, respectively, which is an excellent performance compared with many current methods and demonstrates the effectiveness and superiority of the proposed network model.

CTRNet: An Automatic Modulation Recognition Based on Transformer-CNN Neural Network

Deep learning (DL) has brought new perspectives and methods to automatic modulation recognition (AMR), enabling AMR systems to operate more efficiently and reliably in modern wireless communication environments through its powerful feature learning and complex pattern recognition capabilities. However, convolutional neural networks (CNNs) and recurrent neural networks (RNNs), which are used for sequence recognition tasks, face two main challenges, respectively: the ineffective utilization of global information and slow processing speeds due to sequential operations. To address these issues, this paper introduces CTRNet, a novel automatic modulation recognition network that combines a CNN with Transformer. This combination leverages Transformer’s ability to adequately capture the long-distance dependencies between global sequences and its advantages in sequence modeling, along with the CNN’s capability to extract features from local feature regions of signals. During the data preprocessing stage, the original IQ-modulated signals undergo sliding-window processing. By selecting the appropriate window sizes and strides, multiple subsequences are formed, enabling the network to effectively handle complex modulation patterns. In the embedding module, token vectors are designed to integrate information from multiple samples within each window, enhancing the model’s understanding and modeling ability of global information. In the feedforward neural network, a more effective Bilinear layer is employed for processing to capture the higher-order relationship between input features, thereby enhancing the ability of the model to capture complex patterns. Experiments conducted on the RML2016.10A public dataset demonstrate that compared with the existing algorithms, the proposed algorithm not only exhibits significant advantages in terms of parameter efficiency but also achieves higher recognition accuracy under various signal-to-noise ratio (SNR) conditions. In particular, it performs relatively well in terms of accuracy, precision, recall, and F1-score, with clearer classification of higher-order modulations and notable overall accuracy improvement.

A dataset of acoustic measurements from soundscapes collected worldwide during the COVID-19 pandemic

Political responses to the COVID-19 pandemic led to changes in city soundscapes around the globe. From March to October 2020, a consortium of 261 contributors from 35 countries brought together by the Silent Cities project built a unique soundscape recordings collection to report on local acoustic changes in urban areas. We present this collection here, along with metadata including observational descriptions of the local areas from the contributors, open-source environmental data, open-source confinement levels and calculation of acoustic descriptors. We performed a technical validation of the dataset using statistical models run on a subset of manually annotated soundscapes. Results confirmed the large-scale usability of ecoacoustic indices and automatic sound event recognition in the Silent Cities soundscape collection. We expect this dataset to be useful for research in the multidisciplinary field of environmental sciences.

Open-Set Recognition Model for SAR Target Based on Capsule Network with the KLD

Synthetic aperture radar (SAR) automatic target recognition (ATR) technology has seen significant advancements. Despite these advancements, the majority of research still operates under the closed-set assumption, wherein all test samples belong to classes seen during the training phase. In real-world applications, however, it is common to encounter targets not previously seen during training, posing a significant challenge to the existing methods. Ideally, an ATR system should not only accurately identify known target classes but also effectively reject those belonging to unknown classes, giving rise to the concept of open set recognition (OSR). To address this challenge, we propose a novel approach that leverages the unique capabilities of the Capsule Network and the Kullback-Leibler divergence (KLD) to distinguish unknown classes. This method begins by deeply mining the features of SAR targets using the Capsule Network and enhancing the separability between different features through a specially designed loss function. Subsequently, the KLD of features between a testing sample and the center of each known class is calculated. If the testing sample exhibits a significantly larger KLD compared to all known classes, it is classified as an unknown target. The experimental results of the SAR-ACD dataset demonstrate that our method can maintain a correct identification rate of over 95% for known classes while effectively recognizing unknown classes. Compared to existing techniques, our method exhibits significant improvements.

Assessing Usefulness, Ease of Use and Recognition Performance of Semi-Automatic Mulsemedia Authoring

Mulsemedia (Multiple Sensorial Media) authoring poses a considerable challenge as authors navigate the intricate task of identifying moments to activate sensory effects within multimedia content. A novel proposal is to integrate content recognition algorithms that use machine learning (ML) into authoring tools to alleviate the authoring effort. As author subjectivity is very important, it is imperative to allow users to define which sensory effects should be automatically extracted. This paper conducts a twofold evaluation of the proposed semi-automatic authoring. The first is from a user perspective within the STEVE 2.0 mulsemedia authoring tool, employing the Goal-Question-Metric (GQM) methodology and a user feedback questionnaire. Our user evaluation indicates that users perceive the semi-automatic authoring approach as a positive enhancement to the authoring process. The second evaluation targets sensory effect recognition using two different content recognition modules, quantifying their automatic recognition capabilities against manual authoring. Metrics such as precision, recall, and F1 scores provide insights into the strengths and nuances of each module. Differences in label assignments underscore the need for ML module result combination methodologies. These evaluations contribute to a comprehensive understanding of the effectiveness of sensory effect recognition modules in enhancing mulsemedia content authoring.

An Interpretable Target-Aware Vision Transformer for Polarimetric HRRP Target Recognition with a Novel Attention Loss

Polarimetric high-resolution range profile (HRRP), with its rich polarimetric and spatial information, has become increasingly important in radar automatic target recognition (RATR). This study proposes an interpretable target-aware vision Transformer (ITAViT) for polarimetric HRRP target recognition with a novel attention loss. In ITAViT, we initially fuse the polarimetric features and the amplitude of polarimetric HRRP with a polarimetric preprocessing layer (PPL) to obtain the feature map as the input of the subsequent network. The vision Transformer (ViT) is then used as the backbone to automatically extract both local and global features. Most importantly, we introduce a novel attention loss to optimize the alignment between the attention map and the HRRP span. Thus, it can improve the difference between the target and the background, and enable the model to more effectively focus on real target areas. Experiments on a simulated X-band dataset demonstrate that our proposed ITAViT outperforms comparative models under various experimental conditions. Ablation studies highlight the effectiveness of polarimetric preprocessing and attention loss. Furthermore, the visualization of the self-attention mechanism suggests that attention loss enhances the interpretability of the network.

Study on automatic and rapid diagnosis of distal radius fracture by X-ray

This article aims to combine deep learning with image analysis technology and propose an effective classification method for distal radius fracture types. Firstly, an extended U-Net three-layer cascaded segmentation network was used to accurately segment the most important joint surface and non joint surface areas for identifying fractures. Then, the images of the joint surface area and non joint surface area separately were classified and trained to distinguish fractures. Finally, based on the classification results of the two images, the normal or ABC fracture classification results could be comprehensively determined. The accuracy rates of normal, A-type, B-type, and C-type fracture on the test set were 0.99, 0.92, 0.91, and 0.82, respectively. For orthopedic medical experts, the average recognition accuracy rates were 0.98, 0.90, 0.87, and 0.81, respectively. The proposed automatic recognition method is generally better than experts, and can be used for preliminary auxiliary diagnosis of distal radius fractures in scenarios without expert participation.

Automatic recognition of active landslides by surface deformation and deep learning

Catastrophic landslides are generally evolved from potential active landslides, and early identification of active landslides over an extensive region is vital to effective prevention and control of disastrous landslides in urban areas. Interferometric Synthetic Aperture Radar (InSAR) has immense potential in mapping active landslides. However, artificial interpretation of InSAR measurements and manual recognition of active landslides are very laborious and time-consuming, with a relatively high missing and false alarms. That hinders the application of InSAR technique and the identification of active landslides in wide areas. Automatic recognition of active landslides has always been a great challenge and has been relatively rarely investigated by previous studies. This work establishes comprehensive identification indices of geoenvironmental, disaster-triggering, and surface deformation features. Moreover, it suggests a novel deep learning algorithm of SDeepFM to conduct automatic identification of active landslides across a vast and landslide-serious area of Hualong County. Some new viewpoints are suggested as follows. (1) The identification indices consist of disaster-controlling, disaster-inducing, and active deformation characteristics and are constructed in terms of the cause characteristics of active landslides. Thus, it can effectively decrease the false alarms of active landslide identification. (2) The proposed SDeepFM algorithm features a spatial-perception ability and can adequately extract and fuse the low-level and high-level semantic features. It outperforms the classification and regression tree (CART), multi-layer perceptron (MLP), convolutional neural network (CNN), and deep neural network (DNN) algorithms. The test accuracy attains 0.91, 99.73%, 90.21%, 0.92, 0.96, and 0.91 in F1-score, Accuracy, Precision, Recall, AUC, and Kappa, respectively. (3) A total of 164 active landslides are exactly recognized, and 39 active landslides are newly identified in this work. (4) In Hualong County, the characteristics of slope deformation, spatial context, lithology, tectonic movement, human activity, and topography play important roles in active landslide identification. River distribution and rainfall also contribute to active landslide recognition.

Kyrtos: A methodology for automatic deep analysis of graphic charts with curves in technical documents

Deep Understanding of Technical Documents (DUTD) has become a very attractive field with great potential due to large amounts of accumulated documents and the valuable knowledge contained in them. In addition, the holistic understanding of technical documents depends on the accurate analysis of its particular modalities, such as graphics, tables, diagrams, text, etc. and their associations. In this paper, we introduce the Kyrtos methodology for the automatic recognition and analysis of charts with curves in graphics images of technical documents. The recognition processing part adopts a clustering based approach to recognize middle-points that delimit the line-segments that construct the illustrated curves. The analysis processing part parses the extracted line-segments of curves to capture behavioral features such as direction, trend and etc. These associations assist the conversion of recognized segments’ relations into attributed graphs, for the preservation of the curves’ structural characteristics. The graph relations are also are expressed into natural language (NL) text sentences, enriching the document’s text and facilitating their conversion into Stochastic Petri-net (SPN) graphs, which depict the internal functionality represented in the chart image. Extensive evaluation results demonstrate the accuracy of Kyrtos’ recognition and analysis methods by measuring the structural similarity between input chart curves and the approximations generated by Kyrtos for charts with multiple functions.