Pattern Recognition Framework Research Articles

Semantic-aware frame-event fusion based pattern recognition via large vision–language models

Pattern recognition through the fusion of RGB frames and Event streams has emerged as a novel research area in recent years. Current methods typically employ backbone networks to individually extract the features of RGB frames and event streams, and subsequently fuse these features for pattern recognition. However, we posit that these methods may suffer from two key issues: (1). They attempt to directly learn a mapping from the input vision modality to the semantic labels. This approach often leads to sub-optimal results due to the disparity between the input and semantic labels; (2). They utilize small-scale backbone networks for the extraction of RGB and Event input features, thus these models fail to harness the recent performance advancements of large-scale visual-language models. In this study, we introduce a novel pattern recognition framework that consolidates the semantic labels, RGB frames, and event streams, leveraging pre-trained large-scale vision–language models. Specifically, given the input RGB frames, event streams, and all the predefined semantic labels, we employ a pre-trained large-scale vision model (CLIP vision encoder) to extract the RGB and event features. To handle the semantic labels, we initially convert them into language descriptions through prompt engineering and polish using ChatGPT, and then obtain the semantic features using the pre-trained large-scale language model (CLIP text encoder). Subsequently, we integrate the RGB/Event features and semantic features using multimodal Transformer networks. The resulting frame and event tokens are further amplified using self-attention layers. Concurrently, we propose to enhance the interactions between text tokens and RGB/Event tokens via cross-attention. Finally, we consolidate all three modalities using self-attention and feed-forward layers for recognition. Comprehensive experiments on the HARDVS and PokerEvent datasets fully substantiate the efficacy of our proposed SAFE model. The source code has been released at https://github.com/Event-AHU/SAFE_LargeVLM.

A robust myoelectric pattern recognition framework based on individual motor unit activities against electrode array shifts

Background and objectiveElectrode shift is always one of the critical factors to compromise the performance of myoelectric pattern recognition (MPR) based on surface electromyogram (SEMG). However, current studies focused on the global features of SEMG signals to mitigate this issue but it is just an oversimplified description of the human movements without incorporating microscopic neural drive information. The objective of this work is to develop a novel method for calibrating the electrode array shifts toward achieving robust MPR, leveraging individual motor unit (MU) activities obtained through advanced SEMG decomposition. MethodsAll of the MUs from decomposition of SEMG data recorded at the original electrode array position were first initialized to train a neural network for pattern recognition. A part of decomposed MUs could be tracked and paired with MUs obtained at the original position based on spatial distribution of their MUAP waveforms, so as to determine the shift vector (describing both the orientation and distance of the shift) implicated consistently by these multiple MU pairs. Given the known shift vector, the features of the after-shift decomposed MUs were corrected accordingly and then fed into the network to finalize the MPR task. The performance of the proposed method was evaluated with data recorded by a 16 × 8 electrode array placed over the finger extensor muscles of 8 subjects performing 10 finger movement patterns. ResultsThe proposed method achieved a shift detection accuracy of 100 % and a pattern recognition accuracy approximating to 100 %, significantly outperforming the conventional methods with lower shift detection accuracies and lower pattern recognition accuracies (p < 0.05). ConclusionsOur method demonstrated the feasibility of using decomposed MUAP waveforms’ spatial distributions to calibrate electrode shift. This study provides a new tool to enhance the robustness of myoelectric control systems via microscopic neural drive information at an individual MU level.

Carbon footprint tracing and pattern recognition framework based on visual analytics

With growing concerns about global warming, industrial carbon footprints have garnered increased attention due to the energy-intensive and uninterrupted operation of industrial equipment. Fine-grained modeling and visual analytics of industrial carbon footprints can reveal the mechanisms behind the formation and evolution of carbon chains. However, the mechanisms underlying industrial carbon emissions remain unclear, leading to a lack of accuracy and specificity in current carbon quantification models. To address these gaps, we developed a comprehensive quantitative model that considers specific pathways involved in industrial processes, providing more accurate estimations of carbon emissions. We also designed an innovative visual analytical framework that uncovers implicit patterns and spatiotemporal distributions of industrial carbon footprints. By comparing our approach with state-of-the-art studies, we validated the superiority of our method in terms of its intuitiveness and interactivity. Empirical studies revealed potential emission patterns and spatiotemporal dynamics that traditional studies could not identify. We identified four consistent patterns in industrial carbon emissions: normal, high-emission, low-emission, and dedicated patterns. Our findings also led to optimization suggestions for different emission patterns, highlighting the system’s capability in extracting valuable insights from workshop carbon emission data. Our research showcases a unified visual analytical approach that supports exploratory analysis, and we believe it will uncover implicit knowledge within industrial carbon data, providing valuable insights for optimization.

Pattern recognition of urban traffic accidents: Application of cluster analysis and support vector machine in accident pattern recognition

Urban traffic accidents impose a significant threat to public safety because of its frequent occurrence and potential for severe injuries and fatalities. Hence, an effective analysis of accident patterns is crucial for designing accident prevention strategies. Recent advancement in data analytics have provided opportunities to improve the pattern of urban traffic accidents. However, the existing works face several challenges in adapting the complex dynamics, and heterogeneity of the accident data. To overcome these challenges, we proposed an innovative solution by combining the K-means clustering and Support Vector Machine to precisely predict the traffic accident patterns. By leveraging the efficiencies of clustering technique and machine learning, this work intends to identify the intricate patterns within the traffic database. Initially, a traffic accident database was collected and fed into the system. The collected database was pre-processed to improve and standardize the raw dataset. Further, cluster analysis is employed to identify distinct patterns within the dataset and group similar accidents into clusters. This clustering enables the system to recognize common accident scenarios and identify recent accident trends. Subsequently, a Support Vector Machine is deployed to classify accidents into distinct categories through intensive training with identified clusters. The combination enables the system to understand the complex relationships among diverse accident variables, making it an effective framework for real-time pattern recognition. The proposed strategy is implemented in Python and validated using the publicly available traffic accident database. The experimental results manifest that the proposed method achieved 99.65% accuracy, 99.53% precision, 99.62% recall, and 99.57% f-measure. Finally, the comparison with the existing techniques shows that the developed strategy offers improved accuracy, precision, recall, and f-measure compared to existing ones. shows that the developed strategy offers improved accuracy, precision, recall, and f-measure compared to existing ones.

Variable horizon multivariate driving pattern recognition framework based on vehicle-road two-dimensional information for electric vehicle

Driving condition is the core factor influencing vehicle energy management and power output strategies, hence realistic and accurate driving pattern recognition is crucial for realizing intelligent control of Electric Vehicles in complex scenarios. However, driving patterns mismatch with real scenarios caused by lack of road environment information and the reduced global recognition accuracy triggered by the fixed recognition horizon together constrain the development of intelligent control strategies. In this study, a framework for variable horizon multivariate driving pattern recognition strategy based on vehicle-road two-dimensional (2D) information is proposed. Specifically, multivariate driving pattern extraction is carried out based on real vehicle-road 2D data, and the extraction data serves as input for training offline driving pattern recognizer. To meet the challenge of large differences and frequent changes in driving conditions during the online application, a variable horizon recognition strategy is proposed based on the optimal horizon matching regularity. Finally, test results show that the proposed approach achieves an average recognition accuracy of 88.42% under different cycles, including 88.39% for complex driving cycles. Furthermore, the recognition accuracy is improved by 63.14% and 47.51% compared with the fixed long/short horizon. The framework can provide a basis for intelligent control strategies, thus broadening the Electric Vehicle application scenarios.

Improving the Machine Learning Performance for Image Recognition Using a New Set of Mountain Fourier Moments

The orthogonal moments are giving relevant results of these last years within the framework of object detection, pattern recognition and image reconstruction. This article is based on orthogonal functions called "Orthogonal Mountain functions (OMFs)" and we introduce a new set of moments called the multichannel Mountain Fourier moments (MMFMs), their performance is in reconstruction, noise invariants, rotation, scale and translation for image color. To validate these proposed techniques, we made several experimental tests to analyse images. We compare the results obtained from invariant moments and other current orthogonal invariant moments; the experiments show the power of the proposed moments.

An efficient deep learning framework for mixed-type wafer map defect pattern recognition

Defect detection on wafers holds immense significance in producing micro- and nano-semiconductors. As manufacturing processes grow in complexity, wafer maps may display a mixture of defect types, necessitating the utilization of more intricate deep learning models for effective feature learning. However, sophisticated models come with a demand for substantial computational resources. In this paper, we propose an efficient deep learning framework designed explicitly for mix-type wafer map defect pattern recognition. Our proposed model incorporates several crucial design elements, including lightweight convolutions, bottleneck residual connections, efficient channel attention mechanisms, and optimized activation functions, enabling it to learn spatial and channel features efficiently. We conduct evaluations on a real-world dataset containing 38 mixed-type defect patterns. The experimental results demonstrate that our framework maintains a high level of accuracy while possessing a compact parameter size and remarkably fast inference speed.

A novel framework for generic Spark workload characterization and similar pattern recognition using machine learning

Comprehensive workload characterization plays a pivotal role in comprehending Spark applications, as it enables the analysis of diverse aspects and behaviors. This understanding is indispensable for devising downstream tuning objectives, such as performance improvement. To address this pivotal issue, our work introduces a novel and scalable framework for generic Spark workload characterization, complemented by consistent geometric measurements. The presented approach aims to build robust workload descriptors by profiling only quantitative metrics at the application task-level, in a non-intrusive manner. We expand our framework for downstream workload pattern recognition by incorporating unsupervised machine learning techniques: clustering algorithms and feature selection. These techniques significantly improve the process of grouping similar workloads without relying on predefined labels. We effectively recognize 24 representative Spark workloads from diverse domains, including SQL, machine learning, web search, graph, and micro-benchmarks, available in HiBench. Our framework achieves a high accuracy F-Measure score of up to 90.9% and a Normalized Mutual Information of up to 94.5% in similar workload pattern recognition. These scores significantly outperform the results obtained in a comparative analysis with an established workload characterization approach in the literature.

USING OF THE MODELS IN PRACTICE-ORIENTED EDUCATION DURING LEARNING OF PEDIATRIC DENTISTRY

Teachers of clinical departments of establishments of higher medical education are faced with the problem of practical training of students, and many methods have been proposed for its improvement.&#x0D; The purpose of the study was to consider the models of practice-oriented education that can be used&#x0D; during the teaching of pediatric therapeutic dentistry.&#x0D; One Minute Preceptor model is used by the teacher during a 1-5-minute conversation with the student to guide diagnostic and treatment processes; initially a student examines a patient, then presents the clinical case, after which the teacher manages his behavioral reactions.&#x0D; SNAPPS model (Summarize + Narrow + Analyze + Probe + Plan + Select) describes 6 stages of the clinical training. The model includes analysis of the medical history and examination methods, choice of the diagnoses for differential diagnostic, proper differential diagnosis, questions to the teacher about uncertainties, discussion of patient’s treatment plan and questions and sources related to the clinical case for independent processing.&#x0D; Within the framework of pattern recognition and reproduction model (standard cases), student talks about the main problem that worries the patient and offers his diagnosis based on key symptoms during 30 seconds. Teacher personally examines patient, makes a diagnosis or discusses why it cannot be made with certainty, and evaluates the student’s actions.&#x0D; All described models of practice-oriented learning improve the expression of reasoning without much impact on the time spent for case presentation. Pattern recognition and reproduction model is especially useful in dentistry to recognize widespread clinical cases. One Minute Preceptor model, which involves the initiative and mentoring role of the teacher, fits well for beginners, while SNAPPS model, that focused on independent work of students, for strong students.

Optimal reconstruction and recognition of images by Jacobi Fourier moments and artificial bee colony (ABC) algorithm

The orthogonal moments giving relevant results of these last years within the framework of object detection, pattern recognition and image reconstruction, this work based on orthogonal functions called Orthogonal Jacobi Polynomials (OJPs), and we introduce a new set of moments called Generalized Jacobi Fourier Moments (GJFMs), these polynomials are characterized by parameters . However, it was very important to optimize these parameters in order to obtain a good result, in this context; this study used a new approach to optimized Jacobi Fourier parameters using the artificial bee colony algorithm (ABC) in order to improves the quality of reconstruction of images of large sizes. On the one hand, to validate this technique which offers a high image reconstruction quality. On other hand, the comparison carried out with other algorithms clearly indicates the advantage of the proposed method.

A Novel Approach to Surface EMG-based Gesture Classification Using a Vision Transformer Integrated with Convolutive Blind Source Separation.

A robust pattern recognition framework is required for ideal real-time human-machine interface (HMI) applications. Convolutional neural networks and recurrent neural networks have been widely used for the classification of gestures based on electromyography (EMG), but few studies have demonstrated the effectiveness of using a vision transformer for this purpose. Additionally, the accuracy achieved is influenced by the efficacy of the preprocessing pipeline. This study assessed ViT with and without an attention mechanism for precise motor intent decoding by investigating various input features and integrating convolutive blind source separation (BSS) preprocessing. All investigations were carried out with two open-access high-density surface EMG datasets of 34 and 21 hand gestures recorded from 20 and 5 healthy subjects respectively. Integration of centering and optimal extension factors resulted in better performance with raw input. However, spatial whitening increased the model's sensitivity to noise. The best-performing BSS-integrated convolution vision transformer model (BSS-CViT) model yielded an accuracy of 96.61% and 91.98% on test datasets one and two. This is a promising result for future studies in real-time HMI applications. The code implementation results reported in this study are available on GitHub. https://github.com/deremustapha/BSS-ViT.

Future Projections of Low‐Level Atmospheric Circulation Patterns Over South Tropical South America: Impacts on Precipitation and Amazon Dry Season Length

AbstractThe last few decades have shown evidence of a lengthening dry season in southern Amazonia, which is associated with a delay in the onset of the South American Monsoon System (SAMS). Using a pattern recognition framework of atmospheric circulation patterns (CPs), previous studies have identified specific atmospheric situations related to the onset of the SAMS. Here, we analyze the future changes in the CPs that largely define the main hydro‐climatological states of Tropical South America. We evaluated the CP changes that occurred between two periods: historical (1970–2000) and future (2040–2070), using six General Circulation Models (GCMs) from the Coupled Model Intercomparison Project Phase 6. Future GCM projections show significant spatio‐temporal changes in the CPs associated with the dry season in southern Amazonia during the mid‐21st century. These changes are related to both a late onset of the SAMS and an early demise of the SAMS. Particularly, the CP methodology allowed for a better understanding of the behavior of the southern Amazon dry season under future conditions, showing an increase in the frequency of the CPs typically observed during the dry season. The occurrence of dry days in the Amazon basin during the austral winter of the mid‐21st century increases by 19.4% on average, with respect to the historical period. This methodology also identified a future increase in the frequency of dry CPs, both at the beginning of the dry‐to‐wet transition period (8%) and at the end of the wet‐to‐dry transition season (11%).

Multi-day activity pattern recognition based on semantic embeddings of activity chains

Understanding residents' multi-day activity patterns is crucial for urban transportation planning. However, traditional travel surveys have limitations in observing the recurring regularity of residents' activities over multiple days, resulting in limited insights into long-term activity patterns. This study aims to provide a general and practical analytic framework for multi-day activity pattern recognition by fusing mobile phone data, travel survey data, and built environment data. First, a rule-based model is proposed to infer home/work activities, while a machine learning model is proposed to infer non-home/work activities of mobile phone users. Then, their multi-day activity chains are reconstructed with sequences of activity units which are labeled by inferred activity types. To measure the differences and similarities among these activity chains, this study develops a word embedding model to represent activity units and a sentence embedding model to represent activity chains in numerical vector space. Finally, regular multi-day activity patterns of mobile phone users can be recognized based on the clustering of vectorized activity chains. A case study in Shanghai is conducted to validate the proposed analytic framework for understanding the multi-day activity patterns of residents. The results indicate the effective recognition of 39 representative patterns of 7-day activity chains in Shanghai, reflecting the various lifestyles of residents. In summary, the main contribution of this study is to transform the problem of multi-day activity pattern recognition into similarity measurement of sentence vectors of activity chains by employing techniques from the field of natural language processing.

A Unified User-Generic Framework for Myoelectric Pattern Recognition: Mix-up and Adversarial Training for Domain Generalization and Adaptation.

To address cross-user variability problem in the myoelectric pattern recognition, a novel method for domain generalization and adaptation using both mix-up and adversarial training strategies, termed MAT-DGA, is proposed in this paper. This method enables integration of domain generalization (DG) with unsupervised domain adaptation (UDA) into a unified framework. The DG process highlights user-generic information in the source domain for training a model expected to be suitable for a new user in a target domain, where the UDA process further improves the model performance with a few unlabeled testing data from the new user. In this framework, both mix-up and adversarial training strategies were also applied to each of both the DG and UDA processes by exploiting their complementary advantages towards enhanced integration of both processes. The performance of the proposed method was evaluated via experiments of classifying seven hand gestures using high-density myoelectric data recorded from extensor digitorum muscles of eight intact-limbed subjects. It yielded a high accuracy of 95.71±4.17% and outperformed other UDA methods significantly (p<0.05) under cross-user testing scenarios. Moreover, it reduced the number of calibration samples required in the UDA process (p<0.05) after its initial performance had already been lifted by the DG process. The proposed method provides an effective and promising way of establishing cross-user myoelectric pattern recognition control systems. Our work helps to promote development of user-generic myoelectric interfaces, with wide applications in motor control and health.

Machine Learning–Based Time in Patterns for Blood Glucose Fluctuation Pattern Recognition in Type 1 Diabetes Management: Development and Validation Study

Background Continuous glucose monitoring (CGM) for diabetes combines noninvasive glucose biosensors, continuous monitoring, cloud computing, and analytics to connect and simulate a hospital setting in a person’s home. CGM systems inspired analytics methods to measure glycemic variability (GV), but existing GV analytics methods disregard glucose trends and patterns; hence, they fail to capture entire temporal patterns and do not provide granular insights about glucose fluctuations. Objective This study aimed to propose a machine learning–based framework for blood glucose fluctuation pattern recognition, which enables a more comprehensive representation of GV profiles that could present detailed fluctuation information, be easily understood by clinicians, and provide insights about patient groups based on time in blood fluctuation patterns. Methods Overall, 1.5 million measurements from 126 patients in the United Kingdom with type 1 diabetes mellitus (T1DM) were collected, and prevalent blood fluctuation patterns were extracted using dynamic time warping. The patterns were further validated in 225 patients in the United States with T1DM. Hierarchical clustering was then applied on time in patterns to form 4 clusters of patients. Patient groups were compared using statistical analysis. Results In total, 6 patterns depicting distinctive glucose levels and trends were identified and validated, based on which 4 GV profiles of patients with T1DM were found. They were significantly different in terms of glycemic statuses such as diabetes duration (P=.04), glycated hemoglobin level (P&lt;.001), and time in range (P&lt;.001) and thus had different management needs. Conclusions The proposed method can analytically extract existing blood fluctuation patterns from CGM data. Thus, time in patterns can capture a rich view of patients’ GV profile. Its conceptual resemblance with time in range, along with rich blood fluctuation details, makes it more scalable, accessible, and informative to clinicians.

A generalized adaptive robust distance metric driven smooth regularization learning framework for pattern recognition

In this work, we propose a novel generalized adaptive robust distance metric called Lδ(u). Compared with other distance metrics, Lδ(u) has some desirable salient properties, such as symmetry, boundedness, robustness, nonconvexity, and adaptivity, with both first-order and higher-order moments from samples. On the other hand, Lδ(u) can pick different robust distance metrics for different learning tasks during the learning process by the adaptive parameter δ. Furthermore, we apply Lδ(u) to twin extreme learning machine (TELM) and develop an smooth regularized TELM learning framework for supervised and semi-supervised classification. By introducing the structural risk minimization (SRM) principle and smoothing techniques, the learning framework perfectly overcomes the computational burden associated with the matrix inversion operation required during TELM solving, while also significantly improving performance. More importantly, the proposed learning framework not only improves the robustness of TELM, but also can effectively use the geometric information of the marginal distribution embedded in the unlabeled samples to construct a more reasonable classifier. Finally, the globally convergent and quadratic convergent fast Newton-Armijo algorithm and DC (difference of convex functions) programming algorithm (DCA) are designed to solve the proposed methods. Experimental results demonstrate the effectiveness and robustness of our methods.

Unfavorable geology recognition in front of shallow tunnel face using machine learning

Subsoil profile mapping is typically based on spatially discrete borehole logs obtained from site geotechnical investigations. During the mapping, soil information between two boreholes is indeed unknown and must be interpolated. The interpolation is not always able to identify geological anomalies, posing additional risks to tunneling. To supplement this need, this study develops a time-efficient pattern recognition framework for detecting potential geological anomalies ahead of the tunnel face during the advancement of Tunnel Boring Machine (TBM). Three basic unfavorable geology scenarios are defined, including a soil type change with varying inclination, an interlayer, and a boulder. A series of three-dimensional finite element shallow tunneling models are built to generate synthetic spatial and temporal data of ground settlements and pore water pressures during tunneling with respect to the three basic scenarios. Three machine learning methods, i.e., random forest (RF), support vector machine (SVM), and artificial neural network (ANN) are developed to recognize the geological anomalies using inputs comprised of the synthetic data combined with Gaussian noise that is added to enhance regularization and model robustness. The result shows that the proposed pattern recognition approach exhibits a powerful capability in classifying the unfavorable geology as early as 4.5-8D (D represents tunnel diameter) ahead of the target anomalies. Finally, the efficiency and effectiveness of the proposed pattern recognition framework are demonstrated in a real tunneling project. This study is of value in the mitigation of tunneling risks.

Progressive Tandem Learning for Pattern Recognition With Deep Spiking Neural Networks.

Spiking neural networks (SNNs) have shown clear advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency, due to their event-driven nature and sparse communication. However, the training of deep SNNs is not straightforward. In this paper, we propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition, which is referred to as progressive tandem learning. By studying the equivalence between ANNs and SNNs in the discrete representation space, a primitive network conversion method is introduced that takes full advantage of spike count to approximate the activation value of ANN neurons. To compensate for the approximation errors arising from the primitive network conversion, we further introduce a layer-wise learning method with an adaptive training scheduler to fine-tune the network weights. The progressive tandem learning framework also allows hardware constraints, such as limited weight precision and fan-in connections, to be progressively imposed during training. The SNNs thus trained have demonstrated remarkable classification and regression capabilities on large-scale object recognition, image reconstruction, and speech separation tasks, while requiring at least an order of magnitude reduced inference time and synaptic operations than other state-of-the-art SNN implementations. It, therefore, opens up a myriad of opportunities for pervasive mobile and embedded devices with a limited power budget.

DLPR: Deep Learning-Based Enhanced Pattern Recognition Frame-Work for Improved Myoelectric Prosthesis Control

In EMG based pattern recognition (EMG-PR), deep learning-based techniques have become more prominent for their self-regulating capability to extract discriminant features from large data-sets. Moreover, the performance of traditional machine learning-based methods show limitation to categorize over a certain number of classes and degrades over a period of time. In this paper, an accurate, robust, and fast convolutional neural network-based framework for EMG pattern identification is presented. To assess the performance of the proposed system, five publicly available and benchmark data-sets of upper limb activities were used. This data-set contains 49 to 52 upper limb motions (NinaPro DB1, NinaPro DB2, and NinaPro DB3), Data with force variation, and data with arm position variation for intact and amputated subjects. The classification accuracies of 92.18% (53 classes), 91.56% (49 classes), 84.98% (49 classes of amputees), 95.67% (6 classes with force variation), and 99.12% (8 classes with arm position variation) have been observed during the testing. The performance of the proposed system is compared with the state of art techniques in the literature. The findings demonstrate that classification accuracy and time complexity have been improved significantly. For signal pre-processing and deep learning techniques, Keras which is a high-level API for TensorFlow is utilised to build profound learning models. The proposed method has been tested on the Intel Core i7 3.5GHz, 7th Gen CPU with 8GB DDR4 RAM.

A novel pattern recognition framework based on ensemble of handcrafted features on images

A novel pattern recognition framework based on ensemble of handcrafted features on images