Recognition Framework Research Articles

Subsampled Randomized Hadamard Transformation-based Ensemble Extreme Learning Machine for Human Activity Recognition

Extreme Learning Machine (ELM) is becoming a popular learning algorithm due to its diverse applications, including Human Activity Recognition (HAR). In ELM, the hidden node parameters are generated at random, and the output weights are computed analytically. However, even with a large number of hidden nodes, feature learning using ELM may not be efficient for natural signals due to its shallow architecture. Due to noisy signals of the smartphone sensors and high dimensional data, substantial feature engineering is required to obtain discriminant features and address the “curse-of-dimensionality”. In traditional ML approaches, dimensionality reduction and classification are two separate and independent tasks, increasing the system’s computational complexity. This research proposes a new ELM-based ensemble learning framework for human activity recognition to overcome this problem. The proposed architecture consists of two key parts: (1) Self-taught dimensionality reduction followed by classification. (2) they are bridged by “Subsampled Randomized Hadamard Transformation” (SRHT). Two different HAR datasets are used to establish the feasibility of the proposed framework. The experimental results clearly demonstrate the superiority of our method over the current state-of-the-art methods.

Synergetic Multichiral Covalent Organic Framework for Enantioselective Recognition and Separation.

In enantiomer recognition and separation, a highly enantioselective approach with universal applicability is urgently desired but hard to realize, especially in the case of chiral molecules. To resolve the trade-off between enantioselectivity and universality, a glutathione (GSH) and methylated cyclodextrins (MCD)-functionalized covalent organic framework (GSH-MCD COF) with porosity and abundant chiral surfaces is presented that was designed and synthesized for recognition and separation of various enantiomers. As expected, the GSH-MCD COF can be used as chiral stationary phases for the separation of various enantiomers, including aromatic alcohols, aromatic acids, amides, amino acids, and organic acids, with performance and versatility even superior to some widely used commercial chiral chromatographic columns. Furthermore, the synthesized GSH-MCD COF shows high enantioselectivity for the rapid recognition and identification of enantiomers and chiral metabolites when coupling to Raman spectroscopy. Molecular simulations suggest that the COF provides a confined microenvironment for cyclodextrins and peptides that dictates the separation and recognition capability. This work provides a strategy to synthesize synergetic multichiral COF and achieve separations and recognitions of enantiomers in complex samples.

A Review of Pattern Recognition and Machine Learning

This article aims to provide a concise overview of diverse methodologies employed at different stages of a pattern recognition system, highlighting contemporary research challenges and applications in this dynamic field. The integration of machine learning techniques has played a pivotal role in converging pattern recognition frameworks in academic literature. The process relies heavily on supervised or unsupervised categorization methods to achieve its objectives, with a notable focus on statistical approaches. More recently, there is a growing emphasis on incorporating neural network methodologies and insights from statistical learning theory. Designing an effective recognition system necessitates careful consideration of various factors, including pattern representation, pattern class definition, feature extraction, sensing environment, feature selection, classifier learning and design, cluster analysis, test and training sample selection, and performance assessment.

Design and Implementation of Novel Hybrid and Multiscale- Assisted CNN and ResNet Using Heuristic Advancement of Adaptive Deep Segmentation for Iris Recognition

Due to its significant applications in security, the iris recognition process has been considered as the most active research area over the last few decades. In general, the iris recognition framework has been crucially utilized for various security applications because it includes a set of features as well as does not alter its character according to the time. In recent times, emerging deep learning techniques have attained huge success, particularly in the field of the iris recognition framework model. Moreover, in considering the field of iris recognition, there is no possibility for the remarkable capability of the deep learning model as well as to attain superior performance. To handle the issues in the conventional model of iris recognition, a novel heuristic-aided deep learning framework has been implemented for recognizing the iris system. Initially, the required source iris images are gathered from the data sources. It is then followed by the pre-processing stage, where the pre-processed image is obtained. Consequently, the image segmentation process is carried out by Adaptive Deeplabv3+layers, in which the parameters are optimized using the Modified Weighted Flow Direction Algorithm (MWFDA). Finally, the iris recognition is accomplished by hybrid Hybridization of Multiscale Dilated-Assisted Learning (MDAL) that will be composed of both a Convolutional Neural Network (CNN) and a Residual Network (ResNet). To achieve optimal recognition results, the parameters in CNN and ResNet are tuned optimally by using MWFDA. The experimental results are estimated with the help of distinct measures. Contrary to conventional methods, the empirical results prove that the recommended model achieves the desired value to enhance the recognition performance.

Deep temporal clustering features for speech emotion recognition

Deep clustering is a popular unsupervised technique for feature representation learning. We recently proposed the chunk-based DeepEmoCluster framework for speech emotion recognition (SER) to adopt the concept of deep clustering as a novel semi-supervised learning (SSL) framework, which achieved improved recognition performances over conventional reconstruction-based approaches. However, the vanilla DeepEmoCluster lacks critical sentence-level temporal information that is useful for SER tasks. This study builds upon the DeepEmoCluster framework, creating a powerful SSL approach that leverages temporal information within a sentence. We propose two sentence-level temporal modeling alternatives using either the temporal-net or the triplet loss function, resulting in a novel temporal-enhanced DeepEmoCluster framework to capture essential temporal information. The key contribution to achieving this goal is the proposed sentence-level uniform sampling strategy, which preserves the original temporal order of the data for the clustering process. An extra network module (e.g., gated recurrent unit) is utilized for the temporal-net option to encode temporal information across the data chunks. Alternatively, we can impose additional temporal constraints by using the triplet loss function while training the DeepEmoCluster framework, which does not increase model complexity. Our experimental results based on the MSP-Podcast corpus demonstrate that the proposed temporal-enhanced framework significantly outperforms the vanilla DeepEmoCluster framework and other existing SSL approaches in regression tasks for the emotional attributes arousal, dominance, and valence. The improvements are observed in fully-supervised learning or SSL implementations. Further analyses validate the effectiveness of the proposed temporal modeling, showing (1) high temporal consistency in the cluster assignment, and (2) well-separated emotional patterns in the generated clusters.

Attention-Mechanism-Based Face Feature Extraction Model for WeChat Applet on Mobile Devices

Face recognition technology has been widely used with the WeChat applet on mobile devices; however, facial images are captured on mobile devices and then transmitted to a server for feature extraction and recognition in most existing systems. There are significant security risks related to personal information leakage with these transmissions. Therefore, we propose a face recognition framework for the WeChat applet in which face features are extracted in WeChat by the proposed Face Feature Extraction Model based on Attention Mechanism (FFEM-AM), and only the extracted features are transmitted to the server for recognition. In order to balance the prediction accuracy and model complexity, the structure of the proposed FFEM-AM is lightweight, and Efficient Channel Attention (ECA) was introduced to improve the prediction accuracy. The proposed FFEM-AM was evaluated using a self-built database and the WeChat applet on mobile devices. The experiments show that the prediction accuracy of the proposed FFEM-AM was 98.1%, the running time was less than 100 ms, and the memory cost was only 6.5 MB. Therefore, this demonstrates that the proposed FFEM-AM has high prediction accuracy and can also be deployed with the WeChat applet.

A Time-Local Weighted Transformation Recognition Framework for Steady State Visual Evoked Potentials Based Brain-Computer Interfaces.

Canonical correlation analysis (CCA), Multivariate synchronization index (MSI), and their extended methods have been widely used for target recognition in Brain-computer interfaces (BCIs) based on Steady State Visual Evoked Potentials (SSVEP), and covariance calculation is an important process for these algorithms. Some studies have proved that embedding time-local information into the covariance can optimize the recognition effect of the above algorithms. However, the optimization effect can only be observed from the recognition results and the improvement principle of time-local information cannot be explained. Therefore, we propose a time-local weighted transformation (TT) recognition framework that directly embeds the time-local information into the electroencephalography signal through weighted transformation. The influence mechanism of time-local information on the SSVEP signal can then be observed in the frequency domain. Low-frequency noise is suppressed on the premise of sacrificing part of the SSVEP fundamental frequency energy, the harmonic energy of SSVEP is enhanced at the cost of introducing a small amount of high-frequency noise. The experimental results show that the TT recognition framework can significantly improve the recognition ability of the algorithms and the separability of extracted features. Its enhancement effect is significantly better than the traditional time-local covariance extraction method, which has enormous application potential.

A scalable multimodal ensemble learning framework for automatic modulation recognition

A scalable multimodal ensemble learning framework for automatic modulation recognition

Active Multiview Fusion Framework for SAR Automatic Target Recognition

Active Multiview Fusion Framework for SAR Automatic Target Recognition

A Hierarchical and Multi-modal Framework for Place Recognition with a Learnable Metric

A Hierarchical and Multi-modal Framework for Place Recognition with a Learnable Metric

Transformer-Based Self-Supervised Multimodal Representation Learning for Wearable Emotion Recognition

Recently, wearable emotion recognition based on peripheral physiological signals has drawn massive attention due to its less invasive nature and its applicability in real-life scenarios. However, how to effectively fuse multimodal data remains a challenging problem. Moreover, traditional fully-supervised based approaches suffer from overfitting given limited labeled data. To address the above issues, we propose a novel self-supervised learning (SSL) framework for wearable emotion recognition, where efficient multimodal fusion is realized with temporal convolution-based modality-specific encoders and a transformer-based shared encoder, capturing both intra-modal and inter-modal correlations. Extensive unlabeled data is automatically assigned labels by five signal transforms, and the proposed SSL model is pre-trained with signal transformation recognition as a pretext task, allowing the extraction of generalized multimodal representations for emotion-related downstream tasks. For evaluation, the proposed SSL model was first pre-trained on a large-scale self-collected physiological dataset and the resulting encoder was subsequently frozen or fine-tuned on three public supervised emotion recognition datasets. Ultimately, our SSL-based method achieved state-of-the-art results in various emotion classification tasks. Meanwhile, the proposed model was proved to be more accurate and robust compared to fully-supervised methods on low data regimes.

Convolution-Enhanced Bi-Branch Adaptive Transformer With Cross-Task Interaction for Food Category and Ingredient Recognition.

Recently, visual food analysis has received more and more attention in the computer vision community due to its wide application scenarios, e.g., diet nutrition management, smart restaurant, and personalized diet recommendation. Considering that food images are unstructured images with complex and unfixed visual patterns, mining food-related semantic-aware regions is crucial. Furthermore, the ingredients contained in food images are semantically related to each other due to the cooking habits and have significant semantic relationships with food categories under the hierarchical food classification ontology. Therefore, modeling the long-range semantic relationships between ingredients and the categories-ingredients semantic interactions is beneficial for ingredient recognition and food analysis. Taking these factors into consideration, we propose a multi-task learning framework for food category and ingredient recognition. This framework mainly consists of a food-orient Transformer named Convolution-Enhanced Bi-Branch Adaptive Transformer (CBiAFormer) and a multi-task category-ingredient recognition network called Structural Learning and Cross-Task Interaction (SLCI). In order to capture the complex and unfixed fine-grained patterns of food images, we propose a query-aware data-adaptive attention mechanism called Bi-Branch Adaptive Attention (BiA-Attention) in CBiAFormer, which consists of a local fine-grained branch and a global coarse-grained branch to mine local and global semantic-aware regions for different input images through an adaptive candidate key/value sets assignment for each query. Additionally, a convolutional patch embedding module is proposed to extract the fine-grained features which are neglected by Transformers. To fully utilize the ingredient information, we propose SLCI, which consists of cross-layer attention to model the semantic relationships between ingredients and two cross-task interaction modules to mine the semantic interactions between categories and ingredients. Extensive experiments show that our method achieves competitive performance on three mainstream food datasets (ETH Food-101, Vireo Food-172, and ISIA Food-200). Visualization analyses of CBiAFormer and SLCI on two tasks prove the effectiveness of our method. Codes will be released upon publication. Code and models are available at https://github.com/Liuyuxinict/CBiAFormer.

SecureSense: Defending Adversarial Attack for Secure Device-Free Human Activity Recognition

Deep neural networks have empowered accurate device-free human activity recognition, which has wide applications. Deep models can extract robust features from various sensors and generalize well even in challenging situations such as data-insufficient cases. However, these systems could be vulnerable to input perturbations, i.e. adversarial attacks. We empirically demonstrate that both black-box Gaussian attacks and modern adversarial white-box attacks can render their accuracies to plummet. In this paper, we firstly point out that such phenomenon can bring severe safety hazards to device-free sensing systems, and then propose a novel learning framework, SecureSense, to defend common attacks. SecureSense aims to achieve consistent predictions regardless of whether there exists an attack on its input or not, alleviating the negative effect of distribution perturbation caused by adversarial attacks. Extensive experiments demonstrate that our proposed method can significantly enhance the model robustness of existing deep models, overcoming possible attacks. The results validate that our method works well on wireless human activity recognition and person identification systems. To the best of our knowledge, this is the first work to investigate adversarial attacks and further develop a novel defense framework for wireless human activity recognition in mobile computing research.

A New Causal Inference Framework for SAR Target Recognition

A New Causal Inference Framework for SAR Target Recognition

Online Active Continual Learning for Robotic Lifelong Object Recognition.

In real-world applications, robotic systems collect vast amounts of new data from ever-changing environments over time. They need to continually interact and learn new knowledge from the external world to adapt to the environment. Particularly, lifelong object recognition in an online and interactive manner is a crucial and fundamental capability for robotic systems. To meet this realistic demand, in this article, we propose an online active continual learning (OACL) framework for robotic lifelong object recognition, in the scenario of both classes and domains changing with dynamic environments. First, to reduce the labeling cost as much as possible while maximizing the performance, a new online active learning (OAL) strategy is designed by taking both the uncertainty and diversity of samples into account to protect the information volume and distribution of data. In addition, to prevent catastrophic forgetting and reduce memory costs, a novel online continual learning (OCL) algorithm is proposed based on the deep feature semantic augmentation and a new loss-based deep model and replay buffer update, which can mitigate the class imbalance between the old and new classes and alleviate confusion between two similar classes. Moreover, the mistake bound of the proposed method is analyzed in theory. OACL allows robots to select the most representative new samples to query labels and continually learn new objects and new variants of previously learned objects from a nonindependent and identically distributed (i.i.d.) data stream without catastrophic forgetting. Extensive experiments conducted on real lifelong robotic vision datasets demonstrate that our algorithm, even trained with fewer labeled samples and replay exemplars, can achieve state-of-the-art performance on OCL tasks.

A Siamese neural network framework for glass transition recognition.

A Siamese neural network, which is a deep learning technique, was applied to investigate phase transitions based on polarising microscopic textures of liquid crystals like: antiferroelectric smectic CA* phase and its glass, smectic I phase and its glass, and smectic G and its glass. It is an example of a subtle transition without significant structural changes, where textures above and below the glass transition temperature are similar. The Siamese neural network could distinguish textures of the chosen liquid crystal phases from a glass of that phase. This publication provides details of the Siamese neural network and its implementation based on three different convolutional neural networks has been tested.

Federated Learning Framework for Human Activity Recognition Using Smartphones

Human activity recognition plays a vital role in various applications like healthcare, sports, and smart environments. … With the increase in the use of smart wearables and sensors, the concern for privacy has increased drastically in recognizing the activities of individuals. The legacy techniques send the data from millions of users to a central server where the processing and model generation happens. This increases the load on the server. Federated Learning is a promising method to handle sensitive and personal data in an efficient manner as well as use distributed techniques to reduce challenges that are faced in legacy approaches. Edge devices like smartphones and embedded devices extract the raw data, preprocess it, and generate a personalized model. These models are then aggregated at a central server using federated learning methods and their aggregate model is sent back to clients. Long Short-Term Memory (LSTM) is used for model generation as it can effectively capture temporal dependencies in sequential data. In order to show a real-world scenario, an AWS EC2 instance is used as a server, and laptops and smartphones are used as clients. Our results demonstrate that such methods can be adopted and can be used on a much larger scale. The performance of the global and local models is comparable to those attained by conventional methods. The utilization of federated learning shows a significant step forward in the development of human activity recognition. This approach allows for collaborative learning across multiple devices while also ensuring the privacy of users is maintained.

A deep learning framework for disaster recognition and classification of the damaged regions

A deep learning framework for disaster recognition and classification of the damaged regions

An Effective Multi-Scale Framework for Driver Behavior Recognition With Incomplete Skeletons

An Effective Multi-Scale Framework for Driver Behavior Recognition With Incomplete Skeletons

Feature Aggregation and Compensation-Based Domain Adaptation Framework for Cross-Resolution Target Recognition in SAR Imagery

Feature Aggregation and Compensation-Based Domain Adaptation Framework for Cross-Resolution Target Recognition in SAR Imagery