Recognition Approach Research Articles

Facial Biosignals Time–Series Dataset (FBioT): A Visual–Temporal Facial Expression Recognition (VT-FER) Approach

Visual biosignals can be used to analyze human behavioral activities and serve as a primary resource for Facial Expression Recognition (FER). FER computational systems face significant challenges, arising from both spatial and temporal effects. Spatial challenges include deformations or occlusions of facial geometry, while temporal challenges involve discontinuities in motion observation due to high variability in poses and dynamic conditions such as rotation and translation. To enhance the analytical precision and validation reliability of FER systems, several datasets have been proposed. However, most of these datasets focus primarily on spatial characteristics, rely on static images, or consist of short videos captured in highly controlled environments. These constraints significantly reduce the applicability of such systems in real-world scenarios. This paper proposes the Facial Biosignals Time–Series Dataset (FBioT), a novel dataset providing temporal descriptors and features extracted from common videos recorded in uncontrolled environments. To automate dataset construction, we propose Visual–Temporal Facial Expression Recognition (VT-FER), a method that stabilizes temporal effects using normalized measurements based on the principles of the Facial Action Coding System (FACS) and generates signature patterns of expression movements for correlation with real-world temporal events. To demonstrate feasibility, we applied the method to create a pilot version of the FBioT dataset. This pilot resulted in approximately 10,000 s of public videos captured under real-world facial motion conditions, from which we extracted 22 direct and virtual metrics representing facial muscle deformations. During this process, we preliminarily labeled and qualified 3046 temporal events representing two emotion classes. As a proof of concept, these emotion classes were used as input for training neural networks, with results summarized in this paper and available in an open-source online repository.

Enhanced Aiot Multi‐Modal Fusion for Human Activity Recognition in Ambient Assisted Living Environment

ABSTRACTMethodologyHuman activity recognition (HAR) has emerged as a fundamental capability in various disciplines, including ambient assisted living, healthcare, human‐computer interaction, etc. This study proposes a novel approach for activity recognition by integrating IoT technologies with Artificial Intelligence and Edge Computing. This work presents a fusion HAR approach that combines data readings from wearable sensors such as accelerometer and gyroscope sensors and Images captured by vision‐based sensors such as cameras incorporating the capabilities of Long Short‐Term Memory (LSTM) and Convolutional Neural Network (CNN) models. The aim of fusing these models is to capture and extract the temporal and spatial information, improving the accuracy and resilience of activity identification systems. The work uses the CNN model to find spatial features from the images that represent the contextual information of the activities and the LSTM model for processing sequential accelerometer and gyroscope sensor data to extract the temporal dynamics from the human activities.ResultsThe performance of our fusion approach is evaluated through different experiments using varying parameters and applies the best‐suited parameters for our model. The results demonstrate that the fusion of LSTM and CNN models outperforms standalone models and traditional fusion methods, achieving an accuracy of 98%, which is almost 9% higher than standalone models.ConclusionThe fusion of LSTM and CNN models enables the integration of complementary information from both data sources, leading to improved performance. The computation tasks are performed at the local edge device resulting to enhanced privacy and reduced latency. Our approach greatly impacts real‐world applications where accurate and reliable HAR systems are essential for enhancing human‐machine interaction and monitoring human activities in various domains.

A New Fault Method Detection for Wireless Sensor Networks using “Autoencoder and LS-SVM”

This study focuses on the issue of fault detection in WSNs while not disturbing the flow of data; and it presents a comprehensive, and new approach to dealing with the problem. The first steps in the context of the developed methodology for application to the data of stock exchanges include scaling of samples by the method of min-max, transformation of windows of samples as part of data preparation, as well as preliminary data cleaning and accurate division of data into sections. And these steps are important for dataset preparation for further analysis. The proposed method relies on the integration of Autoencoders put alongside Least Squares Support Vector Machines (LSSVM). An Autoencoder network was developed and the size of the hidden nodes was later adjusted to identify internal parameters in the dataset. It was helpful for the subsequent reconstructions of the data scene and allowed to obtain high-level features required for fault detection. With the help of these extracted features, LSSVM model was developed towards classifying no+rmal and anomalous condition in WSNs, The training outcome exhibited high effectiveness where anticipated indexes of training data set were 99.77% and for the test data set were 99%. The above outcomes support the feasibility and accuracy of the applied approach in fault recognition. The thesis greatly helps in the progression of the field by providing a methodical way of addressing the important problem of fault detection in WSNs and providing experimental evidence and analysis for the stated problem.

Machine Learning and Deep Learning Approaches for Arabic Sign Language Recognition: A Decade Systematic Literature Review.

Sign language (SL) is a means of communication that is used to bridge the gap between the deaf, hearing-impaired, and others. For Arabic speakers who are hard of hearing or deaf, Arabic Sign Language (ArSL) is a form of nonverbal communication. The development of effective Arabic sign language recognition (ArSLR) tools helps facilitate this communication, especially for people who are not familiar with ArSLR. Although researchers have investigated various machine learning (ML) and deep learning (DL) methods and techniques that affect the performance of ArSLR systems, a systematic review of these methods is lacking. The objectives of this study are to present a comprehensive overview of research on ArSL recognition and present insights from previous research papers. In this study, a systematic literature review of ArSLR based on ML/DL methods and techniques published between 2014 and 2023 is conducted. Three online databases are used: Web of Science (WoS), IEEE Xplore, and Scopus. Each study has undergone the proper screening processes, which include inclusion and exclusion criteria. Throughout this systematic review, PRISMA guidelines have been appropriately followed and applied. The results of this screening are divided into two parts: analysis of all the datasets utilized in the reviewed papers, underscoring their characteristics and importance, and discussion of the ML/DL techniques' potential and limitations. From the 56 articles included in this study, it was noticed that most of the research papers focus on fingerspelling and isolated word recognition rather than continuous sentence recognition, and the vast majority of them are vision-based approaches. The challenges remaining in the field and future research directions in this area of study are also discussed.

Object Recognition in Foggy and Hazy Conditions Using Dark Channel Prior-Based Fringe-Adjusted Joint Transform Correlator

Extreme weather conditions like fog and haze present substantial challenges to object recognition systems. Reduced visibility and contrast degradation significantly affect the auto-correlation process, often leading to failure in object recognition. To address this critical issue and to make object recognition accurate and invincible, we propose a hybrid digital–optical correlator specifically designed to perform under adverse weather conditions. This approach integrates the dark channel prior (DCP) with the fringe-adjusted joint transform correlator (FJTC), promising significant potential to enhance the robustness of the object recognition process under challenging environmental conditions. The proposed scheme presents a unique and alternative approach for object recognition under bad weather conditions. The incoming input scenes are processed with the DCP, enabling the FJTC to perform optical correlation on the refined images. The effectiveness of the proposed method is evaluated using several performance metrics like the structural similarity index measure (SSIM), peak signal-to-noise ratio (PSNR), correlation peak intensity (CPI), processing time, and recognition accuracy. To validate the performance of the proposed study, numerical simulation along with hybrid digital–optical demonstrations have been conducted.

An Ensemble Kernelized-based Approach for Precise Emotion Recognition in Depressed People

As the COVID-19 pandemic created serious challenges for mental health worldwide, with a noticeable increase in depression cases, it has become important to quickly and accurately assess emotional states. Facial expression recognition technology is a key tool for this task. To address this need, this study proposes a new approach to emotion recognition using the Ensemble Kernelized Learning System (EKLS). Nonverbal cues, such as facial expressions, are crucial in showing emotional states. This study uses the Extended Cohn-Kanade (CK+) dataset, which was enhanced with images and videos from the COVID-19 era related to depression. Each of these images and videos is manually labeled with the corresponding emotions, creating a strong dataset for training and testing the proposed model. Facial feature detection techniques were used along with key facial measurements to aid in emotion recognition. EKLS is a flexible machine-learning framework that combines different techniques, including Support Vector Machines (SVMs), Self-Organizing Maps (SOMs), kernel methods, Random Forest (RF), and Gradient Boosting (GB). The ensemble model was thoroughly trained and fine-tuned to ensure high accuracy and consistency. EKLS is a powerful tool for real-time emotion recognition in both images and videos, achieving an impressive accuracy of 99.82%. This study offers a practical and effective approach to emotion recognition and makes a significant contribution to the field.

SAT: A Selective Adversarial Training Approach for WiFi-Based Human Activity Recognition

SAT: A Selective Adversarial Training Approach for WiFi-Based Human Activity Recognition

An optimization based deep learning approach for human activity recognition in healthcare monitoring

An optimization based deep learning approach for human activity recognition in healthcare monitoring

Fusion of residual network and t-SNE-CS for 2D visualization of open set recognition

Open set recognition (OSR) is a technique employed to ascertain whether unknown data belongs to a class in a database when the training class is incomplete. In addressing the OSR challenge associated with ADS-B leading pulse signals, this paper proposes a two-dimensional visualization of open set recognition (VOSR) approach that encompasses the stages of feature extraction, feature selection, and feature learning levels. At the feature extraction level, the I/Q features and phase features of the signal are selected; at the feature selection level, feature similarity analysis and mean decrease impurity-based random forest are employed; at the feature learning level, the framework of fusion residual network and the t-distributed stochastic neighbor embedding and circular surfaces (t-SNE-CS) strategy is constructed, and experiments are carried out on the close set data containing 20 classes of 10,229 samples and open set data containing 10 classes of 1,688 samples. Results show that the accuracy of the optimal combination of the residual network and the constructed features is 94.63% for the test set for the close set classification task. For the VOSR task, the accuracy of the test set is 93.69%, the open set recognition accuracy is 53.97% and Macro-F1 scores is 91.8%.

Machine Learning Algorithms for Emotion Recognition Using Audio and Text Data

Over the years, emotion recognition has become one of the cornerstones of affective computing, enabling machines to recognize and be responsive to human emotions. The current study will present approaches to accurately classify emotions by exploiting multimodal data, namely audio and text. Challenges faced in this domain are the noisy speech signals and inherently ambiguous textual expressions that generally reduce the accuracy of unimodal systems. Classic approaches cannot make good use of the complementary nature of these modalities and, therefore, require a robust and combined framework. This study proposes a method called SVM-ERATI, Support Vector Machine (SVM) based emotion recognition (ER) approach that inputs audio and text information (ATI). Extracted audio features in this regard will include Mel-frequency cepstral coefficients (MFCCs) and prosody-like pitch and energy related to the acoustic properties of emotions. Meanwhile, semantic embeddings obtained from transformer models like BERT serve to analyze text data. A feature-level fusion scheme is then followed, whereby the feature vectors from both audio and text are combined into an integrated representation. Then, features after fusion will be classified by the multi-class SVM with a proper radial basis function (RBF) kernel function that is most appropriate to capture the non-linear relationships inherent in the multimodal emotional data. Experiments on benchmark datasets such as CMU-MOSEI demonstrate that the proposed multimodal approach using SVM significantly outperforms unimodal baselines by about 12%. The findings highlight SVM's effectiveness in combining audio and text data for emotion recognition, which has exciting implications for AI in AI-powered mental health diagnostics and AI-powered intelligent virtual assistants.

Automated layered personification methodology of structural drawings with embedded design information

As artificial intelligence continues to reshape various industries, the demand for digitalization and intelligent design in the fields of architecture and structural engineering is increasingly pronounced. In the pursuit of digital and intelligent design, it is necessary to identify and extract information on various structural components from the construction drawings. In the current stages of intelligent structural design, the preprocessing of construction component drawings predominantly relies on manual efforts, highlighting a notable gap in automated solutions. To address this challenge, this paper introduces an innovative automated layered method to mimic human efforts while reading the architectural and structural drawings. The proposed method could both identify and embed the relevant information of the structural components within bitmap drawings. The proposed method employs the Connected Component Labeling algorithm, a novel Intersection Recognition and Connection approach, which is a straightforward method for information embedding. This approach automates the process of preprocessing and integrating basic information into the structural drawings, facilitating the transition from manual to intelligent systems. Then, the proposed method was validated by case studies and algorithm performance evaluations for its effectiveness and efficiency.

Unsupervised prototype self-calibration based on hybrid attention contrastive learning for enhanced few-shot action recognition

The collection and annotation of large-scale video data pose significant challenges, prompting the exploration of few-shot models to recognize unseen actions with limited training samples. However, most existing models rely on complex temporal alignment strategies, neglecting the necessity of constructing accurate class prototypes in action recognition. To address this limitation, we propose an unsupervised prototype self-calibration based on hybrid attention contrastive learning (UPSHC), aiming to refine prototypes through various strategies. Our network comprises three innovative components: a hybrid attention network (HAN), a dual adaptive contrastive learning mechanism (DACL), and an unsupervised prototype self-calibration mechanism (UPSC). The HAN integrates self-attention and cross-attention mechanisms to highlight important spatiotemporal regions, reducing intra-class variations and enhancing inter-class variations. The DACL mechanism constructs prototype-centered and query-centered contrastive losses to deeply explore the similarities between prototypes and samples, improving the model’s sensitivity to subtle inter-class differences. Additionally, DACL employs adaptive margins to dynamically adjust the distances between positive and negative samples, mitigating the unreliability of fixed margins. The UPSC module utilizes unlabeled query samples in an unsupervised manner to optimize target prototypes without requiring additional training. Experimental results on three benchmark datasets demonstrate that UPSHC achieves state-of-the-art performance, highlighting the superiority of our prototype enhancement approach in few-shot action recognition.

Enhanced Speech Emotion Recognition based on Convolution Neural Networks using MLP Classifier and KNN

The paper aims to recognize emotions from speech using CNNs, leveraging well-known databases such as EmoDB and RAVDESS. It proposes a CNN-based architecture for emotion recognition. The proposed model is trained and evaluated in both speaker-dependent and speaker-independent settings. Speaker-dependent means the model is trained and tested on the same speaker's data, while speaker-independent means testing is done on speakers not seen during training. Techniques such as data augmentation and advanced pre-processing are included to enhance model performance. The results indicate that the proposed CNN architecture achieves comparable performance with state-of-the-art methods. It surpasses traditional systems in terms of accuracy or other relevant metrics. The paper contributes by proposing a CNN-based approach for speech emotion recognition using MLP classifier, which is evaluated on established databases and shows promising results compared to existing methods. It also explores various aspects of dataset properties, speech signal analysis, and classifier methods relevant to emotion recognition tasksdocuments in various Scopus indexed journals was examined bibliometrically, for the years 2014-2024.

Deep Fusion of Skeleton Spatial-Temporal and Dynamic Information for Action Recognition.

Focusing on the issue of the low recognition rates achieved by traditional deep-information-based action recognition algorithms, an action recognition approach was developed based on skeleton spatial-temporal and dynamic features combined with a two-stream convolutional neural network (TS-CNN). Firstly, the skeleton's three-dimensional coordinate system was transformed to obtain coordinate information related to relative joint positions. Subsequently, this relevant joint information was encoded as a color texture map to construct the spatial-temporal feature descriptor of the skeleton. Furthermore, physical structure constraints of the human body were considered to enhance class differences. Additionally, the speed information for each joint was estimated and encoded as a color texture map to achieve the skeleton motion feature descriptor. The resulting spatial-temporal and dynamic features were further enhanced using motion saliency and morphology operators to improve their expression ability. Finally, these enhanced skeleton spatial-temporal and dynamic features were deeply fused via TS-CNN for implementing action recognition. Numerous results from experiments conducted on the publicly available datasets NTU RGB-D, Northwestern-UCLA, and UTD-MHAD demonstrate that the recognition rates achieved via the developed approach are 86.25%, 87.37%, and 93.75%, respectively, indicating that the approach can effectively improve the accuracy of action recognition in complex environments compared to state-of-the-art algorithms.

KoHMT: A Multimodal Emotion Recognition Model Integrating KoELECTRA, HuBERT with Multimodal Transformer

With the advancement of human-computer interaction, the role of emotion recognition has become increasingly significant. Emotion recognition technology provides practical benefits across various industries, including user experience enhancement, education, and organizational productivity. For instance, in educational settings, it enables real-time understanding of students’ emotional states, facilitating tailored feedback. In workplaces, monitoring employees’ emotions can contribute to improved job performance and satisfaction. Recently, emotion recognition has also gained attention in media applications such as automated movie dubbing, where it enhances the naturalness of dubbed performances by synchronizing emotional expression in both audio and visuals. Consequently, multimodal emotion recognition research, which integrates text, speech, and video data, has gained momentum in diverse fields. In this study, we propose an emotion recognition approach that combines text and speech data, specifically incorporating the characteristics of the Korean language. For text data, we utilize KoELECTRA to generate embeddings, and for speech data, we extract features using HuBERT embeddings. The proposed multimodal transformer model processes text and speech data independently, subsequently learning interactions between the two modalities through a Cross-Modal Attention mechanism. This approach effectively combines complementary information from text and speech, enhancing the accuracy of emotion recognition. Our experimental results demonstrate that the proposed model surpasses single-modality models, achieving a high accuracy of 77.01% and an F1-Score of 0.7703 in emotion classification. This study contributes to the advancement of emotion recognition technology by integrating diverse language and modality data, suggesting the potential for further improvements through the inclusion of additional modalities in future work.

Intelligent open-set MIMO recognition in OWC using a Siamese neural network.

Multiple-input multiple-output (MIMO) technology, a core component of 6G, has been widely adopted in optical wireless communication (OWC) systems. Accurate recognition of different MIMO types is essential for MIMO selection and demodulation. In this Letter, we propose an open-set MIMO recognition method for OWC systems using a Siamese neural network (SNN). Simulation results show that the SNN significantly outperforms other recognition approaches, including convolutional neural networks (CNNs) and traditional machine learning techniques. For SNN-based recognition, over 90% accuracy is achieved with training based on only nine fixed sampling points in both 2 × 2 and 4 × 4 MIMO-OWC systems.

MCCA-VNet: A Vit-Based Deep Learning Approach for Micro-Expression Recognition Based on Facial Coding.

In terms of facial expressions, micro-expressions are more realistic than macro-expressions and provide more valuable information, which can be widely used in psychological counseling and clinical diagnosis. In the past few years, deep learning methods based on optical flow and Transformer have achieved excellent results in this field, but most of the current algorithms are mainly concentrated on establishing a serialized token through the self-attention model, and they do not take into account the spatial relationship between facial landmarks. For the locality and changes in the micro-facial conditions themselves, we propose the deep learning model MCCA-VNET on the basis of Transformer. We effectively extract the changing features as the input of the model, fusing channel attention and spatial attention into Vision Transformer to capture correlations between features in different dimensions, which enhances the accuracy of the identification of micro-expressions. In order to verify the effectiveness of the algorithm mentioned, we conduct experimental testing in the SAMM, CAS (ME) II, and SMIC datasets and compared the results with other former best algorithms. Our algorithms can improve the UF1 score and UAR score to, respectively, 0.8676 and 0.8622 for the composite dataset, and they are better than other algorithms on multiple indicators, achieving the best comprehensive performance.

EXPLOITING REMOTE SENSING IMAGERY FOR VEHICLE DETECTION AND CLASSIFICATION USING ROBUST COMPETITIVE ALGORITHM WITH DEEP CONVOLUTIONAL NEURAL NETWORK

Fast and robust vehicle recognition from remote sensing images (RSIs) has excellent economic analysis, emergency management, and traffic surveillance applications. Additionally, vehicle density and location data are vital for intelligent transportation systems. However, correct and robust vehicle recognition in RSIs has been complex. Conventional vehicle recognition approaches depend on handcrafted extracted features in sliding windows with distinct scales. Recently, the convolutional neural network can be executed for aerial image object recognition, and it has accomplished promising outcomes. This research projects an automatic vehicle detection and classification utilizing an imperialist competitive algorithm with a deep convolutional neural network (VDC-ICADCNN) technique. The primary purpose of the VDC-ICADCNN technique is to develop the RSI and apply deep learning (DL) models for the recognition and identification of vehicles. Three main procedures were involved in the presented VDC-ICADCNN technique. At the primary stage, the VDC-ICADCNN technique employs the EfficientNetB7 approach for the feature extractor. Then, the hyperparameter fine-tuning of the EfficientNet approach takes place utilizing ICA, which aids in attaining improved performance in the classification process. The VDC-ICADCNN technique utilizes a variational autoencoder (VAE) model for the vehicle recognition method. Extensive experiments can be implemented to establish the superior solution of the VDC-ICADCNN technique. The obtained outcomes of the VDC-ICADCNN technique highlighted a superior accuracy value of 96.77% and 98.59% with other DL approaches under Vehicle Detection in Aerial Imagery (VEDAI) and Potsdam datasets.

Utilization of Artificial Intelligence for the automated recognition of fine arts

Fine art recognition, traditionally dependent on human expertise, is undergoing a significant transformation with the integration of Artificial Intelligence (AI) and deep learning. This article introduces a novel AI-based approach for fine art recognition, utilizing Convolutional Neural Networks (CNNs) and advanced feature extraction techniques. Addressing the inherent challenges within this domain, we present a systematic methodology to enhance automated fine art recognition. By leveraging critical dataset characteristics such as objective type, genre, material, technique, and department, our method exhibits exceptional performance in classifying fine art pieces across diverse attributes. Our approach significantly improves accuracy and efficiency by integrating advanced feature extraction techniques with a customized CNN architecture. Experimental validation on a benchmark dataset highlights the efficacy of our method, indicating substantial contributions to the interdisciplinary field of fine art analysis.

Wi-Fi Sensing Techniques for Human Activity Recognition: Brief Survey, Potential Challenges, and Research Directions

Recent advancements in wireless communication technologies have made Wi-Fi signals indispensable in both personal and professional settings. The utilization of these signals for Human Activity Recognition (HAR) has emerged as a cutting-edge technology. By leveraging the fluctuations in Wi-Fi signals for HAR, this approach offers enhanced privacy compared to traditional visual surveillance methods. The essence of this technique lies in detecting subtle changes when Wi-Fi signals interact with the human body, which are then captured and interpreted by advanced algorithms. This paper initially provides an overview of the key methodologies in HAR and the evolution of non-contact sensing, introducing sensor-based recognition, computer vision, and Wi-Fi signal-based approaches, respectively. It then explores tools for Wi-Fi-based HAR signal collection and lists several high-quality datasets. Subsequently, the paper reviews various sensing tasks enabled by Wi-Fi signal recognition, highlighting the application of deep learning networks in Wi-Fi signal detection. The fourth section presents experimental results that assess the capabilities of different networks. The findings indicate significant variability in the generalization capacities of neural networks and notable differences in test accuracy for various motion analyses.