CNN Features Research Articles

Sustainable Construction of Higher Education MOOCs Using CNN Feature Extraction

The attention time of students studying in MOOC (Massive Open Online Courses) classroom was analyzed to optimize and further improve their performance. On this basis, a student class model based on convolutional neural networks (CNN) feature extraction was proposed. Through Pr (Adobe Premiere) technology, students' class videos were processed by framing, and relevant features were extracted based on changes in students' eye movement trajectories. Then, 10 class videos of ten different experimenters were selected for comparative experiments. After comparing the results, it was found that the test scores of the experimental personnel using MOOC model for assisted learning were significantly different from those before using MOOC model. The final test scores of the students using MOOC model for learning increased to 5-10 points, which had a certain positive impact on the learning results. In the context of sustainable development of higher education, the construction and application of the MOOC model require more favorable promotion and practice.

Beyond Granularity: Enhancing Continuous Sign Language Recognition with Granularity-Aware Feature Fusion and Attention Optimization

The advancement of deep learning techniques has significantly propelled the development of the continuous sign language recognition (cSLR) task. However, the spatial feature extraction of sign language videos in the RGB space tends to focus on the overall image information while neglecting the perception of traits at different granularities, such as eye gaze and lip shape, which are more detailed, or posture and gestures, which are more macroscopic. Exploring the efficient fusion of visual information of different granularities is crucial for accurate sign language recognition. In addition, applying a vanilla Transformer to sequence modeling in cSLR exhibits weak performance because specific video frames could interfere with the attention mechanism. These limitations constrain the capability to understand potential semantic characteristics. We introduce a feature fusion method for integrating visual features of disparate granularities and refine the metric of attention to enhance the Transformer’s comprehension of video content. Specifically, we extract CNN feature maps with varying receptive fields and employ a self-attention mechanism to fuse feature maps of different granularities, thereby obtaining multi-scale spatial features of the sign language framework. As for video modeling, we first analyze why the vanilla Transformer failed in cSLR and observe that the magnitude of the feature vectors of video frames could interfere with the distribution of attention weights. Therefore, we utilize the Euclidean distance among vectors to measure the attention weights instead of scaled-dot to enhance dynamic temporal modeling capabilities. Finally, we integrate the two components to construct the model MSF-ET (Multi-Scaled feature Fusion–Euclidean Transformer) for cSLR and train the model end-to-end. We perform experiments on large-scale cSLR benchmarks—PHOENIX-2014 and Chinese Sign Language (CSL)—to validate the effectiveness.

A Comparative Analysis of Attention Mechanism in RNN-LSTMs for Improved Image Captioning Performance

Image captioning which links computer vision with NATURAL LANGUAGE PROCESSING is critical in providing descriptions for the image. The proposed solution in this research is a hierarchical attention model which includes use of CNN features on images and LSTM networks with attention mechanisms for generating captions. By utilizing both object level and image level features, our method enhances the quality and relevance of captions, enhancing the variability of the automated image description.

A Lightweight Method for Breast Cancer Detection Using Thermography Images with Optimized CNN Feature and Efficient Classification.

Breast cancer is a prominent cause of death among women worldwide. Infrared thermography, due to its cost-effectiveness and non-ionizing radiation, has emerged as a promising tool for early breast cancer diagnosis. This article presents a hybrid model approach for breast cancer detection using thermography images, designed to process and classify these images into healthy or cancerous categories, thus supporting disease diagnosis. Multiple pre-trained convolutional neural networks are employed for image feature extraction, and feature filter methods are proposed for feature selection, with diverse classifiers utilized for image classification. Evaluating the DRM-IR test set revealed that the combination of ResNet34, Chi-square ( ) filter, and SVM classifier demonstrated superior performance, achieving the highest accuracy at . Furthermore, the highest accuracy improvement obtained was when using the SVM classifier and Chi-square filter compared to regular convolutional neural networks. The results confirmed that the proposed method, with its high accuracy and lightweight model, outperforms state-of-the-art breast cancer detection from thermography image methods, making it a good choice for computer-aided diagnosis.

The Structure-sharing Hypergraph Reasoning Attention Module for CNNs

Attention mechanisms improve the performance of models by selectively processing relevant information. However, existing attention mechanisms for CNNs do not utilize the high-order semantic similarity between different channels in the input when inferring attention. To address this issue, in this paper, we propose the Structure-sharing Hypergraph Reasoning Attention Module (SHRA Module) to explore the high-order similarity among nodes via hypergraph learning. SHRA Module transforms the input CNN feature maps into hypergraph node representations, which are used to reason attention under a set of learnable hypergraph convolutions. When performing the hypergraph convolution, the SHRA Module utilizes our proposed structure-sharing hypergraph convolution (SHGCN) to perform hypergraph convolutions, where the hypergraphs from different groups and the weight matrices for hypergraph convolutions are conducted in a right-shifted-permutation sequence of hypergraphs. As a result, the weights matrices can be shared with all groups of hypergraphs while performing hypergraph convolution, thus the global information can be used by the module to have a deep look into the input feature. We evaluate SHRA Module with models in object detection, lesion segmentation, and image classification tasks to demonstrate its effectiveness. Experimental results show that SHRA Module highly significantly enhances model performance, surpassing that of classic attention modules.

Smart grid stability prediction model using two-way attention based hybrid deep learning and MPSO

In smart grid management, precise stability prediction is a complicated task that adds to the effective allocation of resources with grid stability. Specifically, demand-side management is considered an essential element of the overall Smart Grids system. Hence, predicting future energy demands is crucial to regulating consumption by aligning utility offerings with consumer demand. This research presents a hybrid deep learning model (Convolutional Neural Network [CNN] with Bi-LSTM) with a two-way attention method and a multi-objective particle swarm optimization method (MPSO) for short-term load prediction from a smart grid. The proposed hybrid model utilizes a two-way attention method at its encoding and decoding stages, in which an encoding attention layer helps to recognize all the essential features from an input vector, and a decoding attention layer helps to resolve the fixed context vector problem by offering better memory capacity. A CNN and Bi-LSTM are used to capture the essential features from the dataset. We also utilize a t-Nearest Neighbours algorithm to pre-process the initial dataset. An MPSO method combines the features of CNN and Bi-LSTM methods, resulting in better prediction accuracy. As far as we know, it is the first work to suggest a dynamic short-term load prediction model that considers different significant features and enables precise predicting outcomes. The performance of the proposed model and existing well-known deep learning models such as Recurrent Neural Network, Gated Recurrent Unit, Long Short-Term Memory (LSTM), Time Series Transformer, CNN-LSTM and various performance measuring parameters MAE, MSE, MAPE and RMSE are calculated on online UCI dataset (Electrical Grid Stability Simulated Dataset). The proposed hybrid model achieved a better prediction result, which proves the efficiency of the proposed model.

Evaluation of precipitation forecasting methods and an advanced lightweight model

Abstract Precipitation forecasting is crucial for warning systems and disaster management. This study focuses on deep learning-based methods and categorizes them into three categories: Recurrent Neural Network (RNN-RNN-RNN), Convolutional Neural Network (CNN-CNN-CNN), and CNN-RNN-CNN methods. Then, we conduct a comprehensive evaluation of typical methods in these three categories using the SEVIR precipitation dataset. The results show that RNN-RNN-RNN suffers from instability in long-term forecasts due to error accumulation, CNN-CNN-CNN struggles to capture temporal signals but produces relatively stable forecasts, and CNN-RNN-CNN significantly increases model complexity and inherits the drawbacks of RNN, leading to worse forecasts. Here, we propose an advanced lightweight precipitation forecasting model (ALPF) based on CNN. Experimental results demonstrate that ALPF can effectively forecast spatial-temporal features, maintaining CNN’s feature extraction capabilities while avoiding error accumulation in RNN’s propagation. ALPF achieves long-term stable precipitation forecasts and can better capture large precipitation amounts.

Bidirectional Interaction of CNN and Transformer Feature for Visual Tracking

Bidirectional Interaction of CNN and Transformer Feature for Visual Tracking

Short-Term Photovoltaic System Output Power Prediction Based on Integrated Deep Learning Algorithms in the Clean Energy Sector

Photovoltaic power generation system plays an important role in renewable energy. Therefore, accurately predicting the short-term output power of photovoltaic system has become a key challenge for real-time power grid management. This study focuses on Yingli's green energy photovoltaic system, and uses the convolution neural network and long-term and short-term memory network fusion model (CNN-LSTM) to predict the short-term power. The model integrates CNN's data feature extraction and LSTM's time series prediction ability, showing high accuracy and stability. The experimental results show that CNN-LSTM model has a low mean and variance of prediction error, and the prediction is stable and reliable, and it is consistent in different scenarios. This provides theoretical support for the output power prediction of photovoltaic system based on deep learning.

Fusion of transformer attention and CNN features for skin cancer detection

Melanoma is a skin cancer that spreads quickly and has serious risks. Early diagnosis is essential, but since the symptoms of skin lesions in the early stages are vague and similar, they can be difficult for specialists to detect. Therefore, machine learning-based alternative diagnostic methods can be developed in addition to existing ones. This study proposes a new deep learning model, a modified lightweight vision transformer (ViT) architecture, and a hybrid framework developed with an integrated deep learning model and an Ensemble Learning (EL) model for the early-stage diagnosis of skin lesions. The proposed deep learning model was developed based on convolution layers and transformers. The model is called multi-head attention block depthwise separable convolution network (MABSCNET). The proposed hybrid framework was developed by combining modern deep learning and EL models pre-trained with the ImageNet dataset along with the MABSCNET model. In the experimental process, the effectiveness of the proposed methods was evaluated on the ISIC 2020 dataset. Additionally, additional experiments were conducted on ISIC 2018 and a Kaggle dataset to analyze the proposed hybrid framework's classification performance. Image enhancement techniques were used in the datasets. In the ISIC 2020 dataset, the MABSCNET model reached 78.63 % accuracy, the ViT model obtained 76.50 %, and the hybrid framework reached 92.74 % accuracy. Moreover, the proposed hybrid framework achieved 100 % on the ISIC 2018 dataset and 94.24 % on the Kaggle dataset.

C-Hybrid-NET: A self-attention-based COVID-19 screening model based on concatenated hybrid 2D-3D CNN features from chest X-ray images

C-Hybrid-NET: A self-attention-based COVID-19 screening model based on concatenated hybrid 2D-3D CNN features from chest X-ray images

CL-TransFER: Collaborative learning based transformer for facial expression recognition with masked reconstruction

Facial expression recognition (FER) has attracted intensive attention due to its critical role in various computer vision tasks. However, existing FER approaches suffer from either noisy annotations or expression ambiguity (high inter-class and low intra-class similarity), limiting the FER performance. To this end, we propose a robust end-to-end collaborative learning based transformer for FER (CL-TransFER) in this paper. Specifically, CL-TransFER co-trains a CNN feature extractor and a transformer feature extractor jointly to extract both rich local semantic features as well as global structural information from facial images. By enforcing the consensus between the predictions of two extractors, the CL-TransFER could suppress the influence of noisy annotations. To further tackle the expression ambiguity problem, we design a simple yet efficient self-supervised masked reconstruction (SSMR) task to pre-train the transformer feature extractor of CL-TransFER. This enhances the model's capability of learning fine-grained discriminative representations. Extensive experiments on three popular benchmarks have demonstrated the effectiveness and superiority of our method.

Temporal graph convolutional network for multi-agent reinforcement learning of action detection

Most action detection techniques process untrimmed action videos using temporal context, without explicitly consider the inherent spatio-temporal context information, leading to limited accuracy in the cases with big spatial complexity and temporal redundancy. To this end, by intuitively dealing with the problem following a two-stage ”clip selection + clip classification” scheme, this paper proposes to formulate action detection as a Markov process and builds up a multi-agent reinforcement learning framework capturing global structural relationships of videos to optimize the selection and classification, simultaneously and progressively. In particular, a temporal graph convolutional network is constructed to represent the spatio-temporal correlations of video clips, which are initialized by evenly sampling, and further adjusted via learning the rewards adaptively for multi-agent cooperation. Multi-head dot-product attention mechanism is adopted to integrate the relations of latent CNN features of interacting agents. Our framework is jointly learnt by fusing the objectives of clip selection policy and clip recognition. The proposed method comprises a novel graph convolutional network based spatio-temporal semantic observation module which captures topological features among nearby agents, and a new policy module that segments actions according to the rewards from the objectives of action recognition. Extensive experiments are conducted on ActivityNet v1.3 and THUMOS14, with 30.13% and 55.4% mAP obtained, demonstrate the applicability and superiority of our approach.

GM-DETR: Research on a Defect Detection Method Based on Improved DETR.

Defect detection is an indispensable part of the industrial intelligence process. The introduction of the DETR model marked the successful application of a transformer for defect detection, achieving true end-to-end detection. However, due to the complexity of defective backgrounds, low resolutions can lead to a lack of image detail control and slow convergence of the DETR model. To address these issues, we proposed a defect detection method based on an improved DETR model, called the GM-DETR. We optimized the DETR model by integrating GAM global attention with CNN feature extraction and matching features. This optimization process reduces the defect information diffusion and enhances the global feature interaction, improving the neural network's performance and ability to recognize target defects in complex backgrounds. Next, to filter out unnecessary model parameters, we proposed a layer pruning strategy to optimize the decoding layer, thereby reducing the model's parameter count. In addition, to address the issue of poor sensitivity of the original loss function to small differences in defect targets, we replaced the L1 loss in the original loss function with MSE loss to accelerate the network's convergence speed and improve the model's recognition accuracy. We conducted experiments on a dataset of road pothole defects to further validate the effectiveness of the GM-DETR model. The results demonstrate that the improved model exhibits better performance, with an increase in average precision of 4.9% (mAP@0.5), while reducing the parameter count by 12.9%.

Enhancing Qur'anic Recitation Experience with CNN and MFCC Features for Emotion Identification

In this study, MFCC feature extraction and CNN algorithms are used to examine the identification of emotions in the murottal sounds of the Qur'an. A CNN model with labelled emotions is trained and tested, as well as data collection of Qur'anic murottal voices from a variety of readers using MFCC feature extraction to capture acoustic properties. The outcomes show that MFCC and CNN work together to significantly improve emotion identification. The CNN model attains an accuracy rate of 56 percent with the Adam optimizer (batch size 8) and a minimum of 45 percent with the RMSprop optimizer (batch size 16). Notably, accuracy is improved by using fewer emotional parameters, and the Adam optimizer is stable across a range of batch sizes. With its insightful analysis of emotional expression and user-specific recommendations, this work advances the field of emotion identification technology in the context of multitonal music.

Deep Learning-Based Iris Recognition System Using Unprocessed Images

The iris image is a powerful and distinctive feature in biometrics that serves as a reliable instrument for human identification. Extracting significant features is crucial for developing iris-based recognition systems. While preprocessing for iris region detection is typically the first step in such systems, it can often fail in real acquisition conditions, leading to decreased performance. This study proposes a new iris-based recognition system that directly exploits original (noisy) images for feature extraction, avoiding the bottleneck of preprocessing. Additionally, a multimodal architecture is proposed for both right and left iris images to strengthen the decision step. Pretrained CNN models extract features classified by SoftMax and support vector machines (SVM). The performance of the proposed system is tested on four public datasets collected under different conditions. The results show that classification accuracy for the original iris dataset without pre-processing is higher than for the normalized database due to the rich CNN features that provide more cognitive information, resulting in greater discrimination power. The CNN multi-modal recognition system achieves the best accuracy compared to most state-of-the-art models, demonstrating the strength of the proposed fusion recognition system.

Skin cancer diagnosis using CNN features with Genetic Algorithm and Particle Swarm Optimization methods

Skin cancer is one of the most common types of cancer in the world. If skin cancer is not treated early, it also affects the diseased area under the skin and this threatens the treatment of the disease. In recent years, many diseases have been rapidly detected with high accuracy with artificial intelligence methods, and the treatment process has accelerated. Convolutional neural networks, one of the artificial intelligence methods, provide very detailed information about images, and extremely successful results are obtained in classifying images. In this study, first the data set was trained with the EfficientNetB0 model, which is one of the convolutional neural networks models. Then, with the fully connected layer of this model, deep features of the images were obtained. These deep features were obtained by selecting Particle Swarm Optimization and Genetic Algorithm optimization, and different feature combinations were created. Each of these selected feature sets was classified by the support vector machines method, and the best performance results were tried to be obtained. As a result, the success of the proposed model has been proven by obtaining an accuracy rate of 89.17%.

DTDO: Driving Training Development Optimization enabled deep learning approach for brain tumour classification using MRI

ABSTRACT A brain tumour is an abnormal mass of tissue. Brain tumours vary in size, from tiny to large. Moreover, they display variations in location, shape, and size, which add complexity to their detection. The accurate delineation of tumour regions poses a challenge due to their irregular boundaries. In this research, these issues are overcome by introducing the DTDO-ZFNet for detection of brain tumour. The input Magnetic Resonance Imaging (MRI) image is fed to the pre-processing stage. Tumour areas are segmented by utilizing SegNet in which the factors of SegNet are biased using DTDO. The image augmentation is carried out using eminent techniques, such as geometric transformation and colour space transformation. Here, features such as GIST descriptor, PCA-NGIST, statistical feature and Haralick features, SLBT feature, and CNN features are extricated. Finally, the categorization of the tumour is accomplished based on ZFNet, which is trained by utilizing DTDO. The devised DTDO is a consolidation of DTBO and CDDO. The comparison of proposed DTDO-ZFNet with the existing methods, which results in highest accuracy of 0.944, a positive predictive value (PPV) of 0.936, a true positive rate (TPR) of 0.939, a negative predictive value (NPV) of 0.937, and a minimal false-negative rate (FNR) of 0.061%.

Drowsiness Detection and Alert System

Drowsiness detection is a solution for identifying signs of fatigue or sleepiness in individuals. One of the key features of our model is that it can detect drowsiness at night as well using Mobile cameras (infrared sensors). The system captures infrared images of the person's face and analyzes the physiological and behavioral cues related to drowsiness. Infrared sensors allow for drowsiness detection in low-light conditions, making it particularly useful for night-time scenarios such as night driving. The system can trigger alerts or interventions if drowsiness is detected, helping to prevent accidents or mistakes. We will be using libraries like OpenCV, TensorFlow, CNN, and VGG19 features in our model. By combining the accessibility of Android devices with the advanced capabilities of the Deep Learning algorithm, drowsiness detection using infrared sensors has the potential to greatly improve the safety and productivity of individuals in their daily lives.

Multimodal Score Fusion with Sparse Low-rank Bilinear Pooling for Egocentric Hand Action Recognition

With the advent of egocentric cameras, there are new challenges where traditional computer vision is not sufficient to handle this kind of video. Moreover, egocentric cameras often offer multiple modalities that need to be modeled jointly to exploit complimentary information. In this article, we propose a sparse low-rank bilinear score pooling approach for egocentric hand action recognition from RGB-D videos. It consists of five blocks: a baseline CNN to encode RGB and depth information for producing classification probabilities; a novel bilinear score pooling block to generate a score matrix; a sparse low-rank matrix recovery block to reduce redundant features, which is common in bilinear pooling; a one-layer CNN for frame-level classification; and an RNN for video-level classification. We proposed to fuse classification probabilities instead of traditional CNN features from RGB and depth modality, involving an effective yet simple sparse low-rank bilinear score pooling to produce a fused RGB-D score matrix. To demonstrate the efficacy of our method, we perform extensive experiments over two large-scale hand action datasets, namely, THU-READ and FPHA, and two smaller datasets, GUN-71 and HAD. We observe that the proposed method outperforms state-of-the-art methods and achieves accuracies of 78.55% and 96.87% over the THU-READ dataset in cross-subject and cross-group settings, respectively. Further, we achieved accuracies of 91.59% and 43.87% over the FPHA and Gun-71 datasets, respectively.