Multi-level Convolutional Neural Network Research Articles

Concrete Defect Localization Based on Multilevel Convolutional Neural Networks

Concrete Defect Localization Based on Multilevel Convolutional Neural Networks

Railway wagon bearing fault diagnosis method based on improved sparrow search algorithm optimizing variational mode decomposition and multi-level convolutional neural network.

Ensuring the safe operation of trains hinges on precise bearing condition monitoring, given the pivotal role bearings play in railway wagons. The status and maintenance of wagon bearings are of paramount concern, necessitating a shift from traditional maintenance approaches reliant on schedules and experience, which often lack real-time precision and efficiency. To address this challenge, our research focuses on enhancing the sparrow search algorithm by incorporating logistic chaos mapping and the levy flight strategy. This enhanced algorithm optimizes variational mode decomposition parameters, utilizing intrinsic mode components' average dispersion entropy as the fitness function. This optimization is integrated with a multi-level convolutional neural network for bearing fault diagnosis. Our findings demonstrate the improved algorithm's enhanced spatial search capabilities and reduced modal aliasing in the frequency components. Experimental validation on public datasets and the group's experimental platform for railway wagons shows that multi-level convolutional neural networks have higher diagnostic accuracy and faster convergence speeds than traditional models such as LeNet-5, AlexNet, and convolutional neural network. Our research introduces a highly accurate and widely applicable methodology for mechanical equipment fault diagnosis, aligning with the requirements of the "smart" era.

A network structure for industrial process fault diagnosis based on hyper feature extraction and stacked LSTM

Traditional feature extraction methods are highly dependent on manual work, and manual extraction will inevitably ignore some potentially useful features. For complex systems, there is no simple one-to-one correspondence between faults and symptoms, and fault characteristics often have complex characteristics such as hierarchy, propagation correlation, and delay. In the existing research results, the convolutional neural network (CNN) has been proven to be an effective network structure. But the existing network structure is more like a classifier, and only some features of industrial data are considered in the feature extraction stage, which will cause some feature information to be lost during the training process. To effectively solve the above problems, this paper proposes an industrial process-oriented hyper feature extraction network structure. This method first extracts multi-level CNN fault features, aggregates the hierarchical feature maps and compresses them in a unified feature space to form hyper features. Then the hyper features formed by feature fusion are input to the stacked Long Short Term Memory (LSTM) network for further feature extraction. In this way, the information in the time series can be effectively transmitted, and at the same time, the problem of gradient disappearance caused by long-term training can be solved. We conducted experiments on the Tennessee-Eastman (TE) benchmark process and industrial coking oven datasets. The experimental results show that the proposed method can effectively improve the fault diagnosis accuracy of the industrial production process system.

MLCNNwav: Multilevel Convolutional Neural Network With Wavelet Transformations for Sensor-Based Human Activity Recognition

Human activity recognition (HAR) is a rapidly growing field of research that aims to automatically identify and classify human motions and activities from different tracking devices, such as cameras and sensors. One of the most widely used sensor modalities for HAR is the smartphone, which has various sensors, such as gyroscopes, accelerometers, and GPS, that can provide rich information about a person’s movements and actions. HAR applications are essential for the Internet of Things (IoT) and smart home industries. We used the recent advances in deep learning techniques to develop a new HAR model for wearable sensors. The proposed model, MLCNNwav, relies on residual convolutional neural networks and one-dimensional trainable discrete wavelet transform. The multilevel CNN is designed to capture global features, whereas the wavelet transformation enhances the representation and generalization by learning activity-related features. Several deep learning approaches are compared to assess the superiority of the developed model. Four public benchmarks HAR datasets were used for the evaluation. The outcomes confirmed that the developed MLCNNwav recorded high accuracy rates on all datasets.

MulCNN: An efficient and accurate deep learning method based on gene embedding for cell type identification in single-cell RNA-seq data.

Advancements in single-cell sequencing research have revolutionized our understanding of cellular heterogeneity and functional diversity through the analysis of single-cell transcriptomes and genomes. A crucial step in single-cell RNA sequencing (scRNA-seq) analysis is identifying cell types. However, scRNA-seq data are often high dimensional and sparse, and manual cell type identification can be time-consuming, subjective, and lack reproducibility. Consequently, analyzing scRNA-seq data remains a computational challenge. With the increasing availability of well-annotated scRNA-seq datasets, advanced methods are emerging to aid in cell type identification by leveraging this information. Deep learning neural networks have great potential for analyzing single-cell data. This paper proposes MulCNN, a multi-level convolutional neural network that uses a unique cell type-specific gene expression feature extraction method. This method extracts critical features through multi-scale convolution while filtering noise. Extensive testing using datasets from various species and comparisons with popular classification methods show that MulCNN has outstanding performance and offers a new and scalable direction for scRNA-seq analysis.

Multimodal Hierarchical CNN Feature Fusion for Stress Detection

Stress is one of the most severe concerns in modern life. High-level stress can create various diseases or loss of focus and productivity at work. Being under stress prevents people from recognizing their stress levels, so early stress detection is essential. Recently, multimodal fusion has enhanced the performance of stress detection models using Deep Learning (DL) techniques. The low, mid, and high-level features of a Convolutional Neural Network (CNN) are discriminative. A comprehensive feature representation can be obtained by fusing all three levels of CNN’s features. This study mainly focuses on detecting stress by exploiting these advantages using a multimodal hierarchical CNN feature fusion. The two multimodal physiological signals used in this study are Electrodermal activity (EDA) and Electrocardiogram (ECG). We develop a hierarchical feature set by concatenating multi-level CNN features for each modality. Multimodal fusion on both hierarchical feature sets is performed using the Multimodal Transfer Module (MMTM). The experiments are carried out with raw frequency domain data and the features from the frequency bands to study the effectiveness of both. The model’s performance is compared to the different combinations of hierarchical features from low, mid, and high levels. To verify the generalizability, the proposed approach has been evaluated on four benchmark datasets - ASCERTAIN, CLAS, MAUS, and WAUC. The proposed method showed its effectiveness by outperforming existing models by 1-2%, respectively, on frequency band features. It is observed that the hierarchical feature set from all three levels performed better than all other combinations by 2-4%. As a result, this strategy can be a useful addition to stress detection.

MCNN: a multi-level CNN model for the classification of brain tumors in IoT-healthcare system.

The classification of brain tumors is significantly important for diagnosing and treating brain tumors in IoT healthcare systems. In this work, we have proposed a robust classification model for brain tumors employing deep learning techniques. In the design of the proposed method, an improved Convolutional neural network is used to classify Meningioma, Glioma, and Pituitary types of brain tumors. To test the multi-level convolutional neural network model, brain magnetic resonance image data is utilized. The MCNN model classification results were improved using data augmentation and transfer learning methods. In addition, hold-out and performance evaluation metrics have been employed in the proposed MCNN model. The experimental results show that the proposed model obtained higher outcomes than the state-of-the-art techniques and achieved 99.89% classification accuracy. Due to the higher results of the proposed approach, we recommend it for the identification of brain cancer in IoT-healthcare systems.

An Adaptive Job Shop Scheduler Using Multilevel Convolutional Neural Network and Iterative Local Search

This manuscript proposed an effective hybrid method based on multi-level convolutional neural network (ML-CNN) and iterative local search (ILS) to solve job shop scheduling problems (JSSP) with small-scale training samples and less time. In the proposed method, ML-CNN is proposed to find the global path of JSSP instance from the genetic algorithm (GA); ML-CNN learned global path is fed into ILS to search for the best local path, it is the key to attach excellent adaptivity. In order to find the global path from optimal solutions, the proposed ML-CNN treated the JSSP as some sub-classification tasks first. Each subclass corresponding to one sub-operation prioritizes a particular machine according to the production environment. Significantly, the proposed ML-CNN designed two-level inputs to represent production statutes. The detailed-level input channel records fourteen detailed statutes, while the system-level input channel records four systematic statutes. Two-level inputs are fed into one dimensional (1-D) CNN to extract rich hidden features to predict the priority of each suboperation using a support vector machine (SVM) classifier. At last, the global path (priority sequences) could be obtained, which is encoded as the input of ILS to find the best local path. The authors trained and tested the proposed method on 82 public JSSP instances. The results indicated that the proposed method could obtain optimal solutions for small-scale instances and outperform others regarding makespan and computation time for large-scale JSSP instances.

A Novel Hybrid Machine Learning Approach for Classification of Brain Tumor Images

Abnormal growth of brain tissues is the real cause of brain tumor. Strategy for the diagnosis of brain tumor at initial stages is one of the key step for saving the life of a patient. The manual segmentation of brain tumor magnetic resonance images (MRIs) takes time and results vary significantly in low-level features. To address this issue, we have proposed a ResNet-50 feature extractor depended on multilevel deep convolutional neural network (CNN) for reliable images segmentation by considering the low-level features of MRI. In this model, we have extracted features through ResNet-50 architecture and fed these feature maps to multi-level CNN model. To handle the classification process, we have collected a total number of 2043 MRI patients of normal, benign, and malignant tumor. Three model CNN, multi-level CNN, and ResNet-50 based multi-level CNN have been used for detection and classification of brain tumors. All the model results are calculated in terms of various numerical values identified as precision (P), recall (R), accuracy (Acc) and f1-score (F1-S). The obtained average results are much better as compared to already existing methods. This modified transfer learning architecture might help the radiologists and doctors as a better significant system for tumor diagnosis.

Facial Expression Recognition Using a Temporal Ensemble of Multi-Level Convolutional Neural Networks

Emotion recognition is indispensable in human-machine interaction systems. It comprises locating facial regions of interest in images and classifying them into one of seven classes: angry, disgust, fear, happy, neutral, sad, and surprise. Despite several breakthroughs in image classification, particularly in facial expression recognition, this research area is still challenging, as sampling in the wild is a demanding task. In this study, a two-stage method is proposed for recognizing facial expressions given a sequence of images. At the first stage, all face regions are extracted in each frame, and essential information that would be helpful and related to human emotion is obtained. Then, the extracted features from the previous step are considered temporal data and are assigned to one of the seven basic emotions. In addition, a study of multi-level features is conducted in a convolutional neural network for facial expression recognition. Moreover, various network connections are introduced to improve the classification task. By combining the proposed network connections, superior results are obtained compared to state-of-the-art methods on the FER2013 dataset. Furthermore, the performance of our temporal model is better than that of the single architecture of the 2017 EmotiW challenge winner on the AFEW 7.0 dataset.

An Approach Based on Multilevel Convolution for Sentence-Level Element Extraction of Legal Text

In the judicial field, with the increase of legal text data, the extraction of legal text elements plays a more and more important role. In this paper, we propose a sentence-level model of legal text element extraction based on the structure of multilabel text classification. Our proposed model contains an encoder and an improved decoder. The encoder applies multilevel convolutional neural networks (CNN) and Long Short-Term Memory (LSTM) as feature extraction networks to extract local neighborhood and context information from legal text, and a decoder applies LSTM with multiattention and full connection layer with an improved initialization method to decode and generate label sequences. To our best knowledge, it is one of the first attempts to apply a multilabel classification algorithm for element extraction of legal text. In order to verify the effectiveness of our model, we conduct experiments not only on three real legal text datasets but also on a general multilabel text classification dataset.The experimental results demonstrate that our proposed model outperforms baseline models on legal text datasets, and our model is competitive to baseline models on the general text multilabel classification dataset, which indicates that our proposed model is useful for multilabel classification tasks of ordinary texts and legal texts with an uncertain number of characters in words and short lengths.

A Multilevel Convolutional Recurrent Neural Network for Blade Icing Detection of Wind Turbine

Blade icing detection becomes increasingly significant as it can avoid revenue loss and power degradation. Conventional methods are usually limited by additional costs, and model-driven methods heavily depend on prior domain knowledge. Data-driven methods, especially deep learning approaches without needing the time-consuming handcraft feature engineering, offer a promising solution for blade icing detection. However, the monitoring signals normally have complex and diverse features as wind turbine operates in complex environments, thus effective model is needed for data analyzing. Additionally, the distribution of monitoring data is imbalanced, which causes the abnormal data mining inadequate. In this work, a multilevel convolutional recurrent neural network (MCRNN), is proposed for blade icing detection. Specifically, discrete wavelet decomposition is leveraged to obtain multilevel features both from the time domain and the frequency domain. A parallel structure combining an LSTM branch and a CNN branch is established in each level for feature extraction. To alleviate the severe data imbalance, two mechanisms, including data resampling algorithm and class-rebalanced loss function, are investigated. Furthermore, a multi-step accumulation strategy is proposed to enhance the accuracy of real-time detection. Extensive studies demonstrate that the proposed MCRNN can achieve up 38.8% and 42.9% higher F1-score over the best baseline on the balanced data sets processed by data resampling algorithm and 23.9% and 30.6% higher on imbalanced data sets with MCRNN optimized by the class-rebalanced loss function. The real-time detection verifies the applicability of the proposed method and indicates that the proposed multi-step accumulation strategy can improve the accuracy of icing detection.

AMFB: Attention based multimodal Factorized Bilinear Pooling for multimodal Fake News Detection

Fake news is the information or stories that are intentionally created to deceive or mislead the readers. In recent times, Fake news detection has attracted the attention of researchers and practitioners due to its many-fold benefits, including bringing in preventive measures to tackle the dissemination of misinformation that could otherwise disturb the social fabrics. Social media in recent times are heavily loaded with multimedia news and information. People prefer online news reading and find it more informative and convenient if they have access to multimedia content in the forms of text, images, audio, and videos. In early studies, researchers have proposed several fake news detection mechanisms that mostly utilize the textual features and not proper to learn multimodal (textual + visual) shared representation.To overcome these limitations, in this paper, we propose a multimodal fake news detection framework with appropriate multimodal feature fusion that leverages information from text and image and tries to maximize the correlation between them to get the efficient multimodal shared representation. We empirically show that text, when combined with the image, can improve the performance of the model. The model detects the post once it is introduced into the network in an early stage. At the early stage of a news post’s introduction into the network, the model takes the text and image of the post as input and decides whether this is fake or genuine. Since this model only analyzes news contents, It does not require any prior information regarding the user and network details. This framework has four different sub-modules viz.Attention Based Stacked Bidirectional Long Short Term Memory (ABS-BiLSTM) for textual feature representation, Attention Based Multilevel Convolutional Neural Network–Recurrent Neural Network (ABM-CNN–RNN) for visual feature extraction, multimodal Factorized Bilinear Pooling (MFB) for feature fusion and finally Multi-Layer Perceptron (MLP) for the classification. We perform experiments on two publicly available datasets, viz. Twitter and Weibo. Evaluation results show the efficacy of our proposed approach that performs significantly better compared to the state-of-the-art models. It shows to outperform the current state-of-the-art by approximately 10 points for the Twitter dataset. In contrast, the Weibo dataset achieves an overall better performance with balanced F1-scores between fake and real classes. Furthermore, the complexity of our proposed model is significantly lower than the state-of-the-art.

Incorporating multi-level CNN and attention mechanism for Chinese clinical named entity recognition

Named entity recognition (NER) is a fundamental task in Chinese natural language processing (NLP) tasks. Recently, Chinese clinical NER has also attracted continuous research attention because it is an essential preparation for clinical data mining. The prevailing deep learning method for Chinese clinical NER is based on long short-term memory (LSTM) network. However, the recurrent structure of LSTM makes it difficult to utilize GPU parallelism which to some extent lowers the efficiency of models. Besides, when the sentence is long, LSTM can hardly capture global context information. To address these issues, we propose a novel and efficient model completely based on convolutional neural network (CNN) which can fully utilize GPU parallelism to improve model efficiency. Moreover, we construct multi-level CNN to capture short-term and long-term context information. We also design a simple attention mechanism to obtain global context information which is conductive to improving model performance in sequence labeling tasks. Besides, a data augmentation method is proposed to expand the data volume and try to explore more semantic information. Extensive experiments show that our model achieves competitive performance with higher efficiency compared with other remarkable clinical NER models.

Real-time vehicle counting in complex scene for traffic flow estimation using multi-level convolutional neural network

Real-time vehicle counting in complex scene for traffic flow estimation using multi-level convolutional neural network

Convolutional Neural Networks-Based Lung Nodule Classification: A Surrogate-Assisted Evolutionary Algorithm for Hyperparameter Optimization

This article investigates deep neural networks (DNNs)-based lung nodule classification with hyperparameter optimization. Hyperparameter optimization in DNNs is a computationally expensive problem, and a surrogate-assisted evolutionary algorithm has been recently introduced to automatically search for optimal hyperparameter configurations of DNNs, by applying computationally efficient surrogate models to approximate the validation error function of hyperparameter configurations. Different from existing surrogate models adopting stationary covariance functions (kernels) to measure the difference between hyperparameter points, this article proposes a nonstationary kernel that allows the surrogate model to adapt to functions whose smoothness varies with the spatial location of inputs. A multilevel convolutional neural network (ML-CNN) is built for lung nodule classification, and the hyperparameter configuration is optimized by the proposed nonstationary kernel-based Gaussian surrogate model. Our algorithm searches with a surrogate for optimal setting via a hyperparameter importance-based evolutionary strategy, and the experiments demonstrate our algorithm outperforms manual tuning and several well-established hyperparameter optimization methods, including random search, grid search, the tree-structured parzen estimator (TPE) approach, Gaussian processes (GP) with stationary kernels, and the recently proposed hyperparameter optimization via RBF and dynamic (HORD) coordinate search.

Semantic Segmentation of Anaemic RBCs Using Multilevel Deep Convolutional Encoder-Decoder Network

Pixel-level analysis of blood images plays a pivotal role in diagnosing blood-related diseases, especially Anaemia. These analyses mainly rely on an accurate diagnosis of morphological deformities like shape, size, and precise pixel counting. In traditional segmentation approaches, instance or object-based approaches have been adopted that are not feasible for pixel-level analysis. The convolutional neural network (CNN) model required a large dataset with detailed pixel-level information for the semantic segmentation of red blood cells in the deep learning domain. In current research work, we address these problems by proposing a multi-level deep convolutional encoder-decoder network along with two state-of-the-art healthy and Anaemic-RBC datasets. The proposed multi-level CNN model preserved pixel-level semantic information extracted in one layer and then passed to the next layer to choose relevant features. This phenomenon helps to precise pixel-level counting of healthy and anaemic-RBC elements along with morphological analysis. For experimental purposes, we proposed two state-of-the-art RBC datasets, i.e., Healthy-RBCs and Anaemic-RBCs dataset. Each dataset contains 1000 images, ground truth masks, relevant, complete blood count (CBC), and morphology reports for performance evaluation. The proposed model results were evaluated using crossmatch analysis with ground truth mask by finding IoU, individual training, validation, testing accuracies, and global accuracies using a 05-fold training procedure. This model got training, validation, and testing accuracies as 0.9856, 0.9760, and 0.9720 on the Healthy-RBC dataset and 0.9736, 0.9696, and 0.9591 on an Anaemic-RBC dataset. The IoU and BFScore of the proposed model were 0.9311, 0.9138, and 0.9032, 0.8978 on healthy and anaemic datasets, respectively.

Automatic fault instance segmentation based on mask propagation neural network

Fault interpretation plays a critical role in understanding the crustal development and exploring the subsurface reservoirs such as gas and oil. Recently, significant advances have been made towards fault semantic segmentation using deep learning. However, few studies employ deep learning in fault instance segmentation. We introduce mask propagation neural network for fault instance segmentation. Our study focuses on the description of the differences and relationships between each fault profile and the consistency of fault instance segmentations with adjacent profiles. Our method refers to the reference-guided mask propagation network, which is firstly used in video object segmentation: taking the seismic profiles as video frames while the seismic data volume as a video sequence along the inline direction we can achieve fault instance segmentation based on the mask propagation method. As a multi-level convolutional neural network, the mask propagation network receives a small number of user-defined tags as the guidance and outputs the fault instance segmentation on 3D seismic data, which can facilitate the fault reconstruction workflow. Compared with the traditional deep learning method, the introduced mask propagation neural network can complete the fault instance segmentation work under the premise of ensuring the accuracy of fault detection. • We demonstrate the results of fault instance segmentation in seismic profiles on the first try. • We show fault reconstruction based on fault instance segmentation. • The effectiveness of our method is verified by experiments.

Deep Learning Method for Grasping Novel Objects Using Dexterous Hands.

Robotic grasping ability lags far behind human skills and poses a significant challenge in the robotics research area. According to the grasping part of an object, humans can select the appropriate grasping postures of their fingers. When humans grasp the same part of an object, different poses of the palm will cause them to select different grasping postures. Inspired by these human skills, in this article, we propose new grasping posture prediction networks (GPPNs) with multiple inputs, which acquire information from the object image and the palm pose of the dexterous hand to predict appropriate grasping postures. The GPPNs are further combined with grasping rectangle detection networks (GRDNs) to construct multilevel convolutional neural networks (ML-CNNs). In this study, a force-closure index was designed to analyze the grasping quality, and force-closure grasping postures were generated in the GraspIt! environment. Depth images of objects were captured in the Gazebo environment to construct the dataset for the GPPNs. Herein, we describe simulation experiments conducted in the GraspIt! environment, and present our study of the influences of the image input and the palm pose input on the GPPNs using a variable-controlling approach. In addition, the ML-CNNs were compared with the existing grasp detection methods. The simulation results verify that the ML-CNNs have a high grasping quality. The grasping experiments were implemented on the Shadow hand platform, and the results show that the ML-CNNs can accurately complete grasping of novel objects with good performance.

A deep learning fusion model for brain disorder classification: Application to distinguishing schizophrenia and autism spectrum disorder.

Deep learning has shown a great promise in classifying brain disorders due to its powerful ability in learning optimal features by nonlinear transformation. However, given the high-dimension property of neuroimaging data, how to jointly exploit complementary information from multimodal neuroimaging data in deep learning is difficult. In this paper, we propose a novel multilevel convolutional neural network (CNN) fusion method that can effectively combine different types of neuroimage-derived features. Importantly, we incorporate a sequential feature selection into the CNN model to increase the feature interpretability. To evaluate our method, we classified two symptom-related brain disorders using large-sample multi-site data from 335 schizophrenia (SZ) patients and 380 autism spectrum disorder (ASD) patients within a cross-validation procedure. Brain functional networks, functional network connectivity, and brain structural morphology were employed to provide possible features. As expected, our fusion method outperformed the CNN model using only single type of features, as our method yielded higher classification accuracy (with mean accuracy >85%) and was more reliable across multiple runs in differentiating the two groups. We found that the default mode, cognitive control, and subcortical regions contributed more in their distinction. Taken together, our method provides an effective means to fuse multimodal features for the diagnosis of different psychiatric and neurological disorders.