Single-layer Convolutional Neural Network Research Articles

Stacking of BERT and CNN Models for Arabic Word Sense Disambiguation

We propose a new approach for Arabic Word Sense Disambiguation (AWSD) by hybridization of single-layer Convolutional Neural Network (CNN) with contextual representation (BERT). WSD is the task of automatically detecting the correct meaning of a word used in a given context. WSD can be performed as a classification task, and the context is generally a short sentence. Kim [ 26 ] proved that combining a CNN with an RNN (recurrent neural network) provides a good result for text classification. Here, we use a concatenation of BERT models as a word embedding to get simultaneously the target and context representation. Our approach improves the performance of WSD in Arabic languages. The experimental results show that our model outperforms the state-of-the-art approaches and improves the accuracy of 96.42% on the Arabic WordNet dataset.

Feature identification in complex fluid flows by convolutional neural networks

Recent advancements have established machine learning’s utility in predicting nonlinear fluid dynamics, with predictive accuracy being a central motivation for employing neural networks. However, the pattern recognition central to the networks function is equally valuable for enhancing our dynamical insight into the complex fluid dynamics. In this paper, a single-layer convolutional neural network (CNN) was trained to recognize three qualitatively different subsonic buffet flows (periodic, quasi-periodic and chaotic) over a high-incidence airfoil, and a near-perfect accuracy was obtained with only a small training dataset. The convolutional kernels and corresponding feature maps, developed by the model with no temporal information provided, identified large-scale coherent structures in agreement with those known to be associated with buffet flows. Sensitivity to hyperparameters including network architecture and convolutional kernel size was also explored. The coherent structures identified by these models enhance our dynamical understanding of subsonic buffet over high-incidence airfoils over a wide range of Reynolds numbers.

Predicting CircRNA-Disease Associations via Feature Convolution Learning With Heterogeneous Graph Attention Network.

Exploring the relationship between circular RNA (circRNA) and disease is beneficial for revealing the mechanisms of disease pathogenesis. However, a blind search for all possible associations between circRNAs and diseases through biological experiments is time-consuming. Although some prediction methods have been proposed, they still have limitations. In this study, a novel computational framework, called GATCL2CD, is proposed to forecast unknown circRNA-disease associations (CDAs). First, we calculate Gaussian interactive profile kernel (GIP) similarity and semantic similarity for diseases, circRNA sequence similarity and function similarity, and GIPs for circRNAs. Then, we combine them to construct a heterogeneous graph. Thereafter, GATCL2CD proposes a feature convolution learning framework, that uses a multi-head dynamic attention mechanism to obtain different aggregated representations of features that correspond to the nodes in the heterogeneous graph. Then, it extracts rich higher-order features from the stacked feature representations of each node by using of a single-layer convolutional neural network with filter kernels of different sizes. Finally, a pairwise element-wise product operation is implemented to capture the interactions of higher-order feature representations, and a multilayer perceptron neural network is introduced as an efficient classifier for inferring potential CDAs. Major experimental results under 5-fold cross-validation (5-fold CV) on three different datasets show that GATCL2CD is superior to five other state-of-the-art methods. Furthermore, case studies demonstrate the suitability of GATCL2CD as a useful tool for identifying potential disease-related circRNAs.

Public's Mental Health Monitoring via Sentimental Analysis of Financial Text Using Machine Learning Techniques.

Public feelings and reactions associated with finance are gaining significant importance as they help individuals, public health, financial and non-financial institutions, and the government understand mental health, the impact of policies, and counter-response. Every individual sentiment linked with a financial text can be categorized, whether it is a headline or the detailed content published in a newspaper. The Guardian newspaper is considered one of the most famous and the biggest websites for digital media on the internet. Moreover, it can be one of the vital platforms for tracking the public’s mental health and feelings via sentimental analysis of news headlines and detailed content related to finance. One of the key purposes of this study is the public’s mental health tracking via the sentimental analysis of financial text news primarily published on digital media to identify the overall mental health of the public and the impact of national or international financial policies. A dataset was collected using The Guardian application programming interface and processed using the support vector machine, AdaBoost, and single layer convolutional neural network. Among all identified techniques, the single layer convolutional neural network with a classification accuracy of 0.939 is considered the best during the training and testing phases as it produced efficient performance and effective results compared to other techniques, such as support vector machine and AdaBoost with associated classification accuracies 0.677 and 0.761, respectively. The findings of this research would also benefit public health, as well as financial and non-financial institutions.

Automatic Power Quality Disturbance Diagnosis Based on Residual Denoising Convolutional Auto-Encoder

With the increasing integration of non-linear electronic loads, the diagnosis and classification of power quality are becoming crucial for power grid signal management. This paper presents a novel diagnosis strategy based on unsupervised learning, namely residual denoising convolutional auto-encoder (RDCA), which extracts features from the complex power quality disturbances (PQDs) automatically. Firstly, the time–frequency analysis is applied to isolate frequency domain information. Then, the RDCA with a weight residual structure is utilized to extract the useful features in the contaminated PQD data, where the performance is improved using the residual structure. A single-layer convolutional neural network (SCNN) with an added batch normalization layer is proposed to classify the features. Furthermore, combining with RDCA and SCNN, we further propose a classification framework to classify complex PQDs. To provide a reasonable interpretation of the RDCA, visual analysis is employed to gain insight into the model, leading to a better understanding of the features from different layers. The simulation and experimental tests are conducted to verify the practicability and robustness of the RDCA.

Emotion Recognition Using Electrodermal Activity Signals and Multiscale Deep Convolutional Neural Network.

In this work, an attempt has been made to classify emotional states using electrodermal activity(EDA) signals and multiscale convolutional neural networks. For this, EDA signals are considered from a publicly available "A Dataset for Emotion Analysis using Physiological Signals" (DEAP) database. These signals are decomposed into multiple-scales using the coarse-grained method. The multiscale signals are applied to the Multiscale Convolutional Neural Network (MSCNN) to automatically learn robust features directly from the raw signals. Experiments are performed with the MSCNN approach to evaluate the hypothesis (i) improved classification with electrodermal activity signals, and (ii) multiscale learning captures robust complementary features at a different scale. Results show that the proposed approach is able to differentiate various emotional states. The proposed approach yields a classification accuracy of 69.33% and 71.43% for valence and arousal states, respectively. It is observed that the number of layers and the signal length are the determinants for the classifier performance. The performance of the proposed approach outperforms the single-layer convolutional neural network. The MSCNN approach provides end-to-end learning and classification of emotional states without additional signal processing. Thus, it appears that the proposed method could be a useful tool to assess the difference in emotional states for automated decision making.

Advanced Convolutional Neural Networks for Nonlinearity Mitigation in Long-Haul WDM Transmission Systems

Practical implementation of digital signal processing for mitigation of transmission impairments in optical communication systems requires reduction of the complexity of the underlying algorithms. Here, we investigate the application of convolutional neural networks for compensating nonlinear signal distortions in a 3200 km fiber-optic 11x400-Gb/s WDM PDM-16QAM transmission link with a focus on the optimization of the corresponding algorithmic complexity. We propose a design that includes original initialisation of the weights of the layers by a filter predefined through the training a single-layer convolutional neural network. Furthermore, we use an enhanced activation function that takes into account nonlinear interactions between neighbouring symbols. To increase learning efficiency, we apply a layer-wise training scheme followed by joint optimization of all weights applying additional training to all of them together in the large multi-layer network. We examine application of the proposed convolutional neural network for the nonlinearity compensation using only one sample per symbol and evaluate complexity and performance of the proposed technique.

A convolutional neural network-based model observer for breast CT images.

In this paper, we propose a convolutional neural network (CNN)-based efficient model observer for breast computed tomography (CT) images. We first showed that the CNN-based model observer provided similar detection performance to the ideal observer (IO) for signal-known-exactly and background-known-exactly detection tasks with an uncorrelated Gaussian background noise image. We then demonstrated that a single-layer CNN without a nonlinear activation function provided similar detection performance in breast CT images to the Hotelling observer (HO). To train the CNN-based model observer, we generated simulated breast CT images to produce a training dataset in which different background noise structures were generated using filtered back projection with a ramp, or a Hanning weighted ramp, filter. Circular, elliptical, and spiculated signals were used for the detection tasks. The optimal depth and the number of channels for the CNN-based model observer were determined for each task. The detection performances of the HO and a channelized Hotelling observer (CHO) with Laguerre-Gauss (LG) and partial least squares (PLS) channels were also estimated for comparison. The results showed that the CNN-based model observer provided higher detection performance than the HO, LG-CHO, and PLS-CHO for all tasks. In addition, it was shown that the proposed CNN-based model observer provided higher detection performance than the HO using a smaller training dataset. In the presence of nonlinearity in the CNN, the proposed CNN-based model observer showed better performance than other linear observers.

Towards Cyberbullying-free social media in smart cities: a unified multi-modal approach

Smart cities are shifting the presence of people from physical world to cyber world (cyberspace). Along with the facilities for societies, the troubles of physical world, such as bullying, aggression and hate speech, are also taking their presence emphatically in cyberspace. This paper aims to dig the posts of social media to identify the bullying comments containing text as well as image. In this paper, we have proposed a unified representation of text and image together to eliminate the need for separate learning modules for image and text. A single-layer Convolutional Neural Network model is used with a unified representation. The major findings of this research are that the text represented as image is a better model to encode the information. We also found that single-layer Convolutional Neural Network is giving better results with two-dimensional representation. In the current scenario, we have used three layers of text and three layers of a colour image to represent the input that gives a recall of 74% of the bullying class with one layer of Convolutional Neural Network.