Toxic Comment Classification Based on Bidirectional Gated Recurrent Unit and Convolutional Neural Network

<abstract> <p>Energy operations and schedules are significantly impacted by load and energy forecasting systems. An effective system is a requirement for a sustainable and equitable environment. Additionally, a trustworthy forecasting management system enhances the resilience of power systems by cutting power and load-forecast flaws. However, due to the numerous inherent nonlinear properties of huge and diverse data, the classical statistical methodology cannot appropriately learn this non-linearity in data. Energy systems can appropriately evaluate data and regulate energy consumption because of advanced techniques. In comparison to machine learning, deep learning techniques have lately been used to predict energy consumption as well as to learn long-term dependencies. In this work, a fusion of novel multi-directional gated recurrent unit (MD-GRU) with convolutional neural network (CNN) using global average pooling (GAP) as hybridization is being proposed for load and energy forecasting. The spatial and temporal aspects, along with the high dimensionality of the data, are addressed by employing the capabilities of MD-GRU and CNN integration. The obtained results are compared to baseline algorithms including CNN, Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (Bi-LSTM), Gated Recurrent Unit (GRU), and Bidirectional Gated Recurrent Unit (Bi-GRU). The experimental findings indicate that the proposed approach surpasses conventional approaches in terms of accuracy, Mean Absolute Percentage Error (MAPE), and Root Mean Square Error (RSME).</p> </abstract>

Breast cancer accounts for 30% of all female cancers. Accurately distinguishing dangerous malignant tumors from benign harmless ones is key to ensuring patients receive lifesaving treatments on time. However, as doctors currently do not identify 10% to 30% of breast cancers during regular assessment, automated methods to detect malignant tumors are desirable. Although several computerized methods for breast cancer classification have been proposed, convolutional neural networks (CNNs) have demonstrably outperformed other approaches. In this paper, we propose an automated method for the binary classification of breast cancer tumors as either malignant or benign that utilizes a bag of deep multi-resolution convolutional features (BoDMCF) extracted from histopathological images at four resolutions (40×, 100×, 200× and 400×) by three pre-trained state-of-the-art deep CNN models: ResNet-50, EfficientNetb0, and Inception-v3. The BoDMCF extracted by the pre-trained CNNs were pooled using global average pooling and classified using the support vector machine (SVM) classifier. While some prior work has utilized CNNs for breast cancer classification, they did not explore using CNNs to extract and pool a bag of deep multi-resolution features. Other prior work utilized CNNs for deep multi-resolution feature extraction from chest X-ray radiographs to detect other conditions such as pneumoconiosis but not for breast cancer detection from histopathological images. In rigorous evaluation experiments, our deep BoDMCF feature approach with global pooling achieved an average accuracy of 99.92%, sensitivity of 0.9987, specificity (or recall) of 0.9797, positive prediction value (PPV) or precision of 0.99870, F1-Score of 0.9987, MCC of 0.9980, Kappa of 0.8368, and AUC of 0.9990 on the publicly available BreaKHis breast cancer image dataset. The proposed approach outperforms the prior state of the art for histopathological breast cancer classification as well as a comprehensive set of CNN baselines, including ResNet18, InceptionV3, DenseNet201, EfficientNetb0, SqueezeNet, and ShuffleNet, when classifying images at any individual resolutions (40×, 100×, 200× or 400×) or when SVM is used to classify a BoDMCF extracted using any single pre-trained CNN model. We also demonstrate through a carefully constructed set of experiments that each component of our approach contributes non-trivially to its superior performance including transfer learning (pre-training and fine-tuning), deep feature extraction at multiple resolutions, global pooling of deep multiresolution features into a powerful BoDMCF representation, and classification using SVM.

Multispectral (MS) pansharpening is defined as the fusion of spatial information in panchromatic (PAN) image and spectral information in MS image. In this work, we propose a MS pansharpening based on multi-sequence convolutional recurrent neural network (MCRNN). The proposed MCRNN contains two sub-networks (shallow feature extraction sub-network and deep feature fusion sub-network). In the shallow feature extraction sub-network, PAN and MS images are superimposed in the spectral dimension as multi-sequence data. A convolutional neural network (CNN) based on residual learning is then used to obtain the feature maps from multi-sequence data. In the deep feature fusion sub-network, since MS and PAN images are highly correlated, a convolutional recurrent neural network (ConvGRU) belonging to RNN is used to model adjacent and across-band relationships between these feature maps to capture the local and global correlations of the features in different bands. The global average pooling is then performed on the output results to yield the pansharpening result. Several datasets are tested at reduced and full resolution experiments, the experimental results show that the performance of the proposed MCRNN is superior to the traditional pansharpening methods. The source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/HZC-1998/Multi-Sequence-Convolutional-Recurrent-Network-for-Pansharpening</uri> .

Splicing and copy-move are two well-known methods of image tampering, while detection of image splicing and copy-move forgery is an important research topic in image forensics. In this paper, a method based on convolutional neural network with global average pooling was proposed for splicing and copy-move tampering detection. To detect image tampering, the inconsistency between the authentic images and the tampered images should be captured regardless of the image contents. So, the existing strategy using high-pass filter in SRM as initialization of the first layer was improved to reduce the influence of image content and make the features more diverse on each channel at the same time. In order to reduce the number of parameters in the fully connected layers and avoid overfitting, global average pooling was utilized before fully connected layers in the proposed model. Experiments on three public image tampering datasets demonstrated that the proposed method outperformed some state-of-the-art methods.

This paper develops a novel soft fault diagnosis approach for analog circuits. The proposed method employs the backward difference strategy to process the data, and a novel variant of convolutional neural network, i.e., convolutional neural network with global average pooling (CNN-GAP) is taken for feature extraction and fault classification. Specifically, the measured raw domain response signals are firstly processed by the backward difference strategy and the first-order and the second-order backward difference sequences are generated, which contain the signal variation and the rate of variation characteristics. Then, based on the one-dimensional convolutional neural network, the CNN-GAP is developed by introducing the global average pooling technical. Since global average pooling calculates each input vector’s mean value, the designed CNN-GAP could deal with different lengths of input signals and be applied to diagnose different circuits. Additionally, the first-order and the second-order backward difference sequences along with the raw domain response signals are directly fed into the CNN-GAP, in which the convolutional layers automatically extract and fuse multi-scale features. Finally, fault classification is performed by the fully connected layer of the CNN-GAP. The effectiveness of our proposal is verified by two benchmark circuits under symmetric and asymmetric fault conditions. Experimental results prove that the proposed method outperforms the existing methods in terms of diagnosis accuracy and reliability.

Brain tumors can cause serious health complications and lead to death if not detected accurately. Therefore, early-stage detection of brain tumors and accurate classification of types of brain tumors play a major role in diagnosis. Recently, deep convolutional neural network (DCNN) based approaches using brain magnetic resonance imaging (MRI) images have shown excellent performance in detection and classification tasks. However, the accuracy of DCNN architectures depends on the training of data samples since it requires more precise data for better output. Thus, we propose a transfer learning-based DCNN framework to classify brain tumors for example meningioma tumors, glioma tumors, and pituitary tumors. We use a pre-trained DCNN architecture VGGNet which is previously trained on huge datasets and used to transfer its learning parameters to the target dataset. Also, we employ transfer learning aspects such as fine-tune the convolutional network and freeze the layers of the convolutional network for better performance. Further, this proposed approach uses a Global Average Pooling (GAP) layer at the output to avoid overfitting issues and vanishing gradient problems. The proposed architecture is assessed and compared with competing deep learning based brain tumor classification approaches on the Figshare dataset. Our proposed approach produces 98.93% testing accuracy and outperforms the contemporary learning-based approaches.

Convolutional neural networks (CNN), recurrent neural networks (RNN), attention, and their variants are extensively applied in the sentiment analysis, and the effect of fusion model is expected to be better. However, fusion model is confronted with some problems such as complicated structure, excessive trainable parameters, and long training time. The classification effect of traditional model with cross entropy loss as loss function is undesirable since sample category imbalance as well as ease and difficulty of sample classification is not taken into account. In order to solve these problems, the model BiGRU-Att-HCNN is proposed on the basis of bidirectional gated recurrent unit (BiGRU), attention, and hybrid convolutional neural networks. In this model, BiGRU and self-attention are combined to acquire global information, and key information weight is supplemented. Two parallel convolutions (dilated convolution and standard convolution) are used to obtain multi-scale characteristic information with relatively less parameters, and the standard convolution is replaced with depthwise separable convolution with two-step calculations. Traditional max-pooling and average-pooling are discarded, and global average pooling is applied to substitute the pooling layer and the fully-connected layer simultaneously, making it possible to substantially decrease the number of model parameters and reduce over-fitting. In our model, focal loss is used as the loss function to tackle the problems of unbalanced sample categories and hard samples. Experimental results illustrate that in terms of multiple indicators, our model outperforms the 15 benchmark models, even with intermediate number of trainable parameters.

Depression is a mental disorder which with mild effect may cause the feeling of sadness feeling and at worst effect may lead to sucicide. Beside these effects of depression are least talked diseases due to social stigma, unawareness about it. Twitter is a social media platform for self-disclosure of the feeling, people express their happiness and sadness on the platform. So twitter is a good source of sentimental text.Foregoing research for the classification of tweet data for the depressive and non depressive class is performed with SVM,RNN GRU, and CNN. The present paper aims to analyse the behaviour of different deep learning models as a convolutional neural network with global max pooling, convolutional neural network with global average pooling, for the classification of tweet data as depressive and non depressive classes at different epochs 5, 10, 15. The Result concludes that the LSTM model achieves the accuracy of 99.19% followed by LSTM with CNN with 99.16%.

Object detection is an important component of computer vision. Most of the recent successful object detection methods are based on convolutional neural networks (CNNs). To improve the performance of these networks, researchers have designed many different architectures. They found that the CNN performance benefits from carefully increasing the depth and width of their structures with respect to the spatial dimension. Some researchers have exploited the cardinality dimension. Others have found that skip and dense connections were also of benefit to performance. Recently, attention mechanisms on the channel dimension have gained popularity with researchers. Global average pooling is used in SENet to generate the input feature vector of the channel-wise attention unit. In this work, we argue that channel-wise attention can benefit from both global average pooling and global max pooling. We designed three novel attention units, namely, an adaptive channel-wise attention unit, an adaptive spatial-wise attention unit and an adaptive domain attention unit, to improve the performance of a CNN. Instead of concatenating the output of the two attention vectors generated by the two channel-wise attention sub-units, we weight the two attention vectors based on the output data of the two channel-wise attention sub-units. We integrated the proposed mechanism with the YOLOv3 and MobileNetv2 framework and tested the proposed network on the KITTI and Pascal VOC datasets. The experimental results show that YOLOv3 with the proposed attention mechanism outperforms the original YOLOv3 by mAP values of 2.9 and 1.2% on the KITTI and Pascal VOC datasets, respectively. MobileNetv2 with the proposed attention mechanism outperforms the original MobileNetv2 by a mAP value of 1.7% on the Pascal VOC dataset.

Recently, deep learning (DL) models based on convolutional neural networks have achieved satisfactory results in rolling bearing fault diagnosis. However, the bearings usually work in variable loading conditions, and their feature distribution could vary with load. The important features cannot be effectively captured in the convolution process using the existing diagnosis models, resulting in poor generalization performance. In this paper, a novel DL model, named multiscale cascade recurrent dilation convolution network, is proposed by introducing the dilated convolution and global average pooling (GAP) layer. Firstly, a new multiscale cascade structure with different convolution kernel sizes is introduced to extract multiscale features contained in the vibration signal. Secondly, a recurrent dilation convolution strategy is designed in each branch of the multiscale cascade structure to extract abundant feature information. Finally, the GAP is employed to reduce redundant feature vectors and output them, while a classifier of multilayer perceptron is used to automatically identify the fault types. The effectiveness of the proposed algorithm is evaluated by two experimental cases. The results show that the proposed method can successfully identify the labels of fault samples under unknown load conditions using the fault samples with labels under existing load conditions. Compared with other methods, this method exhibits excellent robustness and generalization performance for bearing fault diagnosis under cross-load conditions.

Compared with global average pooling in existing deep convolutional neural networks (CNNs), global covariance pooling can capture richer statistics of deep features, having potential for improving representation and generalization abilities of deep CNNs. However, integration of global covariance pooling into deep CNNs brings two challenges: (1) robust covariance estimation given deep features of high dimension and small sample size; (2) appropriate usage of geometry of covariances. To address these challenges, we propose a global Matrix Power Normalized COVariance (MPN-COV) Pooling. Our MPN-COV conforms to a robust covariance estimator, very suitable for scenario of high dimension and small sample size. It can also be regarded as Power-Euclidean metric between covariances, effectively exploiting their geometry. Furthermore, a global Gaussian embedding network is proposed to incorporate first-order statistics into MPN-COV. For fast training of MPN-COV networks, we implement an iterative matrix square root normalization, avoiding GPU unfriendly eigen-decomposition inherent in MPN-COV. Additionally, progressive 1×1 convolutions and group convolution are introduced to compress covariance representations. The proposed methods are highly modular, readily plugged into existing deep CNNs. Extensive experiments are conducted on large-scale object classification, scene categorization, fine-grained visual recognition and texture classification, showing our methods outperform the counterparts and obtain state-of-the-art performance.

BackgroundCorona Virus Disease 2019 (COVID-19) first appeared in December 2019, and spread rapidly around the world. COVID-19 is a pneumonia caused by novel coronavirus infection in 2019. COVID-19 is highly infectious and transmissible. By 7 May 2021, the total number of cumulative number of deaths is 3,259,033. In order to diagnose the infected person in time to prevent the spread of the virus, the diagnosis method for COVID-19 is extremely important. To solve the above problems, this paper introduces a Multi-Level Enhanced Sensation module (MLES), and proposes a new convolutional neural network model, MLES-Net, based on this module.MethodsAttention has the ability to automatically focus on the key points in various information, and Attention can realize parallelism, which can replace some recurrent neural networks to a certain extent and improve the efficiency of the model. We used the correlation between global and local features to generate the attention mask. First, the feature map was divided into multiple groups, and the initial attention mask was obtained by the dot product of each feature group and the feature after the global pooling. Then the attention masks were normalized. At the same time, there were two scaling and translating parameters in each group so that the normalize operation could be restored. Then, the final attention mask was obtained through the sigmoid function, and the feature of each location in the original feature group was scaled. Meanwhile, we use different classifiers on the network models with different network layers.ResultsThe network uses three classifiers, FC module (fully connected layer), GAP module (global average pooling layer) and GAPFC module (global average pooling layer and fully connected layer), to improve recognition efficiency. GAPFC as a classifier can obtain the best comprehensive effect by comparing the number of parameters, the amount of calculation and the detection accuracy. The experimental results show that the MLES-Net56-GAPFC achieves the best overall accuracy rate (95.27%) and the best recognition rate for COVID-19 category (100%).ConclusionsMLES-Net56-GAPFC has good classification ability for the characteristics of high similarity between categories of COVID-19 X-Ray images and low intra-category variability. Considering the factors such as accuracy rate, number of network model parameters and calculation amount, we believe that the MLES-Net56-GAPFC network model has better practicability.

Sarcasm is a form of emotional expression in which someone speaks or writes something entirely contrary to what is truly intended with humor underlying. Detecting sarcasm is a difficult task as the actual meaning of a text is not consistent. In Bangla language, not enough work has been done in this field. Therefore, we have experimented with some of the state of the art NLP's techniques, machine learning and deep learning models in an existing dataset. In this paper, we have proposed a sarcasm detection AI for Bangla language based on deep learning architecture which is an ensemble model consists gated recurrent unit (GRU) and convolutional neural network (CNN). Our proposed model achieved 96% F <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> score and 96% of accuracy which surpasses the existing traditional machine learning models performance.

Deep neural networks have achieved state-of-the-art performance for a variety of pattern-recognition tasks. In particular, the deep convolutional neural network (CNN), which is composed of several convolutional layers with a nonlinear activation function, pooling layers, and fully connected layers or an optional global average pooling layer, has received significant attention and is widely used in computer vision. Some research is now replacing a top fully connected layer with global pooling to avoid overfitting in the fully connected layers and to achieve regularization. This replacement is very important because global pooling with additional convolutional layers can eliminate restrictions on the necessity for fixed-size or fixed-length input in the fully connected layers. In this paper, the top global pooling layer is focused on, which is used in place of the fully connected layer and creates a simple and effective pooling operation called random crop (RC) pooling. Additionally, how to attain regularization in the top RC pooling layer is discussed. RC pooling randomly crops the feature maps so that only the images with sufficiently scaled and centered objects can be well-trained. This approach achieves comparable accuracy on the CIFAR-10/100 and MNIST.

In this paper, we explore the effect of using different convolutional layers, batch normalization and the global average pooling layer upon a convolutional neural network (CNN) based gaze tracking system. A novel method is proposed to label the participant’s face images as gaze points retrieved from eye tracker while watching videos for building a training dataset that is closer to human visual behavior. The participants can swing their head freely; therefore, the most real and natural images can be obtained without too many restrictions. The labeled data are classified according to the coordinate of gaze and area of interest on the screen. Therefore, varied network architectures are applied to estimate and compare the effects including the number of convolutional layers, batch normalization (BN) and the global average pooling (GAP) layer instead of the fully connected layer. Three schemes, including the single eye image, double eyes image and facial image, with data augmentation are used to feed into neural network to train and evaluate the efficiency. The input image of the eye or face for an eye tracking system is mostly a small-sized image with relatively few features. The results show that BN and GAP are helpful in overcoming the problem to train models and in reducing the amount of network parameters. It is shown that the accuracy is significantly improved when using GAP and BN at the mean time. Overall, the face scheme has a highest accuracy of 0.883 when BN and GAP are used at the mean time. Additionally, comparing to the fully connected layer set to 512 cases, the number of parameters is reduced by less than 50% and the accuracy is improved by about 2%. A detection accuracy comparison of our model with the existing George and Routray methods shows that our proposed method achieves better prediction accuracy of more than 6%.