Fully-connected Layers Research Articles

Classification of Electrocardiography Hybrid Convolutional Neural Network-Long Short Term Memory with Fully Connected Layer

Electrocardiography (ECG) is a technique for observing and recording the electrical activity of the human heart. The usage of an ECG signal is common among clinical professionals in the collection of time data for the examination of any rhythmic conditions associated with a subject. The investigation was carried out in order to computerize the assignment by exhibiting the issue using encoder-decoder techniques, creating the information that was simply typical of it, and utilising misfortune appropriation to anticipate standard or anomalous information. On a broad variety of applications such as voice recognition and prediction, the long short-term memory (LSTM) fully connected layer (FCL) and the two convolutional neural networks (CNNs) have shown superior performance over deep learning networks (DLNs). DNNs are suitable for making high points for a more divisible region and CNNs are suitable for reducing recurrence types, LSTMs are appropriate for temporary displays, in the same way as CNNs are appropriate for reducing recurrence types. The CNN, LSTM, and DNN algorithms are acceptable for viewing. The complementarity of DNNs, CNNs, and LSTMs was investigated in this research by bringing them all together under the single architectural company. The researchers got the ECG data from the MIT-BIH arrhythmia database as a result of the investigation. Our results demonstrate that the approach proposed may expressively describe ECG series and identify abnormalities via scores that outperform existing supervised and unsupervised methods in both the short term and long term. The LSTM network and FCL additionally demonstrated that the unbalanced datasets associated with the ECG beat detection problem could be consistently resolved and that they were not susceptible to the accuracy of ECG signals. It is recommended that cardiologists employ the unique technique to aid them in performing reliable and impartial interpretation of ECG data in telemedicine settings.

Prediction of Retail Price of Sporting Goods Based on LSTM Network.

Commodity prices play a unique role as a lever to regulate the economy. Price forecasting is an important part of macrodecision-making and micromanagement. Because there are many factors affecting the price of goods, price prediction has become a difficulty in research. According to the characteristics that price data are also affected by other factors except for time series, a multifactor LSTM price prediction method is proposed based on the long-term and short-term memory network (LSTM) deep learning algorithm. This method not only makes use of the memory of LSTM to historical data but also introduces the influence of external factors on price through the full connection layer, which provides a new idea for solving the problem of price prediction. Compared with BP neural network, the experimental results show that this method has higher accuracy and better stability. Analyze the commodity description and commodity price characteristics, find out the commodities similar to the target commodity, complete the commodity price data by using the historical price data of similar commodities, and establish the training set to verify the validity of the proposed method.

Hybrid deep learning models for mapping surface NO2 across China: One complicated model, many simple models, or many complicated models?

While deep learning is an emerging trend in modeling spatiotemporal dynamics of air pollution, designing a high-performance architecture with various neural network components is difficult. This study aims to identify a suitable deep learning architecture for deriving spatiotemporal distributions of surface NO2 across China. Based on satellite retrievals and geographic variables, we compared the performance of a series of deep learning models and a benchmark model (i.e., random forest) in predicting daily NO2 during 2016–2017 on a 0.1° grid. The deep learning models tested were hybrids of full connection layers (FC), autoencoders (AE), residual networks (RES), and convolution layers (CNN). Their ensemble forms were also comparatively evaluated. The integrated results of the sample-, cell-, and date-based holdout tests show that the ensemble FC-AE-RES-CNN model (R2 = 0.71; RMSE = 9.9 μg/m3) outperformed the random forest (R2 = 0.67; RMSE = 10.8 μg/m3), but none of the single-form networks did (R2 ranged from 0.56 to 0.61, and RMSE ranged from 11.6 to 12.2 μg/m3). The superior performance of the ensemble FC-AE-RES-CNN model could be attributed to its lower bias and lower variance, which benefited from the sophisticated model structure and the ensemble strategy. Consequently, the ensemble FC-AE-RES-CNN model predictions demonstrated a spatiotemporal distribution of surface NO2 with higher fidelity. In light of the comprehensive comparisons, we expect that ensemble deep learning models will be substituting for traditional machine learning models in environmental spatiotemporal mapping applications.

Image classification of vaginal microecology detection based on gabor texture and LSTM model.

Gynecological diseases threaten women's health, and vaginal microecological testing is a common method for detecting gynecological diseases. Efficient and accurate microecological testing methods have always been the goal pursued by gynecologists. In order to automatically identify different types of microbial images in vaginal micromorphology detection, this paper proposes a vaginal microecological image recognition method based on Gabor texture analysis combined with long and short-term memory network (LSTM) model. Firstly, we denoise the microecological morphological im-ages, which selects the area of interest and sets the label of the microorganism according to the doctors label. Secondly, texture analysis is carried out for the region of interest, which uses Gabor filters with 8 directions and 5 scales to filter the region of interest to extract the texture features on the image. Comparing the differences between different microbial image features, and screening suitable features to reduce the number of features. Then, we design an LSTM model to analyze the relationship of image features in different categories of microorganisms. Finally, we use the full connection layer and Softmax function to realize the automatic recognition of different microbial images. The experimental results show that the image classification accuracy of 8 common microorganisms is 81.26%. Texture analysis combined with LSTM network strategy can identify different kinds of vaginal micro ecological images. Gabor-LSTM model has better classification effect on imbalanced data sets.

An End-to-End Classifier Based on CNN for In-Air Handwritten-Chinese-Character Recognition

A convolutional neural network (CNN) has been successfully applied to in-air handwritten-Chinese-character recognition (IAHCCR). However, the existing models based on CNN for IAHCCR need to convert the coordinate sequence of a character into images. This conversion process increases training and classifying time, and leads to the loss of information. In order to solve this problem, we propose an end-to-end classifier based on CNN for IAHCCR in this paper, which, to knowledge, is novel for online handwritten-Chinese-character recognition (OLHCCR). Specifically, our model based on CNN directly takes the original coordinate sequence of an in-air handwritten-Chinese-character as input, and the output of the full connection layer is pooled by global average pooling to form a fixed-size feature vector, which is sent to softmax for classification. Our model can not only directly process coordinate sequences such as the models based on recurrent neural network (RNN), but can also obtain the global structure information of characters. We conducted experiments on two datasets, IAHCC-UCAS2016 and SCUT-COUCH2009. The experimental results show a comparison with existing CNN models based on image processing or RNN-based methods; our method does not require data augmentation techniques nor an ensemble of multiple trained models, and only uses a more compact structure to obtain higher recognition accuracy.

Computational Intelligence-Based Method for Automated Identification of COVID-19 and Pneumonia by Utilizing CXR Scans.

Chest X-ray (CXR) scans are emerging as an important diagnostic tool for the early spotting of COVID and other significant lung diseases. The recognition of visual symptoms is difficult and can take longer time by radiologists as CXR provides various signs of viral infection. Therefore, artificial intelligence-based method for automated identification of COVID by utilizing X-ray images has been found to be very promising. In the era of deep learning, effective utilization of existing pretrained generalized models is playing a decisive role in terms of time and accuracy. In this paper, the benefits of weights of existing pretrained model VGG16 and InceptionV3 have been taken. Base model has been created using pretrained models (VGG16 and InceptionV3). The last fully connected (FC) layer has been added as per the number of classes for classification of CXR in binary and multi-class classification by appropriately using transfer learning. Finally, combination of layers is made by integrating the FC layer weights of both the models (VGG16 and InceptionV3). The image dataset used for experimentation consists of healthy, COVID, pneumonia viral, and pneumonia bacterial. The proposed weight fusion method has outperformed the existing models in terms of accuracy, achieved 99.5% accuracy in binary classification over 20 epochs, and 98.2% accuracy in three-class classification over 100 epochs.

An ECG Signal Denoising Method Using Conditional Generative Adversarial Net.

In this paper, a novel denoising method for electrocardiogram (ECG) signal is proposed to improve performance and availability under multiple noise cases. The method is based on the framework of conditional generative adversarial network (CGAN), and we improved the CGAN framework for ECG denoising. The proposed framework consists of two networks: a generator that is composed of the optimized convolutional auto-encoder (CAE) and a discriminator that is composed of four convolution layers and one full connection layer. As the convolutional layers of CAE can preserve spatial locality and the neighborhood relations in the latent higher-level feature representations of ECG signal, and the skip connection facilitates the gradient propagation in the denoising training process, the trained denoising model has good performance and generalization ability. The extensive experimental results on MIT-BIH databases show that for single noise and mixed noises, the average signal-to-noise ratio (SNR) of denoised ECG signal is above 39 dB, and it is better than that of the state-of-the-art methods. Furthermore, the denoised classification results of four cardiac diseases show that the average accuracy increased above 32 % under multiple noises under SNR=0 dB. So, the proposed method can remove noise effectively as well as keep the details of the features of ECG signals.

2D&3DHNet for 3D Object Classification in LiDAR Point Cloud

Accurate semantic analysis of LiDAR point clouds enables the interaction between intelligent vehicles and the real environment. This paper proposes a hybrid 2D and 3D Hough Net by combining 3D global Hough features and 2D local Hough features with a classification deep learning network. Firstly, the 3D object point clouds are mapped into the 3D Hough space to extract the global Hough features. The generated global Hough features are input into the 3D convolutional neural network for training global features. Furthermore, a multi-scale critical point sampling method is designed to extract critical points in the 2D views projected from the point clouds to reduce the computation of redundant points. To extract local features, a grid-based dynamic nearest neighbors algorithm is designed by searching the neighbors of the critical points. Finally, the two networks are connected to the full connection layer, which is input into fully connected layers for object classification.

Auxiliary diagnosis study of integrated electronic medical record text and CT images

Abstract At present, most of the research in the field of medical-assisted diagnosis is carried out based on image or electronic medical records. Although there is some research foundation, they lack the comprehensive consideration of comprehensive image and text modes. Based on this situation, this article proposes a fusion classification auxiliary diagnosis model based on GoogleNet model and Bi-LSTM model, uses GoogleNet to process brain computed tomographic (CT) images of ischemic stroke patients and extract CT image features, uses Bi-LSTM model to extract the electronic medical record text, integrates the two features using the full connection layer network and Softmax classifier, and obtains a method that can assist the diagnosis from two modes. Experiments show that the proposed scheme on average improves 3.05% in accuracy compared to individual image or text modes, and the best performing GoogleNet + Bi-LSTM model achieves 96.61% accuracy; although slightly less in recall, it performs better on F1 values, and has provided feasible new ideas and new methods for research in the field of multi-model medical-assisted diagnosis.

A Model for the Prediction of Lifetime Profit Estimate of Dairy Cattle (Student Abstract)

In livestock management, the decision of animal replacement requires an estimation of the lifetime profit of the animal based on multiple factors and operational conditions. In Dairy farms, this can be associated with the profit corresponding to milk production, health condition and herd management costs, which in turn may be a function of other factors including genetics and weather conditions. Estimating the profit of a cow can be expressed as a spatio-temporal problem where knowing the first batch of production (early-profit) can allow to predict the future batch of productions (late-profit). This problem can be addressed either by a univariate or multivariate time series forecasting. Several approaches have been designed for time series forecasting including Auto-Regressive approaches, Recurrent Neural Network including Long Short Term Memory (LSTM) method and a very deep stack of fully-connected layers. In this paper, we proposed a LSTM based approach coupled with attention and linear layers to better capture the dairy features. We compare the model, with three other architectures including NBEATs, ARIMA, MUMU-RNN using dairy production of 292181 dairy cows. The results highlight the performence of the proposed model of the compared architectures. They also show that a univariate NBEATs could perform better than the multi-variate approach there are compared to. We also highlight that such architecture could allow to predict late-profit with an error less than 3$ per month, opening the way of better resource management in the dairy industry.

A novel differential dynamic gradient descent optimization algorithm for resource allocation and offloading in the COMEC system

The multiuser cooperative offloading mobile edge computing (COMEC) system has attracted much attention because it can realize delay-sensitive tasks. However, in the coupling optimization of offloading decision and resource allocation, the existing numerical optimization algorithms are difficult to obtain high-quality optimization solutions. In this paper, we propose a differential dynamic gradient descent (DDGD) optimization algorithm to solve the above optimization problems. DDGD algorithm decomposes the constrained NP-hard optimization problem into two network layers and integrates the constraint function into a larger end-to-end training network. These two-layer networks encode the dependencies and optimization constraints between parameter hidden states, which cannot be captured by a numerical optimization model or a full connection layer neural network. Because the ring learning and self-repeating learning architecture are adopted and the information is stored in the differential dynamics network, the proposed algorithm can achieve better and more intelligent decision-making in searching the solution trajectory without setting accurate parameters in advance and reduce the complexity of the network. We show that compared with the baseline method, the DDGD method has superior optimization performance in the energy consumption optimization of COMEC.

BERT-BiGRU Intelligent Classification of Metro On-Board Equipment Faults Based on Key Layer Fusion

Metro train operation may result in massive and complex unstructured fault text data. To solve the problem of low classification accuracy and incomplete classification effect of unstructured fault data automatic classification task, a BERT-BiGRU fault text classification model based on key layer fusion is proposed. Firstly, the unstructured text data is processed into word vectors with location information in the word embedding layer and then input into the BERT layer. Based on the traditional 12-layer BERT model, the semantic information is fully obtained by encoding the two-way transformer encoder in layers 2, 4, 6, 8, and 12 for fusion and dimensionality reduction, which is then input into the BiGRU layer to extract the context information to obtain the high-level feature representation of the text. After that, the final classification results are output in the output layer through the full connection layer FC and softmax functions. This model is tested with other mainstream models on the fault text data of metro on-board equipment. The experiment results show that on the same data set, the F 1 -score of this model is 7%~8% higher than that of the current mainstream classification model, that is, about 1~2% higher than that of other PLMs, and the F 1 -score of this model is higher than that of BERT model with different transformer layers, reaching the highest value of 0.9272, and the convergence speed in the training process is fast.

HCFNN: High-order coverage function neural network for image classification

Recent advances in deep neural networks (DNNs) have mainly focused on innovations in network architecture and loss function. In this paper, we introduce a flexible high-order coverage function (HCF) neuron model to replace the fully-connected (FC) layers. The approximation theorem and proof for the HCF are also presented to demonstrate its fitting ability. Unlike the FC layers, which cannot handle high-dimensional data well, the HCF utilizes weight coefficients and hyper-parameters to mine underlying geometries with arbitrary shapes in an n-dimensional space. To explore the power and potential of our HCF neuron model, a high-order coverage function neural network (HCFNN) is proposed, which incorporates the HCF neuron as the building block. Moreover, a novel adaptive optimization method for weights and hyper-parameters is designed to achieve effective network learning. Comprehensive experiments on nine datasets in several domains validate the effectiveness and generalizability of the HCF and HCFNN. The proposed method provides a new perspective for further developments in DNNs and ensures wide application in the field of image classification. The source code is available at https://github.com/Tough2011/HCFNet.git

Multiple Ocular Disease Diagnosis Using Fundus Images Based on Multi-Label Deep Learning Classification

Designing computer-aided diagnosis (CAD) systems that can automatically detect ocular diseases (ODs) has become an active research field in the health domain. Although the human eye might have more than one OD simultaneously, most existing systems are designed to detect specific eye diseases. Therefore, it is crucial to develop new CAD systems that can detect multiple ODs simultaneously. This paper presents a novel multi-label convolutional neural network (ML-CNN) system based on ML classification (MLC) to diagnose various ODs from color fundus images. The proposed ML-CNN-based system consists of three main phases: the preprocessing phase, which includes normalization and augmentation using several transformation processes, the modeling phase, and the prediction phase. The proposed ML-CNN consists of three convolution (CONV) layers and one max pooling (MP) layer. Then, two CONV layers are performed, followed by one MP and dropout (DO). After that, one flatten layer is performed, followed by one fully connected (FC) layer. We added another DO once again, and finally, one FC layer with 45 nodes is performed. The system outputs the probabilities of all 45 diseases in each image. We validated the model by using cross-validation (CV) and measured the performance by five different metrics: accuracy (ACC), recall, precision, Dice similarity coefficient (DSC), and area under the curve (AUC). The results are 94.3%, 80%, 91.5%, 99%, and 96.7%, respectively. The comparisons with the existing built-in models, such as MobileNetV2, DenseNet201, SeResNext50, InceptionV3, and InceptionresNetv2, demonstrate the superiority of the proposed ML-CNN model.

4G/5G Cell-level multi-indicator forecasting based on dense-MLP

With the development of 4G/5G, the rapid growth of traffic has caused a large number of cell indicators to exceed the warning threshold, and network quality has deteriorated. It is necessary for operators to solve the congestion in advance and effectively to guarantee the quality of user experience. Cell-level multi-indicator forecasting is the foundation task for proactive complex network optimization. In this paper, we propose the 4G/5G Cell-level multi-indicator forecasting method based on the dense-Multi-Layer Perceptron (MLP) neural network, which adds additional fully-connected layers between non-adjacent layers in an MLP network. The model forecasted the following week's traffic indicators of 13000 cells according to the six-month historical indicators of 65000 cells in the 4G&5G network, which got the highest weighted MAPE score (0.2484) in the China Mobile problem statement in the ITU-T AI/ML in 5G Challenge 2021. Furthermore, the proposed model has been integrated into the AsiaInfo 4G/5G energy-saving system and deployed in Jiangsu Province of China.

A Novel CovidDetNet Deep Learning Model for Effective COVID-19 Infection Detection Using Chest Radiograph Images

The suspected cases of COVID-19 must be detected quickly and accurately to avoid the transmission of COVID-19 on a large scale. Existing COVID-19 diagnostic tests are slow and take several hours to generate the required results. However, on the other hand, most X-rays or chest radiographs only take less than 15 min to complete. Therefore, we can utilize chest radiographs to create a solution for early and accurate COVID-19 detection and diagnosis to reduce COVID-19 patient treatment problems and save time. For this purpose, CovidDetNet is proposed, which comprises ten learnable layers that are nine convolutional layers and one fully-connected layer. The architecture uses two activation functions: the ReLu activation function and the Leaky Relu activation function and two normalization operations that are batch normalization and cross channel normalization, making it a novel COVID-19 detection model. It is a novel deep learning-based approach that automatically and reliably detects COVID-19 using chest radiograph images. Towards this, a fine-grained COVID-19 classification experiment is conducted to identify and classify chest radiograph images into normal, COVID-19 positive, and pneumonia. In addition, the performance of the proposed novel CovidDetNet deep learning model is evaluated on a standard COVID-19 Radiography Database. Moreover, we compared the performance of our approach with hybrid approaches in which we used deep learning models as feature extractors and support vector machines (SVM) as a classifier. Experimental results on the dataset showed the superiority of the proposed CovidDetNet model over the existing methods. The proposed CovidDetNet outperformed the baseline hybrid deep learning-based models by achieving a high accuracy of 98.40%.

Hybrid convolutional network based on hyperspectral imaging for wheat seed varieties classification

Seed quality and safety are related to national food security, and seed variety purity is an important indicator in seed quality detection. In agricultural production, seeds of higher purity are more accessible with stable genetic traits. Thus, it is necessary to rapidly and non-destructively detect the seed purity. In this study, a novel hybrid convolutional network coupled with hyperspectral imaging was used for the sorting of wheat seeds. Hyperspectral imaging acquires spectral and spatial information simultaneously, which provides a collection of internal and external features of samples. To take full advantage of the spatial and spectral information, the hyperspectral images of wheat seeds were analyzed by hybrid convolution. Firstly, the average spectra were extracted from the region of interests (ROIs) and processed with one-dimensional (1D) convolution to extract spectral features. Meanwhile, the two-dimensional (2D) convolution was used to directly analyze hyperspectral images focusing more on extracting spatial features. Then, the features extracted by hybrid convolutional were integrated into the full connection layer of the network structure. The performance of the proposed models was validated with 8000 samples from the eight varieties of wheat seed. Experiments demonstrate that the hybrid convolutional network outperforms the reference model and gets excellent classification accuracy of 95.65% for wheat seeds.

A semantic analysis-driven customer requirements mining method for product conceptual design

Precise customer requirements acquisition is the primary stage of product conceptual design, which plays a decisive role in product quality and innovation. However, existing customer requirements mining approaches pay attention to the offline or online customer comment feedback and there has been little quantitative analysis of customer requirements in the analogical reasoning environment. Latent and innovative customer requirements can be expressed by analogical inspiration distinctly. In response, this paper proposes a semantic analysis-driven customer requirements mining method for product conceptual design based on deep transfer learning and improved latent Dirichlet allocation (ILDA). Initially, an analogy-inspired verbal protocol analysis experiment is implemented to obtain detailed customer requirements descriptions of elevator. Then, full connection layers and a softmax layer are added to the output-end of Chinese bidirectional encoder representations from Transformers (BERT) pre-training language model. The above deep transfer model is utilized to realize the customer requirements classification among functional domain, behavioral domain and structural domain in the customer requirement descriptions of elevator by fine-tuning training. Moreover, the ILDA is adopted to mine the functional customer requirements that can represent customer intention maximally. Finally, an effective accuracy of customer requirements classification is acquired by using the BERT deep transfer model. Meanwhile, five kinds of customer requirements of elevator and corresponding keywords as well as their weight coefficients in the topic-word distribution are extracted. This work can provide a novel research perspective on customer requirements mining for product conceptual design through natural language processing.

Water Quality Classification for Inland Lakes and Ponds with Few Color Image Samples Based on Triple-GAN and CSNN

Water color is an important representation reflecting the characteristics of its quality in inland lakes or ponds; however, sufficient water color image samples are often difficult to obtain due to the limitation of fishery production. For few color image samples, the existing data enhancement methods based on the depth generation model have the problems of low quality of generated data, difficulty of network training, and so on; moreover, for image classification, traditional methods based on convolutional neural network (CNN) cannot effectively extract the potential manifold structure features in the image and the full connection layer in CNN cannot simulate biological neurons well, resulting in high time cost and low efficiency. In this paper, a water quality classification method has been proposed to solve the above problems, the improved semisupervised triple-generation adversarial network (triple-GAN) algorithm is used to enhance the few water color image samples, and the feature data can then be extracted from enhanced data by manifold learning method t-distributed stochastic neighborhood embedding (t-SNE). Moreover, convolutional spiking neural network (CSNN), in which spiking neural network (SNN) has replaced the original full connection layer of CNN, is used for final water quality classification. The main contribution of this paper is to build a new algorithm framework, introduce triple-GAN and CSNN into the field of classification of few water color image samples for the first time, and make an exploration of integrating artificial intelligence (AI) and water quality analysis problems. By comparing with traditional methods, the proposed method is proved to have the advantages of less time-consuming, low operation cost, and high classification accuracy.

An Improved MobileNet Network with Wavelet Energy and Global Average Pooling for Rotating Machinery Fault Diagnosis.

In recent years, neural networks have shown good performance in terms of accuracy and efficiency. However, along with the continuous improvement in diagnostic accuracy, the number of parameters in the network is increasing and the models can often only be run in servers with high computing power. Embedded devices are widely used in on-site monitoring and fault diagnosis. However, due to the limitation of hardware resources, it is difficult to effectively deploy complex models trained by deep learning, which limits the application of deep learning methods in engineering practice. To address this problem, this article carries out research on network lightweight and performance optimization based on the MobileNet network. The network structure is modified to make it directly suitable for one-dimensional signal processing. The wavelet convolution is introduced into the convolution structure to enhance the feature extraction ability and robustness of the model. The excessive number of network parameters is a challenge for the deployment of networks and also for the running performance problems. This article analyzes the influence of the full connection layer size on the total network. A network parameter reduction method is proposed based on GAP to reduce the network parameters. Experiments on gears and bearings show that the proposed method can achieve more than 97% classification accuracy under the strong noise interference of −6 dB, showing good anti-noise performance. In terms of performance, the network proposed in this article has only one-tenth of the number of parameters and one-third of the running time of standard networks. The method proposed in this article provides a good reference for the deployment of deep learning intelligent diagnosis methods in embedded node systems.