Famous Neural Network Research Articles

Stock Price Prediction Using Deep-Learning Models: CNN, RNN, and LSTM

With the rapid development of the economy, stock markets or equity markets have an important role nowadays. More and more people participate in stock investment, the rise or the fall in prices is vital and closely related to investors’ earnings. The basic way uses linear or non-linear algorithms, but the stock market has many factors, so it is highly non-linear prediction, so it is helpless to use one simple model, so this paper proposes to figure out a good deep-learning model to capture and analyze the data of six companies from Yahoo Finance by comparing the fitness of three famous neural network: CNN, RNN, and LSTM. The Sliding-Window model was applied to make future predictions in time series. The results of the models were calculated by using MSE, MAE, and MAPE.

Stochastic simulation of deltas based on a concurrent multi-stage VAE-GAN model

The stochastic simulation of deltas has always been one of the significant problems in the numerical simulation of reservoirs. Three problems of deltaic simulations are non-stationary geological characteristics in training images (TIs), insufficient quantity of TIs in certain regions and unavailable reuse of training parameters and models even after the first training, which have caused big challenges for the stochastic simulation of deltas especially in the circumstances of only heterogeneous TIs are available and the simulation quantity is quite large. As one of the important numerical methods for the simulation of deltas, multiple-point statistics (MPS) obtains the statistical characteristics through the patterns in TIs to perform simulation. However, due to the non-stationarity of deltaic TIs, the traditional MPS cannot extract the non-stationary characteristics of TIs smoothly and is incapable of reusing the extracted probability information, making the simulation process quite time-consuming if multiple simulations are performed successively. Thanks to the powerful feature extraction capability brought by deep learning, the delta simulation possibly will be largely improved. The generative adversarial network (GAN) is an important deep learning method for image generation, but it needs large-quantity training samples and its training process is unstable, but another famous neural network variational auto-encoder (VAE) is more stable. On the other hand, VAE-generated images are often a little blurred but GAN-generated images are clearer. Therefore, based on VAE, GAN and a multi-stage idea, a concurrent multi-stage VAE-GAN model is proposed for the stochastic simulation of deltas to hopefully address the three problems in deltaic simulation by concurrently performing multi-stage simulation based on GAN and VAE. The comparison of our method with some typical MPS methods and deep learning methods has shown its good performance in deltaic simulation.

Prediction of RNA secondary structure with pseudoknots using coupled deep neural networks

Noncoding RNAs play important roles in cell and their secondary structures are vital for understanding their tertiary structures and functions. Many prediction methods of RNA secondary structures have been proposed but it is still challenging to reach high accuracy, especially for those with pseudoknots. Here we present a coupled deep learning model, called 2dRNA, to predict RNA secondary structure. It combines two famous neural network architectures bidirectional LSTM and U-net and only needs the sequence of a target RNA as input. Benchmark shows that our method can achieve state-of-the-art performance compared to current methods on a testing dataset. Our analysis also shows that 2dRNA can learn structural information from similar RNA sequences without aligning them.

Deep Learning Approach for Left Atrium Segmentation

Background: Cardiovascular disease is one of the most common causes of death in the world. Cardiovascular risk is often assessed by analyzing parameters extracted from image data taken from different parts of the heart. Transthoracic echocardiography is a non-invasive imaging modality that is usually the first step of diagnosis procedure. Extracting more information from epical and parasternal views can lead the doctor to early diagnosis. The examination analysis, although has a valuable role, relies on the operator experience. To have a more standard analysis, it is important to reduce the variability of the examination analysis. Automatic image analysis, which is very helpful in this context, can be implemented in image segmentation, parameter measurement, and even diagnosis levels. Left atrial measurements (mostly the atrial volume) are strong predictors of cardiac events. Elevated atrial pressure or increased flow can lead to atrial enlargement. There is evidence that patients with high left atrial volume are at risk of ischemic stroke, heart failure, and atrial fibrillation. Objectives: The purpose of this paper was to conduct automatic segmentation of left atrium in four-chamber view images to make measurements reliable and independent from expert experience. Generally, most methods of automatic segmentation are based on image processing algorithms that are sometimes too complicated and less accurate. Deep learning models have been widely used in different computer vision areas. These models are mathematically simple and are proven to have better accuracy in computer vision problems. Accordingly, we used one of the famous neural network structures named Unet for left atrium segmentation in 2D echocardiography. Comparing the trained network’s results with the ground truth segmentations made by experts, we showed that neural networks were strong choices for automation. Methods: The dataset used in this research was the public echocardiography image dataset published in CAMUS (Cardiac Acquisitions for Multi-structure Ultrasound Segmentation) Challenge. The data included four-chamber view end-systole and end-diastole frames from 450 patients. Unet is a kind of pyramidal network with encoding and decoding paths. The encoder or contraction path was used to capture the context and features in the image. The second path was the symmetric expanding or decoder path that was used to enable precise localization. The whole task of the network was to classify each pixel in the image to the background or left atrium classes. The loss function in the learning procedure was defined as the Dice coefficient, which depicted the ratio of the number of pixels that were correctly classified. For comparing the method with experts, five-fold cross-validation was used. Each time, 10% of the dataset was used as a test set and 90% as a train set. In the training step, the network learned the train set annotations and in the testing procedure, the performance of the trained network was evaluated based on its predictions on the test set. Dice coefficient, Hausdorff distance, and mean absolute distance (MAD) were used to evaluate the accuracy of the method in echocardiography images. Results: The calculated metrics were dice coefficient of 91.17%, Hausdorff distance of 3.31, and MAD of 0.71 for the train set and dice coefficient of 88.14%, Hausdorff distance of 4.75, and MAD of 1.67 for the test set. Conclusion: We showed that using Unet architecture in four-chamber view echocardiography images for bordering left atrium can be a powerful alternative to remove the variability between different experts. Evaluation metrics analysis proved that we can rely on the method predictions and have a more accurate shape of the left atrium for further analysis and measurements such as volume and other features measurement.

TGV Upsampling: A Making-Up Operation for Semantic Segmentation.

With the widespread use of deep learning methods, semantic segmentation has achieved great improvements in recent years. However, many researchers have pointed out that with multiple uses of convolution and pooling operations, great information loss would occur in the extraction processes. To solve this problem, various operations or network architectures have been suggested to make up for the loss of information. We observed a trend in many studies to design a network as a symmetric type, with both parts representing the “encoding” and “decoding” stages. By “upsampling” operations in the “decoding” stage, feature maps are constructed in a certain way that would more or less make up for the losses in previous layers. In this paper, we focus on upsampling operations, make a detailed analysis, and compare current methods used in several famous neural networks. We also combine the knowledge on image restoration and design a new upsampled layer (or operation) named the TGV upsampling algorithm. We successfully replaced upsampling layers in the previous research with our new method. We found that our model can better preserve detailed textures and edges of feature maps and can, on average, achieve 1.4–2.3% improved accuracy compared to the original models.

Improved synchronization analysis of competitive neural networks with time-varying delays

Synchronization and control are two very important aspects of any dynamical systems. Among various kinds of nonlinear systems, competitive neural network holds a very important place due to its application in diverse fields. The model is general enough to include, as subclass, the most famous neural network models such as competitive neural networks, cellular neural networks and Hopfield neural networks. In this paper, the problem of feedback controller design to guarantee synchronization for competitive neural networks with time-varying delays is investigated. The goal of this work is to derive an existent criterion of the controller for the exponential synchronization between drive and response neutral-type competitive neural networks with time-varying delays. The method used in this brief is based on feedback control gain matrix by using the Lyapunov stability theory. The synchronization conditions are given in terms of LMIs. To the best of our knowledge, the results presented here are novel and generalize some previous results. Some numerical simulations are also represented graphically to validate the effectiveness and advantages of our theoretical results.

Time series forecasting based on wavelet decomposition and feature extraction

Time series forecasting is one of the most important issues in numerous applications in real life. The objective of this study was to propose a hybrid neural network model based on wavelet transform (WT) and feature extraction for time series forecasting. The motivation of the proposed model, which is called PCA-WCCNN, is to establish a single simplified model with shorter training time and satisfactory forecasting performance. This model combines the principal component analysis (PCA) and WT with artificial neural networks (ANNs). Given a forecasting sequence, order of the original forecasting model is determined firstly. Secondly, the original time series is decomposed into approximation and detail components by employing WT technique. Then, instead of using all the components as inputs, feature inputs are extracted from all the sub-series obtained from the above step. Finally, based on the extracted features and all the sub-series, a famous neural network construction method called cascade-correlation algorithm is applied to train neural network model to learn the dynamics. As an illustration, the proposed model is compared with two classical models and two hybrid models, respectively. They are the traditional cascade-correlation neural network, back-propagation neural network, wavelet-based cascade-correlation network using all the wavelet components as inputs to establish one model (WCCNN) and wavelet-based cascade-correlation network with combination of each sub-series model (WCCNN multi-models). Results obtained from this study indicate that the proposed method improves the accuracy of ANN and can yield better efficiency than other four neural network models.

Global asymptotic stability of nonautonomous Cohen–Grossberg neural network models with infinite delays

For a general Cohen–Grossberg neural network model with potentially unbounded time-varying coefficients and infinite distributed delays, we give sufficient conditions for its global asymptotic stability. The model studied is general enough to include, as subclass, the most of famous neural network models such as Cohen–Grossberg, Hopfield, and bidirectional associative memory. Contrary to usual in the literature, in the proofs we do not use Lyapunov functionals. As illustrated, the results are applied to several concrete models studied in the literature and a comparison of results shows that our results give new global stability criteria for several neural network models and improve some earlier publications.

Common nature of learning between BP-type and Hopfield-type neural networks

Being two famous neural networks, the error back-propagation (BP) algorithm based neural networks (i.e., BP-type neural networks, BPNNs) and Hopfield-type neural networks (HNNs) have been proposed, developed, and investigated extensively for scientific research and engineering applications. They are different from each other in a great deal, in terms of network architecture, physical meaning and training pattern. In this paper of literature-review type, we present in a relatively complete and creative manner the common natures of learning between BP-type and Hopfield-type neural networks for solving various (mathematical) problems. Specifically, comparing the BPNN with the HNN for the same problem-solving task, e.g., matrix inversion as well as function approximation, we show that the BPNN weight-updating formula and the HNN state-transition equation turn out to be essentially the same. Such interesting phenomena promise that, given a neural-network model for a specific problem solving, its potential dual neural-network model can thus be developed.

Programming squat wall strengths and tuning associated codes with pruned modular neural network

This study designed a four-layer modular neural network (MNN) to predict and program squat wall strength values. Results generated by the proposed MNN include predictions and programmed formulas that are similar in form to modular polynomials, which permit MNN programming to interpret training results in a meaningful way that offers significant advantages over famous neural networks. This study employed particle swarm optimization for MNN parameter learning and structure learning in order to prune MNN to avoid overfitting and increase programmed formula concision. To extend the uses of MNN programming, this paper further employed MNN tuning to refine existing analytical methods and codes. Case studies focused on squat wall strength analyses. Study results demonstrated that MNN programming uniquely uses a programmed formula to deliver good prediction accuracy. MNN tuning further improved the studied methods. Programmed formulas also provided insights into input parameter impacts and significant modular functions.

Synchronization for an Array of Coupled Cohen‐Grossberg Neural Networks with Time‐Varying Delay

This paper makes some great attempts to investigate the global exponential synchronization for arrays of coupled delayed Cohen‐Grossberg neural networks with both delayed coupling and one single delayed one. By resorting to free‐weighting matrix and Kronecker product techniques, two novel synchronization criteria via linear matrix inequalities (LMIs) are presented based on convex combination, in which these conditions are heavily dependent on the bounds of both the delay and its derivative. Owing to that the addressed system can include some famous neural network models as the special cases, the proposed methods can extend and improve those earlier reported ones. The efficiency and applicability of the presented conditions can be demonstrated by two numerical examples with simulations.

Synchronization control for arrays of coupled discrete-time delayed Cohen–Grossberg neural networks

This paper investigates the global exponential synchronization for an array of coupled discrete-time Cohen–Grossberg neural networks (CGNNs) with time-varying delay, in which both the constant coupling and delayed one are considered. Through constructing an improved Lyapunov–Krasovskii functional, the delay-dependent sufficient condition is obtained to guarantee the global synchronization based on linear matrix inequality (LMI) approach. The criterion is presented in terms of LMIs and its feasibility can be easily checked by resorting to Matlab LMI Toolbox. Moreover, the addressed system can include some famous neural network models as its special cases, which can help extend those present results. Finally, the effectiveness of the proposed method can be further illustrated with the help of two numerical examples.