Fully-connected Artificial Neural Networks Research Articles

A miRNA-disease association prediction model based on tree-path global feature extraction and fully connected artificial neural network with multi-head self-attention mechanism

BackgroundMicroRNAs (miRNAs) emerge in various organisms, ranging from viruses to humans, and play crucial regulatory roles within cells, participating in a variety of biological processes. In numerous prediction methods for miRNA-disease associations, the issue of over-dependence on both similarity measurement data and the association matrix still hasn’t been improved. In this paper, a miRNA-Disease association prediction model (called TP-MDA) based on tree path global feature extraction and fully connected artificial neural network (FANN) with multi-head self-attention mechanism is proposed. The TP-MDA model utilizes an association tree structure to represent the data relationships, multi-head self-attention mechanism for extracting feature vectors, and fully connected artificial neural network with 5-fold cross-validation for model training.ResultsThe experimental results indicate that the TP-MDA model outperforms the other comparative models, AUC is 0.9714. In the case studies of miRNAs associated with colorectal cancer and lung cancer, among the top 15 miRNAs predicted by the model, 12 in colorectal cancer and 15 in lung cancer were validated respectively, the accuracy is as high as 0.9227.ConclusionsThe model proposed in this paper can accurately predict the miRNA-disease association, and can serve as a valuable reference for data mining and association prediction in the fields of life sciences, biology, and disease genetics, among others.Graphical

An Advance Study of an Efficient CNN-Grounded Deep Learning Classification Technique for the Diagnosis of IoT based Cardiac Arrhythmias

Deep Learning, or DL for short, is an emerging subfield within the larger discipline of machine learning in today's world. The study being conducted in this area is progressing at an immediate stride, and the discoveries are contributing to the progression of technology. Deep learning (DL) methods were developed with the intention of developing a general-purpose learning method that would enable the gradual learning of characteristics at multiple levels without relying on human-engineered features. This was the goal of deep learning. Because of this, the system is able to acquire intricate purposes and directly map input to output by making use of the data that it has acquired which is based on Internet of things (IoTs). This study places an emphasis on the application of CNN (Convolutional Neural Networks), which are a subcategory of DNN (Deep Neural Networks), and it develops an efficient layered CNN for the classification of ECG arrhythmias. Even while FC-ANNs (Fully Connected Artificial Neural Networks), which are sometimes referred to as Multilayer-Perceptron networks, are effective in categorising ECG arrhythmias, the optimization process for many classification networks takes a significant amount of time in terms of computation. In addition, the features extracted by engineers are what define the accuracy of the categorization of ECG arrhythmias. An improved CNN based filtering, feature abstraction, and classification prototypical is established in order to conduct an accurate analysis of an electrocardiogram (ECG). When measured against ANN, the performance was found to have an accuracy rating of 99.6%. Consequently, the CNN model that was suggested is useful to doctors in arriving at the definitive diagnosis of AFL (atrial flutter), AFIB (atrial fibrillation), VFL (ventricular flutter), and VT (ventricular tachycardia). It includes denoising, feature extraction, and categorization as part of its functionality.

On the quality of training artificial neural networks without initial data preprocessing under conditions of their limited set

The task of training artificial neural networks (ANNs) for the diagnosis of Alzheimer's disease based on the magnetic susceptibility of 10 brain veins under conditions of limited initial data (81 patients) is considered. In a recent study, fully-connected ANNs with one and two hidden layers were successfully trained in a recent study after reducing the number of veins to 4 and determining the number of synaptic connections comparable to a neural network based on a decision root. The known results gave rise to the hypothesis that training ANNs on the same data without preprocessing would also yield high-quality neural networks. To assess the quality of ANNs, the following indicators were used: mean square errors of training and generalization (testing), coefficients of determination (R2) on the sets of values used in training and generalization (testing), as well as the number of trained ANNs satisfying a certain value of the above indicators. For graphical representation of the results of repeated experiments on training and testing of ANNs, a new graph is proposed to visualize the quality of ANNs. Thus, the paper demonstrates the quality indicators of 800 ANNs trained without data preprocessing, among which there were 7 neural networks with one hidden layer and 9 neural networks with two hidden layers, for each of which 50 experiments were conducted. For comparison, we present the quality indices of 2400 ANNs trained with preprocessing.

ВЫБОР АРХИТЕКТУРЫ ИСКУССТВЕННОЙ НЕЙРОННОЙ СЕТИ ДЛЯ ОЦЕНКИ СОСТОЯНИЯ ПОПУЛЯЦИИ ГЕНЕТИЧЕСКОГО АЛГОРИТМА В ЗАДАЧЕ СТРУКТУРНО-ПАРАМЕТРИЧЕСКОГО СИНТЕЗА ИМИТАЦИОННЫХ МОДЕЛЕЙ БИЗНЕС-ПРОЦЕССОВ

This article proposes a study aimed at determining the architecture of artificial neural networks to solve the problem of determining the population state of a genetic algorithm adapted to solve the problem of structuralparametric synthesis of simulation models of business processes. As the initial data for training the artificial neural network, we used the results of computational experiments obtained when operating a genetic algorithm model based on mathematical nested Petri nets, which solves the problem of synthesizing business process models (Petri net models) based on a given behavior. As examples of artificial neural network architectures for managing the process of finding solutions based on an evolutionary procedure, the following are considered: fully connected artificial neural network (FCNN), simple recurrent artificial neural network (Simple RNN), long shortterm memory recurrent network (LSTN), closed recurrent recurrent network block (GRU) and bidirectional LSTM (Bidirectional LSTM). The deep learning algorithms used were: Support Vector Classifier, Decision Tree Classifier and Random Forest Classifier. The article discusses the presented architectures of artificial neural networks and various training methods. Based on the computational experiments carried out and the analysis of the results obtained, conclusions were drawn about the feasibility of using artificial neural networks with RNN architecture to solve the problem of recognizing the state of the population and controlling the process of synthesis of solutions.

Deep-Learning-Based Earthquake Detection for Fiber-Optic Distributed Acoustic Sensing

In this paper, deep learning models trained with real seismic data are proposed and proven to detect earthquakes in fiber-optic distributed acoustic sensor (DAS) measurements. The proposed neural network architectures cover the three classical deep learning paradigms: fully connected artificial neural networks (FC-ANNs), convolutional neural networks (CNNs) and recurrent neural networks (RNNs). Results demonstrate that training these networks with seismic waveforms measured by traditional broadband seismometers can extract and learn relevant features of earthquakes, enabling the reliable detection of seismic waves in DAS measurements. The intrinsic differences between DAS and seismograph waveforms, and eventual errors in the labelling of the DAS data, slightly reduce the performance of the models when tested with the distributed acoustic measurements. Despites of that, trained models can still reach up to 96.94% accuracy in the case of CNN and 93.86% in the case of CNN+RNN. The method and results here reported could represent an important contribution to the development of an early warning earthquake system based on DAS technology.

A Comparative Study of a Fully-Connected Artificial Neural Network and a Convolutional Neural Network in Predicting Bridge Maintenance Costs

The cost assessment of bridge maintenance is a difficult topic to study, but it is critical for a bridge life cycle cost analysis. The maintenance costs sample database was established in this study according to actual engineering data, and a bridge maintenance cost prediction model was developed using a fully-connected artificial neural network (ANN) and convolutional neural network (CNN), respectively. First, eight main factors affecting maintenance costs were evaluated based on the random forest method, and the evaluation results were verified by an exploratory data analysis. The original data were then screened based on the isolation forest principle, and the recent gross domestic product (GDP) growth rate was used to illustrate the relationship between economic development and bridge maintenance costs. Finally, these two neural networks were used to establish maintenance cost prediction models, respectively. The results from the two models were compared and their prediction accuracies were analyzed. The prediction performance of the CNN model for bridge maintenance costs was found to be better than that of the traditional fully-connected ANN model. The results of this study will enhance the opportunity for bridge managers to balance lifecycle maintenance costs.

Data-driven method for real-time prediction and uncertainty quantification of fatigue failure under stochastic loading using artificial neural networks and Gaussian process regression

Various engineering systems such as naval and aerial vehicles, offshore structures, and mechanical components of motorized systems, are exposed to fatigue failures due to stochastic loadings. Methods for early failure prediction are essential for engineering, military, and civil applications. In addition to the prediction of time to failure (TtF), uncertainty quantification (UQ) is of major importance for real-time decision-making purposes. Usually, time domain or frequency domain methods are used for fatigue prediction, such as rainflow counting and Miner’s rule or Dirlik’s method. However, those methods suffer from over-simplistic modeling and inaccurate failure predictions under stochastic loadings. During the last years, several data-driven models were suggested for offline fatigue failure. However, most of them are not capable of both accurate real-time fatigue prediction and UQ. In the current work, a probabilistic data-driven model is introduced. A hybrid architecture of a fully connected artificial neural network (FC-ANN) and Gaussian process regression (GPR) is proposed to ensure enhanced predictive abilities and simultaneous UQ of the predicted TtF. The real-time prediction and UQ performances of the suggested model are validated using both synthetic and experimental data. This novel hybrid method is fully data-driven and extends the forecasting capabilities of existing time-domain and machine learning-based methods for fatigue prediction. It paves the way towards the development of a preventive system that provides real-time safety and operational instructions and insights for structural health monitoring (SHM) purposes, allowing prevention of environmental damage, and loss of human lives.

Non-uniform adaptive angular spectrum method and its application to neural network assisted coherent beam combining

Increasing the number of laser beams that can be coherently combined requires accurate and fast algorithms for compensating phase and alignment errors. The paper proposes to use a Fully Connected Artificial Neural Network (FCANN) to correct the beam positioning perturbations by evaluating the beam shifts and tilts from two images taken at slightly different locations. Then, since it is practically impossible to have a large enough experimental dataset to train the neural network, this approach required developing an accurate and fast simulation method to evaluate the beam propagation in arbitrary directions, overcoming the limitations occurring when the computation must be repeated a large number of times. The numerical approach is a variant of the Angular Spectrum (AS) method, called Non Uniform ADaptive Angular Spectrum (NUADAS) method, which relies on the combination of non-uniform and adaptive Fourier transform algorithms to allow the computation of an arbitrary field distribution in a plane that is shifted and tilted with respect to the source. The parallel implementation of the NUADAS method is discussed and the numerical and experimental validations are presented. Then, an FCANN is trained using the synthetic dataset generated with the NUADAS method and the results are discussed, demonstrating the viability of the proposed approach not only for coherent beam combing, but also in other beam alignment applications.

Towards Detecting Red Palm Weevil Using Machine Learning and Fiber Optic Distributed Acoustic Sensing

Red palm weevil (RPW) is a detrimental pest, which has wiped out many palm tree farms worldwide. Early detection of RPW is challenging, especially in large-scale farms. Here, we introduce the combination of machine learning and fiber optic distributed acoustic sensing (DAS) techniques as a solution for the early detection of RPW in vast farms. Within the laboratory environment, we reconstructed the conditions of a farm that includes an infested tree with ∼12 day old weevil larvae and another healthy tree. Meanwhile, some noise sources are introduced, including wind and bird sounds around the trees. After training with the experimental time- and frequency-domain data provided by the fiber optic DAS system, a fully-connected artificial neural network (ANN) and a convolutional neural network (CNN) can efficiently recognize the healthy and infested trees with high classification accuracy values (99.9% by ANN with temporal data and 99.7% by CNN with spectral data, in reasonable noise conditions). This work paves the way for deploying the high efficiency and cost-effective fiber optic DAS to monitor RPW in open-air and large-scale farms containing thousands of trees.

Optimized neural networks in industrial data analysis

The research work presented here tackles the problem of finding the optimum value for hyper-parameters, such as number of layers and number of neurons per layer, for a fully-connected Artificial Neural Network (ANN), particularly in regression problems. A proposed optimization strategy is tested on different datasets related to diverse industrial applications: (1) prediction of the performance of exploration algorithms for mobile robots, (2) prediction of the compressive strength of concrete, (3) prediction of energy output from a power plant; and (4) prediction of wine quality. Different evaluation metrics, such as Pearson correlation coefficient (R), Square Correlation Coefficient (R2), Absolute Relative Error (RAE), Root Mean Square Error (RMSE), and Mean Absolute Error (MAE), are used to determine the best performing prediction model when the hyper-parameter optimization algorithms are used. That result is compared to the performance of the best model previously reported in the literature on the same dataset in order to determine the gains achieved by using the hyper-parameter optimization strategy under test.

РЕШЕНИЕ ОБРАТНОЙ ЗАДАЧИ МЕТОДА МНОГОКАНАЛЬНОГО АНАЛИЗА ПОВЕРХНОСТНЫХ ВОЛН НА ОСНОВЕ ИСКУССТВЕННОЙ НЕЙРОННОЙ СЕТИ

The study is devoted to improving the efficiency of the method of multichannel analysis of surface waves due to the implementation of a new algorithm for the selection of 1D-models of the shear wave velocity by inverse dispersion curves based on machine learning. It is proposed to use a multilayer fully-connected artificial neural network (ANN) for the estimate of horizontally layered models composed of interpolated velocity values on a uniform grid with an arbitrary step. Various types of correlation dependence of the parameters of the velocity model on dispersion curves were investigated. The optimal type of the objective function was determined for ANN learning on the basis of these studies. The advantages of using a trained ANN for the problem of inversion dispersion curves are: the ability to estimate layered models with sufficient accuracy for their use at subsequent stages of seismic data processing, resistance to noise, no significant requirements for computational resources and the absence of the need for additional adjustment of parameters. The tests of the trained ANN, with the architecture proposed in this study, show high accuracy in validation (about 96%), which indicates the success of training and the actually perspective of using neural networks for inversion problems.

Progressive Operational Perceptrons

There are well-known limitations and drawbacks on the performance and robustness of the feed-forward, fully-connected Artificial Neural Networks (ANNs), or the so-called Multi-Layer Perceptrons (MLPs). In this study we shall address them by Generalized Operational Perceptrons (GOPs) that consist of neurons with distinct (non-)linear operators to achieve a generalized model of the biological neurons and ultimately a superior diversity. We modified the conventional back-propagation (BP) to train GOPs and furthermore, proposed Progressive Operational Perceptrons (POPs) to achieve self-organized and depth-adaptive GOPs according to the learning problem. The most crucial property of the POPs is their ability to simultaneously search for the optimal operator set and train each layer individually. The final POP is, therefore, formed layer by layer and in this paper we shall show that this ability enables POPs with minimal network depth to attack the most challenging learning problems that cannot be learned by conventional ANNs even with a deeper and significantly complex configuration. Experimental results show that POPs can scale up very well with the problem size and can have the potential to achieve a superior generalization performance on real benchmark problems with a significant gain.

Evaluation of global and local training techniques over feed-forward neural network architecture spaces for computer-aided medical diagnosis

In this paper, we investigate the performance of global vs. local techniques applied to the training of neural network classifiers for solving medical diagnosis problems. The presented methodology of the investigation involves systematic and exhaustive evaluation of the classifier performance over a neural network architecture space and with respect to training depth for a particular problem. In this study, the architecture space is defined over feed-forward, fully-connected artificial neural networks (ANNs) which have been widely used in computer-aided decision support systems in medical domain, and for which two popular neural network training methods are explored: conventional backpropagation (BP) and particle swarm optimization (PSO). Both training techniques are compared in terms of classification performance over three medical diagnosis problems ( breast cancer, heart disease, and diabetes) from Proben1 benchmark dataset and computational and architectural analysis are performed for an extensive assessment. The results clearly demonstrate that it is not possible to compare and evaluate the performance of the two algorithms over a single network and with a fixed set of training parameters, as most of the earlier work in this field has been carried out, since training and test classification performances vary significantly and depend directly on the network architecture, the training depth and method used and the available dataset. We, therefore, show that an extensive evaluation method such as the one proposed in this paper is basically needed to obtain a reliable and detailed performance assessment, in that, we can conclude that the PSO algorithm has usually a better generalization ability across the architecture space whereas BP can occasionally provide better training and/or test classification performance for some network configurations. Furthermore, we can in general say that the PSO, as a global training algorithm, is capable of achieving minimum test classification errors regardless of the training depth, i.e. shallow or deep, and its average classification performance shows less variations with respect to network architecture. In terms of computational complexity, BP is in general superior to PSO for the entire architecture space used.

Evolutionary artificial neural networks by multi-dimensional particle swarm optimization

In this paper, we propose a novel technique for the automatic design of Artificial Neural Networks (ANNs) by evolving to the optimal network configuration(s) within an architecture space. It is entirely based on a multi-dimensional Particle Swarm Optimization (MD PSO) technique, which re-forms the native structure of swarm particles in such a way that they can make inter-dimensional passes with a dedicated dimensional PSO process. Therefore, in a multidimensional search space where the optimum dimension is unknown, swarm particles can seek both positional and dimensional optima. This eventually removes the necessity of setting a fixed dimension a priori, which is a common drawback for the family of swarm optimizers. With the proper encoding of the network configurations and parameters into particles, MD PSO can then seek the positional optimum in the error space and the dimensional optimum in the architecture space. The optimum dimension converged at the end of a MD PSO process corresponds to a unique ANN configuration where the network parameters (connections, weights and biases) can then be resolved from the positional optimum reached on that dimension. In addition to this, the proposed technique generates a ranked list of network configurations, from the best to the worst. This is indeed a crucial piece of information, indicating what potential configurations can be alternatives to the best one, and which configurations should not be used at all for a particular problem. In this study, the architecture space is defined over feed-forward, fully-connected ANNs so as to use the conventional techniques such as back-propagation and some other evolutionary methods in this field. The proposed technique is applied over the most challenging synthetic problems to test its optimality on evolving networks and over the benchmark problems to test its generalization capability as well as to make comparative evaluations with the several competing techniques. The experimental results show that the MD PSO evolves to optimum or near-optimum networks in general and has a superior generalization capability. Furthermore, the MD PSO naturally favors a low-dimension solution when it exhibits a competitive performance with a high dimension counterpart and such a native tendency eventually yields the evolution process to the compact network configurations in the architecture space rather than the complex ones, as long as the optimality prevails.

Application of neural nets to ultrasound tissue characterization

There are well-known limitations and drawbacks on the performance and robustness of the feed-forward, fully-connected Artificial Neural Networks (ANNs), or the so-called Multi-Layer Perceptrons (MLPs). In this study we shall address them by Generalized Operational Perceptrons (GOPs) that consist of neurons with distinct (non-)linear operators to achieve a generalized model of the biological neurons and ultimately a superior diversity. We modified the conventional back-propagation (BP) to train GOPs and furthermore, proposed Progressive Operational Perceptrons (POPs) to achieve self-organized and depth-adaptive GOPs according to the learning problem. The most crucial property of the POPs is their ability to simultaneously search for the optimal operator set and train each layer individually. The final POP is, therefore, formed layer by layer and in this paper we shall show that this ability enables POPs with minimal network depth to attack the most challenging learning problems that cannot be learned by conventional ANNs even with a deeper and significantly complex configuration. Experimental results show that POPs can scale up very well with the problem size and can have the potential to achieve a superior generalization performance on real benchmark problems with a significant gain.