Connection Weights Of Neural Network Research Articles

Neural network operators of generalized fractional integrals equipped with a vector-valued function

In this paper, motivated by the recent development of the approximation capability of neural network operators, we aim to construct neural network (NN) operators for multivariate fractional integrals of order α, which are equipped with a vector-valued function ψ and its Jacobian matrix. In this system, the connection strengths between output neurons are represented by fractional mean values, which depend on the vector-valued function ψ, of the approximated multivariate function. To estimate the rate of convergence (learning rate), the connection weights of NN are equipped with a decreasing sequence (λn). The activation function of the system is constructed using a linear collection ΩΨ, which consists of density functions generated by multivariate sigmoidal functions. Our goal is to construct a flexible and productive hybrid system by leveraging a diverse selection of the function ψ, the parameter α, the sequence (λn) and the activation function ΩΨ. The quantitative estimates of the operators are examined by means of the multivariate modulus of continuity. Moreover, we provide some illustrative examples with graphs to demonstrate the approximation performance of the operators through the selected activation functions. Finally, we present numerical results consisting of the maximum absolute errors of approximation for the proposed operators based on various selections of the vector-valued function ψ.

Intelligent Diagnosis Model of Alzheimer's Disease Based on PSO Algorithm Optimized BP Neural Network

In this paper, an intelligent diagnosis model of Alzheimer's disease is established based on the PSO algorithm optimized BP neural network. After descriptive analysis and normalization of patient-specific information characteristic data, the model was established by parameter selection of BP neural network, optimization of BP neural network by PSO algorithm and sensitivity analysis of neural network connection weights, and the intelligent diagnostic model of Alzheimer's disease was tested. The results showed that the model has strong applicability in terms of the degree of Alzheimer's disease in patients, and provides a certain reference basis for accurate diagnosis of the developmental stage of Alzheimer's disease.

Multistrategy Improved Sparrow Search Algorithm Optimized Deep Neural Network for Esophageal Cancer.

Deep neural network is a complex pattern recognition network system. It is widely favored by scholars for its strong nonlinear fitting ability. However, training deep neural network models on small datasets typically realizes worse performance than shallow neural network. In this study, a strategy to improve the sparrow search algorithm based on the iterative map, iterative perturbation, and Gaussian mutation is developed. This optimized strategy improved the sparrow search algorithm validated by fourteen benchmark functions, and the algorithm has the best search accuracy and the fastest convergence speed. An algorithm based on the iterative map, iterative perturbation, and Gaussian mutation improved sparrow search algorithm is designed to optimize deep neural networks. The modified sparrow algorithm is exploited to search for the optimal connection weights of deep neural network. This algorithm is implemented for the esophageal cancer dataset along with the other six algorithms. The proposed model is able to achieve 0.92 under all the eight scoring criteria, which is better than the performance of the other six algorithms. Therefore, an optimized deep neural network based on an improved sparrow search algorithm with iterative map, iterative perturbation, and Gaussian mutation is an effective approach to predict the survival rate of esophageal cancer.

A new adaptive sliding mode controller based on the RBF neural network for an electro-hydraulic servo system

Accuracy and robust trajectory tracking for electro-hydraulic servo systems in the presence of load disturbances and model uncertainties are of great importance in many fields. In this work, a new adaptive sliding mode control method based on the RBF neural networks (SMC–RBF) is proposed to improve the performances of a robotic excavator. Model uncertainties and load disturbances of the electro-hydraulic servo system are approximated and compensated using the RBF neural networks. Adaptive mechanisms are designed to adjust the connection weights of the RBF neural networks in real time to guarantee the stability. A nonlinear term is introduced into the sliding mode to design an adaptive terminal sliding mode control structure to improve dynamic performances and the convergence speed. Moreover, a sliding mode chattering reduction method is proposed to suppress the chattering phenomenon. Three types of step, ramp and sine signals are used as the simulation reference trajectories to compare different controllers on a co-simulation platform. Experiments with leveling and triangle conditions are presented on a robotic excavator. Results show that the proposed SMC–RBF controller is superior to existing proportional integral derivative (PID) and sliding mode controller (SMC) in terms of tracking accuracy and disturbance rejection.

Robust design of compact axial compressor

The method of connection weights in neural networks was used to analyze the sensitivity of the compressor rotor, and the Back Propagation Neural Network (BPNN) was used to construct the analysis relationship between the compressor rotor's geometries and the performance based on the training and learning of the data base, and the prediction accuracy can reach more than 99.99%. Then the modified Grason Algorithm based on the neural network connect weights was used to quantify the contribution of the geometrical effects on its performance. The result shows that the tip clearance contributes 11.43% (efficiency sensitivity analysis) and 10.18% (pressure ratio sensitivity analysis) to compressor performance changes. This study focuses mainly on the robust optimization of tip clearance. Non-intrusive probability collection point method (NIPC) was adopted for the uncertainty propagation. The robust optimization method based on BPNN agent model coupled with multi-objective genetic algorithm Non-dominated sorting genetic algorithm-II (NSGA II) was used to perform the optimization. Compared to the design prototype, the variance of robust compressor rotor's efficiency could be reduced by 21.04%.

Optimizing Connection Weights in Neural Networks Using Hybrid Metaheuristics Algorithms

The learning process of artificial neural networks is an important and complex task in the supervised learning field. The main difficulty of training a neural network is the process of fine-tuning the best set of control parameters in terms of weight and bias. This paper presents a new training method based on hybrid particle swarm optimization with Multi-Verse Optimization (PMVO) to train the feedforward neural networks. The hybrid algorithm is utilized to search better in solution space which proves its efficiency in reducing the problems of trapping in local minima. The performance of the proposed approach was compared with five evolutionary techniques and the standard momentum backpropagation and adaptive learning rate. The comparison was benchmarked and evaluated using six bio-medical datasets. The results of the comparative study show that PMVO outperformed other training methods in most datasets and can be an alternative to other training methods.

Niche-based cooperative co-evolutionary ensemble neural network for classification

Recently, artificial neural networks have been widely used for classification. It is important to optimize the weight parameters and topological structure of the neural network simultaneously. These two tasks are interdependent and should be solved at the same time to achieve a better result. However, existing works cannot balance the accuracy and diversity of neural networks very well. In this paper, a cooperative co-evolutionary algorithm is proposed to simultaneously evolve artificial neural network topology, neuron attributes, and connection weights. In the proposed algorithm, two effective strategies are proposed. First, the niche-based strategy is used in the evolutionary and cooperative process to refine the local search ability. In this way, a set of candidate networks with a higher level of output diversity is obtained. Second, a two-step comparison scheme is designed to acquire a compact ensemble network. Moreover, a fully connected weights matrix crossover scheme is used to avoid destroying the network structure. The proposed algorithm is tested on the benchmark classification problems in the UCI machine learning repository and compared with other state-of-the-art methods. The experimental results show that the proposed niche-based cooperative co-evolutionary ensemble neural network has a higher capability of generalization compared with other methods in six of nine kinds of classification problems. Furthermore, the proposed ensemble neural network has relatively low complexity.

Further Results on Exponentially Robust Stability of Uncertain Connection Weights of Neutral-Type Recurrent Neural Networks

Further results on the robustness of the global exponential stability of recurrent neural network with piecewise constant arguments and neutral terms (NPRNN) subject to uncertain connection weights are presented in this paper. Estimating the upper bounds of the two categories of interference factors and establishing a measuring mechanism for uncertain dual connection weights are the core tasks and challenges. Hence, on the one hand, the new sufficient criteria for the upper bounds of neutral terms and piecewise arguments to guarantee the global exponential stability of NPRNN are provided. On the other hand, the allowed enclosed region of dual connection weights is characterized by a four-variable transcendental equation based on the preceding stable NPRNN. In this way, two interference factors and dual uncertain connection weights are mutually restricted in the model of parameter-uncertainty NPRNN, which leads to a dynamic evolution relationship. Finally, the numerical simulation comparisons with stable and unstable cases are provided to verify the effectiveness of the deduced results.

Research on Mathematical Genetic Algorithms Based on Evolutionary Computer Neural Network of Distribution Function Parameters

Genetic algorithm and neural network are both ideas and algorithm skills. The flexibility of this method provides a stage for many researchers and applications. This paper discusses the neural network model based on the improved genetic algorithm under the parameters of the distribution function and its application in multi-variable, multi-step nonlinear economic forecasting. First of all, this article discusses the method of using improved genetic algorithm to learn neural network connection weights, and proposes a secondary optimization strategy. The main innovations or improvement techniques include: initial population generation strategy, crossover operation secondary optimization strategy, mutation operation Secondary optimization strategy, optimal chromosome inventory strategy, etc. This article uses VB for programming. The developed software has a certain versatility and can be used for prediction problems similar to nonlinear economic data.

Performance Analysis of Wireless Location and Velocity Tracking of Digital Broadcast Signals Based on Extended Kalman Filter Algorithm

In order to improve the accuracy and reliability of wireless location in NLOS environment, a wireless location algorithm based on artificial neural network (ANN) is proposed for NLOS positioning error caused by non-line-of-sight (NLOS) propagation, such as occlusion and signal reflection. The mapping relationship between TOA and TDOA measurement data and coordinates is established. The connection weights of neural network are estimated as the state variables of nonlinear dynamic system. The multilayer perceptron network is trained by the real-time neural network training algorithm based on extended Kalman (EKF). Combined with the statistical characteristics of NLOS error, the state component NLOS bias estimation is modified to realize TDOA data reconstruction. Simulation and experimental data analysis show that the algorithm can effectively weaken the influence of NLOS error. The localization method does not depend on the specific NLOS error distribution, nor does it need LOS and NLOS recognition. It can significantly improve the mobile positioning accuracy.

Advanced Meta-Heuristics, Convolutional Neural Networks, and Feature Selectors for Efficient COVID-19 X-Ray Chest Image Classification.

The chest X-ray is considered a significant clinical utility for basic examination and diagnosis. The human lung area can be affected by various infections, such as bacteria and viruses, leading to pneumonia. Efficient and reliable classification method facilities the diagnosis of such infections. Deep transfer learning has been introduced for pneumonia detection from chest X-rays in different models. However, there is still a need for further improvements in the feature extraction and advanced classification stages. This paper proposes a classification method with two stages to classify different cases from the chest X-ray images based on a proposed Advanced Squirrel Search Optimization Algorithm (ASSOA). The first stage is the feature learning and extraction processes based on a Convolutional Neural Network (CNN) model named ResNet-50 with image augmentation and dropout processes. The ASSOA algorithm is then applied to the extracted features for the feature selection process. Finally, the Multi-layer Perceptron (MLP) Neural Network’s connection weights are optimized by the proposed ASSOA algorithm (using the selected features) to classify input cases. A Kaggle chest X-ray images (Pneumonia) dataset consists of 5,863 X-rays is employed in the experiments. The proposed ASSOA algorithm is compared with the basic Squirrel Search (SS) optimization algorithm, Grey Wolf Optimizer (GWO), and Genetic Algorithm (GA) for feature selection to validate its efficiency. The proposed (ASSOA + MLP) is also compared with other classifiers, based on (SS + MLP), (GWO + MLP), and (GA + MLP), in performance metrics. The proposed (ASSOA + MLP) algorithm achieved a classification mean accuracy of (99.26%). The ASSOA + MLP algorithm also achieved a classification mean accuracy of (99.7%) for a chest X-ray COVID-19 dataset tested from GitHub. The results and statistical tests demonstrate the high effectiveness of the proposed method in determining the infected cases.

Improved Crow Search Algorithm Optimized Extreme Learning Machine Based on Classification Algorithm and Application

In view of the problems of the connection weights and thresholds of the extreme learning machine are randomly generated before training and remain unchanged during the training process, the number of hidden layer nodes is pre-allocated, and the hidden layer parameters are randomly selected. Too many hidden layer nodes not only make the network more complex but also reduce the generalization ability of the algorithm. Aiming at this problem, an improved crow search algorithm is proposed to optimize the extreme learning machine. Based on the analysis of the limitations of the original crow search algorithm, a particle swarm algorithm search strategy is proposed to enhance the global search capability. In the latter part of the algorithm iteration, Gaussian function is added, and the penalty coefficient of the function is used for local disturbance, gradually reducing the amplitude of the search trajectory, and then adaptively adjusting the parameters to avoid being attracted by local extremum. Finally, the improved crow search algorithm is used to optimize the hidden layer neurons and connection weights of the extreme learning machine neural network, so as to obtain accurate prediction results. Through function fitting, regression data set fitting and classification data set for classification experiment verification, the proposed algorithm has higher training speed and efficiency. At the same time, this method is not only significantly higher than the traditional ELM method, but also obtains a more compact network structure, which is an effective neural network optimization algorithm.

Optimizing connection weights of functional link neural network using APSO algorithm for medical data classification

Classification is a common problem in various fields of life, and the key challenging task in data mining. The primary objective of the classification process is to classify the given dataset in a defined class label for all data. Many research papers widely used classification in the medical sector. Various researches have been carried out to classify medical data using different techniques, such as High Order Neural Network (HONN) combined with Back-Propagation Neural Network (BPNN) as a learning algorithm. Due to the increased data complexity, the back-propagation algorithm faced problems of slow convergence. The Back-Propagation (BP) using the gradient descent technique has the possibility of getting stuck in local minima. So, the BP algorithm may cause difficulties in finding the global minima of the error function. This paper proposed a high-order Functional Link Neural Network (FLNN). The proposed FLNN model is integrated with a metaheuristic-based searching algorithm called Accelerated Particle Swarm Optimization (APSO). The performance of the proposed Accelerated Particle Swarm Optimization Functional Link Neural Network (APSOFLNN) model is validated with various medical datasets and compared to traditional techniques such as Accelerated Particle Swarm Optimization Functional Link Back-Propagation (APSOFLBP), and Artificial Bee Colony Functional Link Neural Network (ABCFLNN). The simulation results showed that the proposed APSOFLNN algorithm for the used benchmarked datasets shows good results in terms of the mean square error (MSE) and accuracy in comparison to the traditional techniques.

Analysis and Forecast of Urban Economic Vitality in Northeast China

Taking into account the passage of time, the original economic vitality index will vary with changes in social development, we use the BP neural network nearly a decade as the original GDP data for the next 30 years the GDP forecast. BP neural network in 1985, proposed by Rumelhart, the algorithm solves the system of learning problems multilayer neural network connection weights hidden layer [1].It consists of an input layer, a hidden layer, and an output layer. The principle is to continuously adjust the network weights and thresholds by transmitting errors backward and then correcting the errors to achieve the desired input-output mapping.

The Study of a Sales Forecast Model Based on SA-LSTM

Sales forecast is an indispensable link in the business activities of enterprises, and the accuracy of prediction is directly related to the effectiveness of enterprise sales and operation activities. In order to improve the prediction accuracy, a sales forecasting model based on LSTM is proposed. The model uses SA to optimize the initial connection weights of LSTM neural network, which solves the problem that the LSTM neural network converges to the local optimal, thus improving the network performance, and then makes an empirical analysis of the construction of the sales forecasting model based on SA-LSTM. The results show that the sales forecasting model improves the prediction accuracy, also reduces the number of iterations, and obtains a good prediction effect.

Machine learning methods for better water quality prediction

In any aquatic system analysis, the modelling water quality parameters are of considerable significance. The traditional modelling methodologies are dependent on datasets that involve large amount of unknown or unspecified input data and generally consist of time-consuming processes. The implementation of artificial intelligence (AI) leads to a flexible mathematical structure that has the capability to identify non-linear and complex relationships between input and output data. There has been a major degradation of the Johor River Basin because of several developmental and human activities. Therefore, setting up of a water quality prediction model for better water resource management is of critical importance and will serve as a powerful tool. The different modelling approaches that have been implemented include: Adaptive Neuro-Fuzzy Inference System (ANFIS), Radial Basis Function Neural Networks (RBF-ANN), and Multi-Layer Perceptron Neural Networks (MLP-ANN). However, data obtained from monitoring stations and experiments are possibly polluted by noise signals as a result of random and systematic errors. Due to the presence of noise in the data, it is relatively difficult to make an accurate prediction. Hence, a Neuro-Fuzzy Inference System (WDT-ANFIS) based augmented wavelet de-noising technique has been recommended that depends on historical data of the water quality parameter. In the domain of interests, the water quality parameters primarily include ammoniacal nitrogen (AN), suspended solid (SS) and pH. In order to evaluate the impacts on the model, three evaluation techniques or assessment processes have been used. The first assessment process is dependent on the partitioning of the neural network connection weights that ascertains the significance of every input parameter in the network. On the other hand, the second and third assessment processes ascertain the most effectual input that has the potential to construct the models using a single and a combination of parameters, respectively. During these processes, two scenarios were introduced: Scenario 1 and Scenario 2. Scenario 1 constructs a prediction model for water quality parameters at every station, while Scenario 2 develops a prediction model on the basis of the value of the same parameter at the previous station (upstream). Both the scenarios are based on the value of the twelve input parameters. The field data from 2009 to 2010 was used to validate WDT-ANFIS. The WDT-ANFIS model exhibited a significant improvement in predicting accuracy for all the water quality parameters and outperformed all the recommended models. Also, the performance of Scenario 2 was observed to be more adequate than Scenario 1, with substantial improvement in the range of 0.5% to 5% for all the water quality parameters at all stations. On validating the recommended model, it was found that the model satisfactorily predicted all the water quality parameters (R2 values equal or bigger than 0.9).

Evolving Deep Convolutional Neural Networks for Image Classification

Evolutionary paradigms have been successfully applied to neural network designs for two decades. Unfortunately, these methods cannot scale well to the modern deep neural networks due to the complicated architectures and large quantities of connection weights. In this paper, we propose a new method using genetic algorithms for evolving the architectures and connection weight initialization values of a deep convolutional neural network to address image classification problems. In the proposed algorithm, an efficient variable-length gene encoding strategy is designed to represent the different building blocks and the potentially optimal depth in convolutional neural networks. In addition, a new representation scheme is developed for effectively initializing connection weights of deep convolutional neural networks, which is expected to avoid networks getting stuck into local minimum that is typically a major issue in the backward gradient-based optimization. Furthermore, a novel fitness evaluation method is proposed to speed up the heuristic search with substantially less computational resource. The proposed algorithm is examined and compared with 22 existing algorithms on nine widely used image classification tasks, including the state-of-the-art methods. The experimental results demonstrate the remarkable superiority of the proposed algorithm over the state-of-the-art designs in terms of classification error rate and the number of parameters (weights).

The weights initialization methodology of unsupervised neural networks to improve clustering stability

A study on initialization of connection weights of neural networks is expected to be needed because various deep neural networks based on deep learning have attracted much attention recently. However, studies on the relation between the output value of the active function and the learning performance of the neural network with respect to the connection weight value have been conducted mainly on the supervised learning model. This paper focused on improving the efficiency of autonomous neural network model by studying the connection weight initialization as the neural network model of supervised learning. Adaptive resonance theory (ART) is a major model of autonomous neural network that tries to solve the stability–plasticity dilemma by using bottom-up weights and top-down weights. The conventional weights initialization method of ART was to uniformly set all weights, but the proposed method is to initialize by using pre-trained weights. Experiments show that the ART, which initializes the connectivity weights through the proposed method, performs clustering more reliably.

Fault diagnosis of neural network classified signal fractal feature based on SVM

The fault diagnosis method based on neural network has many defects, such as complicated network, long training time and slow convergence speed. The particle swarm optimization and neural network integration fault diagnosis methods are proposed to improve the fault diagnosis capability. Firstly, the heuristic global optimization capability of particle swarm optimization is used to optimize the neural network connection weights; then the transformer fault samples are trained and tested by using non-linear processing capacity of neural network. The test results show that such algorithm can effectively avoid unstable neural network, easily falling into local minimum and lower diagnostic accuracy etc. and can effectively increase the convergence speed and fault diagnosis efficiency compared with traditional fault diagnosis method.

Selection of criteria for multi-criteria decision making of reservoir flood control operation

In reservoir flood control operation, selection of criteria is an important part of the multi-criteria decision making (MCDM) procedure. This paper proposes a method to select criteria for MCDM of reservoir flood control operation based on the back-propagation (BP) neural network. According to the concept of ideal and anti-ideal points, we propose a method to generate training samples of the BP neural network via stochastic simulation. The topological structure of a three-layer BP neural network used for criteria selection is established. The relative importance of criteria is derived via the learned connection weights of a trained BP neural network, and its calculation method is proposed. The sensitivity curve method is employed to conduct sensitivity analysis, and the relative contribution ratio is defined to quantify the relative sensitivity strength of each criterion. We present the principle and threshold value of criteria selection based on the comprehensive discrimination index defined by the combination of the relative importance and relative contribution ratio. The Pubugou reservoir is selected as the case study. The results show that the proposed method can provide an effective tool for decision makers to select criteria before MCDM modeling of reservoir flood control operation.