Neural Network Topology Optimization Research Articles

Adaptive Neural Network Structure Optimization Algorithm Based on Dynamic Nodes.

Large-scale artificial neural networks have many redundant structures, making the network fall into the issue of local optimization and extended training time. Moreover, existing neural network topology optimization algorithms have the disadvantage of many calculations and complex network structure modeling. We propose a Dynamic Node-based neural network Structure optimization algorithm (DNS) to handle these issues. DNS consists of two steps: the generation step and the pruning step. In the generation step, the network generates hidden layers layer by layer until accuracy reaches the threshold. Then, the network uses a pruning algorithm based on Hebb’s rule or Pearson’s correlation for adaptation in the pruning step. In addition, we combine genetic algorithm to optimize DNS (GA-DNS). Experimental results show that compared with traditional neural network topology optimization algorithms, GA-DNS can generate neural networks with higher construction efficiency, lower structure complexity, and higher classification accuracy.

Minimal neural network topology optimization for aesthetic classification

In this article, a minimum neural network topology in terms of units and connections (neurons and weights), making visual aesthetically categorized images, will be identified. For this purpose, an analysis to compact the initial neural network will be conducted by reducing the amount of connections and neurons required for an a esthetic classification function in hidden layers. A collection of photographs and aesthetic evaluations from an online voting site for photographs such as Photo.net was used. Such pictures are defined by ad-hoc metrics used in previous studies. In this way, the input data, the images, are interpreted within a minimum complexity scheme by complexity estimators (internal representations). The results obtained show that the optimized topology found retains both its performance and its ability to generalize. As part of this study, it has been found a relationship concerning the input data and minimum topology necessary for its correct representation, demonstrating statistically a performance comparable to other not minimized topologies.

Artificial Neural Network Topology Optimization using K-Fold Cross Validation for Spray Drying of Coconut Milk

In this study, the development of an optimized topology neural network model for spray drying coconut milk is investigated using K-fold cross validation technique. Performance between standalone ANN and ANN with K-fold cross validation is compared, as K-fold cross validation method is integrated into neural network to overcome the limitations of restricted dataset. With inlet temperature (140 °C-180 °C), concentration of maltodextrin and sodium caseinate (0 w/w %- 10w/w %) are established as the input parameters, while moisture content (3.64%-5.1%), outlet temperature (76.5 °C-104.5 °C) and surface free fat percentage (0.35%-34.51%) are the output parameters for the neural network. Experimental data from the spray drying process is used to develop the neural network. Selection from the best training algorithm (gradient descent backpropagation, gradient descent with momentum, resilience backpropagation, conjugate gradient backpropagation with Polak-Riebre restarts, conjugate gradient backpropagation with Fletcher-Reeves, scaled conjugate gradient, Broyden-Flectcher-Goldfard-Shanno backpropagation algorithm and Levenberg-Marquardt backpropagation), transfer function (tansig, logsig, purelin and satlin), number of training runs (1000-5000), number of hidden layers (1-3) and nodes (5-15) have significant effect on the performance of the ANN models based on the lowest MSE values and R2 values. Overall, the optimum topology ANN model with k-fold cross validation outperformed the recorded lowest MSE value of 0.064 and highest R2 value of 0.855 compared to the optimum standalone ANN model with MSE value of 0.082 and R2 value of 0.832. The optimum ANN with K-fold cross validation implements the Levenberg-Marquart training algorithm with hyperbolic tangent sigmoid transfer function using 4500 times training runs with optimal topology configuration of 3-8-2-3. Result concludes that the developed neural network using K-fold cross validation represents the spray drying process as a highly reliable model with high degree of accuracy.

Topology optimization of neural networks based on a coupled genetic algorithm and particle swarm optimization techniques (c-GA–PSO-NN)

In this short paper, a coupled genetic algorithm and particle swarm optimization technique was used to supervise neural networks where the applied operators and connections of layers were tracked by genetic algorithm and numeric values of biases and weights of layers were examined by particle swarm optimization to modify the optimal network topology. The method was applied for a previously studied case, and results were analyzed. The convergence to the optimal topology was highly fast and efficient, and the obtained weights and biases revealed great reliability in reproduction of data. The optimal topology of neural networks was obtained only after seven iterations, and an average square of the correlation (R2) of 0.9989 was obtained for the studied cases. The proposed method can be used for fast and reliable topology optimization of neural networks.

Effects of phenotypic redundancy in structure optimization

Concepts from graph theory and molecular evolution are proposed for analyzing the redundancy in the genotype-phenotype mapping in structure optimization stemming from graph isomorphism. Evolutionary topology optimization of neural networks serves as an example. By means of analytical and random-walk methods, it is shown that rare and frequent structures influence the search process: operators that are unbiased in genotype space may have a remarkable bias in phenotype space. In particular, if the desired structures are rare, the probability that an evolutionary algorithm evolves them may decrease. This is verified experimentally by comparing evolutionary structure optimization algorithms with and without search operators that take the redundancy of phenotypes into account. Further, it is shown how different encodings and restrictions on the search space lead to qualitatively different distributions of rare and frequent structures.