Final Connection Weights Research Articles

Neural self-compressor: Collective interpretation by compressing multi-layered neural networks into non-layered networks

Abstract The present paper proposes a new method called “neural self-compressors” to compress multi-layered neural networks into the simplest possible ones (i.e., without hidden layers) to aid in the interpretation of relations between inputs and outputs. Though neural networks have shown great success in improving generalization, the interpretation of internal representations becomes a serious problem as the number of hidden layers and their corresponding connection weights becomes larger and larger. To overcome this interpretation problem, we introduce a method that compresses multi-layered neural networks into ones without hidden layers. In addition, this method simplifies entangled weights as much as possible by maximizing mutual information between inputs and outputs. In this way, final connection weights can be interpreted as easily as by the logistic regression analysis. The method was applied to four data sets: a symmetric data set, ovarian cancer data set, restaurant data set, and credit card holders’ default data set. In the first set, the symmetric data set, we tried to explain how the present method could produce interpretable outputs intuitively. In all the other cases, we succeeded in compressing multi-layered neural networks into their simplest forms with the help of mutual information maximization. In addition, by de-correlating outputs, we were able to transform connection weights from those close to the regression coefficients to ones with more explicit features.

Landslide susceptibility assessment in the Hoa Binh province of Vietnam: A comparison of the Levenberg–Marquardt and Bayesian regularized neural networks

This study investigates the potential application of artificial neural networks in landslide susceptibility mapping in the Hoa Binh province of Vietnam. A landslide inventory map of the study area was prepared by combining landslide locations investigated through three projects during the last 10years. Some recent landslide locations were identified based on SPOT satellite images, field surveys, and existing literature. The images have a spatial resolution of 2.5m. Ten landslide conditioning factors were utilized in the multilayer feed-forward neural network analysis: slope, aspect, relief amplitude, lithology, land use, soil type, rainfall, distance to roads, distance to rivers and distance to faults. Two back-propagation training algorithms, Levenberg–Marquardt and Bayesian regularization, were utilized to determine synoptic weights using a training dataset. Relative importance of each landslide conditioning factor was assessed using the above mentioned synoptic weights. The final connection weights obtained in the training phase were applied to the entire study area to produce landslide susceptibility indexes. The results were then imported to a GIS and landslide susceptibility maps were constructed. Landslide locations not used in the training phase were used to verify and compare the results of the landslide susceptibility maps. Finally, the two landslide susceptibility maps were validated using the prediction-rate method. Subsequently, areas under the prediction curves were assessed. The prediction accuracy of landslide susceptibility maps produced by the Bayesian regularization neural network and the Levenberg–Marquardt neural network were 90.3% and 86.1% respectively. These results indicate that the two models seem to have good predictive capability. The Bayesian regularization network model appears more robust and efficient than the Levenberg–Marquardt network model for landslide susceptibility mapping.

Unifying cost and information in information-theoretic competitive learning

In this paper, we introduce costs into the framework of information maximization and try to maximize the ratio of information to its associated cost. We have shown that competitive learning is realized by maximizing mutual information between input patterns and competitive units. One shortcoming of the method is that maximizing information does not necessarily produce representations faithful to input patterns. Information maximizing primarily focuses on some parts of input patterns that are used to distinguish between patterns. Therefore, we introduce the cost, which represents average distance between input patterns and connection weights. By minimizing the cost, final connection weights reflect input patterns well. We applied the method to a political data analysis, a voting attitude problem and a Wisconsin cancer problem. Experimental results confirmed that, when the cost was introduced, representations faithful to input patterns were obtained. In addition, improved generalization performance was obtained within a relatively short learning time.