Principles Of Annealing Research Articles

Convolutional adaptive denoising autoencoders for hierarchical feature extraction

Convolutional neural networks (CNNs) are typical structures for deep learning and are widely used in image recognition and classification. However, the random initialization strategy tends to become stuck at local plateaus or even diverge, which results in rather unstable and ineffective solutions in real applications. To address this limitation, we propose a hybrid deep learning CNN-AdapDAE model, which applies the features learned by the AdapDAE algorithm to initialize CNN filters and then train the improved CNN for classification tasks. In this model, AdapDAE is proposed as a CNN pre-training procedure, which adaptively obtains the noise level based on the principle of annealing, by starting with a high level of noise and lowering it as the training progresses. Thus, the features learned by AdapDAE include a combination of features at different levels of granularity. Extensive experimental results on STL-10, CIFAR-10, andMNIST datasets demonstrate that the proposed algorithm performs favorably compared to CNN (random filters), CNNAE (pre-training filters by autoencoder), and a few other unsupervised feature learning methods.

Adaptive Noise Level for Stacked Denoising Autoencoder

The stacked denoising autoencoder (SDAE) is a slight modification of stacked autoencoder, which is trained to reconstruct a clean version of an input from its corrupted version. It has been successfully used to learn new representations for an unsupervised framework. However, the noise level in the SDAE is determined by experience and remains fixed throughout the training process. To address this limitation, we present an adaptive stacked denoising autoencoder based on the principle of annealing (AdaptiveSDAE), a novel method of adaptively obtaining the noise level. This is achieved by first computing the average noise level for each epoch using a linear average noise level function based on the principle of annealing; and second calculating the noise level for each input neuron based on the average noise level, and the contribution of the input neuron to the activation of hidden neurons (which depend on the input neuron’s value and the weights). Thus, the network includes a combination of features at multiple scales. The experimental results show that our proposed AdaptiveSDAE performed better than SDAE and other unsupervised feature learning methods.

An Accessibility-Maximization Approach

Accessibility is a key factor in defining the quality of life and potential for development of both cities and regions. This paper presents an accessibility-maximization approach to inter-urban road network planning. The approach is based on a non-linear combinatorial optimization model. Two heuristics have been developed for solving the model, respectively based on local search and simulated annealing principles. The efficiency of the heuristics was analyzed on a representative sample of test problems involving 10-, 20-, and 40-node networks. In the analysis both solution quality and computing effort were taken into account. The approach was used to analyze the ongoing transformation of the Portuguese main road network. The transformation started in 1985, the year when Portugal joined the European Economic Community.