Random Initialization Research Articles

A Brain Storm and Chaotic Accelerated Particle Swarm Optimization Hybridization

Brain storm optimization (BSO) and particle swarm optimization (PSO) are two popular nature-inspired optimization algorithms, with BSO being the more recently developed one. It has been observed that BSO has an advantage over PSO regarding exploration with a random initialization, while PSO is more capable at local exploitation if given a predetermined initialization. The two algorithms have also been examined as a hybrid. In this work, the BSO algorithm was hybridized with the chaotic accelerated particle swarm optimization (CAPSO) algorithm in order to investigate how such an approach could serve as an improvement to the stand-alone algorithms. CAPSO is an advantageous variant of APSO, an accelerated, exploitative and minimalistic PSO algorithm. We initialized CAPSO with BSO in order to study the potential benefits from BSO’s initial exploration as well as CAPSO’s exploitation and speed. Seven benchmarking functions were used to compare the algorithms’ behavior. The chosen functions included both unimodal and multimodal benchmarking functions of various complexities and sizes of search areas. The functions were tested for different numbers of dimensions. The results showed that a properly tuned BSO–CAPSO hybrid could be significantly more beneficial over stand-alone BSO, especially with respect to computational time, while it heavily outperformed stand-alone CAPSO in the vast majority of cases.

Quantum-PSO based unsupervised clustering of users in social networks using attributes.

Unsupervised cluster detection in social network analysis involves grouping social actors into distinct groups, each distinct from the others. Users in the clusters are semantically very similar to those in the same cluster and dissimilar to those in different clusters. Social network clustering reveals a wide range of useful information about users and has many applications in daily life. Various approaches are developed to find social network users' clusters, using only links or attributes and links. This work proposes a method for detecting social network users' clusters based solely on their attributes. In this case, users' attributes are considered categorical values. The most popular clustering algorithm used for categorical data is the K-mode algorithm. However, it may suffer from local optimum due to its random initialization of centroids. To overcome this issue, this manuscript proposes a methodology named the Quantum PSO approach based on user similarity maximization. In the proposed approach, firstly, dimensionality reduction is conducted by performing the relevant attribute set selection followed by redundant attribute removal. Secondly, the QPSO technique is used to maximize the similarity score between users to get clusters. Three different similarity measures are used separately to perform the dimensionality reduction and similarity maximization processes. Experiments are conducted on two popular social network datasets; ego-Twitter, and ego-Facebook. The results show that the proposed approach performs better clustering results in terms of three different performance metrics than K-Mode and K-Mean algorithms.

Discovering New Prognostic Features for the Harmonic Reducer in Remaining Useful Life Prediction

Vibration and current signals are widely used in fault diagnosis and life prediction of electromechanical transmission systems. However, due to the complex working environment of harmonic reducer in the industrial robot, using a single signal for failure analysis or life prediction may risk to false alarming due to lacking of fault information. In the early operation stage of equipment, fault information contained in the signal is too weak to be extracted. In addition, there is certain correlation among the features, which can lead to meaningless superposition of the calculation process. Therefore, a new health index construction method is proposed integrating the current signal and vibration signal and reducing the redundancy among the features in multidomains and can effectively enhance the fault information. In addition, in view of the local optimum and slow speed caused by the random initialization of the network model, an improved life prediction method is proposed to optimize BP neural network to improve the prediction performance. The proposed method is verified by the test data of the harmonic reducer. Results show that the proposed method can predict the remaining useful life of the harmonic reducer more accurately and effectively.

A Comparative Study of Transfer Learning based Models for Lung Cancer Histopathology Classification

Lung cancer is the deadliest form of cancer, which attracted a lot of attention in the past. Transfer learning is a very popular approach in deep learning as it can apply the knowledge obtained from a previous task to improve the performance in another. In this research, four transfer learning models with different complexity are utilized to detect lung cancer, which are AlexNet, VGG16, ResNet50 and Inception-v3. Since the early detection and histopathological diagnosis can considerably decrease the likelihood of mortality, the lung cancer histopathological images dataset is considered. This dataset contains histopathological images of 3 classes, all of them are considered in this study. Firstly, the four models are trained on the histopathological database from random initialization for 10 epochs. Next, the four models are first pre-trained on ImageNet and then trained on the histopathological dataset for 10 epochs. For each epoch, the testing accuracy is recorded so as to find the optimal number of epochs and determine whether transfer learning models are capable in lung cancer detection. Then, various evaluation metrics e.g., accuracy and precision are used to measure and compare the four models’ performance. The study’s finding shows that AlexNet, VGG16, ResNet50 and Inception-v3 pre-trained on ImageNet are adequate in lung cancer detection. There corresponding accuracy rates are 99.367%, 99.800%, 100% and 100% respectively, which are much higher than that trained from random initialization. Among the four transfer learning models, ResNet50 and Inception-v3 perform the best on lung cancer classification.

Squid Game Optimizer (SGO): a novel metaheuristic algorithm

In this paper, Squid Game Optimizer (SGO) is proposed as a novel metaheuristic algorithm inspired by the primary rules of a traditional Korean game. Squid game is a multiplayer game with two primary objectives: attackers aim to complete their goal while teams try to eliminate each other, and it is usually played on large, open fields with no set guidelines for size and dimensions. The playfield for this game is often shaped like a squid and, according to historical context, appears to be around half the size of a standard basketball court. The mathematical model of this algorithm is developed based on a population of solution candidates with a random initialization process in the first stage. The solution candidates are divided into two groups of offensive and defensive players while the offensive player goes among the defensive players to start a fight which is modeled through a random movement toward the defensive players. By considering the winning states of the players of both sides which is calculated based on the objective function, the position updating process is conducted and the new position vectors are produced. To evaluate the effectiveness of the proposed SGO algorithm, 25 unconstrained mathematical test functions with 100 dimensions are used, alongside six other commonly used metaheuristics for comparison. 100 independent optimization runs are conducted for both SGO and the other algorithms with a pre-determined stopping condition to ensure statistical significance of the results. Statistical metrics such as mean, standard deviation, and mean of required objective function evaluations are calculated. To provide a more comprehensive analysis, four prominent statistical tests including the Kolmogorov–Smirnov, Mann–Whitney, and Kruskal–Wallis tests are used. Meanwhile, the ability of the suggested SGOA is assessed through the cutting-edge real-world problems on the newest CEC like CEC 2020, while the SGO demonstrate outstanding performance in dealing with these complex optimization problems. The overall assessment of the SGO indicates that the proposed algorithm can provide competitive and remarkable outcomes in both benchmark and real-world problems.

Pathological Images Classification based on the Pretrained Convolutional Neural Network

Convolutional neural network is more and more used to solve the problem of images classification, but the scale of the dataset in this filed is too small compared with that of natural image recognition, which leads to the slow convergence speed of the neural network, and the instability of the model, which is difficult to be used to predict the results. In this paper, the residual network is applied to the PCam dataset to realize the recognition of tumour cells, and pre-trained through the ImageNet dataset. In the same training epochs, the performance of the pre-train network is much higher than that of the randomly initialized network, which significantly improves the convergence speed of the neural network. In addition, the performance of the pre-train network can converge faster when the amount of data is sufficient, and in the case oof medical image recognition with less data, the performance of the pre-train network is much higher than that of the random initialization network.

Hybrid Path Planning Using a Bionic-Inspired Optimization Algorithm for Autonomous Underwater Vehicles

This research presents a hybrid approach for path planning of autonomous underwater vehicles (AUVs). During path planning, static obstacles affect the desired path and path distance which result in collision penalties. In this study, the merits of grey wolf optimization (GWO) and genetic algorithm (GA) of bionic-inspired algorithms are integrated to implement a hybrid grey wolf optimization (HGWO) algorithm which allows AUVs to reach their destination safely in an obstacle rich environment. The proposed hybrid path planner is employed for path planning of a single AUV based on collision avoidance. It uses the GA as an initialization generator to overcome the random initialization problem of GWO. In this research, the total cost is considered to be a function of path distance and collision penalties. Further, the application of the proposed hybrid path planner is extended for cooperative path planning of AUVs while avoiding collision using communication consensus. Simulation results are obtained for both a single AUV and multiple AUV path planning in a 3D obstacle rich environment using a proportional-derivative controller. The Kruskal–Wallis test is employed for a non-parametric statistical analysis, where the independence of the results given by the algorithms is demonstrated.

Comparative Study of Support Vector Machine and Naive Bayes for Sentiment Analysis on Lecturer Performance

This study addresses the challenge of sentiment analysis within the Information Technology study program at Universitas Timor, aiming to compare the performance of Support Vector Machines (SVM) and Naive Bayes (NB) through 100 iterations. The dataset, comprising 21 instances of negative sentiment and 18 instances of positive sentiment, is analyzed using both methods, with accuracy and Area Under the ROC Curve (AUC) serving as key metrics. The sample size consists of 39 instances, and the results indicate significant variability in both accuracy and AUC, emphasizing the sensitivity of the models to dataset characteristics and random initialization. On average, SVM outperforms NB, with an accuracy of 0.5846 compared to 0.5075 and an AUC of 0.5916 compared to 0.4607.

A data-driven approach to neural architecture search initialization

AbstractAlgorithmic design in neural architecture search (NAS) has received a lot of attention, aiming to improve performance and reduce computational cost. Despite the great advances made, few authors have proposed to tailor initialization techniques for NAS. However, the literature shows that a good initial set of solutions facilitates finding the optima. Therefore, in this study, we propose a data-driven technique to initialize a population-based NAS algorithm. First, we perform a calibrated clustering analysis of the search space, and second, we extract the centroids and use them to initialize a NAS algorithm. We benchmark our proposed approach against random and Latin hypercube sampling initialization using three population-based algorithms, namely a genetic algorithm, an evolutionary algorithm, and aging evolution, on CIFAR-10. More specifically, we use NAS-Bench-101 to leverage the availability of NAS benchmarks. The results show that compared to random and Latin hypercube sampling, the proposed initialization technique enables achieving significant long-term improvements for two of the search baselines, and sometimes in various search scenarios (various training budget). Besides, we also investigate how an initial population gathered on the tabular benchmark can be used for improving search on another dataset, the So2Sat LCZ-42. Our results show similar improvements on the target dataset, despite a limited training budget. Moreover, we analyse the distributions of solutions obtained and find that that the population provided by the data-driven initialization technique enables retrieving local optima (maxima) of high fitness and similar configurations.

Deep learning models for generation of precipitation maps based on numerical weather prediction

Abstract. Numerical weather prediction (NWP) models are atmospheric simulations that imitate the dynamics of the atmosphere and provide high-quality forecasts. One of the most significant limitations of NWP is the elevated amount of computational resources required for its functioning, which limits the spatial and temporal resolution of the outputs. Traditional meteorological techniques to increase the resolution are uniquely based on information from a limited group of interest variables. In this study, we offer an alternative approach to the task where we generate precipitation maps based on the complete set of variables of the NWP to generate high-resolution and short-time precipitation predictions. To achieve this, five different deep learning models were trained and evaluated: a baseline, U-Net, two deconvolution networks and one conditional generative model (Conditional Generative Adversarial Network; CGAN). A total of 20 independent random initializations were performed for each of the models. The predictions were evaluated using skill scores based on mean absolute error (MAE) and linear error in probability space (LEPS), equitable threat score (ETS), critical success index (CSI) and frequency bias after applying several thresholds. The models showed a significant improvement in predicting precipitation, showing the benefits of including the complete information from the NWP. The algorithms doubled the resolution of the predictions and corrected an over-forecast bias from the input information. However, some new models presented new types of bias: U-Net tended to mid-range precipitation events, and the deconvolution models favored low rain events and generated some spatial smoothing. The CGAN offered the highest-quality precipitation forecast, generating realistic outputs and indicating possible future research paths.

Structural Design of Aerostatic Bearing Based on Multi-Objective Particle Swarm Optimization Algorithm

Aerostatic bearings are considered crucial components that can improve the measurement accuracy of ground simulation tests of space equipment. A structural optimization design method is proposed to enhance the static performance of aerostatic bearings. A mathematical model which can quickly calculate the aerostatic bearing capacity and gas consumption is established, and the influence of structural parameters on bearing performance is analyzed using simulation software. By comparing the convergence time and convergence results of the algorithm using different initialization methods, the Latin hypercube initialization method is selected instead of the random initialization method. The multi-objective particle swarm optimization algorithm is used to obtain the optimal solution set distributed in the objective space. It is found that the optimized structural parameters meet the requirements of improving the capacity and reducing gas consumption, which verifies the method’s effectiveness in designing the structural parameters of aerostatic bearings.

Strong Overall Error Analysis for the Training of Artificial Neural Networks Via Random Initializations

Although deep learning-based approximation algorithms have been applied very successfully to numerous problems, at the moment the reasons for their performance are not entirely understood from a mathematical point of view. Recently, estimates for the convergence of the overall error have been obtained in the situation of deep supervised learning, but with an extremely slow rate of convergence. In this note, we partially improve on these estimates. More specifically, we show that the depth of the neural network only needs to increase much slower in order to obtain the same rate of approximation. The results hold in the case of an arbitrary stochastic optimization algorithm with i.i.d. random initializations.

Adaptive, problem-tailored variational quantum eigensolver mitigates rough parameter landscapes and barren plateaus

Variational quantum eigensolvers (VQEs) represent a powerful class of hybrid quantum-classical algorithms for computing molecular energies. Various numerical issues exist for these methods, however, including barren plateaus and large numbers of local minima. In this work, we consider the Adaptive, Problem-Tailored Variational Quantum Eiegensolver (ADAPT-VQE) ansätze, and examine how they are impacted by these local minima. We find that while ADAPT-VQE does not remove local minima, the gradient-informed, one-operator-at-a-time circuit construction accomplishes two things: First, it provides an initialization strategy that can yield solutions with over an order of magnitude smaller error compared to random initialization, and which is applicable in situations where chemical intuition cannot help with initialization, i.e., when Hartree-Fock is a poor approximation to the ground state. Second, even if an ADAPT-VQE iteration converges to a local trap at one step, it can still “burrow” toward the exact solution by adding more operators, which preferentially deepens the occupied trap. This same mechanism helps highlight a surprising feature of ADAPT-VQE: It should not suffer optimization problems due to barren plateaus and random initialization. Even if such barren plateaus appear in the parameter landscape, our analysis suggests that ADAPT-VQE avoids such regions by design.

Implicit Seismic Full Waveform Inversion With Deep Neural Representation

AbstractFull waveform inversion (FWI) is arguably the current state‐of‐the‐art amongst methodologies for imaging subsurface structures and physical parameters with seismic data; however, important challenges are faced in its implementation and use. Keys amongst these are (a) building a suitable initial model, from which a local minimum is unlikely to be reached, and (b) availability of tools for evaluation of uncertainty. An algorithm we refer to as implicit full waveform inversion (IFWI), designed using continuously and implicitly defined deep neural representations, appears in principle to address both of these issues. We observe in IFWI, with its random initialization and deep learning optimization, improved convergence relative to standard FWI model initialization and optimization. Models close to the global minimum, capturing relatively high‐resolution subsurface structures, are obtained. In addition, uncertainty analysis, though not solved in IFWI, is meaningfully addressed by approximating Bayesian inference with the addition of dropout neurons. Numerical experimentation with a range of 2D geological models is suggestive that IFWI exhibits a strong capacity for generalization, and is likely well‐suited for multi‐scale joint geophysical inversion.

A wavelet neural operator based elastography for localization and quantification of tumors

Background and ObjectivesThe application of intelligent imaging techniques and deep learning in the field of computer-aided diagnosis and medical imaging have improved and accelerated the early diagnosis of many diseases. Elastography is an imaging modality where an inverse problem is solved to extract the elastic properties of tissues and subsequently mapped to anatomical images for diagnostic purposes. In the present work, we propose a wavelet neural operator-based approach for correctly learning the non-linear mapping of elastic properties directly from measured displacement field data. MethodsThe proposed framework learns the underlying operator behind the elastic mapping and thus can map any displacement data from a family to the elastic properties. The displacement fields are first uplifted to a high-dimensional space using a fully connected neural network. On the lifted data, certain iterations are performed using wavelet neural blocks. In each wavelet neural block, the lifted data are decomposed into low, and high-frequency components using wavelet decomposition. To learn the most relevant patterns and structural information from the input, the neural network kernels are directly convoluted with the outputs of the wavelet decomposition. Thereafter the elasticity field is reconstructed from the outputs from convolution. The mapping between the displacement and the elasticity using wavelets is unique and remains stable during training. ResultsThe proposed framework is tested on several artificially fabricated numerical examples, including a benign-cum-malignant tumor prediction problem. The trained model was also tested on real Ultrasound-based elastography data to demonstrate the applicability of the proposed scheme in clinical usage. The proposed framework reproduces the highly accurate elasticity field directly from the displacement inputs. ConclusionsThe proposed framework circumvents different data pre-processing and intermediate steps utilized in traditional methods, hence providing an accurate elasticity map. The computationally efficient framework requires fewer epochs for training, which bodes well for its clinical usability for real-time predictions. The weights and biases from pre-trained models can also be employed for transfer learning, which reduces the effective training time with random initialization.

Structure and performance of fully connected neural networks: Emerging complex network properties

Understanding the behavior of Artificial Neural Networks is one of the main topics in the field recently, as black-box approaches have become usual since the widespread of deep learning. Such high-dimensional models may manifest instabilities and weird properties that resemble complex systems. Therefore, we propose Complex Network (CN) techniques to analyze the structure and performance of fully connected neural networks. For that, we build a dataset with 4 thousand models (varying the initialization seed) and their respective CN properties. This is the first work to explore the CN properties of an ample number of fully connected networks accounting for the variance caused by random weight initialization. The networks are trained in a supervised classification setup considering four vision benchmarks and then approached as a weighted and undirected graph of neurons and synapses (learned weights). Results show that neuronal centrality is highly correlated to network classification performance. We also propose the concept of Bag-Of-Neurons (BoN), a CN-based approach for finding topological signatures linking similar neurons. Results suggest that six neuronal types emerge in such networks, independently of the target domain, and are distributed differently according to classification accuracy. We also tackle specific CN properties related to performance, such as higher subgraph centrality on lower-performing models. Our findings suggest that CN properties play a critical role in the performance of fully connected neural networks, with topological patterns emerging independently on a wide range of models.

Sharp global convergence guarantees for iterative nonconvex optimization with random data

We consider a general class of regression models with normally distributed covariates, and the associated nonconvex problem of fitting these models from data. We develop a general recipe for analyzing the convergence of iterative algorithms for this task from a random initialization. In particular, provided each iteration can be written as the solution to a convex optimization problem satisfying some natural conditions, we leverage Gaussian comparison theorems to derive a deterministic sequence that provides sharp upper and lower bounds on the error of the algorithm with sample splitting. Crucially, this deterministic sequence accurately captures both the convergence rate of the algorithm and the eventual error floor in the finite-sample regime, and is distinct from the commonly used “population” sequence that results from taking the infinite-sample limit. We apply our general framework to derive several concrete consequences for parameter estimation in popular statistical models including phase retrieval and mixtures of regressions. Provided the sample size scales near linearly in the dimension, we show sharp global convergence rates for both higher-order algorithms based on alternating updates and first-order algorithms based on subgradient descent. These corollaries, in turn, reveal multiple nonstandard phenomena that are then corroborated by extensive numerical experiments.

Harmony Search-Based Optimization for Multi-RISs MU-MISO OFDMA Systems

In this letter, we aim to maximize the downlink sum-rate for multi-antenna orthogonal frequency division multiple access (OFDMA) systems with multiple reconfigurable intelligent surfaces (RISs). Due to the high non-convexity of the multiple RISs-aided systems, the sum-rate of the system is often dictated by the initial values of the optimization variables. We propose a novel harmony search-based alternating optimization (AO) algorithm to improve the AO algorithm by searching the high-quality initial value. In the numerical results, the proposed technique outperforms the AO algorithm with random initialization, making the proposed algorithm highly practical.

Recovering Thermodynamics from Spectral Profiles Observed by IRIS. (II). Improved Calculation of the Uncertainties Based on Monte Carlo Experiments

Observations by the Interface Region Imaging Spectrograph (IRIS) of the Mg ii h & k spectral lines have provided a new diagnostic window toward knowledge of the complex physical conditions in the solar chromosphere. Theoretical efforts focused on understanding the behavior of these lines have allowed us to obtain a better and more accurate vision of the chromosphere. These efforts include forward modeling, numerical simulations, and inversions. In this paper, we focus our attention on the uncertainties associated with the thermodynamic model atmosphere obtained after the inversion of the Mg ii h & k lines. We have used ≈50,000 synthetic representative profiles of the IRIS2 database to characterize the most important sources of uncertainties in the inversion process, viz.: the inherent noise of the observations, the random initialization process, and the selection criteria in a high-dimensional space. We have applied a Monte Carlo approach to this problem. Thus, for a given synthetic representative profile, we have created five randomized noise realizations (representative of the most popular exposure times in the IRIS observations), and inverted these profiles five times with different inversion initializations. The resulting 25 inverted profiles, fit to noisy data, and model atmospheres are then used to determine the uncertainty in the model atmosphere, based on the standard deviation and empirical selection criteria for the goodness of fit. With this approach, the new uncertainties of the models available in the IRIS2 database are more reliable at the optical depths where the Mg ii h & k lines are sensitive to changes in the thermodynamics.

Flow distribution optimization of parallel pumps based on improved mayfly algorithm

An improved mayfly algorithm is proposed for the energy saving optimization of parallel chilled water pumps in central air conditioning system, with the minimum energy consumption of parallel pump units as the optimization objective and the speed ratio of each pump as the optimization variable for the solution. For the problem of uneven random initialization of mayflies, the variable definition method of Circle chaotic mapping is used to make the initial position of the population uniformly distributed in the solution space, and the mayfly fitness value and the optimal fitness value are incorporated into the calculation of the weight coefficient, which better balances the global exploration and local exploitation of the algorithm. For the problem that the algorithm is easy to fall into the local optimum at the later stage, a multi-subpopulation cooperative strategy is proposed to improve the global search ability of the algorithm. Finally, the performance of the improved mayfly algorithm is tested with two parallel pumping system cases, and the stability and time complexity of the algorithm are verified. The experiments show that the algorithm can get a better operation strategy in solving the parallel water pump energy saving optimization problem, and can achieve energy saving effect of 0.72% 8.68% compared with other optimization algorithms, and the convergence speed and stability of the algorithm have been significantly improved, which can be better applied to practical needs.