Local Minimization Algorithm Research Articles

Using Optimization Techniques in Grammatical Evolution

The Grammatical Evolution technique has been successfully applied to a wide range of problems in various scientific fields. However, in many cases, techniques that make use of Grammatical Evolution become trapped in local minima of the objective problem and fail to reach the optimal solution. One simple method to tackle such situations is the usage of hybrid techniques, where local minimization algorithms are used in conjunction with the main algorithm. However, Grammatical Evolution is an integer optimization problem and, as a consequence, techniques should be formulated that are applicable to it as well. In the current work, a modified version of the Simulated Annealing algorithm is used as a local optimization procedure in Grammatical Evolution. This approach was tested on the Constructed Neural Networks and a remarkable improvement of the experimental results was shown, both in classification data and in data fitting cases.

Characterization of train kinematics and source wavelets from near-field seismic data

SUMMARY Train traffic is a powerful source of seismic waves, with many applications for passive seismic imaging. Seismic signals were recorded a few metres away from the railway track. These records display harmonious waveforms below 15 Hz for trains driving at speeds of around 100 km hr−1. The sensors record an apparent wavelet emitted by the interaction of the axle on a few of the closest sleepers. From this, we build a simple modelling tool using shifted wavelets to simulate a train signal. The relationship involves the varying train speed, the distances between each axle, and the wavelet emitted by each axle. We propose a nonlinear deconvolution method to invert this relationship. We use a local minimization algorithm with gradients derived by the adjoint state method, and use a frequency continuation technique. A linearized picking-based inversion initializes the nonlinear inversion. On real data, we apply this automatic workflow to 300 train passages, with an excellent match between the best simulation and the data. We identify the trains decelerating as they enter a train station. We also identify the train type based on inverted wheel spacing with centimetric accuracy. The inverted wavelets are consistent with the assumption that trains emit seismic waves by bending the rail above sleepers, although the theory does not explain why the inverted wavelet is not zero phase. This automated kinematic inversion algorithm may allow for contactless railway monitoring, and be used for source characterization for subsurface monitoring below railway tracks.

A new parameter free filled function for solving unconstrained global optimization problems

The filled function method is an efficient approach for finding a global minimizer of global optimization problems. This paper introduces a new filled function which overcomes the drawbacks of sensitivity to parameters, containing exponential or logarithmic terms, discontinuity and non-differentiability for some previous filled functions. It proposes a filled function without any parameters to be adjusted. This filled function has no exponential or logarithmic terms which make the filled function numerically unstable. Also it is continuously differentiable, so gradient information is available in order to use effective local minimization algorithms. Theories of the proposed filled function are investigated and an algorithm for unconstrained global optimization is presented. Numerical results on many test problems with large number of variables are reported. A comparison with some existing algorithms shows that this algorithm is efficient and reliable.

Flow-Based Clustering on Directed Graphs: A Structural Entropy Minimization Approach

In this study, we focus on flow-based clustering on directed graphs and propose a localized algorithm for this problem. Flow-based clustering in networks requires a set of closely related vertices where the flow amount getting into it is larger than that going out of it. It is able to formulate a variety of practical problems, such as fund-raising set detection in financial networks and influential document clustering in citation networks, etc. Methodologically, we propose the new concept of two-dimensional structural entropy on directed graphs, and based on this, a local structural entropy minimization algorithm for detecting the flow-based community structure of networks is designed. We adopt our algorithm for the problem of fund-raising set detection in financial networks, in which vertices represent accounts, edges represent transactions between two accounts, and weights represent money amounts of transactions. In our experiments, the local two-dimensional structure entropy minimization algorithm is devoted to find a fund-raising community which involves a given input account. We conduct experiments on both synthetic and real fund-raising datasets. The experimental results demonstrate that, given a fixed account, our algorithm is able to efficiently locate a fund-raising community (if any) for which the fund flowing into the community is much higher than that flowing out, and the transactions within the community are relatively denser (fund amount based) than that of inter-community. For a synthetic ground-truth fund-raising community, we adjust the parameters to change its fund-raising tendency. The results for the synthetic datasets show that our algorithm obtains higher precision and recall rates as this tendency gets stronger with each single factor varing. For a real fund-raising community embedded in a simulated capital flow network, our algorithm also find it with high precision and recall rates. The experiments for both scenarios verify the effectiveness of our algorithm.

Minimizing Piecewise-Concave Functions Over Polyhedra

We introduce an iterative method for solving linearly constrained optimization problems, whose nonsmooth nonconvex objective function is defined as the pointwise maximum of finitely many concave functions. Such problems often arise from reformulations of certain constraint structures (e.g., binary constraints, finite max-min constraints) in diverse areas of optimization. We state a local optimization strategy, which exploits piecewise-concavity of the objective function, giving rise to a linearized model corrected by a proximity term. In addition we introduce an approximate line-search strategy, based on a curvilinear model, which, similarly to bundle methods, can return either a satisfactory descent or a null-step declaration. Termination at a point satisfying an approximate stationarity condition is proved. We embed the local minimization algorithm into a Variable Neighborhood Search scheme and a Coordinate Direction Search heuristic, whose aim is to improve the current estimate of the global minimizer by exploring different parts of the feasible region. We finally provide computational results obtained on two sets of small- to large-size test problems.

Optimal prediction for sparse linear models? Lower bounds for coordinate-separable M-estimators

For the problem of high-dimensional sparse linear regression, it is known that an $\ell_{0}$-based estimator can achieve a $1/n$ “fast” rate for prediction error without any conditions on the design matrix, whereas in the absence of restrictive conditions on the design matrix, popular polynomial-time methods only guarantee the $1/\sqrt{n}$ “slow” rate. In this paper, we show that the slow rate is intrinsic to a broad class of M-estimators. In particular, for estimators based on minimizing a least-squares cost function together with a (possibly nonconvex) coordinate-wise separable regularizer, there is always a “bad” local optimum such that the associated prediction error is lower bounded by a constant multiple of $1/\sqrt{n}$. For convex regularizers, this lower bound applies to all global optima. The theory is applicable to many popular estimators, including convex $\ell_{1}$-based methods as well as M-estimators based on nonconvex regularizers, including the SCAD penalty or the MCP regularizer. In addition, we show that bad local optima are very common, in that a broad class of local minimization algorithms with random initialization typically converge to a bad solution.

A KIM-compliant potfit for fitting sloppy interatomic potentials: application to the EDIP model for silicon

Fitted interatomic potentials are widely used in atomistic simulations thanks to their ability to compute the energy and forces on atoms quickly. However, the simulation results crucially depend on the quality of the potential being used. Force matching is a method aimed at constructing reliable and transferable interatomic potentials by matching the forces computed by the potential as closely as possible, with those obtained from first principles calculations. The potfit program is an implementation of the force-matching method that optimizes the potential parameters using a global minimization algorithm followed by a local minimization polish. We extended potfit in two ways. First, we adapted the code to be compliant with the KIM Application Programming Interface (API) standard (part of the Knowledgebase of Interatomic Models project). This makes it possible to use potfit to fit many KIM potential models, not just those prebuilt into the potfit code. Second, we incorporated the geodesic Levenberg–Marquardt (LM) minimization algorithm into potfit as a new local minimization algorithm. The extended potfit was tested by generating a training set using the KIM environment-dependent interatomic potential (EDIP) model for silicon and using potfit to recover the potential parameters from different initial guesses. The results show that EDIP is a ‘sloppy model’ in the sense that its predictions are insensitive to some of its parameters, which makes fitting more difficult. We find that the geodesic LM algorithm is particularly efficient for this case. The extended potfit code is the first step in developing a KIM-based fitting framework for interatomic potentials for bulk and two-dimensional materials. The code is available for download via https://www.potfit.net.

Local Minimization Algorithms for Dynamic Programming Equations

The numerical realization of the dynamic programming principle for continuous-time optimal control leads to nonlinear Hamilton--Jacobi--Bellman equations which require the minimization of a nonlinear mapping over the set of admissible controls. This minimization is often performed by comparison over a finite number of elements of the control set. In this paper we demonstrate the importance of an accurate realization of these minimization problems and propose algorithms by which this can be achieved effectively. The considered class of equations includes nonsmooth control problems with $\ell_1$-penalization which lead to sparse controls.

Ramp loss linear programming support vector machine

The ramp loss is a robust but non-convex loss for classification. Compared with other non-convex losses, a local minimum of the ramp loss can be effectively found. The effectiveness of local search comes from the piecewise linearity of the ramp loss. Motivated by the fact that the l1-penalty is piecewise linear as well, the l1-penalty is applied for the ramp loss, resulting in a ramp loss linear programming support vector machine (ramp-LPSVM). The proposed ramp-LPSVM is a piecewise linear minimization problem and the related optimization techniques are applicable. Moreover, the l1-penalty can enhance the sparsity. In this paper, the corresponding misclassification error and convergence behavior are discussed. Generally, the ramp loss is a truncated hinge loss. Therefore ramp-LPSVM possesses some similar properties as hinge loss SVMs. A local minimization algorithm and a global search strategy are discussed. The good optimization capability of the proposed algorithms makes ramp-LPSVM perform well in numerical experiments: the result of ramp-LPSVM is more robust than that of hinge SVMs and is sparser than that of ramp-SVM, which consists of the || ċ || k-penalty and the ramp loss.

Flexible Refinement of Protein-Ligand Docking on Manifolds.

Our work is motivated by energy minimization of biological macromolecules, an essential step in computational docking. By allowing some ligand flexibility, we generalize a recently introduced novel representation of rigid body minimization as an optimization on the [Formula: see text] manifold, rather than on the commonly used Special Euclidean group SE(3). We show that the resulting flexible docking can also be formulated as an optimization on a Lie group that is the direct product of simpler Lie groups for which geodesics and exponential maps can be easily obtained. Our computational results for a local optimization algorithm developed based on this formulation show that it is about an order of magnitude faster than the state-of-the-art local minimization algorithms for computational protein-small molecule docking.

The polymorphs of ROY: application of a systematic crystal structure prediction technique

We investigate the ability of current ab initio crystal structure prediction techniques to identify the polymorphs of 5-methyl-2-[(2-nitrophenyl)amino]-3-thiophenecarbonitrile, also known as ROY because of the red, orange and yellow colours of its polymorphs. We use a methodology combining the generation of a large number of structures based on a computationally inexpensive model using the CrystalPredictor global search algorithm, and the further minimization of the most promising of these structures using the CrystalOptimizer local minimization algorithm which employs an accurate, yet efficiently constructed, model based on isolated-molecule quantum-mechanical calculations. We demonstrate that this approach successfully predicts the seven experimentally resolved structures of ROY as lattice-energy minima, with five of these structures being within the 12 lowest energy structures predicted. Some of the other low-energy structures identified are likely candidates for the still unresolved polymorphs of this molecule. The relative stability of the predicted structures only partially matches that of the experimentally resolved polymorphs. The worst case is that of polymorph ON, whose relative energy with respect to Y is overestimated by 6.65 kJ mol(-1). This highlights the need for further developments in the accuracy of the energy calculations.

A new method for the gradient‐based optimization of molecular complexes

We present a novel method for the local optimization of molecular complexes. This new approach is especially suited for usage in molecular docking. In molecular modeling, molecules are often described employing a compact representation to reduce the number of degrees of freedom. This compact representation is realized by fixing bond lengths and angles while permitting changes in translation, orientation, and selected dihedral angles. Gradient-based energy minimization of molecular complexes using this representation suffers from well-known singularities arising during the optimization process. We suggest an approach new in the field of structure optimization that allows to employ gradient-based optimization algorithms for such a compact representation. We propose to use exponential mapping to define the molecular orientation which facilitates calculating the orientational gradient. To avoid singularities of this parametrization, the local minimization algorithm is modified to change efficiently the orientational parameters while preserving the molecular orientation, i.e. we perform well-defined jumps on the objective function. Our approach is applicable to continuous, but not necessarily differentiable objective functions. We evaluated our new method by optimizing several ligands with an increasing number of internal degrees of freedom in the presence of large receptors. In comparison to the method of Solis and Wets in the challenging case of a non-differentiable scoring function, our proposed method leads to substantially improved results in all test cases, i.e. we obtain better scores in fewer steps for all complexes.

Informative SNP Selection Methods Based on SNP Prediction

The search for the association between complex diseases and single nucleotide polymorphisms (SNPs) or haplotypes has recently received great attention. For these studies, it is essential to use a small subset of informative SNPs, i.e., tag SNPs, accurately representing the rest of the SNPs. Tag SNP selection can achieve: 1) considerable budget savings by genotyping only a limited number of SNPs and computationally inferring all other SNPs or 2) necessary reduction of the huge SNP sets (obtained, e.g., from Affymetrix) for further fine haplotype analysis. In this paper, we show that the tag SNP selection strongly depends on how the chosen tags will be used-advantage of one tag set over another can only be considered with respect to a certain prediction method. We show how to separate tag selection from SNP prediction and propose greedy and local-minimization algorithms for tag SNP selection. We give two novel approaches to SNP prediction based on multiple linear regression (MLR) and support vector machines (SVMs). An extensive experimental study on various datasets including ten regions from hapMap project shows that the MLR prediction combined with stepwise tag selection uses fewer tags than the state-of-the-art method of Halperin et al. The MLR-based method also uses on average 30% fewer tags than IdSelect for statistical covering all SNPs. The tag selection based on SVM SNP prediction uses fewer tags to achieve the same prediction accuracy as the methods of Halldorsson et al.

A new hybrid methodology for cooperative-coevolutionary optimization of radial basis function networks

This paper presents a new multiobjective cooperative–coevolutive hybrid algorithm for the design of a Radial Basis Function Network (RBFN). This approach codifies a population of Radial Basis Functions (RBFs) (hidden neurons), which evolve by means of cooperation and competition to obtain a compact and accurate RBFN. To evaluate the significance of a given RBF in the whole network, three factors have been proposed: the basis function’s contribution to the network’s output, the error produced in the basis function radius, and the overlapping among RBFs. To achieve an RBFN composed of RBFs with proper values for these quality factors our algorithm follows a multiobjective approach in the selection process. In the design process, a Fuzzy Rule Based System (FRBS) is used to determine the possibility of applying operators to a certain RBF. As the time required by our evolutionary algorithm to converge is relatively small, it is possible to get a further improvement of the solution found by using a local minimization algorithm (for example, the Levenberg–Marquardt method). In this paper the results of applying our methodology to function approximation and time series prediction problems are also presented and compared with other alternatives proposed in the bibliography.

Generating inherent structures of liquids: Comparison of local minimization algorithms

Properties of minima obtained by two different local minimization algorithms, the conjugate gradient and the limited-memory Broyden-Fletcher-Goldfarb-Shanno, are compared for a liquid whose particles interact via a modified Lennard-Jones pair interaction. The average properties of inherent structures obtained by the two minimization procedures are shown to agree within statistical error, though for a given starting configuration the two algorithms may not quench to identical minima.

Local sequential minimization of double stranded B-DNA using Monte Carlo annealing.

A software algorithm has been developed to investigate the folding process in B-DNA structures in vacuum under a simple and accurate force field. This algorithm models linear double stranded B-DNA sequences based on a local, sequential minimization procedure. The original B-DNA structures were modeled using initial nucleotide structures taken from the Brookhaven database. The models contain information at the atomic level allowing one to investigate as accurately as possible the structure and characteristics of the resulting DNA structures. A variety of DNA sequences and sizes were investigated containing coding and non-coding, random and real, homogeneous or heterogeneous sequences in the range of 2 to 40 base pairs. The force field contains terms such as angle bend, Lennard-Jones, electrostatic interactions and hydrogen bonding which are set up using the Dreiding II force field and defined to account for the helical parameters such as twist, tilt and rise. A close comparison was made between this local minimization algorithm and a global one (previously published) in order to find out advantages and disadvantages of the different methods. From the comparison, this algorithm gives better and faster results than the previous method, allowing one to minimize larger DNA segments. DNA segments with a length of 40 bases need approximately 4 h, while 2.5 weeks are needed with the previous method. After each minimization the angles between phosphate-oxygen-carbon A1, the oxygen-phosphate-oxygen A2 and the average helical twists were calculated. From the generated fragments it was found that the bond angles are A1=150 degrees +/-2 degrees and A2=130 degrees +/-10 degrees, while the helical twist is 36.6 degrees +/-2 degrees in the A strand and A1=150 degrees +/-6 degrees and A2=130+/-6 degrees with helical twist 39.6 degrees +/-2 degrees in the B strand for the DNA segment with the same sequence as the Dickerson dodecamer.

B-spline method for energy minimization in grid-based molecular mechanics calculations

A method is described for molecular mechanics calculations based on a cubic B-spline approximation of the potential energy. This method is useful when parts of the system are allowed to remain fixed in position so that a potential energy grid can be precalculated and used to approximate the interaction energy between parts of a molecule or between molecules. We adapted and modified the conventional B-spline method to provide an approximation of the Empirical Conformational Energy Program for Peptides (ECEPP) potential energy function. The advantage of the B-spline method over simpler approximations is that the resulting B-spline function is C2 continuous, which allows minimization of the potential energy by any local minimization algorithm. The standard B-spline method provides a good approximation of the electrostatic energy; but in order to reproduce the Lennard–Jones and hydrogen-bonding functional forms accurately, it was necessary to modify the standard B-spline method. This modification of the B-spline method can also be used to improve the accuracy of trilinear interpolation for simulations that do not require continuous derivatives. As an example, we apply the B-spline method to rigid-body docking energy calculations using the ECEPP potential energy function. Energies are calculated for the complex of Phe-Pro-Arg with thrombin. For this system, we compare the performance of the B-spline method to that of the standard pairwise summation in terms of speed, accuracy, and overhead costs for a variety of grid spacings. In our rigid-body docking calculations, the B-spline method provided an accurate approximation of the total energy of the system, and it resulted in an 180-fold reduction in the time required for a single energy and gradient calculation for this system. © 1998 John Wiley & Sons, Inc. J Comput Chem 19: 71–85, 1998

A Multivariate Global Optimization Using Linear Bounding Functions

Recently linear bounding functions (LBFs) were proposed and used to find e-global minima. This paper presents an LBF-based algorithm for multivariate global optimization problems. The algorithm consists of three phases. In the global phase, big subregions not containing a solution are quickly eliminated and those which possibly contain the solution are detected. An efficient scheme for the local phase is developed using our previous local minimization algorithm, which is globally convergent with superlinear/quadratic rate and does not require evaluation of gradients and Hessian matrices. To ensure that the found minimizers are indeed the global solutions or save computation effort, a third phase called the verification phase is often needed. Under adequate conditions the algorithm finds the e-global solution(s) within finite steps. Numerical testing results illustrate how the algorithm works, and demonstrate its potential and feasibility.

New suboptimal multiuser detectors for synchronous CDMA systems

In code-division multiple access (CDMA) systems, the optimal multiuser detection is equivalent to an unconstrained quadratic bivalent optimization problem, and its computational complexity increases exponentially with the number of users. In this paper, we propose several new suboptimal detectors with a greatly reduced computational complexity based on a local minimization algorithm for a synchronous CDMA system. The performances of these detectors are compared with those of the conventional, optimal, and several other suboptimal detectors.

Approach for Optimal Nesting Using Genetic Algorithm and Local Minimization Algorithm.

In layout design problems including blank nesting, the positions and directions of layout elements must be determined so as to minimize the total space. It is difficult and computationally time-consuming to find the optimal solution for such layout problems, because they include numerous underlying combinational conditions. In this paper, we develop an approach for optimal nesting by combining a genetic algorithm and a local minimization algorithm. In the approach, the genetic algorithm is used for handling combinations which are represented in the string, and the local minimization algorithm is used for determining the embodiment layout under fixed combinations so as to minimize the scrap space corresponding to the fitness value in the genetic algorithm. We also present an example for showing the effective nesting result produced by this approach.