Traditional Grid Search Research Articles

Design of tidal range energy generation schemes using a Genetic Algorithm model

One of the key aspects of Tidal Range Schemes globally is identifying the most appropriate site and the optimised design and operation of the scheme, to maximise societal needs and the benefits from electricity generation. Variations in the design parameters of Tidal Range Schemes for electricity generation could therefore lead to a very large number of design and operation scenarios. In this study, a novel Genetic Algorithm model was developed to deliver the complete design of the most optimised Tidal Range Schemes for electricity generation, including the number of turbines, sluicing areas and the maximum amount of electricity that could be generated, through identifying the most optimised operation scheme for a particular site. The Genetic Algorithm model has been used to design a new Tidal Range Scheme proposed for development in the Bristol Channel, UK, with a potential to generate about 7.16 TWh/yr. The design of the scheme was also investigated using a traditional grid search approach for a range of scenarios, together with the model being used to investigate the performance of the complete design of the scheme, evaluated through a comparison of the most optimised design in terms of electricity generation. This comparison has shown that the Genetic Algorithm model was capable of achieving largely the same outcomes and reducing the computational time by approximately 95% to that based on using traditional Grid Search methods.

An improved grid search algorithm to optimize SVR for prediction

Parameter optimization is an important step for support vector regression (SVR), since its prediction performance greatly depends on values of the related parameters. To solve the shortcomings of traditional grid search algorithms such as too many invalid search ranges and sensitivity to search step, an improved grid search algorithm is proposed to optimize SVR for prediction. The improved grid search (IGS) algorithm is used to optimize the penalty parameter and kernel function parameter of SVR by automatically changing the search range and step for several times, and then SVR is trained for the optimal solution. The available of the method is proved by predicting the values of soil and plant analyzer development (SPAD) in rice leaves. To predict SPAD values more quickly and accurately, some dimension reduction methods such as stepwise multiple linear regressions (SMLR) and principal component analysis (PCA) are processed the training data, and the results show that the nonlinear fitting and prediction performance of accuracy of SMLR-IGS-SVR and PCA-IGS-SVR are better than those of IGS-SVR.

Optimizing echo state network through a novel fisher maximization based stochastic gradient descent

Hyperparameter optimization is a challenging process that has the potential to improve machine learning algorithms. Since it creates a remarkable computational burden for machine learning tasks, there have been few works coping with tuning strategies of a specific algorithm. In this paper, an improved Stochastic Gradient Descent (SGD) based on Fisher Maximization is developed for tuning hyperparameters of an Echo State Network (ESN) which has a wide range of applications. The results of the method are then compared with those of traditional Gradient Descent and Grid Search. According to the obtained results; 1) The scale of the data sets greatly affects the reliability of hyperparameter optimization results; 2) Feature selection is critical in terms of mean error of training when hyperparameter optimization is applied on some methods such as ESN; 3) SGD falls in a good local minima if Fisher Maximization is performed to find a good starting point.

Embedding physics domain knowledge into a Bayesian network enables layer-by-layer process innovation for photovoltaics

Process optimization of photovoltaic devices is a time-intensive, trial-and-error endeavor, which lacks full transparency of the underlying physics and relies on user-imposed constraints that may or may not lead to a global optimum. Herein, we demonstrate that embedding physics domain knowledge into a Bayesian network enables an optimization approach for gallium arsenide (GaAs) solar cells that identifies the root cause(s) of underperformance with layer-by-layer resolution and reveals alternative optimal process windows beyond traditional black-box optimization. Our Bayesian network approach links a key GaAs process variable (growth temperature) to material descriptors (bulk and interface properties, e.g., bulk lifetime, doping, and surface recombination) and device performance parameters (e.g., cell efficiency). For this purpose, we combine a Bayesian inference framework with a neural network surrogate device-physics model that is 100× faster than numerical solvers. With the trained surrogate model and only a small number of experimental samples, our approach reduces significantly the time-consuming intervention and characterization required by the experimentalist. As a demonstration of our method, in only five metal organic chemical vapor depositions, we identify a superior growth temperature profile for the window, bulk, and back surface field layer of a GaAs solar cell, without any secondary measurements, and demonstrate a 6.5% relative AM1.5G efficiency improvement above traditional grid search methods.

A grid-quadtree model selection method for support vector machines

In this paper, a new model selection approach for Support Vector Machine (SVM), which integrates the quadtree technique with the grid search, denominated grid-quadtree (GQ) is proposed. The developed method is the first in the literature to apply the quadtree for the SVM parameters optimization. The SVM is a machine-learning technique for pattern recognition whose performance relies on its parameters determination. Thus, the model selection problem for SVM is an important field of study and requires expert and intelligent systems to solve it. Real classification data sets involve a huge number of instances and features, and the greater is the training data set dimension, the larger is the cost of a recognition system. The grid search (GS) is the most popular and the simplest method to select parameters for SVM. However, it is time-consuming, which limits its application for big-sized problems. With this in mind, the main idea of this research is to apply the quadtree technique to the GS to make it faster. Hence, this may lower computational time cost for solving problems such as bio-identification, bank credit risk and cancer detection. Based on the asymptotic behaviors of the SVM, it was noticeably observed that the quadtree is able to avoid the GS full search space evaluation. As a consequence, the GQ carries out fewer parameters analysis, solving the same problem with much more efficiency. To assess the GQ performance, ten classification benchmark data set were used. The obtained results were compared with the ones of the traditional GS. The outcomes showed that the GQ is able to find parameters that are as good as the GS ones, executing 78.8124% to 85.8415% fewer operations. This research points out that the adoption of quadtree expressively reduces the computational time of the original GS, making it much more efficient to deal with high dimensional and large data sets.

Application of Raman spectroscopy in the detection of hepatitis B virus infection.

Detection of hepatitis B virus (HBV) using Raman spectroscopy. Raman spectroscopy was used to examine the serum samples of 500 patients with HBV and 500 non-HBV persons. First, the adaptive iterative weighted penalty least squares method (airPLS) was used to deduct the fluorescence background in Raman spectra. Then, a principal component analysis (PCA) was used to extract the processed Raman spectra, and a support vector machine (SVM) was used for modeling and prediction. The particle swarm optimization (PSO) algorithm was selected to optimize the parameters of the SVM instead of a traditional grid search. Finally, 600 serum samples were detected by Raman spectroscopy, and the results wereverified using a double-blind method. In the Raman spectra, the non-HBV human Raman peaks at 509, 957, 1002, 1153, 1260, 1512, 1648 and 2305 cm-1 were different from those of patients with HBV. The reported accuracy, sensitivity and specificity of the HBV serum model established using airPLS-PCA-PSO-SVM was 93.1%, 100% and 88%, respectively. The two groups were verified by a double-blind method. In the first group sensitivity was 87%, specificity was 92%, and the KAPPA value was 0.79; in the second group sensitivity was 80%, specificity was 79%, and the KAPPA value was 0.59. This preliminary study shows that serum Raman spectroscopy combined with the airPLS-PCA-PSO-SVM model can be used for hepatitis B virus detection.

Quantitative Analysis of Elements in Fertilizer Using Laser-Induced Breakdown Spectroscopy Coupled with Support Vector Regression Model.

The rapid detection of the elements nitrogen (N), phosphorus (P), and potassium (K) is beneficial to the control of the compound fertilizer production process, and it is of great significance in the fertilizer industry. The aim of this work was to compare the detection ability of laser-induced breakdown spectroscopy (LIBS) coupled with support vector regression (SVR) and obtain an accurate and reliable method for the rapid detection of all three elements. A total of 58 fertilizer samples were provided by Anhui Huilong Group. The collection of samples was divided into a calibration set (43 samples) and a prediction set (15 samples) by the Kennard–Stone (KS) method. Four different parameter optimization methods were used to construct the SVR calibration models by element concentration and the intensity of characteristic line variables, namely the traditional grid search method (GSM), genetic algorithm (GA), particle swarm optimization (PSO), and least squares (LS). The training time, determination coefficient, and the root-mean-square error for all parameter optimization methods were analyzed. The results indicated that the LIBS technique coupled with the least squares–support vector regression (LS-SVR) method could be a reliable and accurate method in the quantitative determination of N, P, and K elements in complex matrix like compound fertilizers.

Accurate unconditional p-values for a two-arm study with binary endpoints

ABSTRACTUnconditional exact tests are increasingly used in practice for categorical data to increase the power of a study and to make the data analysis approach being consistent with the study design. In a two-arm study with a binary endpoint, p-value based on the exact unconditional Barnard test is computed by maximizing the tail probability over a nuisance parameter with a range from 0 to 1. The traditional grid search method is able to find an approximate maximum with a partition of the parameter space, but it is not accurate and this approach becomes computationally intensive for a study beyond two groups. We propose using a polynomial method to rewrite the tail probability as a polynomial. The solutions from the derivative of the polynomial contain the solution for the global maximum of the tail probability. We use an example from a double-blind randomized Phase II cancer clinical trial to illustrate the application of the proposed polynomial method to achieve an accurate p-value. We also compare the performance of the proposed method and the traditional grid search method under various conditions. We would recommend using this new polynomial method in computing accurate exact unconditional p-values.

Locating Firefighters in Immediately Dangerous to Life or Health Environments

Locating injured firefighters is critical in immediately dangerous to life or health (IDLH) environments where they can be overcome by smoke, or exposed to other environmental stressors. Auditory and visual communication technologies such as Personal Alert Safety System (PASS) devices and radio communications are often used to locate firefighters entering IDLH environments. However, the reliability of these communication methods can be detrimental in pinpointing the location of an injured firefighter due to unconsciousness, disorientation, or unfamiliarity with the layout of the building. The purpose of this study was to identify the effects of visibility conditions, rescuer familiarity with the search area, and search and rescue method on the time needed to locate victims in an IDLH environment. Results of the study indicated that time needed to locate victims could be significantly reduced when using a radio frequency identification (RFID) based real time locating method compared to a traditional grid search rescue method.

Cross Validation Through Two-Dimensional Solution Surface for Cost-Sensitive SVM.

Model selection plays an important role in cost-sensitive SVM (CS-SVM). It has been proven that the global minimum cross validation (CV) error can be efficiently computed based on the solution path for one parameter learning problems. However, it is a challenge to obtain the global minimum CV error for CS-SVM based on one-dimensional solution path and traditional grid search, because CS-SVM is with two regularization parameters. In this paper, we propose a solution and error surfaces based CV approach (CV-SES). More specifically, we first compute a two-dimensional solution surface for CS-SVM based on a bi-parameter space partition algorithm, which can fit solutions of CS-SVM for all values of both regularization parameters. Then, we compute a two-dimensional validation error surface for each CV fold, which can fit validation errors of CS-SVM for all values of both regularization parameters. Finally, we obtain the CV error surface by superposing K validation error surfaces, which can find the global minimum CV error of CS-SVM. Experiments are conducted on seven datasets for cost sensitive learning and on four datasets for imbalanced learning. Experimental results not only show that our proposed CV-SES has a better generalization ability than CS-SVM with various hybrids between grid search and solution path methods, and than recent proposed cost-sensitive hinge loss SVM with three-dimensional grid search, but also show that CV-SES uses less running time.

Novel application of heuristic optimisation enables the creation and thorough evaluation of robust support vector machine ensembles for machine learning applications.

Today’s researchers have access to an unprecedented range of powerful machine learning tools with which to build models for classifying samples according to their metabolomic profile (e.g. separating diseased samples from healthy controls). However, such powerful tools need to be used with caution and the diagnostic performance of models produced by them should be rigorously evaluated if their output is to be believed. This involves considerable processing time, and has hitherto required expert knowledge in machine learning. By adopting a constrained nonlinear simplex optimisation for the tuning of support vector machines (SVMs) we have reduced SVM training times more than tenfold compared to a traditional grid search, allowing us to implement a high performance R package that makes it possible for a typical bench scientist to produce powerful SVM ensemble classifiers within a reasonable timescale, with automated bootstrapped training and rigorous permutation testing. This puts a state-of-the-art open source multivariate classification pipeline into the hands of every metabolomics researcher, allowing them to build robust classification models with realistic performance metrics.

A new approach to grid search method in slope stability analysis using Box–Behnken statistical design

Grid search method for locating the critical failure surface is extended by deriving additional analytical expressions for slip center grid (xmin, zmin; xmax, zmax), where global minimum of safety factor occurs, including the prediction of minimum and maximum values for safety factor (Fsmin,max) and for slip circle radius (Rmin,max). Derived models are proposed in a form of nonlinear functions of geometrical parameters (slope height H, depth to bedrock d and slope angle β) and soil factors (bulk density γ, cohesion c, angle of internal friction φ and pore water pressure coefficient ru). Research was performed using Box–Behnken experimental design, for which the input data were provided by Spencer limit equilibrium analyzes of different slopes with circular slip surface. Reasonable predictive power of the proposed models was verified both by internal and external validation, latter of which included the analyzes of slopes with random geometrical and soil properties. Regarding the impact of input parameters, β has the strongest influence on response values (Fs, R, x, z), with the predominant linear and quadratic effect. As for the influence of remaining factors, c and φ also have strong impact on Fs, while H and φ have significant influence on slip circle radius and the location of slip center grid. However, due to existence of two-factor interactions, it is shown that the effect of β on Fs, x, z and R is highly dependent on the values of c, φ, ru, H and d/H, including the significant effect of ru×φ, c×H and c×γ. When compared to traditional grid search method, proposed approach could be used to locate the circular slip surface with global minimum of safety factor, without the need for additional slope stability analyzes.

Application of Improved Grid Search Algorithm on SVM for Classification of Tumor Gene

According to the merits and shortcomings of the traditional gridsearch algorithm in parameters optimization of support vector machine (SVM), an improved grid search algorithm is proposed. Dichotomous search algorithm is used to reduce target searching range. First, searching range is determined roughly, and a set of parameters are obtained. Then fine search is applied in reduction the range for searching, and searching the optimum parameters.Three kinds of famous tumor gene data set are used in the comparison experiments to validate the classification accuracy of principal component analysis (PCA)SVM and kernel principal component analysis (KPCA)-SVM. Experiment results and data analysis shows that, comparing with traditional gridsearch algorithm, the proposed method has higher classification accuracy and less search time.

An Improved Grid Search Algorithm for Parameters Optimization on SVM

The algorithm in this paper is based on the basic principle of SVM. The sale data classification SVM classifier is designed using this algorithm. Also three classifiers including traditional grid search algorithm, ZGenetic Algorithm and Particle Swarm Optimization are used to do the comparison experiments of classification. Result shows that the improved grid-search algorithm can reduce the SVM classifier’s computational complexity effectively and improve its performance and classification accuracy.

Intelligent Optimization Methods for High-Dimensional Data Classification for Support Vector Machines

Support vector machine (SVM) is a popular pattern classification method with many application areas. SVM shows its outstanding performance in high-dimensional data classification. In the process of classification, SVM kernel parameter setting during the SVM training procedure, along with the feature selection significantly influences the classification accuracy. This paper proposes two novel intelligent optimization methods, which simultaneously determines the parameter values while discovering a subset of features to increase SVM classification accuracy. The study focuses on two evolutionary computing approaches to optimize the parameters of SVM: particle swarm optimization (PSO) and genetic algorithm (GA). And we combine above the two intelligent optimization methods with SVM to choose appropriate subset features and SVM parameters, which are termed GA-FSSVM (Genetic Algorithm-Feature Selection Support Vector Machines) and PSO-FSSVM(Particle Swarm Optimization-Feature Selection Support Vector Machines) models. Experimental results demonstrate that the classification accuracy by our proposed methods outperforms traditional grid search approach and many other approaches. Moreover, the result indicates that PSO-FSSVM can obtain higher classification accuracy than GA-FSSVM classification for hyperspectral data.

Technical note: Estimating parameters of nonlinear segmented models.

The objective of this technical note is to develop an applied technique to estimate parameters using the Statistical Analysis System's nonlinear procedure (SAS PROC NLIN) for segmented models that have a change point or lag as one of the parameters. The goal is to select good starting values for the parameters near the global minimum or at least near a lower local minimum compared with what might be achieved using traditional starting values. The model used was f1 (t) = B1 + B4 if t < or = B3 and f2(t) = B1*exp[-B2*(t - B3)] + B4 if t > B3, where B1, B2, B3, and B4 are the parameters that require estimates, B3 is the change point or lag, and t = time. This technical note illustrates the solution when a traditional grid search for starting values is used and demonstrates a modified technique where starting values are systemically determined by fixing B3 over a range of reasonable values and then using the parameters from the solution with the lowest residual sums of squares as the starting values for the final solution. The modified method resulted in a lower square root of mean square error compared with the traditional method. The estimates for B3 (lag) were 3.5 for the modified method compared with 4.5 for the traditional method. This technique works well when using the SAS PROC NLIN procedure but can be modified to work with other statistical packages.

Conformational analysis and flexibility of carbohydrates using the CICADA approach with MM3

AbstractThe potential energy hypersurfaces (PES) of several carbohydrate molecules were studied with a new algorithm for conformational searches, CICADA (Channels in Conformational Space Analyzed by Driver Approach) interfaced with the molecular mechanics program MM3(92). The method requires (1) one or a few low‐energy conformations as starting points; and (2) designation of the torsion angles important for understanding the conformational behavior of the molecule. The PES is explored by driving separately each selected torsion angle (in both directions) with a concomitant full‐geometry optimization at each increment (except for the driven angle). When a minimum has been detected, the molecule is freely optimized, and the minima so detected are then stored if not encountered previously. The new minima serve as starting structures for further explorations. The results from CICADA permit prediction of relative and absolute flexibility and conformational softness for both the entire molecule as well as for individual group rotations and local minima. The carbohydrates analyzed were Me‐α‐D‐glucopyranoside, β‐D‐GlcNAc(1‐2)α‐D‐Man, and α‐D‐GalNAc(1‐3)[α‐L‐Fuc(1‐2)]Gal‐O‐Me. All the low‐energy conformers along with the transition states and flexibilities features were characterized. CICADA found all minima and low‐energy conversion pathways for the disaccharide that were found by a traditional grid search. In contrast to the grid search method, CICADA concentrates mostly on the exploration of the low‐energy regions of the PES, thereby saving a significant amount of computational time. The performance of the method opens new routes for exploring conformational space of larger molecules, such as oligosaccharides. © 1995 by John Wiley &amp; Sons, Inc.