Exhaustive Grid Search Research Articles

Using artificial intelligence to overcome over-indebtedness and fight poverty

This research examines how artificial intelligence may contribute to better understanding and to overcome over-indebtedness in contexts of high poverty risk. This research uses Automated Machine Learning (AutoML) in a field database of 1654 over-indebted households to identify distinguishable clusters and to predict its risk factors. First, unsupervised machine learning using Self-Organizing Maps generated three over-indebtedness clusters: low-income (31.27%), low credit control (37.40%), and crisis-affected households (31.33%). Second, supervised machine learning with exhaustive grid search hyperparameters (32,730 predictive models) suggests that Nu-Support Vector Machine had the best accuracy in predicting families’ over-indebtedness risk factors (89.5%). By proposing an AutoML approach on over-indebtedness, our research adds both theoretically and methodologically to current models of scarcity with important practical implications for business research and society. Our findings also contribute to novel ways to identify and characterize poverty risk in earlier stages, allowing customized interventions for different profiles of over-indebtedness.

On-line classification of coal combustion quality using nonlinear SVM for improved neural network NOx emission rate prediction

A nonlinear support vector machine (SVM) uses engineered features to classify the quality of currently combusting coal as it is fired in an operating electric utility generator. The SVM classification result selects a unique neural network regression model to predict NOx emission rate. A two-part exhaustive grid-search and 5-fold cross-validation routine identifies the radial basis kernel as optimal for the SVM, achieving a classification accuracy of greater than 66%. The accuracy of the modified neural network structure improves on the original structure by 40%. This work contributes 1) evidence of feature engineering to enhance raw features in a complex industrial process and to provide otherwise unavailable data, 2) the formulation of a novel hybrid machine learning approach combining SVMs and neural networks with differing objectives harmoniously, and 3) a demonstrated improvement in neural network NOx emission rate prediction accuracy at a live operating electric utility generator due to SVM classification.

TDOA Localization With Unknown Signal Propagation Speed and Sensor Position Errors

In time-difference-of-arrival (TDOA) localization systems, the TDOA measurements and range-difference-of-arrival (RDOA) values can be used interchangeably if the signal propagation speed is known. In some scenarios like underwater acoustic localization, the signal propagation speed is often unknown. This letter addresses joint source location and propagation speed (unknown) estimation using signal TDOA measurements in the presence of sensor position errors. The maximum likelihood estimator (MLE) for this problem is optimal, but it requires a suitable initial value to obtain the global solution or needs exhaustive grid search. We develop a semidefinite programming (SDP) based localization algorithm to effectively solve the MLE problem. The accuracy of the proposed algorithm is close to the Cramer-Rao lower bound and has a superior performance than existing solutions.

Multivariable inversion using exhaustive grid search and high-performance GPU processing: a new perspective

SUMMARYExhaustive searches in regular grids is a traditional and effective method for inversion, that is numerical solution of systems of non-linear equations which cannot be solved using formal algebraic techniques. However, this technique is effective for very few (3–4) variables and is slow. Recently, the first limitation was to a major degree overpassed with the new TOPological INVersion (TOPINV) algorithm which was used for inversion of systems with up to 18, or even more unknown variables. The novelty of this algorithm is that it is not based on the principle of the mean minimum misfit (cost function) between observations and model predictions, used by most inversion techniques. The new algorithm investigates for each gridpoint whether misfits of each observation are within specified uncertainty intervals, and stores clusters of ‘successful’ gridpoints in matrix form. These clusters (ensembles, sets) of gridpoints are tested whether they satisfy certain criteria and are then used to compute one or more optimal statistical solutions. The new algorithm is efficient for highly non-linear problems with high measurement uncertainties (low signal-to-noise ratio, SNR) and poor distribution of observations, that is problems leading to complicated 3-D mean misfit surfaces without dominant peaks, but it is slow when running in common computers. To overcome this limitation, we used GPUs which permit parallel processing in common computers, but faced another computational problem: GPU parallel processing supports only up to three dimensions.To solve this problem, we used CUDA programming and optimized the distribution of the computational load to all GPU cores. This leads up to 100x speedup relative to common CPU processing, as is derived from comparative tests with synthetic data for two typical inversion geophysical problems with up to 18 unknown variables, Mogi magma source modeling and elastic dislocation modeling of seismic faults. This impressive speedup makes the GPU/CUDA implementation of TOPINV practical even for low-latency solution of certain geophysical problems.This speedup in calculations also permitted to investigate the performance of the new algorithm in relation to the density of the adopted grids. We focused on a typical problem of elastic dislocation in unfavorable conditions (poor observations geometry, data with low SNR) and on synthetic observations with noise, so that the difference of each solution from the ‘true’/reference value was known (accuracy-based approach). Application of the algorithm revealed stable, accurate and precise solutions, with quality increasing with the grid density. Solution defects (bias), mainly produced by very coarse grids, can be identified through specific diagnostic criteria, dictating finer search grids.

Localization of a Moving Source by Frequency Measurements

This paper investigates the localization of a moving source in position and velocity, by observing the emitted frequency from the source that is subject to the Doppler shift effect at a number of stationary sensors. Previous attempts rely on exhaustive grid search or numerical polynomial optimization to obtain a solution. We shall propose a constrained optimization to formulate the localization problem, which enables the problem to be solved efficiently using the linear optimization method to reach a closed-form solution or the semi-definite relaxation technique to achieve a noise resilient estimate. The algorithms are developed for the single-time and multiple-time observations. The presence of errors in the source frequency and the sensor positions are considered. The non-cooperative scenario where the source frequency is completely not known is also addressed. Analysis validates the proposed closed-form solution in reaching the Cramer-Rao Lower Bound accuracy under Gaussian noise over the small error region. Simulations support the performance of the proposed solutions.

Weighted Two-Dimensional Root MUSIC for Joint Angle-Doppler Estimation With MIMO Radar

For jointly estimating angle-Doppler information, two-dimen-sional multiple signal classification (MUSIC) is often applied, which requires an exhaustive grid search over angle-Doppler domain. To mitigate the computational complexity and enhance the estimation performance, we modify two-dimensional MUSIC in three steps. First, instead of using two-dimensional spectral search, it is converted to two consecutive polynomial root-finding problems. Second, the degree of the polynomial in the first root-finding stage is undesirably high in general; hence, we approximate it to a lower order polynomial through discrete Fourier transform. Finally, further enhancement can be achieved by having the closest approximation of null-spectrum function by assigning unequal weight coefficients. Cramer–Rao bound is derived for joint angle-Doppler estimation as a benchmark in our performance analysis.

Towards real-time respiratory motion prediction based on long short-term memory neural networks

Radiation therapy of thoracic and abdominal tumors requires incorporating the respiratory motion into treatments. To precisely account for the patient’s respiratory motions and predict the respiratory signals, a generalized model for predictions of different types of patients’ respiratory motions is desired. The aim of this study is to explore the feasibility of developing a long short-term memory (LSTM)-based generalized model for the respiratory signal prediction. To achieve that, 1703 sets of real-time position management (RPM) data were collected from retrospective studies across three clinical institutions. These datasets were separated as the training, internal validity and external validity groups. Among all the datasets, 1187 datasets were used for model development and the remaining 516 datasets were used to test the model’s generality power. Furthermore, an exhaustive grid search was implemented to find the optimal hyper-parameters of the LSTM model. The hyper-parameters are the number of LSTM layers, the number of hidden units, the optimizer, the learning rate, the number of epochs, and the length of time lags. The obtained model achieved superior accuracy over conventional artificial neural network (ANN) models: with the prediction window equaling to 500 ms, the LSTM model achieved an average relative mean absolute error (MAE) of 0.037, an average root mean square error (RMSE) of 0.048, and a maximum error (ME) of 1.687 in the internal validity data, and an average relative MAE of 0.112, an average RMSE of 0.139 and an ME of 1.811 in the external validity data. Compared to the LSTM model trained with default hyper-parameters, the MAE of the optimized model results decreased by 20%, indicating the importance of tuning the hyper-parameters of LSTM models to obtain superior accuracies. This study demonstrates the potential of deep LSTM models for the respiratory signal prediction and illustrates the impacts of major hyper-parameters in LSTM models.

Speed up grid-search for parameter selection of support vector machines

Support vector machine (SVM) has been recently considered as one of the most efficient classifiers. However, the time complexity of kernel SVM, which is quadratic in the number of training patterns, makes it impractical to be applied to large data sets. In such a case, the complexity is further increased when an exhaustive grid search is used to find its optimal parameters (the kernel parameters and the penalty parameter, C). To reduce this extra complexity, a novel approach is proposed that prunes the data points by removing the ones that have extremely small chance of becoming support vectors. This is accomplished by using the support vectors obtained from the training of an SVM with a smaller value of C as the training patterns for an SVM with a slightly larger value. This can serve in reducing the grid-search time for the standard SVM and for the approximate methods that search heuristically for a small range of the kernel parameters first. Experiments showed the effectiveness of the proposed approach in reducing the training time for both methods considerably while achieving a similar accuracy to the standard SVM. In addition, the training time of the latter method was faster than that of the evolutionary techniques based on the particle swarm optimization algorithm.

Parametric direction-of-arrival estimation with three recurrence relations of spherical harmonics.

The eigenbeam estimation of signal parameters via the rotational invariance technique (EB-ESPRIT) is a well-known subspace-based beamforming algorithm for a spherical microphone array. EB-ESPRIT uses a recurrence relation to directly estimate directional parameters expressing directions-of-arrival (DOAs) of sound sources without an exhaustive grid-search. In the conventional EB-ESPRIT, the directional parameter along the elevational direction is given by a tangent function, which inevitably produces two shortcomings. First, the tangent function becomes singular for sources near the equator in spherical coordinates. Furthermore, two sources lying in exactly opposite directions in the spherical coordinates are indistinguishable and a strong ambiguity problem arises. In this work, an EB-ESPRIT technique based on generalized eigenvalue decomposition (GEVD) is proposed to resolve the singularity and ambiguity problems. The proposed technique uses three independent recurrence relations for spherical harmonics, thus the singularity problem due to the tangent function can be completely avoided. A common transformation matrix for extracting DOAs from recurrence relations are found from the GEVD, and the use of cosine and sine functions makes it possible to find DOAs without ambiguity and without extra transforms or angle-pairing processes. It is demonstrated that the proposed method not only overcomes the singularity and ambiguity problems, but also outperforms conventional techniques in terms of DOA accuracy.

A hybrid method for improved stability prediction in construction projects: A case study of stope hangingwall stability

Artificial intelligence (AI) approaches have proliferated in stability prediction of construction projects in the past decade. However, the application of AI approaches did not reach the peak of its potential due to the inappropriate handling of missing data and the omission of state-of-the-art techniques. In the present contribution, we proposed a hybrid method for the improved stability prediction of construction projects based on individual machine learning (ML) algorithms, input missing data imputation, semi-supervised learning and the classifier ensemble. Seven ML algorithms were selected to build individual classifiers for the classifier ensemble. 5-fold cross validation was used as the validation method and the performance measures were chosen to be the confusion matrix, the receiver operating characteristic (ROC) curve and the area under ROC curve (AUC). Exhaustive grid search and firefly algorithm were used for hyper-parameters and weights tuning respectively. The capability of the proposed method was verified using an underground construction dataset, the stope hangingwall (HW) dataset. The case study shows that the input missing data imputation and semi-supervised learning improved the predictive performance of ML algorithms on HW stability prediction. The highest and average AUC values on the testing set were increased to 0.954 and 0.923 respectively on the expanded dataset, compared with 0.879 and 0.860 on the original complete dataset. Further improvement was obtained through the classifier ensemble, with the AUC value being increased to 0.976. Harnessing such method extends recent efforts for stability prediction in construction projects, and can significantly accelerate the project design and stability management.

Design of experiments and response surface methodology to tune machine learning hyperparameters, with a random forest case-study

Most machine learning algorithms possess hyperparameters. For example, an artificial neural network requires the determination of the number of hidden layers, nodes, and many other parameters related to the model fitting process. Despite this, there is still no clear consensus on how to tune them. The most popular methodology is an exhaustive grid search, which can be highly inefficient and sometimes infeasible. Another common solution is to change one hyperparameter at a time and measure its effect on the model’s performance. However, this can also be inefficient and does not guarantee optimal results since it ignores interactions between the hyperparameters. In this paper, we propose to use the Design of Experiments (DOE) methodology (factorial designs) for screening and Response Surface Methodology (RSM) to tune a machine learning algorithm’s hyperparameters. An application of our methodology is presented with a detailed discussion of the results of a random forest case-study using a publicly available dataset. Benefits include fewer training runs, better parameter selection, and a disciplined approach based on statistical theory.

An Efficient Direct Position Determination Method for Multiple Strictly Noncircular Sources.

This paper focuses on the localization methods for multiple sources received by widely separated arrays. The conventional two-step methods extract measurement parameters and then estimate the positions from them. In the contrast to the conventional two-step methods, direct position determination (DPD) localizes transmitters directly from original sensor outputs without estimating intermediate parameters, resulting in higher location accuracy and avoiding the data association. Existing subspace data fusion (SDF)-based DPD developed in the frequency domain is computationally attractive in the presence of multiple transmitters, whereas it does not use special properties of signals. This paper proposes an improved SDF-based DPD algorithm for strictly noncircular sources. We first derive the property of strictly noncircular signals in the frequency domain. On this basis, the observed frequency-domain vectors at all arrays are concatenated and extended by exploiting the noncircular property, producing extended noise subspaces. Fusing the extended noise subspaces of all frequency components and then performing a unitary transformation, we obtain a cost function for each source location, which is formulated as the smallest eigenvalue of a real-valued matrix. To avoid the exhaustive grid search and solve this nonlinear function efficiently, we devise a Newton-type iterative method using matrix Eigen-perturbation theory. Simulation results demonstrate that the proposed DPD using Newton-type iteration substantially reduces the running time, and its performance is superior to other localization methods for both near-field and far-field noncircular sources.

Safe Screening Rules for Accelerating Twin Support Vector Machine Classification.

The twin support vector machine (TSVM) is widely used in classification problems, but it is not efficient enough for large-scale data sets. Furthermore, to get the optimal parameter, the exhaustive grid search method is applied to TSVM. It is very time-consuming, especially for multiparameter models. Although many techniques have been presented to solve these problems, all of them always affect the performance of TSVM to some extent. In this paper, we propose a safe screening rule (SSR) for linear-TSVM, and give a modified SSR (MSSR) for nonlinear TSVM, which contains multiple parameters. The SSR and MSSR can delete most training samples and reduce the scale of TSVM before solving it. Sequential versions of SSR and MSSR are further introduced to substantially accelerate the whole parameter tuning process. One important advantage of SSR and MSSR is that they are safe, i.e., we can obtain the same solution as the original problem by utilizing them. Experiments on eight real-world data sets and an imbalanced data set with different imbalanced ratios demonstrate the efficiency and safety of SSR and MSSR.

Efficient hyper parameter selection for support vector regression using orthogonal array

Support vector regression (SVR) is a nonlinear prediction method using kernel functions and has been widely applied to real-world problems. Although the accuracy of an effectively tuned SVR is high, its performance strongly depends on hyper parameters. Therefore, the determination of the parameters is important when applying SVR to real-world problems. Although the optimum parameters are usually determined by an exhaustive grid search, using this method is not realistic when the sample size is considerably large. To decrease the computational time required to determine the optimum parameters, we employ orthogonal array and propose two efficient methods for SVR parameter tuning based on variable selection in Taguchi method. The proposed methods can reduce the computational time to approximately one-twelfth of that taken by a grid search method. We also validated the actual computational time and accuracy of the proposed methods by applying it to five real datasets in UCI repository.

Efficient hyper parameter selection for support vector regression using orthogonal array

Support vector regression (SVR) is a nonlinear prediction method using kernel functions and has been widely applied to real-world problems. Although the accuracy of an effectively tuned SVR is high, its performance strongly depends on hyper parameters. Therefore, the determination of the parameters is important when applying SVR to real-world problems. Although the optimum parameters are usually determined by an exhaustive grid search, using this method is not realistic when the sample size is considerably large. To decrease the computational time required to determine the optimum parameters, we employ orthogonal array and propose two efficient methods for SVR parameter tuning based on variable selection in Taguchi method. The proposed methods can reduce the computational time to approximately one-twelfth of that taken by a grid search method. We also validated the actual computational time and accuracy of the proposed methods by applying it to five real datasets in UCI repository.

Automatic Scale Parameters in Affinity Matrix Construction for Improved Spectral Clustering

Spectral clustering partitions data into similar groups in the eigenspace of the affinity matrix. The accuracy of the spectral clustering algorithm is affected by the affine equivariance realized in the translation of distance to similarity relationship. The similarity value computed as a Gaussian of the distance between data objects is sensitive to the scale factor [Formula: see text]. The value of [Formula: see text], a control parameter of drop in affinity value, is generally a fixed constant or determined by manual tuning. In this research work, [Formula: see text] is determined automatically from the distance values i.e. the similarity relationship that exists in the real data space. The affinity value of a data pair is determined as a location estimate of the spread of distance values of the data points with the other points. The scale factor [Formula: see text] corresponding to a data point [Formula: see text] is computed as the trimean of its distance vector and used in fixing the scale to compute the affinity matrix. Our proposed automatic scale parameter for spectral clustering resulted in a robust similarity matrix which is affine equivariant with the distance distribution and also eliminates the overhead of manual tuning to find the best [Formula: see text] value. The performance of spectral clustering using such affinity matrices was analyzed using UCI data sets and image databases. The obtained scores for NMI, ARI, Purity and F-score were observed to be equivalent to those of existing works and better for most of the data sets. The proposed scale factor was used in various state-of-the-art spectral clustering algorithms and it proves to perform well irrespective of the normalization operations applied in the algorithms. A comparison of clustering error rates obtained for various data sets across the algorithms shows that the proposed automatic scale factor is successful in clustering the data sets equivalent to that obtained using manually tuned best [Formula: see text] value. Thus the automatic scale factor proposed in this research work eliminates the need for exhaustive grid search for the best scale parameter that results in best clustering performance.

Computationally Efficient Blind Estimation of Carrier Frequency Offset for MIMO-OFDM Systems

In this paper, we propose a new computationally efficient blind carrier frequency offset (CFO) estimator for multi-input multi-output orthogonal frequency division multiplexing systems. A cost function is carefully designed and can be exactly expressed as the superposition of very few harmonically related cosine waves even with the effect of the noise. Using this property, the minimization of the designed cost function can be solved in a quite computationally efficient manner without any exhaustive grid search procedure. It is seen that the proposed estimator can achieve comparable estimation performance as existing competitors with substantially reduced computational burden. Moreover, in order to improve the estimation performance, we further develop an enhanced CFO estimator by introducing an additional one-step adjustment. We find that the enhanced estimator can attain almost the same estimation performance as the existing maximum likelihood estimator but with a lower order of computational burden. We provide both numerical results and theoretical performance analysis to corroborate the proposed studies.

MINFIT: A Spreadsheet-Based Tool for Parameter Estimation in an Equilibrium Speciation Software Program.

Determination of equilibrium constants describing chemical reactions in the aqueous phase and at solid-water interface relies on inverse modeling and parameter estimation. Although there are existing tools available, the steep learning curve prevents the wider community of environmental engineers and chemists to adopt those tools. Stemming from classical chemical equilibrium codes, MINEQL+ has been one of the most widely used chemical equilibrium software programs. We developed a spreadsheet-based tool, which we are calling MINFIT, that interacts with MINEQL+ to perform parameter estimations that optimize model fits to experimental data sets. MINFIT enables automatic and convenient screening of a large number of parameter sets toward the optimal solutions by calling MINEQL+ to perform iterative forward calculations following either exhaustive equidistant grid search or randomized search algorithms. The combined use of the two algorithms can securely guide the searches for the global optima. We developed interactive interfaces so that the optimization processes are transparent. Benchmark examples including both aqueous and surface complexation problems illustrate the parameter estimation and associated sensitivity analysis. MINFIT is accessible at http://minfit.strikingly.com .

Peak‐to‐peak search: fast and accurate DOA estimation method for arbitrary non‐uniform linear array

A peak-to-peak (P-to-P) search-based single source direction of arrival estimation is proposed. The P-to-P method aims to search only at each local minima of a least square cost function, thus quickly acquiring the global optimal solution relative to exhaustive grid search. Performance comparisons with existing methods verify that the proposed method achieves high accuracy, possesses low signal-to-noise ratio threshold and computational cost and can be applicable to any non-uniform linear array.

Modelling fluid accumulation in the neck using simple baseline fluid metrics: implications for sleep apnea.

Obstructive sleep apnea (OSA) is a common respiratory disorder among adults. Recently we have shown that sedentary lifestyle causes an increase in diurnal leg fluid volume (LFV), which can shift into the neck at night when lying down to sleep and increase OSA severity. The purpose of this work was to investigate various metrics that represent baseline fluid retention in the legs and examine their correlation with neck fluid volume (NFV) and to develop a robust model for predicting fluid accumulation in the neck. In 13 healthy awake non-obese men, LFV and NFV were recorded continuously and simultaneously while standing for 5 minutes and then lying supine for 90 minutes. Simple regression was used to examine correlations between baseline LFV, baseline neck circumference (NC) and change in LFV with the outcome variables: change in NC (ΔNC) and in NFV (ΔNFV90) after lying supine for 90 minutes. An exhaustive grid search was implemented to find combinations of input variables which best modeled outcomes. We found strong positive correlations between baseline LFV (supine and standing) and ΔNFV90. Models developed for predicting ΔNFV90 included baseline standing LFV, baseline NC combined with change in LFV after lying supine for 90 minutes. These correlations and the developed models suggest that a greater baseline LFV might contribute to increased fluid accumulation in the neck. These results give more evidence that sedentary lifestyle might play a role in the pathogenesis of OSA by increasing the baseline LFV. The best models for predicting ΔNC include baseline LFV and NC; they improved accuracies of estimating ΔNC over individual predictors, suggesting that a combination of baseline fluid metrics is a good predictor of the change in NC while lying supine. Future work is aimed at adding additional baseline demographic features to improve model accuracy and eventually use it as a screening tool to predict severity of OSA prior to sleep.