Grid Search Cross-validation Research Articles

The Applicability of the Risk Score Approach to Competitive Sport: Development of a Physical Success Score for the Canadian Football League Combine.

Matsuo, H, Funasaki, K, and Yamada, S. The applicability of the risk score approach to competitive sport: development of a physical success score for the Canadian football league combine. J Strength Cond Res 36(3): 695-701, 2022-In the scouting combine of the Canadian Football League (CFL), players are measured for 6 athletic abilities, including 2 measurements of body size. For coaches and their players who desire to play in the CFL, knowing the players metrics that are important for being drafted and their levels will help coaches plan and evaluate their training. Thus, the purpose of this study was to provide a simple scoring system using the predictors of CFL Combines identified by multivariate analysis and their regression coefficients, and reference values. In this study, authors created a scoring system to classify draft success and failure of CFL Combines based on the measurement results. This scoring system was named as the physical success score (PSS). To do so, a repeated grid-search cross-validation for variable selection algorithm was performed on the players (line: n = 211, big skill linebacker, running back and tight end: n = 177, skill [defensive back and wide receiver]: n = 231) who participated in the CFL Combines between 2011 and 2019. The final binary logistic regression models and the reference values were used to generate the PSS for each group. As a result, PSS of the line, big skill, and skill were developed. In each group, all possible total scores and the estimated draft success probability were shown. From the results of this study, it was concluded that the PSS approach can provide useful information for coaches in setting training goals.

A model-based validation study of postoperative complications with considerations on operative timing.

BackgroundSurgery is a highly technical procedure relying on high mental acuity and manual dexterity. The possibility that surgical outcomes and post-operative complications could be subject to influence by fatigue and/or circadian rhythms in surgeons has been investigated with inconsistent results.MethodsWe conducted a retrospective study to assess the significance of operative timing on classifying surgical complications using an interpretable machine learning approach. We trained various linear, generative as well as tree models on the surgical record data collected from a university-affiliated, tertiary teaching hospital in China by performing parameter tuning using grid search cross-validation for optimizing the F1 score.ResultsThe results indicated that XGBoost was the best-performing model overall and its feature importance was shown to provide insight into possible timing-related associations with postoperative complications. We observed that the duration of surgery acted as the strongest indicator, and while surgery initiated at night (between 9 pm and 7 am) also ranked higher on the feature importance scale, it bore less significance than other factors such as the patient’s age, gender, and type of surgery performed.ConclusionsWe showed that surgical records could be used to demonstrate that operative timing might affect the occurrence of postoperative complications, but only in a relatively mild way while potentially entangling with multiple factors.

TOC determination of Zhangjiatan shale of Yanchang formation, Ordos Basin, China, using support vector regression and well logs

Total organic carbon content (TOC) is an important parameter for shale gas resource evaluation. Among the existing TOC prediction methods, ΔlgR method is the most widely used. There are two main problems in this method: 1) assuming a linear relationship between ΔlgR and TOC; 2) manually determining the baseline and introducing human error. Due to the frequent alternating changes in the sedimentary environments of continental shale, the heterogeneity of it is strong. In this case, the relationship between well logs and TOC is complex and nonlinear, which leads to incorrect results for predicted TOC using ΔlgR method. The objective of this paper is to develop a new empirical method to determine the TOC of continental shale using support vector regression (SVR) based on grid search cross-validation. The optimized SVR method can fit the nonlinear relationships between log data and TOC well, and it is suitable for TOC estimation in strong heterogeneity case such as continental shale. Three hundred sixty-four measured TOC samples of Zhangjiatan shale and the corresponding GR, DEN, AC, U logs data were used to construct the SVR model. Compared with the ΔlgR method, the accuracy of the SVR model to calculate TOC is obviously higher.

A novel ensemble learning-based grey model for electricity supply forecasting in China

<abstract><p>Electricity consumption is one of the most important indicators reflecting the industrialization of a country. Supply of electricity power plays an import role in guaranteeing the running of a country. However, with complex circumstances, it is often difficult to make accurate forecasting with limited reliable data sets. In order to take most advantages of the existing grey system model, the ensemble learning is adopted to provide a new stratagy of building forecasting models for electricity supply of China. The nonhomogeneous grey model with different types of accumulation is firstly fitted with multiple setting of acculumation degrees. Then the majority voting is used to select and combine the most accurate and stable models validated by the grid search cross validation. Two numerical validation cases are taken to validate the proposed method in comparison with other well-known models. Results of the real-world case study of forecasting the electricity supply of China indicate that the proposed model outperforms the other 15 exisiting grey models, which illustrates the proposed model can make much more accurate and stable forecasting in such real-world applications.</p></abstract>

An Efficient Cancer Classification Model Using Microarray and High-Dimensional Data.

Cancer can be considered as one of the leading causes of death widely. One of the most effective tools to be able to handle cancer diagnosis, prognosis, and treatment is by using expression profiling technique which is based on microarray gene. For each data point (sample), gene data expression usually receives tens of thousands of genes. As a result, this data is large-scale, high-dimensional, and highly redundant. The classification of gene expression profiles is considered to be a (NP)-Hard problem. Feature (gene) selection is one of the most effective methods to handle this problem. A hybrid cancer classification approach is presented in this paper, and several machine learning techniques were used in the hybrid model: Pearson's correlation coefficient as a correlation-based feature selector and reducer, a Decision Tree classifier that is easy to interpret and does not require a parameter, and Grid Search CV (cross-validation) to optimize the maximum depth hyperparameter. Seven standard microarray cancer datasets are used to evaluate our model. To identify which features are the most informative and relative using the proposed model, various performance measurements are employed, including classification accuracy, specificity, sensitivity, F1-score, and AUC. The suggested strategy greatly decreases the number of genes required for classification, selects the most informative features, and increases classification accuracy, according to the results.

Bayesian Estimation and Optimization for Learning Sequential Regularized Portfolios

This paper incorporates Bayesian estimation and optimization into portfolio selection framework, particularly for high-dimensional portfolio in which the number of assets is larger than the number of observations. We leverage a constrained 𝓁1 minimization approach, called linear programming optimal (LPO) portfolio, to directly estimate effective parameters appearing in the optimal portfolio. We propose two refinements for the LPO strategy. First, we explore improved Bayesian estimates, instead of sample estimates, of the covariance matrix of asset returns. Second, we introduce Bayesian optimization (BO) to replace traditional grid-search cross-validation (CV) in tuning hyperparameters of the LPO strategy. We further propose modifications in the BO algorithm by (1) taking into account time-dependent nature of financial problems and (2) extending commonly used expected improvement (EI) acquisition function to include a tunable trade-off with the improvement's variance (EIVar). Allowing a general case of noisy observations, we theoretically derive the sub-linear convergence rate of BO under the newly proposed EIVar and thus our algorithm has no regret. Our empirical studies confirm that the adjusted BO result in portfolios with higher out-of-sample Sharpe ratio, certainty equivalent, and lower turnover compared to those tuned with CV. This superior performance is achieved with significant reduction in time elapsed, thus also addressing time-consuming issues of CV. Furthermore, LPO with Bayesian estimates outperform original proposal of LPO, as well as the benchmark equally weighted and plug-in strategies.

Experimental Investigation and Optimal Prediction of Maximum Forming Angle and Surface Roughness of an Al/SUS Bimetal Sheet in an Incremental Forming Process Using Machine Learning.

Bimetal sheets have superior properties as they combine different materials with different characteristics. Producing bimetal parts using a single-point incremental forming process (SPIF) has increased recently with the development of industrial requirements. Such types of sheets have multiple functions that are not applicable in the case of monolithic sheets. In this study, the correlation between the operating variables, the maximum forming angle, and the surface roughness is established based on the ensemble learning using gradient boosting regression tree (GBRT). In order to obtain the dataset for the machine learning, a series of experiments with continuous variable angle pyramid shape were carried out based on D-Optimal design. This design is created based on numerical variables (i.e., tool diameter, step size, and feed rate) and categorical variable (i.e., layer arrangement). The grid search cross-validation (CV) method was used to determine the optimum GBRT parameters prior to model training. After the parameter tuning and model selection, the model with a better generalization performance is obtained. The reliability of the predictive models is confirmed by the testing samples. Furthermore, the microstructure of the aluminum/stainless steel (Al/SUS) bimetal sheet is analyzed under different levels of operating parameters and layer arrangements. The microstructure results reveal that severe cracks are attained in the case of a small tool diameter while a clear refinement is observed when a high tool diameter value with small step down is used for both Al and SUS layers.

Prediction of Alzheimer’s disease (AD) Using Machine Learning Techniques with Boruta Algorithm as Feature Selection Method

Alzheimer’s disease (AD) is the most frequent incurable neurodegenerative disease, a general term for memory loss and other cognitive abilities. Early detection of AD can help with proper treatment and prevent brain tissue damage. Traditional medical tests are time consuming, fail to recognize early signs and lack of diagnosis sensitivity and specificity. To achieve promising prediction accuracy, the best predictive machine learning model is selected based on initial pre-processing step followed by vital attributes selection and performance evaluation for five proposed supervised machine learning algorithms. In the pre-processing, all the missing values have been removed since the overall percentage only covered 5.63%. Boruta algorithm as feature selection method resulted Atlas Scaling Factor, Estimated Total Intracranial Volume, Normalized Whole-brain Volume, Mini-Mental State Examination and Clinical Dementia Rating must be included as primary features. With Boruta algorithm, it has been shown that Random Forest Grid Search Cross Validation (RF GSCV) outperformed with 94.39% of accuracy, 88.24% sensitivity, 100.00% specificity and 94.44% AUC among other 12 models that includes conventional and fine-tuned models even for the small OASIS-2 longitudinal MRI dataset. Finally, our developed Graphical User Interface (GUI) prediction tool was evaluated through prediction over OASIS-1 cross-sectional MRI dataset containing 216 samples of imaging sessions that have been pre-processed. Prediction results were closed with the dementia status provided in OASIS cross-sectional data fact sheet.

SCOPE: Selective Cross-Validation over Parameters for Elo

It is crucial to develop reusable methods for explaining and evaluating esports given their popularity and diversity. Quantifying skill in an esport has the potential to improve win prediction, matchmaking, and storytelling for these games. Arpad Elo’s skill modeling system for chess has been adapted to many games and sports. In each instance, the modeler is challenged with tuning parameters to optimize for some metric, usually accuracy. Often these approaches are one-off and lack consistency. We propose SCOPE, a framework that uses grid search cross-validation to select optimal parameters for Elo based on accuracy, calibration, or log loss. We demonstrate this method on a season of Call of Duty World League, a first-person shooter esport, and we demonstrate comparable performance to other more complex, state-of-the-art methods.

Cross-validation pitfalls when selecting and assessing regression and classification models

BackgroundWe address the problem of selecting and assessing classification and regression models using cross-validation. Current state-of-the-art methods can yield models with high variance, rendering them unsuitable for a number of practical applications including QSAR. In this paper we describe and evaluate best practices which improve reliability and increase confidence in selected models. A key operational component of the proposed methods is cloud computing which enables routine use of previously infeasible approaches.MethodsWe describe in detail an algorithm for repeated grid-search V-fold cross-validation for parameter tuning in classification and regression, and we define a repeated nested cross-validation algorithm for model assessment. As regards variable selection and parameter tuning we define two algorithms (repeated grid-search cross-validation and double cross-validation), and provide arguments for using the repeated grid-search in the general case.ResultsWe show results of our algorithms on seven QSAR datasets. The variation of the prediction performance, which is the result of choosing different splits of the dataset in V-fold cross-validation, needs to be taken into account when selecting and assessing classification and regression models.ConclusionsWe demonstrate the importance of repeating cross-validation when selecting an optimal model, as well as the importance of repeating nested cross-validation when assessing a prediction error.Electronic supplementary materialThe online version of this article (doi:10.1186/1758-2946-6-10) contains supplementary material, which is available to authorized users.

A Method of Particle Swarm Optimized SVM Hyper-spectral Remote Sensing Image Classification

Support Vector Machine (SVM) has been proved to be suitable for classification of remote sensing image and proposed to overcome the Hughes phenomenon. Hyper-spectral sensors are intrinsically designed to discriminate among a broad range of land cover classes which may lead to high computational time in SVM mutil-class algorithms. Model selection for SVM involving kernel and the margin parameter values selection which is usually time-consuming, impacts training efficiency of SVM model and final classification accuracies of SVM hyper-spectral remote sensing image classifier greatly. Firstly, based on combinatorial optimization theory and cross-validation method, particle swarm algorithm is introduced to the optimal selection of SVM (PSSVM) kernel parameter σ and margin parameter C to improve the modelling efficiency of SVM model. Then an experiment of classifying AVIRIS in India Pine site of USA was performed for evaluating the novel PSSVM, as well as traditional SVM classifier with general Grid-Search cross-validation method (GSSVM). And then, evaluation indexes including SVM model training time, classification Overall Accuracy (OA) and Kappa index of both PSSVM and GSSVM are all analyzed quantitatively. It is demonstrated that OA of PSSVM on test samples and whole image are 85% and 82%, the differences with that of GSSVM are both within 0.08% respectively. And Kappa indexes reach 0.82 and 0.77, the differences with that of GSSVM are both within 0.001. While the modelling time of PSSVM can be only 1/10 of that of GSSVM, and the modelling. Therefore, PSSVM is an fast and accurate algorithm for hyper-spectral image classification and is superior to GSSVM.