Grid Search Approach Research Articles

Detection of Hidden Low-Frequency Earthquakes in Southern Vancouver Island with Deep Learning

Low-frequency earthquakes (LFEs) are small-magnitude earthquakes that are depleted in high-frequency content relative to traditional earthquakes of the same magnitude. These events occur in conjunction with slow slip events (SSEs) and can be used to infer the space and time evolution of SSEs. However, because LFEs have weak signals, and the methods used to identify them are computationally expensive, LFEs are not routinely cataloged in most places. Here, we develop a deep-learning model that learns from an existing LFE catalog to detect LFEs in 14 years of continuous waveform data in southern Vancouver Island. The result shows significant increases in detection rates at individual stations. We associate the detections and locate them using a grid search approach in a 3D regional velocity model, resulting in over 1 million LFEs during the performing period. Our resulting catalog is consistent with a widely used tremor catalog during periods of large-magnitude SSEs. However, there are time periods where it registers far more LFEs than the tremor catalog. We highlight a 16-day period in May 2010, when our model detects nearly 3,000 LFEs, whereas the tremor catalog contains only one tremor detection in the same region. This suggests the possibility of hidden small-magnitude SSEs that are undetected by current approaches. Our approach improves the temporal and spatial resolution of the LFE activities and provides new opportunities to understand deep subduction zone processes in this region.

Ensemble learning using Gompertz function for leukemia classification

Leukemia, a life-threatening disease primarily affecting children below 15 and individuals aged 55 and above, is characterized by the overproduction of abnormal blood cells. Timely detection of leukemia plays a crucial role in significantly reducing the associated mortality rate. Given the recent developments in deep learning, the development of automated leukemia detection techniques using blood smear images has emerged as an important research area. This manuscript introduces a novel algorithm to classify microscopic blood smear images into five categories: AML, CML, ALL, CLL and normal. The study investigates the use of pre-trained convolutional neural networks powered by a fuzzy ensemble-based classifier. The fuzzy-based ensemble learning technique employs adaptable weights according to the confidence scores derived from the base learner models instead of using fixed weights for the base learner models. Gompertz function with parameters, α, β and γ are considered here. The parameters of the Gompertz function are further optimized using a grid search approach to achieve the most favorable classification outcomes. Conducting a 5-class classification, the proposed algorithm achieved an accuracy of 88.80%, surpassing the performance of traditional ensemble techniques such as average, weighted average and majority voting by a significant margin of 1.83%, 1.83% and 5.71%, respectively.

A method of developing ensemble classifiers for recognizing audio data of various nature

The work developed a method of building ensemble classifiers for recognizing audio data of various nature. The method is a tentative date and requires the following steps. In the first step, the input datasets are selected, which are transformed and divided into training and test samples, respectively. RAVDESS datasets are chosen for the task of audio emotion recognition, music genre recognition is performed on the GTZAN dataset. In the second step, the following seven classifiers were created and investigated as elements of ensemble classifiers: K Nearest Neighbors, Support Vector Machine, Random Forest, XGBoost, Multilayer Perceptron, Convolutional Neural Network and Long Short-term Memory. In the process of training elementary classifiers, the corresponding hyperparameters were adjusted using the Grid Search approach. In the third step, elementary classifiers were combined using the stacking ensemble method with such types of aggregation as soft voting, hard voting, soft voting using the GOMPERTZ function. All possible ensemble combinations starting with three elementary classifiers for recognizing audio emotions and music genres were tested. Therefore, the total number of ensembles studied in the work was 297. The research results for the problem of audio emotion classification showed that the accuracy of recognition according to the Accuracy metric of the best ensemble classifier is 8.1% higher than that of the best elementary classifier in its composition, which is based on on MLP, and according to the F1 metric, this indicator is 8% higher. For the task of recognizing music genres, the corresponding indicators are higher by 5.6 % and 5.2 %, respectively.

Efficient prediction for Blast Furnace Gas holder level using novel preprocessing techniques and weight correction strategy

Accurately forecasting the level of Blast Furnace Gas (BFG) holder, managing the balance of gas supply and demand, strategically allocating BFG brings substantial economic and social benefits. In order to address the issues of volatile fluctuation, strong temporal variability, and high uncertainty of BFG, this paper proposes an efficient system for predicting the BFG holder level. Variational Mode Decomposition is utilized to separate components with different frequencies. The number of modes is determined according to Energy Ratio. Improved Discrete Wavelet Threshold is employed to eliminate noise. The grid search approach adjusts the activation functions. What is more, Particle Swarm Optimization is used to correct the weights of subsequences and emphasize critical information. Experimental results demonstrate that Root Mean Squared Error of the system are 0.0411, 0.0423 and 0.0416 for the three real-time monitoring datasets in April 2021. The average improvement percentage for the second dataset is 67.5% when compared to other commonly models. It is obvious that the prediction errors are minimized, and the stability and generalizability of the hybrid prediction system are enhanced. Additionally, the performance of the system is further verified through Diebold–Mariano test, improvement percentage test, and effectiveness test.

Feasibility of Identifying Shale Sweet Spots by Downhole Microseismic Imaging

Several studies suggest that shale sweet spots are likely associated with a low Poisson’s ratio in the shale layer. Compared with conventional geophysical techniques with active seismic data, it is straightforward and cost-effective to delineate the distribution of 3D Poisson’s ratios using microseismic data. In this study, an alternating method is proposed to determine microseismic event locations, 3D P-wave velocity, and Poisson’s ratio models with data recorded from downhole monitoring arrays. The method combines the improved 3D traveltime tomography, which inverts P and S arrivals for 3D P-wave velocity and Poisson’s ratio structures simultaneously, and a 3D grid search approach for event locations in an iterative fashion. The traveltime tomography directly inverts the Poisson’s ratio structure instead of calculating the Poisson’s ratios from P- and S-wave velocities (i.e., Vp and Vs) that are inverted by conventional traveltime tomography separately. The synthetic results and analysis suggest that the proposed method recovers the true Poisson’s ratio model reasonably. Additionally, we apply the method to a field dataset, which indicates that it may help delineate the reservoir structure and identify potential shale sweet spots.

A data-driven analysis to determine the optimal number of topics 'K' for latent Dirichlet allocation model

Topic modeling is an unsupervised machine learning technique successfully used to classify and retrieve textual data. However, the performance of topic models is sensitive to selecting optimal hyperparameters, the number of topics 'K' and Dirichlet priors 'α' and 'β.' This data-driven analysis aims to determine the optimum number of topics, 'K,' within the latent Dirichlet allocation (LDA) model. This work utilizes three datasets, namely 20-Newsgroups news articles, Wikipedia articles, and Web of Science containing science articles, to assess and compare various 'K' values through the grid search approach. The grid search approach finds the best combination of hyperparameter values by trying all possible combinations to see which performs best. This research seeks to identify the 'K' that optimizes topic relevance, coherence, and model performance by leveraging statistical metrics, such as coherence scores, perplexity, and topic distribution quality. Through empirical analysis and rigorous evaluation, this work provides valuable insights for determining the ideal 'K' for LDA models.

Cloud-based machine learning algorithms for anomalies detection

Gradient boosting machines harnesses the inherent capabilities of decision trees and meticulously corrects their errors in a sequential fashion, culminating in remarkably precise predictions. Word2Vec, a prominent word embedding technique, occupies a pivotal role in natural language processing (NLP) tasks. Its proficiency lies in capturing intricate semantic relationships among words, thereby facilitating applications such as sentiment analysis, document classification, and machine translation to discern subtle nuances present in textual data. Bayesian networks introduce probabilistic modeling capabilities, predominantly in contexts marked by uncertainty. Their versatile applications encompass risk assessment, fault diagnosis, and recommendation systems. Gated recurrent units (GRU), a variant of recurrent neural networks, emerges as a formidable asset in modeling sequential data. Both training and testing are crucial to the success of an intrusion detection system (IDS). During the training phase, several models are created, each of which can recognize typical from anomalous patterns within a given dataset. To acquire passwords and credit card details, "phishing" usually entails impersonating a trusted company. Predictions of student performance on academic tasks are improved by hyper parameter optimization of the gradient boosting regression tree using the grid search approach.

Enhancing tribological performance of hybrid fiber-reinforced composites through machine learning and response surface methodology

This study delves into the significant effects of sodium hydroxide (NaOH) treatment on the tribological properties of hybrid fiber-reinforced composites, specifically focusing on the combination of paddy straw (PS) and pineapple leaf (PALF) in a polyester matrix. By leveraging Artificial Neural Networks (ANNs) to predict the Specific Wear Rate (SWR) and Coefficient of Friction (COF), the research employs a grid search approach for hyperparameter optimization. This optimization process results in an optimal ANN architecture with impressive accuracy, showcasing low mean absolute error and high R-squared values of 0.991 and 0.986 for SWR and COF predictions, respectively. Utilizing the Design of Experiments (DOE), the study systematically analyzes the intricate interplay of disc speed, wear duration, and NaOH treatment percentage, with a specific focus on SWR and COF as pivotal tribological metrics. The Analysis of Variance (ANOVA) results underscore the substantial impact of duration and treatment percentage on wear characteristics. Additionally, quadratic regression models reveal nuanced correlations, highlighting the sensitivity of SWR to NaOH percentage and the influence of disc speed, duration, and treatment percentage on COF. This outcome emphasizes the efficacy of these parameters in achieving superior tribological performance in hybrid composites. Beyond contributing to a profound understanding of wear characteristics, this work introduces an innovative dimension through optimized ANN modeling, ensuring a more accurate and fine-tuned predictive model.

Efficient disease identification using symptom-based ensemble models and bayes-search optimization

Symptom-based disease identification is crucial to the diagnosis of the disease at the early stage. Usage of traditional stacking and blending models i.e., with default values of the models cannot handle the multi-classification data properly. Some of the existing researchers also implemented tuning with the help of a grid search approach but it consumes more time because it checks all the possible combinations. Suppose if the model has n estimators with k values it needs to check (n*k)! elements combination, this makes the learning time high. The proposed model chooses the estimators to train the model with in a considerable amount of time using an advanced tuning technique known as “Bayes-Search” on an ensemble random forest and traditional, support vector machine. The advantage of this model is its capability to store the best evaluation metrics from the previous model and utilise them to store the new model. This model chooses the values of the estimator based on the probability of selection, which reduces the elements in search space i.e., (< (n-k)!). The proposed model defines the objective function with a minimum error rate and predicts the error rate with the selected estimators for different distributions. The model depending on the predicted value decides whether to store the value or to return the value to the optimizer. The Bayes search optimization has achieved +9.21% accuracy than the grid search approach. Among the two approaches random forest has achieved good accuracy and less loss using Bayes search with cross-validation.

A note on the covariate-dependent kink threshold regression model for panel data

This article provides new estimating and testing procedures for the panel covariate-dependent kink threshold regression model. By utilizing a linearization technique, we develop an efficient estimator that offers a computationally viable alternative to the grid-search approach. Then we propose a novel testing procedure based on the score statistic to examine the existence of the covariate-dependent kink effect. The test statistic only requires fitting the model in the absence of varying kink threshold, greatly reducing computing time. Extensive simulation studies demonstrate that the proposed method outperforms existing methodologies. To further illustrate their performance, we utilize a real-data analysis. The results demonstrate the practicality and effectiveness of our approach in a wide range of scenarios.

Optimizing launch window opportunities for ESA's comet Interceptor mission using primer vector theory

Comet Interceptor (Comet-I) was selected in June 2019 as the first ESA F-Class mission and aims to explore a pristine comet, which will visit the inner Solar System for the first time. Comet-I will hitch a ride to a Sun-Earth L2 quasi-halo orbit, as a co-passenger in ESA's M4 ARIEL's launch. At the time of writing, this is scheduled for 2029. It will then remain there awaiting the right departure conditions to leave definitively and intercept a newly discovered comet. Comet-I will be the first mission to be designed and launched without an already identified target. Given the latter, the Comet-I Target Identification Working Group is tasked with the continuous follow-up of newly discovered comets, to analyse their characteristics as virtual targets for Comet-I. The goal of the paper is to inform the decision-making process that will eventually be necessary to commit Comet-I spacecraft to its final target.Within this context, this paper presents a trajectory optimization process based on Primer Vector theory that considers intercept trajectories with multiple revolutions and multiple deep space manoeuvres (DSM). The process is applied to 21 long period comets, which were discovered since July 1, 2020 and whose characteristics are of interest for the Comet-I mission. In fact, one of the latest of these recently discovered comets, C/2023H2, is the most accessible virtual target discovered at the time of writing, with delta-V intercept costs well within Comet-I capabilities. A complete launch window analysis is thus presented for C/2023H2, which accounts not only for the most optimal delta-V intercept transfer, but also considers all launch options that satisfy all Comet-I's current boundary conditions and flyby constraints. A systematic launch window analysis is made possible by a grid search approach on which departure and fly-by dates are fixed and the remaining design variables, including the times for manoeuvres and their Δv impulse vector, are optimized via Primer Vector theory. This allows to understand the latest possible departure date, but also the sensitivity to departure and arrival dates of key encounter characteristics such as relative fly-by speed, the solar aspect angle and the Earth distance.

A personalized semi-automatic sleep spindle detection (PSASD) framework

BackgroundSleep spindles are distinct electroencephalogram (EEG) patterns of brain activity that have been posited to play a critical role in development, learning, and neurological disorders. Manual scoring for sleep spindles is labor-intensive and tedious but could supplement automated algorithms to resolve challenges posed with either approaches alone. New methodsA Personalized Semi-Automatic Sleep Spindle Detection (PSASD) framework was developed to combine the strength of automated detection algorithms and visual expertise of human scorers. The underlying model in the PSASD framework assumes a generative model for EEG sleep spindles as oscillatory components, optimized to EEG amplitude, with remaining signals distributed into transient and low-frequency components. ResultsA single graphical user interface (GUI) allows both manual scoring of sleep spindles (model training data) and verification of automatically detected spindles. A grid search approach allows optimization of parameters to balance tradeoffs between precision and recall measures. Comparison with existing methodsPSASD outperformed DETOKS in F1-score by 19% and 4% on the DREAMS and P-DROWS-E datasets, respectively. It also outperformed YASA in F1-score by 25% in the P-DROWS-E dataset. Further benchmarking analysis showed that PSASD outperformed four additional widely used sleep spindle detectors in F1-score in the P-DROWS-E dataset. Titration analysis revealed that four 30-second epochs are sufficient to fine-tune the model parameters of PSASD. Associations of frequency, duration, and amplitude of detected sleep spindles matched those previously reported with automated approaches. ConclusionsOverall, PSASD improves detection of sleep spindles in EEG data acquired from both younger healthy and older adult patient populations.

Global Model Selection via Solution Paths for Robust Support Vector Machine.

Robust support vector machine (RSVM) using ramp loss provides a better generalization performance than traditional support vector machine (SVM) using hinge loss. However, the good performance of RSVM heavily depends on the proper values of regularization parameter and ramp parameter. Traditional model selection technique with gird search has extremely high computational cost especially for fine-grained search. To address this challenging problem, in this paper, we first propose solution paths of RSVM (SPRSVM) based on the concave-convex procedure (CCCP) which can track the solutions of the non-convex RSVM with respect to regularization parameter and ramp parameter respectively. Specifically, we use incremental and decremental learning algorithms to deal with the Karush-Khun-Tucker violating samples in the process of tracking the solutions. Based on the solution paths of RSVM and the piecewise linearity of model function, we can compute the error paths of RSVM and find the values of regularization parameter and ramp parameter, respectively, which corresponds to the minimum cross validation error. We prove the finite convergence of SPRSVM and analyze the computational complexity of SPRSVM. Experimental results on a variety of benchmark datasets not only verify that our SPRSVM can globally search the regularization and ramp parameters respectively, but also show a huge reduction of computational time compared with the grid search approach.

Comparison of Hyperparameter Optimization Techniques in Hybrid CNN-LSTM Model for Heart Disease Classification

Heart disease, which causes the highest number of deaths worldwide, recorded about 17.9 million cases in 2019, or about 32% of total global deaths, according to the World Health Organization (WHO). The significance of early detection of heart disease drives research to develop effective diagnosis systems utilizing machine learning. The advancement of machine learning in healthcare currently primarily serves as a supporting role in the ability of clinicians or analysts to fulfill their roles, identify healthcare trends, and develop disease prediction models. Meanwhile, deep learning has experienced rapid development and has become the most popular method in recent years, one of which is detecting diseases. The main objective of this research is to optimize the hybrid convolutional neural network (CNN) and long short-term memory (LSTM) model for classifying heart disease by comparing hyperparameter optimization using grid search and random search. Although random search requires less time in hyperparameter tuning, the classification performance results of grid search show higher accuracy. In the test, the hybrid CNN and LSTM model with grid search achieved 91.67% accuracy, 89.66% recall (sensitivity), 93.55% specificity, 92.86% precision, 91.23% f1-score, and 0.9310 AUC value. These results confirm that using a hybrid CNN and LSTM model with a grid search approach is better suited for classifying heart disease.

Improving the Forecasting Accuracy Based on the Lunar Calendar in Modeling Rainfall Levels Using the Bi-LSTM Method through the Grid Search Approach.

Rainfall is one of the climatic factors that influence various human activities and affect decision making in daily life activities. High intensity of rainfall can turn into a threat and cause serious problems such as causing various natural disasters. Therefore, it is essential to conduct rainfall forecasting to anticipate and enable preventive actions and can be used as a decision consideration in increasing the productivity and mobility of human activities. The aim of this study is to compare rainfall accuracy between the Gregorian and the lunar calendars using the bidirectional long short-term memory (Bi-LSTM) machine learning model through the grid search approach. This method was used because it can capture patterns arising from the simultaneous effects of two asynchronous calendars, Gregorian and lunar, which were used in this study by finding the right parameters. Monthly rainfall data from Bogor City, Indonesia, were used from the period of 2001 to 2022. The results show that the MAPE of the lunar calendar is relatively smaller at 14.82% which indicates the better forecasting ability than the Gregorian calendar which is 35.12%.

Axial strength prediction of seawater sea sand concrete-filled circular FRP tubes under alkaline environment based on ensemble learning algorithms

The rapid development of marine and urban infrastructure led to the extensive studies on seawater sea sand concrete (SWSSC) filled fiber reinforced polymer (FRP) / steel tubes. The material property of FRP is a function of fiber orientations, and the confinement of SWSSC by FRP tubes enhances its axial load carrying capacity. And therefore, its strength prediction is very challenging because of various FRP layouts, surrounding harsh environment and complicated failure modes. The existing empirical models do not consider effects of surrounding seawater environment under elevated temperature. Therefore, this study concentrates on evaluating the axial capacity of two types of physical models: (1) SWSSC filled circular FRP tubes immersed in seawater environment (Pu1), and (2) SWSSC filled FRP-steel-FRP circular tubes (Pu2), with the help of machine learning (ML) algorithms namely extreme gradient boosting (XGBoost), support vector machine (SVM) and artificial neural networks (ANN). The experimental results of 138 and 120 tested specimens were used for the training and testing of ML models. The models were trained with the best hyperparameters based on grid search and cross validation approach. The models were evaluated with number of statistical indices i.e., R2, MAE and RMSE accompanied with visual comparison of trend line between experimental and predicted values and predicted to experimental ratios. The ML models were graded on the basis of performance as XGBoost > ANN > SVM for the Model 1, whereas for the Model 2, the order was observed as ANN > XGBoost > SVM. Further, SHAP analysis was conducted based on XGBoost to see the influence of input attributes on ML models.

A physics-guided eXtreme gradient boosting model for predicting the initial productivity of oil wells

The initial productivity of oil wells is an important index in oilfield development. Due to the nonlinear relationship among the numerous affecting factors of productivity, productivity prediction methods based on seepage theory are difficult to characterize this nonlinear relationship. The machine learning method provides a novel theory to solve this problem. However, the data-driven machine learning method neglects the physical-based seepage theory. For balancing the advantage of the seepage model and the data-driven model, the productivity formula was used to guide the machine learning algorithm for achieving nonlinear regression in this study. Therefore, an innovative physics-guided eXtreme Gradient Boosting (XGBoost) trees algorithm was proposed to predict the initial productivity. The productivity formula was employed as the physical constraint and combined with the mean-square error (MSE) loss function of the XGBoost algorithm. A hyperparameter was introduced to balance the productivity formula and the MSE formula for constructing this adaptive physics-guided loss function. Moreover, the random subsampling cross-validation approach was adopted to train the XGBoost model. The grid search approach was introduced to optimize the hyperparameters of the model. The results show that based on the numerical simulator data, the initial productivity prediction accuracy of physics-guided XGBoost was verified as 94.67%, which was higher than the completely data-driven model. Moreover, the robustness of the novel adaptive physics-guided loss function for improving forecasting accuracy had been validated with sample sizes between 50 and 20000. By introducing the physics-guided sampling process and loss function, the applicability of the XGBoost algorithm has been extended to small sample conditions. Furthermore, the practical accuracy of physics-guided XGBoost was verified as 84.74% by 162 oil wells in the Penglai oilfield. Therefore, the application value of combining the physical knowledge with the XGBoost model has been confirmed in oilfield situations.

Performance of Various Naïve Bayes Using GridSearch Approach In Phishing Email Dataset

The background is the increasing cybersecurity threats in the form of phishing attacks that can be detrimental to individuals and organizations. The purpose of this research is to compare the performance of four Naive Bayes variants in classifying phishing emails with a method that involves a data pre-processing stage, phishing emails are collected, cleaned, and converted into appropriate numerical features. Next, the GridSearch approach was used to find the best parameters. This research objective is to understand how each Naive Bayes variant works on phishing email datasets. This phishing detection task is based on the following performance evaluation criteria such as accuracy, precision, recall, and F1-score. In this study, Bernoulli got the best accuracy of 97.34% but when the results obtained a hyperparameter, the results showed an increase with the most optimal results and the best performance is Bernoulli 97.38%. The research results are to provide an in-depth insight into the effectiveness of each variant of Naive Bayes in dealing with phishing email datasets and researchers in selecting the most suitable Naive Bayes variant for phishing detection tasks. In addition, the applied GridSearch method can guide how to find the best parameters for Naive Bayes models in other contexts. In summary, this study focuses on analyzing the performance of four variants of Naive Bayes Gaussian, Multinomial, Complement, and Bernoulli with the best algorithms Bernoulli 97.38%.

Hybrid quantum ResNet for car classification and its hyperparameter optimization

Image recognition is one of the primary applications of machine learning algorithms. Nevertheless, machine learning models used in modern image recognition systems consist of millions of parameters that usually require significant computational time to be adjusted. Moreover, adjustment of model hyperparameters leads to additional overhead. Because of this, new developments in machine learning models and hyperparameter optimization techniques are required. This paper presents a quantum-inspired hyperparameter optimization technique and a hybrid quantum-classical machine learning model for supervised learning. We benchmark our hyperparameter optimization method over standard black-box objective functions and observe performance improvements in the form of reduced expected run times and fitness in response to the growth in the size of the search space. We test our approaches in a car image classification task and demonstrate a full-scale implementation of the hybrid quantum ResNet model with the tensor train hyperparameter optimization. Our tests show a qualitative and quantitative advantage over the corresponding standard classical tabular grid search approach used with a deep neural network ResNet34. A classification accuracy of 0.97 was obtained by the hybrid model after 18 iterations, whereas the classical model achieved an accuracy of 0.92 after 75 iterations.

Theoretical Aspects in Penalty Hyperparameters Optimization

Learning processes play an important role in enhancing understanding and analyzing real phenomena. Most of these methodologies revolve around solving penalized optimization problems. A significant challenge arises in the choice of the penalty hyperparameter, which is typically user-specified or determined through Grid search approaches. There is a lack of automated tuning procedures for the estimation of these hyperparameters, particularly in unsupervised learning scenarios. In this paper, we focus on the unsupervised context and propose a bi-level strategy to address the issue of tuning the penalty hyperparameter. We establish suitable conditions for the existence of a minimizer in an infinite-dimensional Hilbert space, along with presenting some theoretical considerations. These results can be applied in situations where obtaining an exact minimizer is unfeasible. Working on the estimation of the hyperparameter with the gradient-based method, we also introduce a modified version of Ekeland’s principle as a stopping criterion for these methods. Our approach distinguishes from conventional techniques by reducing reliance on random or black-box strategies, resulting in stronger mathematical generalization.