Tree-structured Parzen Estimator Research Articles

Nonparametric modeling of ship maneuvering motions in calm water and regular waves based on R-LSTM hybrid method

It is essential to establish reliable method to predict the ship maneuvering motions in complex navigation environments for navigation efficiency and safety. In this paper, a novel hybrid nonparametric modeling method (R-LSTM) is developed by coupling Bidirectional Long Short-Term Memory (Bi-LSTM) with residual network and attention mechanism. Firstly, the prediction capability of the proposed hybrid method is evaluated in the predictions of zigzag maneuvers in calm water. The model test data of a KVLCC2 tanker model are utilized for modeling and validation, and the optimal parameters are determined by Tree-structured Parzen Estimator algorithm. The results demonstrate that the proposed method has higher prediction accuracy and better generalization ability than the classical LSTM method. On this basis, the online modeling method for predicting the turning motion in regular waves is developed along with sliding time window and time series split cross-validation. The model test data of a KCS container ship and a ONRT tumblehome ship under different wave conditions are used for online modeling and real-time prediction. The results indicate that the ship trajectories with trajectory drift and fluctuation features during the turning motion under the wave impacts are captured well by the developed method with satisfactory confidence intervals.

DNN model development of biogas production from an anaerobic wastewater treatment plant using Bayesian hyperparameter optimization

Deep neural networks have been regarded as accurate models to predict complex fermentation processes due to their capacity to learn from a high number of data sets via artificial intelligence. To enhance the performance of these models the main challenge is to select the appropriate hyperparameters, such as neural network architecture (number of hidden layers and hidden units), activation function, optimizer, learning rate, and others.This study aims to use a hybrid Bayesian optimization with tree-structured Parzen estimator (BO-TPE) to predict biogas production from real wastewater treatment plant data using deep neural network machine learning (DNN) with optimized hyperparameters. Due to the high number of missing sample measurement records, the data preprocessing process has been performed in three sequential steps: the first step is to remove columns with a high percentage of missing values. The second step concerns removing rows with a high number of missing values. The remaining missing values in the dataset were removed in the third step using the dropna function from Pandas' library. Then, on the remaining data, three normalization techniques (MinMaxScaler, RobustScaler, and StandardScaler) were used for 16 relevant features of the anaerobic digestion process (AD) and compared with Non-Normalized data. The RobustScaler technique demonstrated good prediction capabilities of biogas volume produced. The maximum predicted volume was 2236.105 Nm3/d, with the coefficient of determination (R2), the mean absolute error (MAE), and the root mean square error (RMSE) values of 0.712, 164.610, and 223.429, respectively. This value remains slightly higher than the actual biogas volume produced in the wastewater treatment plant, which was 2131 Nm3/d. Adopting StandardScaler, MinMaxScaler, and Non-Normalized data provided statistical performance indicator values of (0.690; 170.518; 231.519), (0.679; 180.575; 235.648), and (0.647; 183.403; 247.203), respectively. These findings offer an autonomous strategy to monitor the effective operational variables of a large-scale digester, ensuring the stability of the complex fermentation process as well as the long-term sustainability and economic viability of the renewable energy approach in the sector of waste treatment.

Free nitrous acid prediction in ANAMMOX process using hybrid deep neural network model

Free nitrous acid (FNA) is a critical metric for stabilization of ANAMMOX but can not be directly and immediately measured by sensors or chemical measurement method, which hinders the effective management and operation for ANAMMOX. This study focuses on FNA prediction using hybrid model based on temporal convolutional network (TCN) combined with attention mechanism (AM) optimized by multiobjective tree-structured parzen estimator (MOTPE), called MOTPE-TCNA. A case study in an ANAMMOX reactor is carried out. Results show that nitrogen removal rate (NRR) is highly correlated with FNA concentration, indicating that it can forecast the operational status by predicting FNA. Then, MOTPE successfully optimizes the hyperparameters of TCN, helping TCN achieve a high prediction accuracy, and AM furtherly improves model accuracy. MOTPE-TCNA obtains the highest prediction accuracy, whose R2 value gets 0.992, increasing 1.71–11.80% compared to other models. As a deep neural network model, MOTPE-TCNA has more advantages than traditional machine learning methods in FNA prediction, which is beneficial to maintain the stable operation and easy control for ANAMMOX process.

Dual-kernel echo state network for nonlinear time series prediction

An echo state network (ESN) is a recurrent neural network (RNN) often applied to nonlinear time series prediction. The traditional ESN randomly generates the weights of the input layer and the reservoir layer and does not change these weights, and generally only learns the weights of the output layer through linear regression, so the training speed is very fast. In this work, we propose a novel kernel echo state network (KESN). In KESN, the random weights of the input layer are removed and a series of gaussian kernels are used to replace the neurons in the input layer. Similar to radial basis function (RBF) neural networks, KESN can use the k-means algorithm to generate the kernel center and estimate the bandwidth of the kernel function. We prove that a KESN has echo state property, which is an important factor of KESN that can normally work. Furthermore, kernel ridge regression (KRR) is used to learn the weights of the output layer instead of a simple linear model. Finally, to obtain the optimal parameters of the model, the tree-structured parzen estimator approach (TPE) is used to optimize the hyperparameters of the model. In a time series prediction experiment, it is proved that KESN is more stable and performs better than the echo state network which randomly generates weights and trains output weights using linear models. We found that the reservoir layer weights are equivalent to a dropout operation, and the KESN is inherently equivalent to a regularized neural network. We call the KRR-based KESN dual-kernel echo state network (DKESN).

A Light Gradient-Boosting Machine algorithm with Tree-Structured Parzen Estimator for breast cancer diagnosis

Breast cancer is a common and potentially life-threatening disease. Early and accurate diagnosis of breast cancer is crucial for effective treatment and improved patient outcomes. This study proposed using the Light Gradient-Boosting Machine (LightGBM) algorithm, Borderline- Synthetic Minority Oversampling Technique (SMOTE), and the Tree-Structured Parzen Estimator (TPE) for hyperparameter tuning to enhance the effectiveness of the Machine Learning (ML) model for diagnosing breast cancer. A 10-fold cross-validated TPE optimized Borderline-SMOTE LightGBM classifier was modelled on the Wisconsin Diagnostic Breast Cancer (WDBC) Dataset and evaluated for its performance compared to a baseline LightGBM model. The TPE-optimized Borderline-SMOTE LightGBM model exhibited a significant improvement in performance over the baseline model, achieving an average accuracy of 99.12%, specificity of 100%, precision of 100%, recall of 97.62%, F1-score of 98.80%, and a Mathews Correlation Coefficient of 98.12%. Compared to previous studies, the TPE-optimized Borderline-SMOTE LightGBM model performed exceptionally well. The study demonstrates the effectiveness of using data augmentation and hyperparameter optimization techniques to improve the performance of ML models for breast cancer diagnosis, which has significant implications for the medical field where the accurate and efficient diagnosis of breast cancer is critical.

Deep learning models for multilabel ECG abnormalities classification: A comparative study using TPE optimization

AbstractThe problem addressed in this study is the limitations of previous works that considered electrocardiogram (ECG) classification as a multiclass problem, despite many abnormalities being diagnosed simultaneously in real life, making it a multilabel classification problem. The aim of the study is to test the effectiveness of deep learning (DL)-based methods (Inception, MobileNet, LeNet, AlexNet, VGG16, and ResNet50) using three large 12-lead ECG datasets to overcome this limitation. The define-by-run technique is used to build the most efficient DL model using the tree-structured Parzen estimator (TPE) algorithm. Results show that the proposed methods achieve high accuracy and precision in classifying ECG abnormalities for large datasets, with the best results being 97.89% accuracy and 90.83% precision for the Ningbo dataset, classifying 42 classes for the Inception model; 96.53% accuracy and 85.67% precision for the PTB-XL dataset, classifying 24 classes for the Alex net model; and 95.02% accuracy and 70.71% precision for the Georgia dataset, classifying 23 classes for the Alex net model. The best results achieved for the optimum model that was proposed by the define-by-run technique were 97.33% accuracy and 97.71% precision for the Ningbo dataset, classifying 42 classes; 96.60% accuracy and 83.66% precision for the PTB-XL dataset, classifying 24 classes; and 94.32% accuracy and 66.97% precision for the Georgia dataset, classifying 23 classes. The proposed DL-based methods using the TPE algorithm provide accurate results for multilabel classification of ECG abnormalities, improving the diagnostic accuracy of heart conditions.

Prediction of the NOx and CO2 emissions from an experimental dual fuel engine using optimized random forest combined with feature engineering

This study was conducted to investigate the performance of a novel Random Forest (RF) model for predicting variables from an original experimental dataset of a diesel engine adapted to work with both compressed natural gas and diesel fuels. The aim was to develop a reliable framework for diesel engine emissions prediction that could assist designers, engineers, and decision-makers in optimizing engine performance and reducing emissions. The engine was modified to run on compressed natural gas as well as diesel fuel, and five variables were studied. Trials were done on a six-cylinder diesel engine to assess the RF model, employing various factors for improving engine performance and emissions, such as fuel injection angles, air-fuel ratio mixtures, diesel-to-gas exchange rates, and fuel rail pressure. A tree structured Parzen estimator and six feature engineering approaches were used to tune the RF model's parameters. In addition, the Shapley Additive explanation (SHAP) approach adapting a concept coming from game theory is employed to interpret the RF model outputs. The results analysis showed that the RF model correctly predicted the output signals of the diesel engine, with determination coefficient R2 of 0.9811, 0.9276, 0.9516, 0.8842, and 0.8944, respectively, for the studied five output variables. The RF regression model's predictive power can be used to generate an efficient modeling framework, and successfully predicts the output signals of the diesel engine, confirming the viability, effectiveness, and competitive performance.

Ultra-short-term electricity load forecasting based on improved random forest algorithm

Electricity load forecasting is one of the important tasks of the power marketing department, and accurate load forecasting is extremely important to ensure real-time dispatch and security of the power system. In order to obtain accurate and reliable load forecasting results, an ultra-short-term power load forecasting model based on an improved random forest regression algorithm is proposed in this paper. First, data pre-processing is performed on the original dataset. Then the pre-processed time data and historical load data are used as inputs to the model, and optimization of the model using the Gaussian mixture-based tree-structured Parzen estimator algorithm is carried out. Finally, the final prediction results were derived. Experimental analysis was conducted with real load data from a region of China, and the experimental results show that the method has better prediction accuracy than the original random forest algorithm and other traditional machine learning algorithms.

Spatial Prediction and Mapping of Soil Water Content by TPE-GBDT Model in Chinese Coastal Delta Farmland with Sentinel-2 Remote Sensing Data

Soil water content is an important indicator used to maintain the ecological balance of farmland. The efficient spatial prediction of soil water content is crucial for ensuring crop growth and food production. To this end, 104 farmland soil samples were collected in the Yellow River Delta (YRD) in China, and the soil water content was determined using the drying method. A gradient boosting decision tree (GBDT) model based on a tree-structured Parzen estimator (TPE) hyperparametric optimization was developed, and then the soil water content was predicted and mapped based on the soil texture and vegetation index from Sentinel-2 remote sensing images. The results of statistical analysis showed that the soil water content had a high coefficient of variation (55.30%), a non-normal distribution, and complex spatial variability. Compared with other models, the TPE-GBDT model had the highest prediction accuracy (RMSE = 6.02% and R2 = 0.71), and its mapping results showed that the areas with high soil water content were distributed on both sides of the river and near the estuary. Furthermore, the results of Shapley additive explanation (SHAP) analysis showed that the soil texture (PC2 and PC5), modified normalized difference vegetation index (MNDVI), and Sentinel-2 red edge position (S2REP) index provided important contributions to the spatial prediction of soil water content. We found that the hydraulic physical properties of soil texture and the vegetation characteristics (such as vegetation coverage, root action, and transpiration) are the key factors affecting the spatial migration and heterogeneity of the soil water content in the study area. The above results show that the TPE algorithm can quickly capture the hyperparameters that are most suitable for the GBDT model, so that the GBDT model can ensure prediction accuracy, reduce the loss function with less training data, and accurately learn of the nonlinear relationship between soil water content and environmental factors. This paper proposes a machine learning method for hyperparameter optimization that shows considerable potential to predict the spatial heterogeneity of soil water content, which can effectively support regional farmland soil and water conservation and high-quality agricultural development.

HCEs-Net: Hepatic cystic echinococcosis classification ensemble model based on tree-structured Parzen estimator and snap-shot approach.

Hepatic cystic echinococcosis (HCE) still has a high misdiagnosis rate, and misdiagnosis may lead to wrong treatments seriously harmful for the patients. Precise diagnosis of HCE relies heavily on the experience of clinical experts with auxiliary diagnostic tools using medical images. This paper intends to improve the diagnostic accuracy for HCE by employing a method which combines deep learning with ensemble method. We proposed a method, namely HCEs-Net, for classification of five HCE subtypes using ultrasound images. It takes first the snap-shot strategy to obtain sub-models from the pre-trained VGG19, ResNet18, ViT-Base, and ConvNeXt-T models, then a stacking process to ensemble those sub-models. Afterwards, it uses the tree-structured Pazren estimator (TPE) to optimize the hyperparameters. The experiments were evaluated by the five-fold cross-validation process. A total of 3083 abdominal ultrasound images from 972 patients covering five subtypes of HCE were utilized in this study. The experiments were conducted to predict the HCE subtype, and results of modeling performance evaluation were reported in terms of precision, recall, F1-score, and AUC. The stacking model based on three ConvNeXt-T sub-models showed the best performance, with precision 85.9%, recall 85.5%, F1-score 85.7%, and AUC 0.971 which are higher than the compared state-of-the-art models. The stacking model of three ConvNeXt-T sub-models shows comparable or superior performance to the other methods, including VGG19, ResNet18 and ViT-Base. It has the potential to enhance clinical diagnosis for HCE.

A new hybrid approach model for predicting burst pressure of corroded pipelines of gas and oil

Accurate prediction of the burst pressure of corroded pipeline is of great significance to pipeline design, reliability analysis and maintenance decision. This paper presents a novel hybrid approach model for predicting the burst pressure of corroded pipelines. First, a new feature space with physical meaning is constructed based on the pipeline geometry, the size of corrosion defects, and the mechanical properties of materials. Then, a fusion mechanism combining empirical formula and ensemble learning is proposed, which makes full use of the prior knowledge in the Tresca criterion and the predictive ability of ensemble learning. The prediction model is built using the Light Gradient Boosting Machine, and the hyper-parameters are tuned using the Tree-structured Parzen Estimator algorithm. Finally, a burst pressure dataset of corroded full-scale oil and gas pipelines ranging from low strength to high strength is established to verify this model. The prediction results demonstrate that the hybrid approach model can significantly improve the prediction accuracy. And compared to other ensemble learning methods such as random forest and XGBoost, the accuracy of this model proposed in this paper is the highest. The correlation coefficient is 0.98163, the mean square error is 0.98087 MPa, the mean absolute error is 0.66500 MPa, and mean absolute percentage error is 0.04480. The model also provides feature importance, enhancing the interpretability of the model. In addition, further experiments demonstrate that this model has good adaptability to corroded pipelines of different strengths grade and obvious advantages compared with the calculation results of five traditional empirical formulas.

Agent-based mathematical model of COVID-19 spread in Novosibirsk region: Identifiability, optimization and forecasting

Abstract The problem of identification of unknown epidemiological parameters (contagiosity, the initial number of infected individuals, probability of being tested) of an agent-based model of COVID-19 spread in Novosibirsk region is solved and analyzed. The first stage of modeling involves data analysis based on the machine learning approach that allows one to determine correlated datasets of performed PCR tests and number of daily diagnoses and detect some features (seasonality, stationarity, data correlation) to be used for COVID-19 spread modeling. At the second stage, the unknown model parameters that depend on the date of introducing of containment measures are calibrated with the usage of additional measurements such as the number of daily diagnosed and tested people using PCR, their daily mortality rate and other statistical information about the disease. The calibration is based on minimization of the misfit function for daily diagnosed data. The OPTUNA optimization framework with tree-structured Parzen estimator and covariance matrix adaptation evolution strategy is used to minimize the misfit function. Due to ill-posedness of identification problem, the identifiability analysis is carried out to construct the regularization algorithm. At the third stage, the identified parameters of COVID-19 for Novosibirsk region and different scenarios of COVID-19 spread are analyzed in relation to introduced quarantine measures. This kind of modeling can be used to select effective anti-pandemic programs.

Structure Optimization of Ensemble Learning Methods and Seasonal Decomposition Approaches to Energy Price Forecasting in Latin America: A Case Study about Mexico

The energy price influences the interest in investment, which leads to economic development. An estimate of the future energy price can support the planning of industrial expansions and provide information to avoid times of recession. This paper evaluates adaptive boosting (AdaBoost), bootstrap aggregation (bagging), gradient boosting, histogram-based gradient boosting, and random forest ensemble learning models for forecasting energy prices in Latin America, especially in a case study about Mexico. Seasonal decomposition of the time series is used to reduce unrepresentative variations. The Optuna using tree-structured Parzen estimator, optimizes the structure of the ensembles through a voter by combining several ensemble frameworks; thus an optimized hybrid ensemble learning method is proposed. The results show that the proposed method has a higher performance than the state-of-the-art ensemble learning methods, with a mean squared error of 3.37 × 10−9 in the testing phase.

Prediction of heating and cooling loads based on light gradient boosting machine algorithms

Machine learning models have been widely used to study the prediction of heating and cooling loads in residential buildings. However, most of these methods use the default hyperparameters, resulting in inaccurate prediction accuracy. In this work, based on hyperparametric optimization algorithms of random search (Random), grid search (Grid), covariance matrix adaptive evolution strategy (CMA-ES), and tree-structured parzen estimator (TPE), were combined with the light gradient boosting machine (LightGBM) model, to construct four hybrid models (Random-LightGBM, Grid-LightGBM, CMA-ES-LightGBM and TPE-LightGBM) for improved prediction accuracy of heating and cooling loads. The LightGBM model was trained using a dataset consisting of building features, cooling set points, and occupant behavior parameters. Feature selection was performed by a random forest-based feature selection method, which determines the input features of the load prediction model. The TPE- LightGBM model achieved the best prediction accuracy among all proposed models with a root mean square error (RMSE) of 0.2714, mean absolute error (MAE) of 0.1416, coefficient of determination (R2) of 0.9981, and mean absolute percentage error (MAPE) of 0.4699% for heating load prediction, and RMSE of 0.1901, MAE of 0.1394, R2 of 0.9924, and MAPE of 2.3509% for cooling load prediction. The proposed TPE-LightGBM model provides an efficient strategy for predicting heating and cooling loads, which can provide better energy efficiency measures at the early design stages of residential buildings.

Hyperparameter Optimization in Convolutional Neural Networks for Maize Seed Classification

Corn farming is of great importance for the continuity of our society. Because corn is a cheap and efficient food, especially for animal feeding. However, with the Doubled-haploid technique, the selection of the haploid seeds necessary for this job to be done efficiently creates a problem. Today, the selection of haploid seeds is usually done by trained technicians. With the development of machine learning methods, the parts expected from technicians can be made by machines. In this study, a new model architecture based on a convolutional neural network (CNN) was produced to perform the selection of haploid seeds and the hyperparameters of this model were optimized with the use of tree-structured parzen estimator algorithm. The newly produced model achieved a 94.66% validation score, higher than the VGG-19 model, which proved to be relatively efficient.

A New Method for Inversion of Dam Foundation Hydraulic Conductivity Using an Improved Genetic Algorithm Coupled with an Unsaturated Equivalent Continuum Model and Its Application.

Seepage is a main cause of dam failure, and its stability analysis is the focus of a dam's design, construction, and management. Because a geological survey can only determine the range of a dam foundation's hydraulic conductivity, hydraulic conductivity inversion is crucial in engineering. However, current inversion methods of dam hydraulic conductivity are either not accurate enough or too complex to be directly used in engineering. Therefore, this paper proposes a new method for the inversion of hydraulic conductivity with high application value in hydraulic engineering using an improved genetic algorithm coupled with an unsaturated equivalent continuum model (IGA-UECM). This method is implemented by a new code that fully considers engineering applicability. In addition to overcoming the premature convergence shortcomings of traditional genetic algorithms, it converges faster than Bayesian optimization and tree-structured Parzen estimator inversion algorithms. This method is verified by comparing the water head from drilling exploration and inversion. The results of the inversion are used to study the influence of a cement grouting curtain layout scheme on the seepage field of the Hami concrete-face rockfill dam in China, which is used as an engineering application case of the IGA-UECM. The law of the seepage field is reasonable, which verifies the validity of the IGA-UECM. The new inversion method of hydraulic conductivity and the proposed cement grouting curtain layout in this study offer possible strategies for the design, construction, and management of concrete-face rockfill dams.

Prediction of compressive strength of recycled aggregate concrete using machine learning and Bayesian optimization methods

With the sustainable development of the construction industry, recycled aggregate (RA) has been widely used in concrete preparation to reduce the environmental impact of construction waste. Compressive strength is an essential measure of the performance of recycled aggregate concrete (RAC). In order to understand the correspondence between relevant factors and the compressive strength of recycled concrete and accurately predict the compressive strength of RAC, this paper establishes a model for predicting the compressive strength of RAC using machine learning and hyperparameter optimization techniques. RAC experimental data from published literature as the dataset, extreme gradient boosting (XGBoost), random forest (RF), K-nearest neighbour (KNN), support vector machine regression Support Vector Regression (SVR), and gradient boosted decision tree (GBDT) RAC compressive strength prediction models were developed. The models were validated and compared using correlation coefficients (R2), Root Mean Square Error (RMSE), mean absolute error (MAE), and the gap between the experimental results of the predicted outcomes. In particular, The effects of different hyperparameter optimization techniques (Grid search, Random search, Bayesian optimization-Tree-structured Parzen Estimator, Bayesian optimization- Gaussian Process Regression) on model prediction efficiency and prediction accuracy were investigated. The results show that the optimal combination of hyperparameters can be searched in the shortest time using the Bayesian optimization algorithm based on TPE (Tree-structured Parzen Estimator); the BO-TPE-GBDT RAC compressive strength prediction model has higher prediction accuracy and generalisation ability. This high-performance compressive strength prediction model provides a basis for RAC’s research and practice and a new way to predict the performance of RAC.

Prediction of the Critical Temperature of Superconductors Based on Two-Layer Feature Selection and the Optuna-Stacking Ensemble Learning Model.

The study of superconductors' critical temperature (T c) has been a matter of interest. A method combining a two-layer feature selection (TL) and Optuna-Stacking ensemble learning model is proposed in the study for predicting T c from physicochemical components. Since most machine-learning models require a large amount of prior knowledge to construct the feature vectors associated with T c manually, they may contain redundant or invalid features that adversely affect the analysis and prediction of T c. The TL model combines the advantages of filtered and packed feature selection. In the first layer, feature importance is ranked by "SHapley Additive explain (SHAP)" in combination with CatBoost, followed by maximum mutual information coefficient (MIC) and distance correlation coefficient (DCC) for initial feature selection in terms of feature importance ranking. The second layer uses a cross-validation-based genetic algorithm (cv-GA) to eliminate the remaining redundant/invalid features. The selected features are fed into the Stacking integrated learning model to achieve prediction of Tc, and the multidimensional hyperparametric optimization of the metamodel is achieved by Optuna, an improved Bayesian hyperparametric optimization framework based on the Tree-structured Parzen Estimator (TPE) and pruning strategy. The model has obvious advantages and generality in terms of prediction performance and feature reduction rate, and it also proves to be suitable for high-temperature superconductor T c prediction. It provides an efficient and cost-effective method for data-driven superconductor research.

Effects of Automatic Hyperparameter Tuning on the Performance of Multi‐Variate Deep Learning‐Based Rainfall Nowcasting

AbstractRainfall nowcasting has become increasingly important as we move into an era where more and more storms are occurring in many countries as a result of climate change. Developing an accurate rainfall nowcasting model could provide insights into rainfall dynamics and ultimately could prevent significant damages. In this paper, deep neural networks (DNNs) and numerical weather predictions (NWPs) are applied for rainfall and runoff forecasting in an urban catchment with a complex drainage system. DNNs are among the most accurate models for rainfall nowcasting. However, the design and training of DNNs are usually complicated. This paper combines different convolutional, long short‐term memory (LSTM)‐based networks and NWPs using ensemble techniques (i.e., bagging, random forest, and adaboost methods) with automatic hyperparameter tuning for multi‐step rainfall nowcasting. The relative humidity, air temperature, and previous rainfall sequences are considered the inputs of the DNNs. We focus on applying two hyperparameter tuning methods (i.e., random search and tree structured Parzen estimator) to improve the performance of the proposed rainfall nowcasting models. The proposed framework was applied to the eastern drainage catchment (EDC) in Tehran city. The results illustrate that the utilization of automatic hyperparameter tuning along with multivariate DNNs, NWPs, and ensemble techniques could improve the nowcasting performance (10%–25%) compared to the traditional univariate models. Also, Adaboost is more accurate than other ensemble techniques in predicting both extreme and normal rainfall events with average RMSE of 0.765, and random forest obtain better results when predict sub normal rainfall events with overall RMSE of 0.315. The proposed framework is applicable to different climates and catchments.

A Novel Three-Probability Spaces Logic Decoupling Distillation for Flip-Chip Defect Detection

Although deep learning has achieved remarkable results in the industry, to achieve the ultimate prediction accuracy improvement, many existing models with poor performance will be added with other modules, which will become more and more bloated. To achieve the deployment of the lightweight network in an industrial system, knowledge distillation (KD) is currently an effective way to reduce model size. This study focused on the research of knowledge transfer logic and decoupled the interactive information into 3-probability spaces, namely the target space (t), the intraclass class space (o\t), and the out-of-class class space (c\o). The 3-probability spaces method is named TOCKD (t-o\t-c\o spaces knowledge distillation), which introduces three components corresponding to 3-probability spaces, that is target knowledge distillation (TKD), intraclass knowledge distillation (OKD), and out-of-class knowledge distillation (CKD). Based on the support of the flip-chip vibration signal with defect information obtained by ultrasonic excitation experiment, the research confirms the primacy of TKD in the information transmission process and the auxiliary properties of the other two components by qualitative analysis. To realize the quantitative analysis of the corresponding coefficients of three components in TOCKD, Tree-structured Parzen Estimator (TPE) is used to search the coefficients. On the basis of it, combined with subjective and objective weight analysis methods, the guide value of coefficients is defined. This research verifies the excellent performance and robustness of TOCKD by analyzing the flip-chip vibration signals, and hopes that TOCKD will be helpful for future research and the application of lightweight technology in the engineering field.