K-fold Cross-validation Method Research Articles

Classification of Normal and Abnormal Heart Sounds Using Empirical Mode Decomposition and First Order Statistic

Analysis of heart sound signals for automatic segmentation and classification has revealed in recent decades that it has the potential to detect pathology accurately in clinical applications. Various audio signal processing techniques have been used to reduce the subjectivity of heart sound analysis. This study aims to classify normal and abnormal heart sound signals. The feature extraction process was optimized by EMD and calculated using five first-order statistical parameters: mean, variance, kurtosis, skewness, and entropy. The classification system is optimized with a mutual information algorithm to select traits that can significantly improve system performance. In addition, the selection of the optimal system configuration also includes the k-fold cross-validation and kNN methods with k values ​​and the proper distance type. Based on the test results, the highest accuracy of 98.2% was obtained when the value of k = 1 and the type of cosine distance on kNN with a five-fold cross-validation system evaluation model.

Identification of High-Risk Patients for Postoperative Myocardial Injury After CME Using Machine Learning: A 10-Year Multicenter Retrospective Study.

The occurrence of myocardial injury, a grave complication post complete mesocolic excision (CME), profoundly impacts the immediate and long-term prognosis of patients. The aim of this inquiry was to conceive a machine learning model that can recognize preoperative, intraoperative and postoperative high-risk factors and predict the onset of myocardial injury following CME. This study included 1198 colon cancer patients, 133 of whom experienced myocardial injury after surgery. Thirty-six distinct variables were gathered, encompassing patient demographics, medical history, preoperative examination characteristics, surgery type, and intraoperative details. Four machine learning algorithms, namely, extreme gradient boosting (XGBoost), random forest (RF), multilayer perceptron (MLP), and k-nearest neighbor algorithm (KNN), were employed to fabricate the model, and k-fold cross-validation, ROC curve, calibration curve, decision curve analysis (DCA), and external validation were employed to evaluate it. Out of the four predictive models employed, the XGBoost algorithm demonstrated the best performance. The ROC curve findings indicated that the XGBoost model exhibited remarkable predictive accuracy, with an area under the curve (AUC) value of 0.997 in the training set and 0.956 in the validation set. For internal validation, the k-fold cross-validation method was utilized, and the XGBoost model was shown to be steady. Furthermore, the calibration curves demonstrated the XGBoost model's high predictive capability. The DCA curve revealed higher benefit rates for patients who underwent interventional treatment under the XGBoost model. The AUC value for the external validation set was 0.74, which indicated that the XGBoost prediction model possessed good extrapolative capacity. The myocardial injury prediction model for patients undergoing CME that was developed using the XGBoost machine learning algorithm in this study demonstrates both high predictive accuracy and clinical utility.

Deep learning for inverse design of low-boom supersonic configurations

Mitigating the sonic boom to an acceptable stage is crucial for the next generation of supersonic transports. The primary way to suppress sonic booms is to develop a low sonic boom aerodynamic shape design. This paper proposes an inverse design approach to optimize the near-field signature of an aircraft, making it close to the shaped ideal ground signature after propagation in the atmosphere. By introducing the Deep Neural Network (DNN) model for the first time, a predicted input of Augmented Burgers equation is inversely achieved. By the K-fold cross-validation method, the predicted ground signature closest to the target ground signature is obtained. Then, the corresponding equivalent area distribution is calculated using the classical Whitham’s F-function theory from the optimal near-field signature. The inversion method is validated using the classic example of the C608 vehicle provided by the Third Sonic Boom Prediction Workshop (SBPW-3). The results show that the design ground signature is consistent with the target signature. The equivalent area distribution of the design result is smoother than the baseline distribution, and it shrinks significantly in the rear section. Finally, the robustness of this method is verified through the inverse design of sonic boom for the non-physical ground signature target.

Applying interpretable machine learning algorithms to predict risk factors for permanent stoma in patients after TME.

The purpose of this study was to develop a machine learning model to identify preoperative and intraoperative high-risk factors and to predict the occurrence of permanent stoma in patients after total mesorectal excision (TME). A total of 1,163 patients with rectal cancer were included in the study, including 142 patients with permanent stoma. We collected 24 characteristic variables, including patient demographic characteristics, basic medical history, preoperative examination characteristics, type of surgery, and intraoperative information. Four machine learning algorithms including extreme gradient boosting (XGBoost), random forest (RF), support vector machine (SVM) and k-nearest neighbor algorithm (KNN) were applied to construct the model and evaluate the model using k-fold cross validation method, ROC curve, calibration curve, decision curve analysis (DCA) and external validation. The XGBoost algorithm showed the best performance among the four prediction models. The ROC curve results showed that XGBoost had a high predictive accuracy with an AUC value of 0.987 in the training set and 0.963 in the validation set. The k-fold cross-validation method was used for internal validation, and the XGBoost model was stable. The calibration curves showed high predictive power of the XGBoost model. DCA curves showed higher benefit rates for patients who received interventional treatment under the XGBoost model. The AUC value for the external validation set was 0.89, indicating that the XGBoost prediction model has good extrapolation. The prediction model for permanent stoma in patients with rectal cancer derived from the XGBoost machine learning algorithm in this study has high prediction accuracy and clinical utility.

Prediction of Microstructure and Mechanical Properties of Atmospheric Plasma-Sprayed 8YSZ Thermal Barrier Coatings Using Hybrid Machine Learning Approaches

The preparation of thermal barrier coatings (TBCs) is a complex process involving the integration of physics and chemistry, mainly involving the flight behavior and deposition behavior of molten particles. The service life and performance of the TBCs were determined by various factors, especially the preparation process parameters. In this work, to set up the quantitative characterization model between the preparation process parameters and the performance characteristic parameters, the ceramic powder particle size, spraying power and spraying distance were treated as the model input parameters, the characteristic parameters of microstructure properties represented by the porosity, circularity and Feret’s diameter and the mechanical property represented by the interfacial binding strength and macrohardness were treated as the model output. The typical back propagation (BP) model and extreme learning machine (ELM) model combined with flower pollination algorithm (FPA) optimization algorithm were employed for modeling analysis. To ensure the robustness of the obtained regression prediction model, the k-fold cross-validation method was employed to evaluate and analyze the regression prediction models. The results showed that the regression coefficient R value of the proposed FPA-ELM hybrid machine learning model was more than 0.94, the root-mean-square error (RMSE) was lower than 2 and showed better prediction accuracy and robustness. Finally, this work provided a novel method to optimize the TBCs preparation process, and was expected to improve the efficiency of TBCs preparation and characterization in the future.

Machine learning approach with various regression models for predicting the ultimate tensile strength of the friction stir welded AA 2050-T8 joints by the K-Fold cross-validation method

Machine learning approach with various regression models for predicting the ultimate tensile strength of the friction stir welded AA 2050-T8 joints by the K-Fold cross-validation method

Smaller is better? Unduly nice accuracy assessments in roof detection using remote sensing data with machine learning and k-fold cross-validation

Deriving the thematic accuracy of models is a fundamental part of image classification analyses. K-fold cross-validation (KCV), as an accuracy assessment technique, can be biased because existing built-in algorithms of software solutions do not handle the high autocorrelation of remotely sensed images, leading to overestimation of accuracies. We aimed to quantify the magnitude of the overestimation of KCV-based accuracies and propose a method to overcome this problem with the example of rooftops using a WorldView-2 (WV2) satellite image, and two orthophotos. Random split to training/testing subsets, independent testing and different types of repeated KCV sampling strategies were used to generate input datasets for classification. Results revealed that applying the random splitting of reference data to training/testing subsets and KCV methods had significantly biased the accuracies by up to 17%; overall accuracies (OAs) can incorrectly reach >99%. We found that repeated KCV can provide similar results to independent testing when spatial sampling is applied with a sufficiently large distance threshold (in our case 10 m). Coarser resolution of WV2 ensured more reliable results (up to a 5–9% increase in OA) than orthophotos. Object-based pixel purity of buildings showed that when using a majority filter for at least of 50% of objects the final accuracy approached 100% with each sampling method. The final conclusion is that KCV-based modelling ensures better accuracy than single models (with better pixel purity on the object level), but the accuracy metrics without spatially filtered sampling are not reliable.

Using Machine Learning Algorithms to Predict High-Risk Factors for Postoperative Delirium in Elderly Patients.

Postoperative delirium (POD) is a common postoperative complication in elderly patients, and it greatly affects the short-term and long-term prognosis of patients. The purpose of this study was to develop a machine learning model to identify preoperative, intraoperative and postoperative high-risk factors and predict the occurrence of delirium after nonbrain surgery in elderly patients. A total of 950 elderly patients were included in the study, including 132 patients with POD. We collected 30 characteristic variables, including patient demographic characteristics, basic medical history, preoperative examination characteristics, type of surgery, and intraoperative information. Three machine learning algorithms, multilayer perceptron (MLP), extreme gradient boosting (XGBoost), and k-nearest neighbor algorithm (KNN), were applied to construct the model, and the k-fold cross-validation method, ROC curve, calibration curve, decision curve analysis (DCA) and external validation were used for model evaluation. XGBoost showed the best performance among the three prediction models. The ROC curve results showed that XGBoost had a high area under the curve (AUC) value of 0.982 in the training set; the AUC value in the validation set was 0.924, and the prediction model was highly accurate. The k-fold cross-validation method was used for internal validation, and the XGBoost model was stable The calibration curve showed high predictive power of the XGBoost model. The DCA curve showed a higher benefit rate for patients who received interventional treatment under the XGBoost model. The AUC value for the external validation set was 0.88, indicating that the predictive model was extrapolative. The prediction model of POD derived from the machine learning algorithm in this study has high prediction accuracy and clinical utility, which is beneficial for clinicians to diagnose and treat patients in a timely manner.

Sentiment Analysis of Flood Disaster Management in Jakarta on Twitter Using Support Vector Machines

Floods can cause negative impacts in various aspects, starting from economic, social, to health aspects. Even quoted from the site gis.bnpb.go.id, during 2022, there have been 1031 cases of flood disasters in Indonesia. Meanwhile in Jakarta, in 2022 there have been 14 cases of floods that caused hundreds of people to lose their homes. Several approaches can be taken to determine public opinion about flooding, one of which is text mining with an analysis of community sentiment. Sentiment analysis aims to determine public opinion regarding flood management in the capital city of Jakarta based on positive, neutral, and negative categories. To get the public sentiment, researchers carried out several stages, including the preprocessing stage. After obtaining public sentiment regarding flood management through the preprocessing stage, then classification is carried out based on public opinion so that it can be used by related parties for evaluation material in flood handling. In this study, the classification method used is the SVM method which is one of the supervised learning methods in machine learning. After classification, the next stage is the testing process using the K-Fold Cross Validation method. From the various sentiments obtained from Twitter data, it can be concluded that there are around 414 positive sentiments and 2464 negative sentiments related to flood handling in DKI Jakarta, while the results obtained from the test results show that the accuracy reaches 88.6%, the precision reaches 88.6% and recall reached 89.4%.

Prediction of Shipping Cost on Freight Brokerage Platform Using Machine Learning

Not having an exact cost standard can present a problem for setting the shipping costs on a freight brokerage platform. Transport brokers who use their high market position to charge excessive commissions can also make it difficult to set rates. In addition, due to the absence of a quantified fare policy, fares are undervalued relative to the labor input. Therefore, vehicle owners are working for less pay than their efforts. This study derives the main variables that influence the setting of the shipping costs and presents the recommended shipping cost given by a price prediction model using machine learning methods. The cost prediction model was built using four algorithms: multiple linear regression, deep neural network, XGBoost regression, and LightGBM regression. R-squared was used as the performance evaluation index. In view of the results of this study, LightGBM was chosen as the model with the greatest explanatory power and the fastest processing. Furthermore, the range of the predicted shipping costs was determined considering realistic usage patterns. The confidence interval was used as the method of calculation for the range of the predicted shipping costs, and, for this purpose, the dataset was classified using the K-fold cross-validation method. This paper could be used to set the shipping costs on freight brokerage platforms and to improve utilization rates.

Prediction of fracture density in a gas reservoir using robust computational approaches

One of the challenges that reservoir engineers, drilling engineers, and geoscientists face in the oil and gas industry is determining the fracture density (FVDC) of reservoir rock. This critical parameter is valuable because its presence in oil and gas reservoirs boosts productivity and is pivotal for reservoir management, operation, and ultimately energy management. This valuable parameter is determined by some expensive operations such as FMI logs and core analysis techniques. As a result, this paper attempts to predict this important parameter using petrophysics logs routinely collected at oil and gas wells and by applying four robust computational algorithms and artificial intelligence hybrids. A total of 6067 data points were collected from three gas wells (#W1, #W2, and #W3) in one gas reservoir in Southwest Asia. Following feature selection, the input variables include spectral gamma ray (SGR); sonic porosity (PHIS); potassium (POTA); photoelectric absorption factor (PEF); neutron porosity (NPHI); sonic transition time (DT); bulk density (RHOB); and corrected gamma ray (CGR). In this study, four hybrids of two networks were used, including least squares support vector machine (LSSVM) and multi-layer perceptron (MLP) with two optimizers particle swarm optimizer (PSO) and genetic algorithm (GA). Four robust hybrid machine learning models were applied, and these are LSSVM-PSO/GA and MLP-PSO/GA, which had not previously used for prediction of FVDC. In addition, the k-fold cross validation method with k equal to 8 was used in this article. When the performance accuracy of the hybrid algorithms for the FVDC prediction is compared, the revealed result is LSSVM-PSO > LSSVM-GA > MLP-PSO > MLP-GA. The study revealed that the best algorithm for predicting FVDC among the four algorithms is LSSVM-PSO (for total dataset RMSE = 0.0463 1/m; R2 = 0.9995). This algorithm has several advantages, including: 1) lower adjustment parameters, 2) high search efficiency, 3) fast convergence speed, 4) increased global search capability, and 5) preventing the local optimum from falling. When compared to other models, this model has the lowest error.

An ensemble learning model for estimating the virtual energy storage capacity of aggregated air-conditioners

Renewable energy resources (RES) pose several challenges due to their natural intermittency when integrated into a distribution network. A smart energy storage system (SESS) alleviates these challenges, which is achieved by integrating thermostatically controlled loads (TCLs) such as air conditioners (ACs). Hence, this study proposes a virtual energy storage system (VESS) by modeling the ACs analogous to an electro-chemical battery. Besides, non-linearity in energy consumption patterns of ACs due to consumer behavior makes predictive energy analysis critical to deplete the discrepancy between supply and demand. Therefore, this study proposes an ensemble learning (EL) technique to predict the virtual energy storage (VES) capacity of ACs. The prediction stage relies on various machine learning (ML) methods, like support vector regression (SVR) and artificial neural network (ANN). Based on these ML techniques, this study proposes an SVRs-ANN-based EL model to predict the VES capacity during the discharging cycle. Each weak learner in the EL model is trained using a significant features subset with the aid of the k-fold cross-validation method to build a more generalized model for unseen data. The performance of the EL model is estimated using minute-based smart meter data set and empirically manifested that the EL model outperforms conventional approaches. The R2 and root mean squared error (RMSE) of the EL model reach 1 and 1.03×10−7, respectively, and results demonstrate lower RMSE values and higher performance than conventional methods. The results obtained have shown the robustness of the proposed EL model to operate ACs in energy-saving mode instead of load shifting. Although ACs are not controlled in the experiments, the experimental validation proved that the EL model accurately predicts the daily VES capacity of ACs without violating consumer satisfaction.

Error reduction in long-term mine planning estimates using deep learning models

The long-term mine planning model (LTMP) and short-term mine planning model (STMP) are two approaches that describe the ore content in a mine; they are essential intangible resources that determine a mining operation and its feasibility. These models are obtained with geostatistical methods and, given their nature, are prone to discrepancies with one another. To reduce these differences, we studied the performance of deep learning (DL)-based models in ore grade estimation for a copper mine in Chile. Specifically, feedforward neural network (FNN), one-dimensional (1D) convolutional neural network (CNN), and long short-term memory (LSTM) models were analyzed. The experiment consisted of a dataset with 732,870 samples, obtained after data cleaning and selection. The use of spatial information in the samples was also studied, adding contextual information for estimation. This led to a dataset of 545,768 samples that were used to evaluate 1D CNN and LSTM models. Architecture tuning was performed by the k-fold cross-validation (CV) method, and hyperparameters such as the number of layers, number of neurons, activation function, and kernel size were optimized. The resulting hyperparameters were used to perform a final evaluation. Performance was measured based on the mean absolute error (MAE), mean squared error (MSE), and coefficient of determination (R2) metrics. A baseline is created using LTMP and STMP estimates to quantify the improvement in the performance of DL-based models. The experimental results revealed the ability of DL-based models to significantly improve copper grade estimates provided by standard mining industry methods in the context under study. For MSE in the final tests, FNN improved by 21%, CNN by 37%, and LSTM by 39% over baselines.

Applying Wide & Deep Learning Model for Android Malware Classification

Android malware has exploded in popularity in recent years, due to the platform’s dominance of the mobile market. With the advancement of deep learning technology, numerous deep learning-based works have been proposed for the classification of Android malware. Deep learning technology is designed to handle a large amount of raw and continuous data, such as image content data. However, it is incompatible with discrete features, i.e., features gathered from multiple sources. Furthermore, if the feature set is already well-extracted and sparsely distributed, this technology is less effective than traditional machine learning. On the other hand, a wide learning model can expand the feature set to enhance the classification accuracy. To maximize the benefits of both methods, this study proposes combining the components of deep learning based on multi-branch CNNs (Convolutional Network Neural) with wide learning method. The feature set is evaluated and dynamically partitioned according to its meaning and generalizability to subsets when used as input to the model’s wide or deep component. The proposed model, partition, and feature set quality are all evaluated using the K-fold cross validation method on a composite dataset with three types of features: API, permission, and raw image. The accuracy with Wide and Deep CNN (WDCNN) model is 98.64%, improved by 1.38% compared to RNN (Recurrent Neural Network) model.

A spatial regression approach to modeling urban land surface temperature

Land surface temperature (LST) is the instantaneous radiative skin temperature of land obtained from satellite sensors. Measured by visible, infrared or microwave sensors, the LST is useful in determining thermal comfort for urban planning. It also serves as a precursor to many underlying impacts such as health, climate change and the likelihood of rainfall. Due to the paucity of observed data because of cloud cover or rain-bearing clouds in the case of microwave sensors, it is necessary to model LST for the purpose of forecasting. Two spatial regression models were employed: the spatial lag model and the spatial error model. Using Landsat 8 and Shuttle Radar Topography Mission (SRTM), these models can be studied and compared in terms of their robustness in reproducing LST. Whereas LST is to be the independent variable, built-up area, water surface, albedo, elevation, and vegetation are to be considered as dependent variables and their relative contributions to LST examined.•Modeling LST based on spatial regression models with calculated LST as independent variable.•Dependent variables to be considered are normalised difference Built-up index (NDBI), normalised difference vegetation index (NDVI), modified normalised difference water index (MNDWI), albedo and elevation.•The models were validated using k-fold cross validation method, mean square error and standard deviation.

Artificial Intelligence (AI)–Based Surface Quality Prediction Model for Carbon Fiber Reinforced Plastics (CFRP) Milling Process

The objective of this study is to develop a surface quality prediction model for CFRP milling based on artificial intelligence and enhance the model performance by proposing a novel feature through signal processing and applying it to machine learning algorithms. In this paper, the data-driven approach is applied based on data that is obtained by various sensors such as a two-axis load cell, an acoustic emission (AE) sensor, and an accelerometer, and surface quality is predicted as one of three grades based on surface roughness value. To establish the models, a novel feature, referred to as the tool rotation frequency (TRF) feature, was introduced as well as the conventional wavelet transform (WT) feature. Then, three machine learning algorithms – artificial neural network (ANN), K-nearest neighbor (KNN), and support vector machine (SVM) – are used to construct the surface quality prediction models. Each model is trained with 450 data sets obtained from 45 experimental cases, and their predictive accuracy is evaluated by the k-fold cross validation method. Next, to confirm each model’s robustness, 200 data sets obtained from additional 20 experimental cases designed with untrained conditions are tested. Finally, the monitoring software is developed for visualizing prediction results of the surface grades.

Hybrid Machine Learning-Based Approach for Anomaly Detection using Apache Spark

Over the past few decades, the volume of data has increased significantly in both scientific institutions and universities, with a large number of students enrolled and a high volume of related data. Furthermore, network traffic has increased with post-pandemic and the use of online learning. Therefore, processing network traffic data is a complex and challenging task that increases the possibility of intrusions and anomalies. Traditional security systems cannot deal with such high-speed and big data traffic. Real-time anomaly detection should be able to process data as quickly as possible to detect abnormal and malicious data. This paper proposes a hybrid approach consisting of supervised and unsupervised learning for anomaly detection based on the big data engine Apache Spark. Initially, the k-means algorithm was implemented in Sparks MLlib for clustering network traffic, then for each cluster, K-nearest neighbors algorithm (KNN) was implemented for classification and anomaly detection. The proposed model was trained and validated against a real dataset from Ibn Zohr University. The results indicate that the proposed model outperformed other well-known algorithms in detecting anomalies based on the aforementioned dataset. The experimental results show that the proposed hybrid approach can reach up to 99.94 % accuracy using the k-fold cross-validation method in the complete dataset with all 48 features.

Implementasi Jaringan Syaraf Tiruan Backpropagation untuk Menentukan Prediksi Jumlah Permintaan Produksi Dodol Apel

Forecasting is importantly in accordance with the planning strategy; therefore it will affect the way of decision making. One of the forecasting methods is Artificial Neural Network with Backpropagation as the algorithm. This research aims to measure the accuracy of the network architecture which is being applied in order to calculate the prediction of the future’s apple paste product monthly demand which was obtained from CV. Bagus Agriseta Mandiri. The data which are being used are 36 monthly data from the year 2017, 2018 and 2019. Furthermore, the data obtained are normalized and divided into two, 66,66% as the data for training process and 33,33% as the data for testing process. Network architecture that is applied in this research is 12 : 10 :1, where 12 are neurons for input layer, 10 are neurons for one hidden layer and 1 is neuron for output layer. The Network with that framework obtained a result 20.161% for MAPE and 79.839% for the accuracy. That model is categorized as good enough for its forecasting ability. Moreover, the network was entirely validated using k-fold cross validation method with . The result obtained as follows: the average of MAPE is 47.079% and the average accuracy is 52.921%. According to it, the entire model can be categorized as good enough in order to run a forecast. As a comparison, another testing has been done with the same fold but different in the network architecture (model 6 – 8 – 1). The second model obtained results as follows: the average of MAPE is 26.74% and the average accuracy is 73.18%, so that the two prediction models’ ability are in the same category, it is good enough to run a forecast.

An Efficient Ensemble Method Using K-Fold Cross Validation for the Early Detection of Benign and Malignant Breast Cancer

In comparison to all other malignancies, breast cancer is the most commonform of cancer, among women. Breast cancer prediction has been studied by a number of researchers, and is considered as a serious threat to women. Clinicians are finding it difficult to create a treatment approach that will help patients live longer, due to the lack of solid predictive models which predicts the outcome in early stages by analyzing history of patient’s data. Rates of this malignancy have been observed to rise, more with industrialization and urbanization, as well as with early detection facilities. It is still considerably more prevalent in very developed countries, but it is rapidly spreading to developing countries as well. The purpose of this work is to offer a report on the disease of breast cancer in which we used available technical breakthroughs to construct breast cancer survivability prediction models. The Machine Learning (ML) techniques, namely Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Decision Tree (DT) Classifier, Random Forests (RF), and Logistic Regression (LR)are used as base Learners and their performance has been compared with the ensemble method, eXtreme Gradient Boosting(XGBoost). For performance comparison, we employed k-fold cross validation method to measure the unbiased estimate of these prediction models. The results indicated that XGBoost outperformed with an accuracy of 97.81% compared to other ML algorithms.

Crystal structural prediction of perovskite materials using machine learning: A comparative study

In this study, Machine Learning (ML) techniques have been exploited to classify the crystal structure of ABO3 perovskite compounds. In the present work, seven different ML algorithms are applied to the experimentally determined crystal structure data. The relevance of the data featured is measured by computing the Chi-Square test and Spearman's correlation matrix. The Z-Score value has been calculated for each attribute to confirm the existence of any outliers in the data. The Synthetic Minority Oversampling (SMOTE) technique is employed to overcome the imbalanced data set. The models' performance is calculated using the stratified k-Fold cross-validation method. Further, to improve the accuracy of the prediction model, the conventional algorithm is supported by boosting algorithm. Comparative model efficiency on prediction of the crystal structure is presented to identify the most suitable model. As per the inferences drawn from the observations, the ensemble model using Xtreme Gradient Boosting (XGBoost) algorithm when applied to the pre-processed and balanced data outperforms the other models.