K-nearest Neighbor Research Articles

Comparative Analysis of Machine Learning Models for Music Recommendation

Abstract. This study is inspired by the Kaggle competition WSDM - KKBoxs Music Recommendation Challenge. The study focuses on doing a comparative study on music recommendation models. Based on the requirements and the given dataset from the Kaggle competition, the problem can be transferred to a classification problem, and therefore, we chose three classification models for the comparative study. The three models are K Nearest Neighbors (KNN), Random Forest, and Light Gradient Boosting Machine (LightGBM). We also did data analysis and data preparation before applying the model and used the handout method and cross-validation method for the validation. For the evaluation, the AUC score is applied to the results and the empirical results show that LightGBM is the best model among these three.

Automated Cough Analysis with Convolutional Recurrent Neural Network.

Chronic cough is associated with several respiratory diseases and is a significant burden on physical, social, and psychological health. Non-invasive, real-time, continuous, and quantitative monitoring tools are highly desired to assess cough severity, the effectiveness of treatment, and monitor disease progression in clinical practice and research. There are currently limited tools to quantitatively measure spontaneous coughs in daily living settings in clinical trials and in clinical practice. In this study, we developed a machine learning model for the detection and classification of cough sounds. Mel spectrograms are utilized as a key feature representation to capture the temporal and spectral characteristics of coughs. We applied this approach to automate cough analysis using 300 h of audio recordings from cough challenge clinical studies conducted in a clinical lab setting. A number of machine learning algorithms were studied and compared, including decision tree, support vector machine, k-nearest neighbors, logistic regression, random forest, and neural network. We identified that for this dataset, the CRNN approach is the most effective method, reaching 98% accuracy in identifying individual coughs from the audio data. These findings provide insights into the strengths and limitations of various algorithms, highlighting the potential of CRNNs in analyzing complex cough patterns. This research demonstrates the potential of neural network models in fully automated cough monitoring. The approach requires validation in detecting spontaneous coughs in patients with refractory chronic cough in a real-life setting.

Effective methods for quantifying complexity based on improved ordinal partition networks: Topological dispersion entropy and weighted topological dispersion entropy

Topological permutation entropy is based on ordinal partition networks (OPNs) to approximate topological entropy of low-dimensional chaotic systems. But the ordinal patterns of OPN ignore the magnitude of the amplitude value. To solve the problem, we propose topological dispersion entropy (TDE) and weighted topological dispersion entropy (WTDE) based on dispersion patterns to characterize the complexity of a system. Furthermore, WTDE strengthens the topological structure analysis of complex networks by weighting the adjacency matrix of the improved ordinal partition networks, thereby more accurately capturing the dynamic evolution of time series. The proposed methods are comprehensively evaluated by numerical experiments. The results show that the performance of both TDE and WTDE is significantly better than TPE. Especially, WTDE has good stability to parameters, data length, and noise. Finally, combining support vector machines and K-Nearest Neighbor, TDE and WTDE are applied to the classification of physiological data and mechanical failure data.

Enhancing Bipolar Disorder Detection using Heterogeneous Ensemble Machine Learning Techniques

This paper introduces a novel Heterogeneous Ensemble Machine Learning (HEML) approach designed to detect bipolar disorder, a significant healthcare challenge that demands precise and prompt diagnosis for effective treatment. The HEML method integrates multiple machines learning models, incorporating various physiological, behavioral, and contextual data from patients. By using a comprehensive feature selection technique, relevant features are extracted from each data source and utilized to train individual classifiers for detecting mental disorders. The classifiers include Adaboost, Decision Tree, K-nearest neighbors, Multilayer Perceptron, Random Forest, Relevance Vector Machine, and XGB, with Logistic Regression serving as the meta-model. This ensemble of classifiers enhances overall performance by capturing a wider range of characteristics related to mental disorders. The research evaluates the HEML method across three bipolar disorder datasets: Dataset1 (a multimodal dataset), Dataset2 (a sensor-based dataset), and Dataset3 (a real-time dataset). The HEML approach surpasses traditional methods, achieving superior accuracy rates of 95.21% with Dataset 1, 99.28% with Dataset 2, and 99% with Dataset 3. It outperforms individual models in detecting bipolar disorder, delivering the best Precision, Recall, F1 score, and Kappa Score. This comparative analysis advances the field of mental health diagnosis by leveraging the strengths of ensemble machine learning to improve accuracy and reliability in detection methods.

Optimizing wind power forecasting with RNN-LSTM models through grid search cross-validation

Wind energy is a crucial renewable resource that supports sustainable development and reduces carbon emissions. However, accurate wind power forecasting is challenging due to the inherent variability in wind patterns. This paper addresses these challenges by developing and evaluating some machine learning (ML) and deep learning (DL) models to enhance wind power forecasting accuracy. Traditional ML models, including Random Forest, k-nearest Neighbors, Ridge Regression, LASSO, Support Vector Regression, and Elastic Net, are compared with advanced DL models, such as Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), Stacked LSTM, Graph Convolutional Networks (GCN), Temporal Convolutional Networks (TCN), and the Informer network, which is well-suited for long-sequence forecasting and large, sparse datasets. Recognizing the complexities of wind power forecasting, such as the need for high-resolution meteorological data and the limitations of ML models like overfitting and computational complexity, a novel hybrid approach is proposed. This approach uses hybrid RNN-LSTM models optimized through GS-CV. The models were trained and validated on a SCADA dataset from a Turkish wind farm, comprising 50,530 instances. Data preprocessing included cleaning, encoding, and normalization, with 70 % of the dataset allocated for training and 30 % for validation. Model performance was evaluated using key metrics such as R², MSE, MAE, RMSE, and MedAE. The proposed hybrid RNN-LSTM Models achieved outstanding results, with the RNN-LSTM model attaining an R² of 99.99 %, significantly outperforming other models. These results demonstrate the effectiveness of the hybrid approach and the Informer network in improving wind power forecasting accuracy, contributing to grid stability, and facilitating the broader adoption of sustainable energy solutions. The proposed model also achieved superior comparable performance when compared to state-of-the-art methods.

Big Data and Machine Learning-Based Iot Models for Sustainable Energy Prediction

Integrating Big Data and Internet of Things (IoT) platforms is the focus of this research, which aims to improve energy management. The problem statement is centered on the potential for development through advanced technologies and the inefficiencies in traditional energy management methods. The objectives are to analyze energy consumption patterns, develop an innovative Home Energy Management System (HEMS) architecture, and offer energy-saving solutions. Synthetic energy consumption data is generated, normalized, and divided into training and testing sets from a methodological perspective. K-nearest neighbors, Decision Trees, Support Vector Regression, and Random Forest are the machine learning models trained and evaluated. The Random Forest model outperforms other models in terms of the accuracy of its predictions of energy consumption. The integration of renewable energy sources with cutting-edge technology to revolutionize energy management practices is the essence of novelty. In conclusion, this investigation underscores the importance of utilizing advanced technologies to promote sustainable energy management, providing practitioners and policymakers with practical insights.

Identification of lu’an Guapian at different picking periods by using excitation–emission matrix fluorescence spectroscopy coupled with chemometrics

Lu’an Guapian (LAGP) is a renowned green tea, with its price significantly higher when picked before the Qingming Festival compared to after, posing risks of confusion and counterfeiting. This study proposed using an excitation-emission matrix (EEM) fluorescence method combined with chemometrics for rapid identification of tea picked before and after Qingming Festival. Firstly, the differences among the EEM fingerprints of different tea samples were analyzed using the alternating trilinear decomposition (ATLD) algorithm. To determine the differences between LAGP before and after Qingming Festival, the total contents of ten main components in tea were detected, and their effects on the EEM fluorescence fingerprint of tea were analyzed using correlation heatmaps. Finally, two chemometric algorithms, partial least squares discriminant analysis (PLS-DA) and k-nearest neighbor (k-NN), were used to classify LAGP before and after Qingming Festival, achieving a classification accuracy of 100% for the training set, test set, and prediction set. To further explore the potential of this method, LAGP was further classified in detail according to four detailed picking periods, achieving an accuracy of over 83%. The same chemometric algorithm was used to classify the data based on the high-performance liquid chromatography (HPLC) method, yielding results comparable to those of the EEM-based method, though slightly inferior. Variable importance projection (VIP) analysis shows that catechin analogs are the main contributors to the classification of LAGP. The results demonstrated the EEM method’s significant potential in identifying the picking time of green tea.

Deep Learning and Statistical Operations Based features extraction for Skin Cancer Detection

Skin cancer is considered as one of the most serious types of cancer that leads to death worldwide. The number of deaths that caused by skin cancer can be reduced if it is diagnosed at early stages. Skin cancer is usually diagnosed using visual inspection, but it is less accurate. Using deep learning-based methods have been proposed to assist the doctors to diagnose the skin cancers at early accurately. The investigation was achieved on 3600 images collected from kaggle. Two deep learning algorithms used in the study: Vgg-19 and Alexnet to extract the layers 6 and 7. Then, the two layers will be merged to generate the statistical operations like: median, lowest, highest and joined of two layers. The following classifiers were used in the investigation: K-Nearest Neighbor, Random Forest, Naïve Bayes and Decision Tree. However, the study considered the following measures: accuracy, precision, recall, and F-Measure. Three training datasets sizes will be used to investigate the influence on classification accuracy. The results of all datasets were slightly similar. This approves that the extracted features and the statistical operations have an influence on the classification accuracy. The results show that Alexnet performs high accuracy and consumes less time that required for training the model compared to vgg-19. The results of classifiers showed that Random Forest scored high classification accuracy (85.6) compared to other ML classifiers

A Comprehensive Evaluation of Machine Learning Algorithms for Digital Soil Organic Carbon Mapping on a National Scale

The aim of this study was to narrow the research gap of ambiguity in which machine learning algorithms should be selected for evaluation in digital soil organic carbon (SOC) mapping. This was performed by providing a comprehensive assessment of prediction accuracy for 15 frequently used machine learning algorithms in digital SOC mapping based on studies indexed in the Web of Science Core Collection (WoSCC), providing a basis for algorithm selection in future studies. Two study areas, including mainland France and the Czech Republic, were used in the study based on 2514 and 400 soil samples from the LUCAS 2018 dataset. Random Forest was first ranked for France (mainland) and then ranked for the Czech Republic regarding prediction accuracy; the coefficients of determination were 0.411 and 0.249, respectively, which was in accordance with its dominant appearance in previous studies indexed in the WoSCC. Additionally, the K-Nearest Neighbors and Gradient Boosting Machine regression algorithms indicated, relative to their frequency in studies indexed in the WoSCC, that they are underrated and should be more frequently considered in future digital SOC studies. Future studies should consider study areas not strictly related to human-made administrative borders, as well as more interpretable machine learning and ensemble machine learning approaches.

Machine learning assisted design and preparation of Fe85Si2B8.5P3.5C1 amorphous/nanocrystalline alloy with high Bs and low Hc

Four machine learning (ML) models including eXtreme Gradient boosting (XGBT), k-Nearest Neighbor (kNN), Gradient Boosting Decision Tree (GBDT) and Artificial Neural Network (ANN) were employed to predict saturation flux density (Bs), coercivity (Hc), grain size, magnetostriction (λ), and Curie temperature (Tc) of Fe-based amorphous/nanocrystalline alloys. To maximize predictive ability of ML models, grid-search and normalization were used to search the most proper parameters of ML and pre-process raw data, respectively. XGBT had best predictive and generalization ability for predicting Bs and Hc with coefficient of determination (R2) of 0.992 and 0.967, respectively. Based on the feature importance analysis from the XGBT model, the Fe85Si2B8.5P3.5C1 amorphous alloy ribbon with good magnetic properties, such as high Bs, low Hc, was designed and prepared by melt spinning. X-ray diffraction (XRD), differential scanning calorimetry (DSC), transmission electron microscopy (TEM), vibrating sample magnetometer (VSM), B–H loop tracer, and magnetostriction instrument were used to identify the phase structure and physical properties of the Fe85Si2B8.5P3.5C1 alloy. It was found that the Fe85Si2B8.5P3.5C1 alloy had good magnetic properties with Bs of 1.82 T and the Hc of 2.02 A/m after annealing at 723 K for 180 s, in good agreement with the designed results by machine learning.

Dialogue between algorithms and soil: Machine learning unravels the mystery of phthalates pollution in soil

Soil is a major environmental sink for the emerging organic pollutants phthalates (PAEs), and the determination of key factors influencing PAEs accumulation in soil is crucial for agricultural sustainability and food security. Aiming at the time-consuming and inefficient characteristics of traditional batch experiments and statistical prediction models in comprehensively capturing PAEs dynamics in soil, an intelligent analysis framework based on machine learning was proposed and developed. In this study, thirty features were incorporated, including soil PAEs-concentrations, pollutant emissions, agricultural inputs, soil physicochemical properties, and climatic parameters. Six data-driven machine learning models were established: Random Forest Regression (RFR), Gradient Boosting Regression Tree (GBRT), Extreme Gradient Boosting (XGBoost), Multilayer Perceptron (MLP), Support Vector Regression (SVR), and k-Nearest Neighbors (KNN). Results showed that the MLP model exhibited optimal performance in predicting soil PAEs concentrations (R²=0.8637), followed by SVR (R²=0.8132) and XGBoost (R²=0.8096). Through feature importance analysis, it was determined that hydrometeorological factors, soil moisture conditions, and nutritional characteristics were the key factors controlling PAEs spatial distribution. Furthermore, non-linear effect analysis elucidated significant synergistic interactions among these environmental covariates. The spatiotemporal prediction model revealed continuous declining trends in PAEs pollution levels in eastern coastal regions over the next 5–10 years, while accumulation tendencies were observed in inland provinces particularly in Guizhou. This study demonstrates the effectiveness and advantages of machine learning in predicting soil PAEs-pollution, providing a new perspective for pollutant risk assessment and management in the era of environmental big data.

Leading Edge Erosion Classification in Offshore Wind Turbines Using Feature Extraction and Classical Machine Learning

Leading edge (LE) erosion is a type of damage that inhibits the aerodynamic performance of a wind turbine, resulting in high operation and maintenance (O&M) costs. This paper makes use of a small dataset consisting of 50 images of LE erosion and healthy blades for feature extraction and the training of four types of classifiers, namely, support vector machine (SVM), random forest, K-nearest neighbour (KNN), and multi-layer perceptron (MLP). Six feature extraction methods were used with these classifiers to train 24 models. The dataset has also been used to train a convolutional neural network (CNN) model developed using Keras. The purpose of this work is to determine whether classical machine learning (ML) classifiers trained with extracted features can produce higher-accuracy results, train faster, and classify faster than deep learning (DL) models for the application of LE damage detection of wind turbine blades. The oriented fast and rotated brief (ORB)-trained SVM achieved an accuracy of 90% ± 0.01, took 80.4 s to train, and achieved inference speeds of 63 frames per second (FPS), compared to the CNN model, which achieved an accuracy of 79.4% ± 2.07, took 4667.4 s to train, and achieved an inference speed of 1.3 FPS. These results suggest that classical ML models can be more accurate and efficient than DL models if the appropriate feature extraction method is used.

Machine learning models for predicting dysphonia following anterior cervical discectomy and fusion: a Swedish Registry Study

BACKGROUNDDysphonia is one of the more common complications following anterior cervical discectomy and fusion (ACDF). ACDF is the gold standard for treating degenerative cervical spine disorders, and identifying high-risk patients is therefore crucial. PURPOSEThis study aimed to evaluate different machine learning models to predict persistent dysphonia after ACDF. STUDY DESIGNA retrospective review of the nationwide Swedish spine registry (Swespine). PATIENT SAMPLEAll adults in the Swespine registry who underwent elective ACDF between 2006 and 2020. OUTCOME MEASURESThe primary outcome was self-reported dysphonia lasting at least 1 month after surgery. Predictive performance was assessed using discrimination and calibration metrics. METHODSPatients with missing dysphonia data at the 1-year follow-up were excluded. Data preprocessing involved one-hot encoding categorical variables, scaling continuous variables, and imputing missing values. Four machine learning models (logistic regression, random forest (RF), gradient boosting, K-nearest neighbor) were employed. The models were trained and tested using an 80:20 data split and 5-fold cross-validation, with performance metrics guiding the selection of the best model for predicting persistent dysphonia. RESULTSIn total, 2,708 were included in the study. Twelve key predictors were identified. Four machine learning models were tested, with the RF model achieving the best performance (AUC=0.794). The most significant predictors across models included preoperative NDI, EQ5Dindex, preoperative neurology, number of operated levels, and use of a fusion cage. The RF model, chosen for its superior performance, showed high sensitivity and consistent accuracy, but a low specificity and positive predictive value. CONCLUSIONSIn this study, machine learning models were employed to identify predictors of persistent dysphonia following ACDF. Among the models tested, the RF classifier demonstrated superior performance, with an AUC value of 0.790. The RF model identified NDI, EQ5Dindex, and number of fused vertebrae as key variables. These findings underscore the potential of machine learning models in identifying patients at increased risk for dysphonia persisting for more than 1 month after surgery.

From Policy to Prediction: Assessing Forecasting Accuracy in an Integrated Framework with Machine Learning and Disease Models.

To improve the forecasting accuracy of the spread of infectious diseases, a hybrid model was recently introduced where the commonly assumed constant disease transmission rate was actively estimated from enforced mitigating policy data by a machine learning (ML) model and then fed to an extended susceptible-infected-recovered model to forecast the number of infected cases. Testing only one ML model, that is, gradient boosting model (GBM), the work left open whether other ML models would perform better. Here, we compared GBMs, linear regressions, k-nearest neighbors, and Bayesian networks (BNs) in forecasting the number of COVID-19-infected cases in the United States and Canadian provinces based on policy indices of future 35 days. There was no significant difference in the mean absolute percentage errors of these ML models over the combined dataset []. In two provinces, a significant difference was observed [], yet posthoc tests revealed no significant difference in pairwise comparisons. Nevertheless, BNs significantly outperformed the other models in most of the training datasets. The results put forward that the ML models have equal forecasting power overall, and BNs are best for data-fitting applications.

Preoperative knee hyperextension is the most relevant predictor for ACL reconstruction objective failure: a machine learning analysis of 680 patients

Objectives To evaluate the predictive performance of different machine learning algorithms for the occurrence of objective failure in ACL reconstruction and to identify the most relevant predictors associated with this outcome. Methods We evaluated 680 patients submitted submitted to ACL reconstruction between January 2012 and July 2021. The study outcome was ACL objective failure, defined as a complete tear confirmed by MRI or arthroscopy evaluation or clinically ACL insufficiency. Routinely collected imaging, clinical, and demographic data were used to train 9 machine learning algorithms (k-nearest neighbors (KNN) classifier, decision tree classifier, random forest classifier, extra trees classifier, gradient boosting classifier, Light Gradient Boosting Machine (LightGBM), eXtreme Gradient Boosting (XGBoost), CatBoost classifier, and logistic regression). We used a random sample of 70% of patients to train the algorithms, and 30% were left for performance assessment, simulating new data. The performance of the models was evaluated with the area under the receiver operating characteristic curve (AUC). Results The predictive performance of all models was good, with AUC’s ranging from 0.81 to 0.88. The models with the best AUC metric were the Gradient Boosting Classifier (0.88 [95% CI, 0.81 to 0.92]) and Random Forest Classifier (0.88 [ 95% CI, 0.81 to 0.92). Knee hyperextension consistently emerged as the primary predictor across all models subjected to our analysis. Conclusion Machine learning algorithms demonstrated good performance to predict ACL reconstruction objective failure. Additionally, knee hyperextension consistently emerged as the primary predictor across all models subjected to our analysis.

Hydrogen Concentration Prediction in a Passive Autocatalytic Recombiner Using Machine Learning Algorithms

Monitoring hydrogen levels within Nuclear Power Plants (NPPs) is crucial to mitigate potential risks during severe accidental scenarios. Passive Autocatalytic Recombiners (PARs), which operate passively through catalytic reactions, are installed in the containment to reduce hydrogen concentration. This study numerically investigates hydrogen behavior within PAR under various accident conditions. As the inlet temperature and hydrogen concentration increase, the reaction rate and maximum catalyst plate temperature rise. As the inflow velocity increases, the reaction amount rises, but the residence time for the reactions decreases, leading to an increase in the outlet hydrogen concentration. Leveraging the operational characteristics of the PAR, the present study develops a data-driven model to identify the correlations among the parameters associated with the PAR's performance and to predict hydrogen concentration at the PAR's outlet by adopting four machine learning algorithms: Artificial Neural Networks (ANN), k-nearest Neighbor (k-NN), Random Forest (RF), and Support Vector Regression (SVR). Using the PAR's inlet variables of flow velocity, temperature, hydrogen concentration, and outlet temperature as input parameters, the ANN model demonstrates good predictive performance with R2 values of about 0.99. The predictive performance of the ANN model remains robust even without the inlet hydrogen information.

Securing DNS over HTTPS traffic: a real-time analysis tool

DNS over HTTPS (DoH) is a developing protocol that uses encryption to secure domain name system (DNS) queries within hypertext transfer protocol secure (HTTPS) connections, thereby improving privacy and security while browsing the web. This study involved the development of a live tool that captures and analyzes DoH traffic in order to classify it as either benign or malicious. We employed machine learning (ML) algorithms such as K-Nearest Neighbors (K-NN), random forest (RF), decision tree (DT), deep neural network (DNN), and support vector machine (SVM) to categorize the data. All of the algorithms, namely KNN, RF, and DT, achieved exceptional performance, with F1 scores of 1.0 or above for both precision and recall. The SVM and DNN both achieved exceptionally high scores, with only slight differences in accuracy. This tool employs a voting mechanism to arrive at a definitive classification decision. By integrating with the Mallory tool, it becomes possible to locally resolve DNS, which in turn allows for more accurate simulation of DoH queries. The evaluation results clearly indicate outstanding performance, confirming the tool's effectiveness in analyzing DoH traffic for network security and threat detection purposes.

MRI-based radiomics virtual biopsy for BCL6 in primary central nervous system lymphoma

To establish a machine learning model based on a radiomic signature for predicting B-cell lymphoma 6 (BCL-6) rearrangement in primary central nervous system lymphoma (PCNSL). Retrospective study on 102 PCNSL patients (31 with BCL-6 rearrangement positive, 71 with BCL-6 rearrangement negative) were randomly divided into the training and validation sets at a ratio of 7:3. Radiomics models based on contrast-enhanced T1-weighted imaging (CE-T1WI) and fluid-attenuated inversion recovery (FLAIR) in different regions, including VOItumour core and VOIperitumoural oedema. Radiomics features were extracted and selected using LASSO regression, and radiomics score (rad-score) were calculated using the weighted coefficients. Four machine learning models (logistic regression, random forest, support vector machine, K-nearest neighbours) were developed and evaluated based on rad-score. The optimal radiomics model was integrated into the clinical or radiological factors to construct a predictive model through logistic regression analysis. A nomogram was constructed based on independent significant features for individualised prediction. All rad-scores based on CE-T1WI and FLAIR sequences were significantly associated with BCL6 rearrangement (p < 0.05) in univariate regression analysis. The logistic regression machine learning model performed best with AUCs of 0.935 (training) and 0.923 (validation). Rad-scores from CE-T1WI tumour core and peritumoural oedema were independent significant predictors. Radiomics signatures based on CE-T1WI and FLAIR sequences have significant value in distinguishing BCL6 rearrangement. The CE-T1WI radiomics model based on VOItumour core and VOIperitumoural oedema are robust markers for identifying BCL6 rearrangement.

Deep learning for prediction and classifying the dynamical behaviour of piecewise-smooth maps

This paper explores novel ways of predicting and classifying the dynamics of piecewise smooth maps using various deep learning models. Moreover, we have used machine learning models such as Decision Tree Classifier, Logistic Regression, K-Nearest Neighbor, Random Forest, and Support Vector Machine for predicting the border collision bifurcation in the 1D normal form map and the 1D tent map. The decision tree classifier best predicts the border collision bifurcation for the 1D normal form map, the random forest, and the 1D tent map. This study introduces a novel application of deep learning models to cobweb diagrams and phase portraits, which provides a new perspective for classifying regular and chaotic behaviour. Further, we classified the regular and chaotic behaviour of the 1D tent map and the 2D Lozi map using deep learning models like Convolutional Neural Network (CNN), ResNet50, and ConvLSTM via cobweb diagram and phase portraits, where CNN exhibits better performance than other models. We also classified the chaotic and hyperchaotic behaviour of the 3D piecewise smooth map using deep learning models such as the Feed Forward Neural Network (FNN), Long Short-Term Memory (LSTM), and Recurrent Neural Network (RNN). We have shown that LSTM performs best for classifying chaotic and hyperchaotic behaviour. Additionally, LSTM outperforms other models in accuracy and computational efficiency, making it highly effective for real-time analysis. Finally, deep learning models such as Long Short Term Memory (LSTM) and Recurrent Neural Network (RNN) are used for reconstructing the two-parameter bifurcation charts of 2D normal form map, in which LSTM is more precise than RNN in reconstructing the two-parametric charts.

Effects of model-overfit on model-assisted forest inventory in boreal forests with remote sensing data

Abstract While remote sensing can be an effective tool in building a forest inventory, field measurements and model fitting can be both expensive and challenging. One strategy to reduce forest inventory costs is to leverage forest inventory models fitted to a different population (external models), although the effectiveness of external models is poorly understood. One concern is that models may predict well to the sample data, but poorly to the population—which is termed ‘overfitting’. The effect of overfit may be especially problematic in attempts to predict for a different population (a forest area not covered by any sample plots). Assessing overfit is difficult and its consequence for estimation are not well understood, especially in the context of prediction using external models. This study assesses how overfitting affects model-assisted forest inventory estimation when using internal and external models. We used field and remotely sensed data (Sentinel-2 images and airborne laser scanning data) from two forest areas in Finland. We evaluated four modeling approaches: ordinary least square regression (OLS), random forest, k-nearest neighbors, and gaussian process regression. Both analytical and bootstrap variance estimators were used to evaluate model-assisted estimation performance. Internal models, especially OLS, were the most affected by model overfitting, leading to bias in the population means and underestimation of variance. Estimates using external models provided unbiased means and realistic intervals except in the case of deliberate excessive overfitting. The bootstrap variance estimator was found to be more robust to overfit than the analytical variance estimator for the internal model, but was not helpful for the external model. Internal models should be parsimonious to generalize well to the population and avoid bias. The bootstrap estimator of variance is recommended for internal models, especially if there is concern about model overfitting.