Distance-weighted K-nearest Neighbor Research Articles

An Adaptive Learning Environment for Programming Based on Fuzzy Logic and Machine Learning

In this paper, we present an Intelligent Tutoring System (ITS), for use in teaching the logic of computer programming and the programming language ‘C’. The aim of the ITS is to adapt the delivered learning material and the lesson sequence to the knowledge level and learning needs of each individual student. The adaptation of the presented ITS is based on fuzzy logic and a machine learning technique. Particularly, the system uses the distance weighted k-nearest neighbor algorithm to detect the learner’s knowledge level and abilities concerning computer programming during her/ his first interaction with the system. Next and during subsequent interactions of the learner with the system, fuzzy logic is used to identify the learner’s current knowledge level and potential misconceptions. The system takes into consideration the knowledge dependencies that exist among the domain concepts of the learning material and, applying fuzzy rules, decides about the learning material that has to be delivered to the learner as well as the lesson sequence. The system has been fully implemented and evaluated through t-tests. The evaluation results show that the combination of machine learning (for initially identifying the student’s learning abilities and needs) with fuzzy logic (for the continuous identification of the learner’s current knowledge level and misconceptions) provides more personalized learning experience, promotes the active participation of students in the learning process and results in decrease in the number of dropouts.

Map-Reduce based Distance Weighted k-Nearest Neighbor Machine Learning Algorithm for Big Data Applications

With the evolution of Internet standards and advancements in various Internet and mobile technologies, especially since web 4.0, more and more web and mobile applications emerge such as e-commerce, social networks, online gaming applications and Internet of Things based applications. Due to the deployment and concurrent access of these applications on the Internet and mobile devices, the amount of data and the kind of data generated increases exponentially and the new era of Big Data has come into existence. Presently available data structures and data analyzing algorithms are not capable to handle such Big Data. Hence, there is a need for scalable, flexible, parallel and intelligent data analyzing algorithms to handle and analyze the complex massive data. In this article, we have proposed a novel distributed supervised machine learning algorithm based on the MapReduce programming model and Distance Weighted k-Nearest Neighbor algorithm called MR-DWkNN to process and analyze the Big Data in the Hadoop cluster environment. The proposed distributed algorithm is based on supervised learning performs both regression tasks as well as classification tasks on large-volume of Big Data applications. Three performance metrics, such as Root Mean Squared Error (RMSE), Determination coefficient (R2) for regression task, and Accuracy for classification tasks are utilized for the performance measure of the proposed MR-DWkNN algorithm. The extensive experimental results shows that there is an average increase of 3% to 4.5% prediction and classification performances as compared to standard distributed k-NN algorithm and a considerable decrease of Root Mean Squared Error (RMSE) with good parallelism characteristics of scalability and speedup thus, proves its effectiveness in Big Data predictive and classification applications.

Optimized machine learning models for natural fractures prediction using conventional well logs

Identifying and characterizing natural fractures is essential for understanding fluid flow and drainage in many oil and gas reservoirs, particularly carbonate. The presence of fractures often enhances fluid recovery but substantially complicates reservoir flow performance. Information from cores and formation imaging logs is constrained by their availability and cost. Reliably predicting fracture density using petrophysical logs and machine learning (ML) is therefore a desirable objective for fields with reservoirs displaying intermittent and sparsely distributed fractures. In the first step, Hybrid ML-optimizer models are developed and applied to a large, high- resolution, dataset (10 petrophysical variables; 3395 data records; ∼12% of the records displaying fractures) from the Asmari fractured carbonate reservoir in Iran’s Marun oil and gas field. Fracture density measured with a formation image log from one well is predicted by supervised learning using five hybrid models. The models use six of the ten petrophysical variables considered, based on feature selection, to predict fracture density. The selected variables are: corrected gamma ray (CGR), neutron porosity (NPHI); compressional sonic transition time (DT); interpreted sonic porosity (PHIS); bulk formation density (RHOB); and, the photoelectric absorption factor (PEF). The models enhance the performance of distance-weighted K-nearest neighbor (DWKNN) and neural network (MLP) with firefly and artificial bee colony optimizers. The novel firefly-KNN model achieves higher fracture density prediction accuracy than the other models. It is further refined by executing it in two layers: the first layer detects fractures; the second layer predicts fracture density. The double-layer-firefly-KNN model (DL-FF-DWKNN) achieves excellent prediction accuracy of fracture density for the Asmari carbonate (Precision = 99%, Recall = 97%, and F1-score = 98%). This correlation-free data matching technique substantially outperforms the correlation-dependent neural network models evaluated for this dataset with sparsely distributed fractured zones. The generalizability of the developed algorithms is tested with datasets from two other Marun field wells achieving prediction results that confirm high accuracy in fracture detection and prediction of fracture density (FVDC).

Intelligent Measurement of Coal Moisture Based on Microwave Spectrum via Distance-Weighted kNN

Realizing the rapid measurement of coal moisture content (MC) is of great significance. However, existing measurement methods are time-consuming and damage the original properties of the samples. To address these concerns, a coal MC intelligent measurement system is designed in this study that integrates microwave spectrum analysis and the distance-weighted k-nearest neighbor (DW-kNN) algorithm to realize rapid and non-destructive measurement of coal MC. Specifically, the measurement system is built using portable microwave analysis equipment, which can efficiently collect the microwave signals of coal. To improve the cleanliness of modeling data, an iterative clipping method based on Mahalanobis distance (MD-ICM) is used to detect and eliminate outliers. Based on multiple microwave frequency bands, various machine learning methods are evaluated, and it is found that coal MC measurement using broad frequency signals of 8.05–12.01 GHz yields the best results. Experiments are also carried out on coals from different regions to examine the regional robustness of the proposed method. The results of on-site testing with 27 additional samples show that the method based on the combination of microwave spectrum analysis and DW-kNN has a potential application prospect in the rapid measurement of coal MC.

Data driven models to predict pore pressure using drilling and petrophysical data

The mud weight window (MW) determination is one of the most important parameters in drilling oil and gas wells, where accurate design can secure the drilled well and deliver a stable borehole. In this paper, novel algorithms based on the most influential set of input features are developed to predict pore pressure, including rate of penetration (ROP), deep resistivity (ILD), density (RHOB), photoelectric index (PEF), corrected gamma ray (CGR), compression-wave velocity (Vp), weight on bit (WOB), shear-wave velocity (Vs) and pore compressibility (Cp). The algorithms used in this study are as follows: 1) machine learning algorithms (ML), these are the K-nearest neighbor (KNN) algorithm, weighted K-Nearest Neighbor (WKKNN), and distance weighted KNN (DWKNN); 2) hybrid machine learning algorithms (HML), which include the combination of three ML with particle swarm optimization (PSO) (KNN-PSO, WKNN-PSO and DWKNN-PSO). The 2875-record dataset used in this study was collected from three wells (S1, S2 and S3) in one of the gas reservoirs (Tabnak field) in Iran. After comparing the performance accuracy of all algorithms, DWKNN-PSO has the best performance accuracy compared to other algorithms presented in this paper (for the total dataset of wells S1 and S2: R2=0.9656 and RMSE = 12.6773 psi). Finally, the generalizability of the best predictive algorithm for PP, DWKNN-PSO, is evaluated by testing the proposed algorithm on an unseen dataset from another well (S3) in the field under study, where the DWKNN-PSO algorithm provides PP predictions in well S3 with high accuracy, R2 = 0.9765 and RMSE = 9.7545 psi, confirming its ability to be used in PP prediction in the studied field.

Generating accurate in silico predictions of acute aquatic toxicity for a range of organic chemicals: Towards similarity-based machine learning methods

There has been an increase in the use of non-animal approaches, such as in silico and/or in vitro methods, for assessing the risks of hazardous chemicals. A number of machine learning algorithms link molecular descriptors that interpret chemical structural properties with their biological activity. These computer-aided methods encounter several challenges, the most significant being the heterogeneity of datasets; more efficient and inclusive computational methods that are able to process large and heterogeneous chemical datasets are needed. In this context, this study verifies the utility of similarity-based machine learning methods in predicting the acute aquatic toxicity of diverse organic chemicals on Daphnia magna and Oryzias latipes. Two similarity-based methods were tested that employ a limited training dataset, most similar to a given fitting point, instead of using the entire dataset that encompasses a wide range of chemicals. The kernel-weighted local polynomial approach had a number of advantages over the distance-weighted k-nearest neighbor (k-NN) algorithm. The results highlight the importance of lipophilicity, electrophilic reactivity, molecular polarizability, and size in determining acute toxicity. The rigorous model validation ensures that this approach is an important tool for estimating toxicity in new or untested chemicals.

Prediction of oil flow rate through orifice flow meters: Optimized machine-learning techniques

Flow measurement is an essential requirement for monitoring and controlling oil movements through pipelines and facilities. However, delivering reliably accurate measurements through certain meters requires cumbersome calculations that can be simplified by using supervised machine learning techniques exploiting optimizers. In this study, a dataset of 6292 data records with seven input variables relating to oil flow through 40 pipelines plus processing facilities in southwestern Iran is evaluated with hybrid machine-learning-optimizer models to predict a wide range of oil flow rates (Qo) through orifice plate meters. Distance-weighted K-nearest-neighbor (DWKNN) and multi-layer perceptron (MLP) algorithms are coupled with artificial-bee colony (ABC) and firefly (FF) swarm-type optimizers. The two-stage ABC-DWKNN Plus MLP-FF model achieved the highest prediction accuracy (root mean square errors = 8.70 stock-tank barrels of oil per day) for oil flow rate through the orifice plates, thereby removing dependence on unreliable empirical formulas in such flow calculations.

Shear modulus prediction of embedded pressurized salt layers and pinpointing zones at risk of casing collapse in oil and gas wells

Rock shear and strain behaviors deep in the sub-surface are dependent on several geomechanical variables. These include various types of stress and rock formation moduli. Deformations of specific rock formations are usually quantified in a series of laboratory tests, which are costly and time-consuming and can only be performed on the few rock formation samples and cores available. Consequently, indirect estimates of geomechanical metrics from well-log data are required to measure variations throughout the entire length of a wellbore. Such measurements and detailed geomechanical models are required to identify zones at risk of casing collapse due to shear stress buildups. Such zones tend to occur in problematic ductile formations such as salt and swelling clays that are prone to creep. Machine-learning algorithms coupled with optimizers, and calibrated with a few mechanical rock test measurements, offer a practical method for predicting geomechanical characteristics along an entire wellbore. Hybrid models combining the optimizers, imperialist competitive (ICA) and gravitation search (GSA) algorithms, with the machine-learning algorithms, distance-weighted K-nearest neighbor (DWKNN) and multi-layer perceptron (MLP) are shown to achieve high prediction accuracy of shear modulus (Gs) from a large dataset (22,325 data records) crossing the problematic salt-rich Gachsaran formation in a wellbore from the Marun oil field (Iran). ICA-MLP, GSA-MLP, ICA-DWKNN and GSA-DWKNN models applied to four independent variables recorded by well logs were calibrated with some laboratory-derived Gs measurements. These models predict Gs with high levels of accuracy achieving coefficients of determination (R2) > 0.96. The GSA-DWKNN hybrid algorithm (R2 = 1) provided the highest prediction accuracy. Gs predictions can then be used to accurately identify zones at risk of casing collapse.

Identifying GPCR-drug interaction based on wordbook learning from sequences

BackgroundG protein-coupled receptors (GPCRs) mediate a variety of important physiological functions, are closely related to many diseases, and constitute the most important target family of modern drugs. Therefore, the research of GPCR analysis and GPCR ligand screening is the hotspot of new drug development. Accurately identifying the GPCR-drug interaction is one of the key steps for designing GPCR-targeted drugs. However, it is prohibitively expensive to experimentally ascertain the interaction of GPCR-drug pairs on a large scale. Therefore, it is of great significance to predict the interaction of GPCR-drug pairs directly from the molecular sequences. With the accumulation of known GPCR-drug interaction data, it is feasible to develop sequence-based machine learning models for query GPCR-drug pairs.ResultsIn this paper, a new sequence-based method is proposed to identify GPCR-drug interactions. For GPCRs, we use a novel bag-of-words (BoW) model to extract sequence features, which can extract more pattern information from low-order to high-order and limit the feature space dimension. For drug molecules, we use discrete Fourier transform (DFT) to extract higher-order pattern information from the original molecular fingerprints. The feature vectors of two kinds of molecules are concatenated and input into a simple prediction engine distance-weighted K-nearest-neighbor (DWKNN). This basic method is easy to be enhanced through ensemble learning. Through testing on recently constructed GPCR-drug interaction datasets, it is found that the proposed methods are better than the existing sequence-based machine learning methods in generalization ability, even an unconventional method in which the prediction performance was further improved by post-processing procedure (PPP).ConclusionsThe proposed methods are effective for GPCR-drug interaction prediction, and may also be potential methods for other target-drug interaction prediction, or protein-protein interaction prediction. In addition, the new proposed feature extraction method for GPCR sequences is the modified version of the traditional BoW model and may be useful to solve problems of protein classification or attribute prediction. The source code of the proposed methods is freely available for academic research at https://github.com/wp3751/GPCR-Drug-Interaction.

Offloading Decision Algorithm Based on Distance Weighted K-Nearest Neighbor in Power Internet of Things

Offloading Decision Algorithm Based on Distance Weighted K-Nearest Neighbor in Power Internet of Things

A Novel Regression Modeling Method for PMSLM Structural Design Optimization Using a Distance-Weighted KNN Algorithm

This paper investigates the modeling methods for the structural design optimization of permanent magnet synchronous linear motors (PMSLMs), which are applied to linear motion machines. First, an initial PMSLM model is established by finite element analysis (FEA), and the data space is obtained for subsequent rapid regression modeling. Second, a powerful regression machine learning method, that is, distance-weighted K-nearest neighbor algorithm (DW-KNNA), is introduced to establish a calculation model by grasping the nonlinear relationships between input structural parameters and output motor performances. Model comparison experiments among analytical method, response surface method, K-nearest neighbor algorithm, and DW-KNNA demonstrate the superiority of the DW-KNNA. Finally, the differential evolution algorithm is used to determine the best combination of structural parameters and performances. The FEA and prototype motor experiments proved the validity and advantages of the proposed method.

Comparison of Different Classification Algorithms for Fault Detection and Fault Isolation in Complex Systems

Due to the lack of sufficient results seen in literature, feature extraction and classification methods of hydraulic systems appears to be somewhat challenging. This paper compares the performance of three classifiers (namely linear support vector machine (SVM), distance-weighted k-nearest neighbor (WKNN), and decision tree (DT) using data from optimized and non-optimized sensor set solutions. The algorithms are trained with known data and then tested with unknown data for different scenarios characterizing faults with different degrees of severity. This investigation is based solely on a data-driven approach and relies on data sets that are taken from experiments on the fuel system. The system that is used throughout this study is a typical fuel delivery system consisting of standard components such as a filter, pump, valve, nozzle, pipes, and two tanks. Running representative tests on a fuel system are problematic because of the time, cost, and reproduction constraints involved in capturing any significant degradation. Simulating significant degradation requires running over a considerable period; this cannot be reproduced quickly and is costly.

Development of a Novel Gas Sensing Algorithm Based on Impedance Spectroscopy

A novel pattern recognition algorithm is developed, which detects concentration of different gases automatically with impedance spectroscopy (IS) using a conducting poly sensor or a WO3 sensor. It consists of adaptive- simulated-annealing-supported-parameter-estimation (ASA-PE) for feature-extraction and committee-machine (CM) for classification. The results of ASA-PE, complex-principle-component-analysis (CPCA) and discriminant- analysis-via-support-vector (SVDA) are compared. All algorithms are satisfied. But ASA-PE ensures minimal loss of information. The classifiers: distance-weighted-k-nearest-neighbor (DW-kNN), multiple-layer-perceptron (MLP), support-vector-machine (SVM) and CM are combined with feature-extraction methods and compared with each other. Recognition result is satisfying. However, if gas or gas condition is unknown, only CM with SVDA showed best performance.

Optimization of reference library used in content-based medical image retrieval scheme.

Building an optimal image reference library is a critical step in developing the interactive computer-aided detection and diagnosis (I-CAD) systems of medical images using content-based image retrieval (CBIR) schemes. In this study, the authors conducted two experiments to investigate (1) the relationship between I-CAD performance and size of reference library and (2) a new reference selection strategy to optimize the library and improve I-CAD performance. The authors assembled a reference library that includes 3153 regions of interest (ROI) depicting either malignant masses (1592) or CAD-cued false-positive regions (1561) and an independent testing data set including 200 masses and 200 false-positive regions. A CBIR scheme using a distance-weighted K-nearest neighbor algorithm is applied to retrieve references that are considered similar to the testing sample from the library. The area under receiver operating characteristic curve (Az) is used as an index to evaluate the I-CAD performance. In the first experiment, the authors systematically increased reference library size and tested I-CAD performance. The result indicates that scheme performance improves initially from Az= 0.715 to 0.874 and then plateaus when the library size reaches approximately half of its maximum capacity. In the second experiment, based on the hypothesis that a ROI should be removed if it performs poorly compared to a group of similar ROIs in a large and diverse reference library, the authors applied a new strategy to identify "poorly effective" references. By removing 174 identified ROIs from the reference library, I-CAD performance significantly increases to Az = 0.914 (p < 0.01). The study demonstrates that increasing reference library size and removing poorly effective references can significantly improve I-CAD performance.

Classification of symbolic objects: A lazy learning approach

Symbolic data analysis aims at generalizing some standard statistical data mining methods, such as those developed for classification tasks, to the case of symbolic objects (SOs). These objects synthesize information concerning a group of individuals of a population, eventually stored in a relational database, and ensure confidentiality of original data. Classifying SOs is an important task in symbolic data analysis. In this paper a lazy-learning approach that extends a traditional distance weighted k-Nearest Neighbor classification algorithm to SOs, is presented. The proposed method has been implemented in the system SO-NN (Symbolic Objects Nearest Neighbor) and evaluated on symbolic datasets.

A Framework for the Initialization of Student Models in Web-based Intelligent Tutoring Systems

Initializing a student model for individualized tutoring in educational applications is a difficult task, since very little is known about a new student. On the other hand, fast and efficient initialization of the student model is necessary. Otherwise the tutoring system may lose its credibility in the first interactions with the student. In this paper we describe a framework for the initialization of student models in Web-based educational applications. The framework is called ISM. The basic idea of ISM is to set initial values for all aspects of student models using an innovative combination of stereotypes and the distance weighted k-nearest neighbor algorithm. In particular, a student is first assigned to a stereotype category concerning her/his knowledge level of the domain being taught. Then, the model of the new student is initialized by applying the distance weighted k-nearest neighbor algorithm among the students that belong to the same stereotype category with the new student. ISM has been applied in a language learning system, which has been used as a test-bed. The quality of the student models created using ISM has been evaluated in an experiment involving classroom students and their teachers. The results from this experiment showed that the initialization of student models was improved using the ISM framework.

Imputing tree-lists from aerial attributes for complex stands of south-eastern British Columbia

The nearest neighbor, k-nearest neighbors, distance-weighted k-nearest neighbor, and class-weighted k-nearest neighbor imputation methods were compared for accuracy in estimating tree-lists (list of species and diameter for each tree) from aerial attributes for complex stands, with up to nine species and a wide range of sizes, in south-eastern British Columbia, Canada. For the four imputation methods, the most similar neighbor distance metric was used, and three neighbors were used for the k-nearest neighbor methods. Ground variables used to represent the tree-list included the number of trees per hectare by species, ranges of diameter by species, and basal area per hectare. Aerial variables included species composition, crown closure (%), elevation, biogeoclimatic ecosystem classification (BEC) zones, height, age, and site class. Sample data were divided, and the imputation methods were compared for accuracy using observed and estimated species composition, stand tables, basal area, and volume per hectare. Also, the imputed tree-list was used to predict yield using a stand level growth model, and this predicted yield was compared to the yield obtained using the actual tree-list. Of the four approaches used, the nearest neighbor was marginally better, but the methods that averaged the three nearest neighbors were somewhat better for the distribution of stems per hectare by diameter for the more sparse hardwood species. Of the three averaging methods, weighting by similarity of the species composition and the BEC zone provided better results. In using the estimated trees lists in a growth and yield model, the average volumes were reasonable at the beginning and end of the period for all methods. However, the volumes for a particular stand could be quite different than that obtained for an observed tree-list.

Estimating defoliation in boreal coniferous forests by combining Landsat TM, aerial photographs and field data

Multitemporal Landsat TM images, panchromatic high altitude aerial photographs and field sample plots of the Finnish national forest inventory (NFI) were used for estimating defoliation in boreal coniferous forests. Satellite images were used for spectral information, aerial photographs for textural features and the NFI sample plots were utilized as reference data of defoliation. A distance weighted k-nearest neighbor classification was applied in the estimation. The defoliation was estimated as a function of distance between the spectral and textural responses of pixels in the inventory area and pixels corresponding to the reference data. Two different spatial scales, plot and stand levels, were examined. The plot level examination was implemented both as single date estimation and estimation of defoliation change. In the plot level, the producer’s accuracy of the three change classes was 26 and 56% overall (Kappa 0.12) for plot and stand levels. When the estimation was done with the changed sample plots only, the accuracy was 61% (Kappa 0.34). The results of the corresponding plot level single date estimation were slightly poorer. The stand level examination was done only with single date estimation. Using the NFI sample plots as reference data, defoliation was estimated for each test stand and compared to the field assessment. When three defoliation classes (none, slight and severe) were applied, the overall accuracy for the test stands was 42% (Kappa 0.11) and with two classes (no defoliation and defoliation) the accuracy was 58% (Kappa 0.15). The most useful variables for estimating defoliation were TM channels 4 and 5.

A Re-Examination of the Distance-Weighted k-Nearest Neighbor Classification Rule

It was previously proved by Bailey and Jain that the asymptotic classification error rate of the (unweighted) k-nearest neighbor (k-NN) rule is lower than that of any weighted k-NN rule. Equations are developed for the classification error rate of a test sample when the number of training samples is finite, and it is argued intuitively that a weighted rule may then in some cases achieve a lower error rate than the unweighted rule. This conclusion is confirmed by analytically solving a particular simple problem, and as an illustration, experimental results are presented that were obtained using a generalized form of a weighting function proposed by Dudani.

A Reappraisal of Distance-Weighted K-Nearest Neighbor Classification for Pattern Recognition with Missing Data

A Reappraisal of Distance-Weighted K-Nearest Neighbor Classification for Pattern Recognition with Missing Data