Fuzzy K-nearest Neighbor Algorithm Research Articles

A fuzzy KNN-based model for significant wave height prediction in large lakes

Summary Some algorithms based on fuzzy set theory (FST) such as fuzzy inference system (FIS) and adaptive-network-based fuzzy inference system (ANFIS) have been successfully applied to significant wave height (SWH) prediction. In this paper, perhaps for the first time, the fuzzy K-nearest neighbor (FKNN) algorithm is utilized to develop a fuzzy wave height prediction model for large lakes, where the fetch length depends on the wind direction. As fetch length (or wind direction) can affect the wave height in lakes, this variable is also considered as one of the inputs of the prediction model. The results of the FKNN model are compared with those of some soft computing techniques such as Bayesian networks (BNs), regression tree induction (named M5P), and support vector regression (SVR). The developed FKNN model is used for SWH prediction in the western part of Lake Superior in North America. The results show that the FKNN and M5P model can outperform the other soft computing techniques.

A novel fuzzy K-nearest neighbor inference model with differential evolution for spatial prediction of rainfall-induced shallow landslides in a tropical hilly area using GIS

This research represents a novel soft computing approach that combines the fuzzy k-nearest neighbor algorithm (fuzzy k-NN) and the differential evolution (DE) optimization for spatial prediction of rainfall-induced shallow landslides at a tropical hilly area of Quy Hop, Vietnam. According to current literature, the fuzzy k-NN and the DE optimization are current state-of-the-art techniques in data mining that have not been used for prediction of landslide. First, a spatial database was constructed, including 129 landslide locations and 12 influencing factors, i.e., slope, slope length, aspect, curvature, valley depth, stream power index (SPI), sediment transport index (STI), topographic ruggedness index (TRI), topographic wetness index (TWI), Normalized Difference Vegetation Index (NDVI), lithology, and soil type. Second, 70 % landslide locations were randomly generated for building the landslide model whereas the remaining 30 % landslide locations was for validating the model. Third, to construct the landslide model, the DE optimization was used to search the optimal values for fuzzy strength (fs) and number of nearest neighbors (k) that are the two required parameters for the fuzzy k-NN. Then, the training process was performed to obtain the fuzzy k-NN model. Value of membership degree of the landslide class for each pixel was extracted to be used as landslide susceptibility index. Finally, the performance and prediction capability of the landslide model were assessed using classification accuracy, the area under the ROC curve (AUC), kappa statistics, and other evaluation metrics. The result shows that the fuzzy k-NN model has high performance in the training dataset (AUC = 0.944) and validation dataset (AUC = 0.841). The result was compared with those obtained from benchmark methods, support vector machines and J48 decision trees. Overall, the fuzzy k-NN model performs better than the support vector machines and the J48 decision trees models. Therefore, we conclude that the fuzzy k-NN model is a promising prediction tool that should be used for susceptibility mapping in landslide-prone areas.

Biometric cattle identification approach based on Weber’s Local Descriptor and AdaBoost classifier

In this paper, we proposed a new and robust biometric-based approach to identify head of cattle. This approach used the Weber Local Descriptor (WLD) to extract robust features from cattle muzzle print images (images from 31 head of cattle were used). It also employed the AdaBoost classifier to identify head of cattle from their WLD features. To validate the results obtained by this classifier, other two classifiers (k-Nearest Neighbor (k-NN) and Fuzzy-k-Nearest Neighbor (Fk-NN)) were used. The experimental results showed that the proposed approach achieved a promising accuracy result (approximately 99.5%) which is better than existed proposed solutions. Moreover, to evaluate the results of the proposed approach, four different assessment methods (Area Under Curve (AUC), Sensitivity and Specificity, accuracy rate, and Equal Error Rate (EER)) were used. The results of all these methods showed that the WLD along with AdaBoost algorithm gave very promising results compared to both of the k-NN and Fk-NN algorithms.

Evolutionary fuzzy k-nearest neighbors algorithm using interval-valued fuzzy sets

One of the most known and effective methods in supervised classification is the k-nearest neighbors classifier. Several approaches have been proposed to enhance its precision, with the fuzzy k-nearest neighbors (fuzzy-kNN) classifier being among the most successful ones. However, despite its good behavior, fuzzy-kNN lacks of a method for properly defining several mechanisms regarding the representation of the relationship between the instances and the classes of the classification problems. Such a method would be very desirable, since it would potentially lead to an improvement in the precision of the classifier.In this work we present a new approach, evolutionary fuzzy k-nearest neighbors classifier using interval-valued fuzzy sets (EF-kNN-IVFS), incorporating interval-valued fuzzy sets for computing the memberships of training instances in fuzzy-kNN. It is based on the representation of multiple choices of two key parameters of fuzzy-kNN: one is applied in the definition of the membership function, and the other is used in the computation of the voting rule. Besides, evolutionary search techniques are incorporated to the model as a self-optimization procedure for setting up these parameters. An experimental study has been carried out to assess the capabilities of our approach. The study has been validated by using nonparametric statistical tests, and remarks the strong performance of EF-kNN-IVFS compared with several state of the art techniques in fuzzy nearest neighbor classification.

Combination of Complementary Features for Automatic Image Annotation

Image annotation is a method for representing an image with a suitable keyword closer to its semantic concept. Automatically assigning relevant text keywords to image is an important problem. Many algorithms and combination of different features have been proposed in the past and achieved good performance. Efforts have focused upon many other fields and some predefined set of features in the area of Automatic image annotation. But properties of features and their complementing combinations have not been well investigated. In this paper the performance of different feature combinations are compared, and find out the one which outperforms the other combinations by applying the Fuzzy Knearest neighbor algorithm as the classification method. General Terms Computer vision, Image Processing.

Fuzzy local discriminant embedding for image feature extraction

In pattern recognition, feature extraction techniques have been widely employed to reduce the high dimensionality of data. In this paper, we propose a novel algorithm called fuzzy local discriminant embedding (FLDE) based on the unsupervised discriminant projection criterion and fuzzy set theory for image feature extraction and recognition. In the proposed method, a membership degree matrix is firstly calculated using the fuzzy k-nearest neighbor (FKNN) algorithm, and then the membership degree and the label information are incorporated into the definition of the weighted matrices to get the fuzzy local scatter and fuzzy nonlocal scatter. After characterizing the fuzzy nonlocal scatter and the fuzzy local scatter, a concise feature extraction criterion is derived via maximizing the ratio between them. Experimental results on the ORL, FERET, and CMU PIE face databases show the effectiveness of the proposed method.

A Swarm-Optimized Fuzzy Instance-based Learning approach for predicting slope collapses in mountain roads

Due to the disastrous consequences of slope failures, forecasting their occurrences is a practical need of government agencies to develop strategic disaster prevention programs. This research proposes a Swarm-Optimized Fuzzy Instance-based Learning (SOFIL) model for predicting slope collapses. The proposed model utilizes the Fuzzy k-Nearest Neighbor (FKNN) algorithm as an instance-based learning method to predict slope collapse events. Meanwhile, to determine the model’s hyper-parameters appropriately, the Firefly Algorithm (FA) is employed as an optimization technique. Experimental results have pointed out that the newly established SOFIL can outperform other benchmarking algorithms. Therefore, the proposed model is very promising to help decision-makers in coping with the slope collapse prediction problem.

IGPCR-drug: a web server for predicting interaction between GPCRs and drugs in cellular networking.

Involved in many diseases such as cancer, diabetes, neurodegenerative, inflammatory and respiratory disorders, G-protein-coupled receptors (GPCRs) are among the most frequent targets of therapeutic drugs. It is time-consuming and expensive to determine whether a drug and a GPCR are to interact with each other in a cellular network purely by means of experimental techniques. Although some computational methods were developed in this regard based on the knowledge of the 3D (dimensional) structure of protein, unfortunately their usage is quite limited because the 3D structures for most GPCRs are still unknown. To overcome the situation, a sequence-based classifier, called “iGPCR-drug”, was developed to predict the interactions between GPCRs and drugs in cellular networking. In the predictor, the drug compound is formulated by a 2D (dimensional) fingerprint via a 256D vector, GPCR by the PseAAC (pseudo amino acid composition) generated with the grey model theory, and the prediction engine is operated by the fuzzy K-nearest neighbour algorithm. Moreover, a user-friendly web-server for iGPCR-drug was established at http://www.jci-bioinfo.cn/iGPCR-Drug/. For the convenience of most experimental scientists, a step-by-step guide is provided on how to use the web-server to get the desired results without the need to follow the complicated math equations presented in this paper just for its integrity. The overall success rate achieved by iGPCR-drug via the jackknife test was 85.5%, which is remarkably higher than the rate by the existing peer method developed in 2010 although no web server was ever established for it. It is anticipated that iGPCR-Drug may become a useful high throughput tool for both basic research and drug development, and that the approach presented here can also be extended to study other drug – target interaction networks.

ICDI-PseFpt: Identify the channel–drug interaction in cellular networking with PseAAC and molecular fingerprints

Many crucial functions in life, such as heartbeat, sensory transduction and central nervous system response, are controlled by cell signalings via various ion channels. Therefore, ion channels have become an excellent drug target, and study of ion channel–drug interaction networks is an important topic for drug development. However, it is both time-consuming and costly to determine whether a drug and a protein ion channel are interacting with each other in a cellular network by means of experimental techniques. Although some computational methods were developed in this regard based on the knowledge of the 3D (three-dimensional) structure of protein, unfortunately their usage is quite limited because the 3D structures for most protein ion channels are still unknown. With the avalanche of protein sequences generated in the post-genomic age, it is highly desirable to develop the sequence-based computational method to address this problem. To take up the challenge, we developed a new predictor called iCDI-PseFpt, in which the protein ion-channel sample is formulated by the PseAAC (pseudo amino acid composition) generated with the gray model theory, the drug compound by the 2D molecular fingerprint, and the operation engine is the fuzzy K-nearest neighbor algorithm. The overall success rate achieved by iCDI-PseFpt via the jackknife cross-validation was 87.27%, which is remarkably higher than that by any of the existing predictors in this area. As a user-friendly web-server, iCDI-PseFpt is freely accessible to the public at the website http://www.jci-bioinfo.cn/iCDI-PseFpt/. Furthermore, for the convenience of most experimental scientists, a step-by-step guide is provided on how to use the web-server to get the desired results without the need to follow the complicated math equations presented in the paper just for its integrity. It has not escaped our notice that the current approach can also be used to study other drug–target interaction networks.

Predict Drug-Protein Interaction in Cellular Networking

Involved with many diseases such as cancer, diabetes, neurodegenerative, inflammatory and respiratory disorders, GPCRs (G-protein-coupled receptors) are the most frequent targets for drug development: over 50% of all prescription drugs currently on the market are actually acting by targeting GPCRs directly or indirectly. Found in every living thing and nearly all cells, ion channels play crucial roles for many vital functions in life, such as heartbeat, sensory transduction, and central nervous system response. Their dysfunction may have significant impact to human health, and hence ion channels are deemed as "the next GPCRs". To develop GPCR-targeting or ion-channel-targeting drugs, the first important step is to identify the interactions between potential drug compounds with the two kinds of protein receptors in the cellular networking. In this minireview, we are to introduce two predictors. One is called iGPCR-Drug accessible at http://www.jci-bioinfo.cn/iGPCR-Drug/; the other called iCDI-PseFpt at http://www.jci-bioinfo.cn/iCDI-PseFpt. The former is for identifying the interactions of drug compounds with GPCRs; while the latter for that with ion channels. In both predictors, the drug compound was formulated by the two-dimensional molecular fingerprint, and the protein receptor by the pseudo amino acid composition generated with the grey model theory, while the operation engine was the fuzzy K-nearest neighbor algorithm. For the convenience of most experimental pharmaceutical and medical scientists, a step-bystep guide is provided on how to use each of the two web-servers to get the desired results without the need to follow the complicated mathematics involved originally for their establishment.

IAMP-2L: A two-level multi-label classifier for identifying antimicrobial peptides and their functional types

Antimicrobial peptides (AMPs), also called host defense peptides, are an evolutionarily conserved component of the innate immune response and are found among all classes of life. According to their special functions, AMPs are generally classified into ten categories: Antibacterial Peptides, Anticancer/tumor Peptides, Antifungal Peptides, Anti-HIV Peptides, Antiviral Peptides, Antiparasital Peptides, Anti-protist Peptides, AMPs with Chemotactic Activity, Insecticidal Peptides, and Spermicidal Peptides. Given a query peptide, how can we identify whether it is an AMP or non-AMP? If it is, can we identify which functional type or types it belong to? Particularly, how can we deal with the multi-type problem since an AMP may belong to two or more functional types? To address these problems, which are obviously very important to both basic research and drug development, a multi-label classifier was developed based on the pseudo amino acid composition (PseAAC) and fuzzy K-nearest neighbor (FKNN) algorithm, where the components of PseAAC were featured by incorporating five physicochemical properties. The novel classifier is called iAMP-2L, where “2L” means that it is a 2-level predictor. The 1st-level is to answer the 1st question above, while the 2nd-level is to answer the 2nd and 3rd questions that are beyond the reach of any existing methods in this area. For the conveniences of users, a user-friendly web-server for iAMP-2L was established at http://www.jci-bioinfo.cn/iAMP-2L.

IEzy-drug: a web server for identifying the interaction between enzymes and drugs in cellular networking.

With the features of extremely high selectivity and efficiency in catalyzing almost all the chemical reactions in cells, enzymes play vitally important roles for the life of an organism and hence have become frequent targets for drug design. An essential step in developing drugs by targeting enzymes is to identify drug-enzyme interactions in cells. It is both time-consuming and costly to do this purely by means of experimental techniques alone. Although some computational methods were developed in this regard based on the knowledge of the three-dimensional structure of enzyme, unfortunately their usage is quite limited because three-dimensional structures of many enzymes are still unknown. Here, we reported a sequence-based predictor, called “iEzy-Drug,” in which each drug compound was formulated by a molecular fingerprint with 258 feature components, each enzyme by the Chou's pseudo amino acid composition generated via incorporating sequential evolution information and physicochemical features derived from its sequence, and the prediction engine was operated by the fuzzy K-nearest neighbor algorithm. The overall success rate achieved by iEzy-Drug via rigorous cross-validations was about 91%. Moreover, to maximize the convenience for the majority of experimental scientists, a user-friendly web server was established, by which users can easily obtain their desired results.

Using Grey System GM(1,1) Model to Predict the Drug-GPCRs Couples

It is an important topic to find and identify molecular drug targets for modern drug research. Naturally, to find new or potential drug-GPCRs couples is very useful for determining the new effective medicine target, because GPCRs are among the most frequent targets of therapeutic drugs and proven evolutionary pharmacology value. To realize this, this paper incorporates chemical functional groups of drugs and grey model GM(1,1) based on digital coding of amino acids to formulate the drug-GPCRs couples for statistical prediction. The overall success rate using the fuzzy K-Nearest Neighbor algorithm by the jackknife test is about 81%. This novel approach will further stimulate the development of scanning target via a computational approach.

Discriminating Outer Membrane Proteins with Fuzzy K-Nearest Neighbor Algorithms Based on the General Form of Chou’s PseAAC

Outer membrane proteins (OMPs) play important roles in cell biology. In addition, OMPs are targeted by multiple drugs. The identification of OMPs from genomic sequences and successful prediction of their secondary and tertiary structures is a challenging task due to short membrane-spanning regions with high variation in properties. Therefore, an effective and accurate silico method for discrimination of OMPs from their primary sequences is needed. In this paper, we have analyzed the performance of various machine learning mechanisms for discriminating OMPs such as: Genetic Programming, K-nearest Neighbor, and Fuzzy K-nearest Neighbor (Fuzzy K-NN) in conjunction with discrete methods such as: Amino acid composition, Amphiphilic Pseudo amino acid composition, Split amino acid composition (SAAC), and hybrid versions of these methods. The performance of the classifiers is evaluated by two datasets using 5-fold crossvalidation. After the simulation, we have observed that Fuzzy K-NN using SAAC based-features makes it quite effective in discriminating OMPs. Fuzzy K-NN achieves the highest success rates of 99.00% accuracy for discriminating OMPs from non-OMPs and 98.77% and 98.28% accuracies from α-helix membrane and globular proteins, respectively on dataset1. While on dataset2, Fuzzy K-NN achieves 99.55%, 99.90%, and 99.81% accuracies for discriminating OMPs from non- OMPs, α-helix membrane, and globular proteins, respectively. It is observed that the classification performance of our proposed method is satisfactory and is better than the existing methods. Thus, it might be an effective tool for high throughput innovation of OMPs.

Face recognition algorithm fusing 2DPCA and fuzzy 2DLDA

A new face image feature extraction method which combined fuzzy set theory and Two-Dimentional Two-Dimentional Principal Component Analysis-Linear Discriminant Analysis((2D)2PCALDA) was proposed.Firstly,Two-Dimentional Principal Component Analysis(2DPCA) was used to extract the optimal projective vectors from face image.Then,the membership degree matrix was calculated by fuzzy K-nearest neighbor algorithm,and it was merged into the process of Two-Dimentional Linear Discriminant Analysis(2DLDA).Finally,fuzziness between-class scatter matrix and fuzziness within-class scatter matrix were obtained.Compared with(2D)2PCALDA,the method made full use of the advantages of(2D)2PCALDA.It not only effectively extracted the row and column recognition information,but also made full use of the distribution information of samples.The experiments based on Yale and FERET face databases show that the method can have better recognition effect than(2D)2PCALDA and(2D)2PCA.

Image based diagnostic aid system for interstitial lung diseases

Automatic classification of lung tissue patterns in high-resolution computed tomography (HRCT) images of patients affected with interstitial lung diseases (ILD) is an important stage in the construction of a computer-aided diagnosis system. In this study, we propose a new image based system for classification of lung tissue patterns. The proposed system comprises three stages. In the first stage, the parenchyma region in HRCT lung images is separated using a set of thresholding, filtering and morphological operators. In the second stage, two sets of overcomplete wavelet filters, namely discrete wavelet frames and rotated wavelet frames, are utilized to extract features from defined regions of interest (ROIs) within parenchyma. Then, in the third stage, the fuzzy k-nearest neighbor algorithm is employed to perform the pattern classification. The proposed method is tested for classifying four different lung tissue patterns (ground glass, honeycombing, reticular, and normal) selected from a database of 339 images from 17 subjects. After applying our technique to classify these patterns in isolated ROIs, we extend the classification scheme to the whole lung in order to produce quantitative scores of abnormalities in lung parenchyma of patients. The performance of the proposed method is compared with two state-of-the-art texture based methods for lung tissue characterization and is also validated against experienced observers. The average kappa statistic of the agreement between two radiologists and the computer was found to be 0.6543 where as the average kappa statistic for the inter-observer agreement was 0.6848. We also performed an experiment to show the correlation between pulmonary function test parameters and quantitative scores of computerized system. Results show that extent of HRCT findings correlates significantly with functional impairment. The computer system is shown to approach the performance of the expert observers in diagnosing regions of interest and can help to produce objective measures of abnormal patterns in lung HRCT images.

A Multi-Fault Diagnosis Method of Rolling Bearing Based on Wavelet-PCA and Fuzzy <i>K</i>-Nearest Neighbor

Rolling bearings are widely used in various areas including aircraft, mining, manufacturing, and agriculture, etc. The breakdowns of the rotational machinery resulted from the rolling bearing failures account for 30%. It is therefore imperative to monitor the rolling bearing conditions in time in order to prevent the malfunctions of the plants. In the present paper is described a fault detection and diagnosis technique for rolling bearing multi-faults based on wavelet-principle component analysis (PCA) and fuzzy k-nearest neighbor (FKNN). In the diagnosis process, the wavelet analysis was firstly employed to decompose the vibration data of the rolling bearings under eight different operating conditions, and for each sample its energy of each sub-band was calculated to obtain the original feature space. Then, the PCA was used to reduce the dimensionality of the original feature vector and hence the most important features could be gotten. Lastly, the FKNN algorithm was employed in the pattern recognition to identify the conditions of the bearings of interest. The experimental results suggest that the sensitive fault features can be extracted efficiently after the wavelet-PCA processing, and the proposed diagnostic system is effective for the rolling bearing multi-fault diagnosis. In addition, the proposed method can achieve higher performance than that without PCA with respect to the classification rate.

Gear Multi-Faults Diagnosis of a Rotating Machinery Based on Independent Component Analysis and Fuzzy <i>K</i>-Nearest Neighbor

Gearboxes are extensively used in various areas including aircraft, mining, manufacturing, and agriculture, etc. The breakdowns of the gearbox are mostly caused by the gear failures. It is therefore crucial for engineers and researchers to monitor the gear conditions in time in order to prevent the malfunctions of the plants. In this paper, a condition monitoring and faults identification technique for rotating machineries based on independent component analysis (ICA) and fuzzy k-nearest neighbor (FKNN) is described. In the diagnosis process, the ICA was initially employed to separate characteristic vibration signal and interference vibration signal from the parallel time series obtained from multi-channel accelerometers mounted on different positions of the gearbox. The wavelet transform (WT) and autoregressive (AR) model method then were performed as the feature extraction technique to attain the original feature vector of the characteristic signal. Meanwhile, the ICA was used again to reduce the dimensionality of the original feature vector. Hence, the useless information in the feature vector could be removed. Finally, the FKNN algorithm was implemented in the pattern recognition process to identify the conditions of the gears of interest. The experimental results suggest that the sensitive fault features can be extracted efficiently after the ICA processing, and the proposed diagnostic system is effective for the gear multi-faults diagnosis, including the gear crack failure, pitting failure, gear tooth broken, compound fault of wear and spalling, etc. In addition, the proposed method can achieve higher performance than that without ICA processing with respect to the classification rate.

A SOFT-LINK SPECTRAL MODEL FOR LINK PREDICTION

Unsupervised spectral clustering methods can yield good performance when identifying crisp clusters with low complexity since the learning algorithm does not rely on finding the local minima of an objective function and rather uses spectral properties of the graph. Nonetheless, the performance of such approaches are usually affected by their uncertain parameters. Using the underlying structure of a general spectral clustering method, in this paper a new soft-link spectral clustering algorithm is introduced to identify clusters based on fuzzy k-nearest neighbor approach. We construct a soft weight matrix of a graph by identifying the upper and lower boundaries of learning parameters of the similarity function, specifically the fuzzifier parameter (fuzziness) of the Fuzzy k-Nearest Neighbor algorithm. The algorithm allows perturbations on the graph Laplace during the learning stage by the changes on such learning parameters. With the empirical analysis using an artificial and a real textual entailment dataset, we demonstrate that our initial hypothesis of implementing soft links for spectral clustering can improve the classification performance of final outcome.

An information‐theoretic approach to the prediction of protein structural class

An information-theoretical approach, which combines a sequence decomposition technique and a fuzzy clustering algorithm, is proposed for prediction of protein structural class. This approach could bypass the process of selecting and comparing sequence features as done previously. First, distances between each pair of protein sequences are estimated using a conditional decomposition technique in information theory. Then, the fuzzy k-nearest neighbor algorithm is used to identify the structural class of a protein given as set of sample sequences. To verify the strength of our method, we choose three widely used datasets constructed by Chou and Zhou. It is shown by the Jackknife test that our approach represents an improvement in the prediction of accuracy over existing methods.