Value Difference Metric Research Articles

DAPLSR: Data Augmentation Partial Least Squares Regression Model via Manifold Optimization

Traditional Partial Least Squares Regression (PLSR) models frequently underperform when handling data characterized by uneven categories. To address the issue, this paper proposes a Data Augmentation Partial Least Squares Regression (DAPLSR) model via manifold optimization. The DAPLSR model introduces the Synthetic Minority Over-sampling Technique (SMOTE) to increase the number of samples and utilizes the Value Difference Metric (VDM) to select the nearest neighbor samples that closely resemble the original samples for generating synthetic samples. In solving the model, in order to obtain a more accurate numerical solution for PLSR, this paper proposes a manifold optimization method that uses the geometric properties of the constraint space to improve model degradation and optimization. Comprehensive experiments show that the proposed DAPLSR model achieves superior classification performance and outstanding evaluation metrics on various datasets, significantly outperforming existing methods.

Weighting features by the value displacement rebound

Learning from examples draws on similarity, a concept which formalisation leads to the notion of instance space. Continuous spaces are easier to embrace since, unlike discrete, they often can be seen as hyper-constructs of 3D. Unsurprisingly, the instance-based learning methods are more developed for continuous domains than for discrete ones. The value difference metric (VDM) is one of the few examples of metrics for discrete spaces. Mixed reports about utility of VDM exist. In this paper VDM is compared with another approach where data features are weighted by the Information Gain. Some vulnerabilities of VDM are identified. A weighting method, nothing like VDM, although inspired by the former, is proposed. The results are in favour of the new weighting scheme with illustration of utility for health diagnostics.

Methods for a similarity measure for clinical attributes based on survival data analysis

BackgroundCase-based reasoning is a proven method that relies on learned cases from the past for decision support of a new case. The accuracy of such a system depends on the applied similarity measure, which quantifies the similarity between two cases. This work proposes a collection of methods for similarity measures especially for comparison of clinical cases based on survival data, as they are available for example from clinical trials.MethodsOur approach is intended to be used in scenarios, where it is of interest to use longitudinal data, such as survival data, for a case-based reasoning approach. This might be especially important, where uncertainty about the ideal therapy decision exists. The collection of methods consists of definitions of the local similarity of nominal as well as numeric attributes, a calculation of attribute weights, a feature selection method and finally a global similarity measure. All of them use survival time (consisting of survival status and overall survival) as a reference of similarity. As a baseline, we calculate a survival function for each value of any given clinical attribute.ResultsWe define the similarity between values of the same attribute by putting the estimated survival functions in relation to each other. Finally, we quantify the similarity by determining the area between corresponding curves of survival functions. The proposed global similarity measure is designed especially for cases from randomized clinical trials or other collections of clinical data with survival information. Overall survival can be considered as an eligible and alternative solution for similarity calculations. It is especially useful, when similarity measures that depend on the classic solution-describing attribute “applied therapy” are not applicable. This is often the case for data from clinical trials containing randomized arms.ConclusionsIn silico evaluation scenarios showed that the mean accuracy of biomarker detection in k = 10 most similar cases is higher (0.909–0.998) than for competing similarity measures, such as Heterogeneous Euclidian-Overlap Metric (0.657–0.831) and Discretized Value Difference Metric (0.535–0.671). The weight calculation method showed a more than six times (6.59–6.95) higher weight for biomarker attributes over non-biomarker attributes. These results suggest that the similarity measure described here is suitable for applications based on survival data.

Hybrid self-organizing feature map (SOM) for anomaly detection in cloud infrastructures using granular clustering based upon value-difference metrics

We have witnessed an increase in the availability of data from diverse sources over the past few years. Cloud computing, big data and Internet-of-Things (IoT) are distinctive cases of such an increase which demand novel approaches for data analytics in order to process and analyze huge volumes of data for security and business use. Cloud computing has been becoming popular for critical structure IT mainly due to cost savings and dynamic scalability. Current offerings, however, are not mature enough with respect to stringent security and resilience requirements. Mechanisms such as anomaly detection hybrid systems are required in order to protect against various challenges that include network based attacks, performance issues and operational anomalies. Such hybrid AI systems include Neural Networks, blackboard systems, belief (Bayesian) networks, case-based reasoning and rule-based systems and can be implemented in a variety of ways. Traffic in the cloud comes from multiple heterogeneous domains and changes rapidly due to the variety of operational characteristics of the tenants using the cloud and the elasticity of the provided services. The underlying detection mechanisms rely upon measurements drawn from multiple sources. However, the characteristics of the distribution of measurements within specific subspaces might be unknown. We argue in this paper that there is a need to cluster the observed data during normal network operation into multiple subspaces each one of them featuring specific local attributes, i.e. granules of information. Clustering is implemented by the inference engine of a model hybrid NN system. Several variations of the so-called value-difference metric (VDM) are investigated like local histograms and the Canberra distance for scalar attributes, the Jaccard distance for binary word attributes, rough sets as well as local histograms over an aggregate ordering distance and the Canberra measure for vectorial attributes. Low-dimensional subspace representations of each group of points (measurements) in the context of anomaly detection in critical cloud implementations is based upon VD metrics and can be either parametric or non-parametric. A novel application of a Self-Organizing-Feature Map (SOFM) of reduced/aggregate ordered sets of objects featuring VD metrics (as obtained from distributed network measurements) is proposed. Each node of the SOFM stands for a structured local distribution of such objects within the input space. The so-called Neighborhood-based Outlier Factor (NOOF) is defined for such reduced/aggregate ordered sets of objects as a value-difference metric of histogrammes. Measurements that do not belong to local distributions are detected as anomalies, i.e. outliers of the trained SOFM. Several methods of subspace clustering using Expectation-Maximization Gaussian Mixture Models (a parametric approach) as well as local data densities (a non-parametric approach) are outlined and compared against the proposed method using data that are obtained from our cloud testbed in emulated anomalous traffic conditions. The results—which are obtained from a model NN system—indicate that the proposed method performs well in comparison with conventional techniques.

Two improved attribute weighting schemes for value difference metric

Due to its simplicity, efficiency and efficacy, value difference metric (VDM) has continued to perform well against more sophisticated newcomers and thus has remained of great interest to the distance metric learning community. Of numerous approaches to improving VDM by weakening its attribute independence assumption, attribute weighting has received less attention (only two attribute weighting schemes) but demonstrated remarkable class probability estimation performance. Among two existing attribute weighting schemes, one is non-symmetric and the other is symmetric. In this paper, we propose two simple improvements for setting attribute weights for use with VDM. One is the non-symmetric Kullback–Leibler divergence weighted value difference metric (KLD-VDM) and the other is the symmetric gain ratio weighted value difference metric (GR-VDM). We performed extensive evaluations on a large number of datasets and found that KLD-VDM and GR-VDM significantly outperform two existing attribute weighting schemes in terms of the negative conditional log likelihood and root relative squared error, yet at the same time maintain the computational simplicity and robustness that characterize VDM.

Independently weighted value difference metric

The majority of the difference metrics used in categorical classification algorithms do not take the dependence structure among attributes into account. Some of these metrics even make strong assumptions on attribute independence which are not realistic for many real-world datasets. In addition, these metrics do not consider attribute importance on the class variable. In this paper, a new difference metric is proposed which is named as Independently Weighted Value Difference Metric (IWVDM). IWVDM includes an embedded Incremental Feature Selection (IFS) phase. The proposed metric does not require attribute independence and it introduces a weighting procedure for attributes depending on the information that they possess on the class variable. A series of experiments is conducted using 30 UCI benchmark datasets for comparing the efficiency of IWVDM with Overlap Metric (OM), Value Difference Metric (VDM) and Frequency Difference Metric (FDM). Experimental results show the superiority of IWVDM over these three metrics.

Toward value difference metric with attribute weighting

In distance metric learning, recent work has shown that value difference metric (VDM) with a strong attribute independence assumption outperforms other existing distance metrics. However, an open q...

Local value difference metric

Value difference metric (VDM) is one of the widely used distance functions designed to work with nominal attributes. Research has indicated that the definition of VDM follows naturally from a simple probabilistic model called a naive Bayes (NB). NB assumes that all the attributes are independent given the class. To further improve the performance of NB, several techniques have been proposed. Among these, an effective technique is local learning. Because VDM has a close relationship with NB, in this paper, we propose a local learning method for VDM. The improved distance function is called local value difference metric (LVDM). When LVDM computes the distance between a test instance and each training instance, the conditional probabilities in VDM are estimated by counting from the neighborhood of the test instance only instead of from all the training data. A modified decision tree algorithm is proposed to determine the neighborhood of the test instance. The experimental results on 43 datasets downloaded from the University of California at Irvine (UCI) show that the proposed LVDM significantly outperforms VDM in terms of the class-probability estimation performance of distance-based learning algorithms.

A NOVEL DISTANCE FUNCTION: FREQUENCY DIFFERENCE METRIC

A high quality distance function that measures the difference between instances is essential in many real-world applications and research fields. For example, in instance-based learning, the distance function plays the most important role. A large number of distance functions have been proposed. For nominal attributes, Value Difference Metric (VDM) is one of the state-of-the-art and widely used distance functions. However, it needs to estimate the conditional probabilities, which drops its efficiency in computing the distance between instances. Besides, a practical issue that arises in estimating the conditional probabilities is that the denominators can be zero or very small. This makes them either undefined or very large. Therefore, an efficient distance function that can measure the difference between two instances but without the practical issue confronting VDM is desirable. In this paper, we propose a novel distance function: Frequency Difference Metric (FDM). FDM is just based on the joint frequencies of class labels and attribute values, instead of the conditional probabilities. Extensive empirical studies show that FDM performs almost as well as VDM in terms of accuracy, but significantly outperforms VDM in terms of efficiency. This work provides a very simple, efficient, and effective distance function that can be widely used in many real-world applications and research fields.

Naive Bayes for value difference metric

The value difference metric (VDM) is one of the best-known and widely used distance functions for nominal attributes. This work applies the instanceweighting technique to improveVDM. An instance weighted value difference metric (IWVDM) is proposed here. Different from prior work, IWVDM uses naive Bayes (NB) to find weights for training instances. Because early work has shown that there is a close relationship between VDM and NB, some work on NB can be applied to VDM. The weight of a training instance x, that belongs to the class c, is assigned according to the difference between the estimated conditional probability ^P(c|x) by NB and the true conditional probability P(c|x), and the weight is adjusted iteratively. Compared with previous work, IWVDM has the advantage of reducing the time complexity of the process of finding weights, and simultaneously improving the performance of VDM. Experimental results on 36 UCI datasets validate the effectiveness of IWVDM.

An Augmented Value Difference Measure

How to learn distances from categorical variables (nominal attributes) is a key problem in instance-based learning and other paradigms of machine learning. Recent work in distance learning has shown that a surprisingly simple Value Difference Metric (VDM), with strong assumptions of independence among attributes, is competitive with state-of-the-art distance functions such as Short and Fukunaga Metric (SFM) and Minimum Risk Metric (MRM). This fact raises the question of whether a distance function with less restrictive assumptions can perform even better. In order to answer this question, we proposed an augmented memory-based reasoning (MBR) transform. Based on our augmented MBR transform, we then developed an Augmented Value Difference Measure (AVDM) for learning distances from categorical variables. We experimentally tested our AVDM using 36 natural domains and three artificial Monk’s domains, taken from the University of California at Irvine repository, and compared it to its competitor such as VDM, SFM, MRM, ODVDM, and MSFM. The compared results show that our AVDM can generally improve accuracy in domains that involve correlated attributes without reducing accuracy in ones that do not.

Selective Value Difference Metric

Value Difference Metric (VDM) is one of the widely used distance functions to define the distance between a pair of instances with nominal attributes only. Many approaches have been proposed to improve the performance of VDM. In this paper, we focus on the attribute selection approach and propose another improved Value Difference Metric. We call it Selective Value Difference Metric (SVDM). In order to learn SVDM, we investigate the attribute independence assumption held by VDM and then single out two effective attribute selection methods for SVDM. The experimental results on 36 UCI benchmark datasets validate the effectiveness of the proposed SVDM.

ONCO-i2b2: improve patients selection through CBR techniques with heterogeneous distance functions

ONCO-i2b2: improve patients selection through CBR techniques with heterogeneous distance functions

A Modified Short and Fukunaga Metric based on the attribute independence assumption

It is well known that the naive Bayesian classifier assumes the attribute independence given the class. According to our observation, some distance functions also assume the attribute independence, such as Value Difference Metric (VDM). Short and Fukunaga Metric (SFM) is another widely used distance function, which does not assume the attribute independence. In this paper, we investigate the attribute independence assumption in VDM, and propose a Modified Short and Fukunaga Metric (MSFM) based on the attribute independence assumption. We find that MSFM is surprisingly similar to VDM. In fact, based on some assumptions, our MSFM can be regarded as a logarithmic modification of VDM. That is to say, in some sense, a logarithmic modification of SFM is equivalent to a logarithmic modification of VDM. Our experimental results on a large number of UCI benchmark datasets show that MSFM significantly outperforms SFM and SF2LOG (another improved version of SFM), and almost ties VDM.

One Dependence Value Difference Metric

Many distance-related algorithms depend upon a good distance metric to be successful. The Value Difference Metric, simply VDM, is proposed to find reasonable distance metric between each pair of instances with nominal attribute values only. In VDM, all of the attributes are assumed to be fully independent, and the difference between two values of an attribute is only considered to be closer if they have more similar correlation with the output classes. It is obvious that the attribute independence assumption in VDM is rarely true in reality, which would harm its performance in the applications with complex attribute dependencies. In this paper, we single out an improved Value Difference Metric by relaxing its unrealistic attribute independence assumption. We call it One Dependence Value Difference Metric, simply ODVDM. In ODVDM, the structure learning algorithms for Bayesian network classifiers, such as tree augmented naive Bayes, are used to find the dependence relationships among the attributes. Our experimental results validate its effectiveness in terms of classification accuracy.

Heterogeneous Distance Measures and Nearest-Neighbor Classification in an Ecological Setting

Deriving a suitable heterogeneous distance measure that mixes continuous and categorical attributes is a difficult problem with a variety of applications. We developed the Scaled Heterogeneous Euclidean Overlap Metric (SHEOM) and we adapted the Interpolated Value Difference Metric (IVDM) from Wilson and Martinez [2, 3]. Our adaptation of the IVDM utilizes output classes given by a continuous response variable. We applied both distance measures in a nearest-neighbor classification for an ecological assessment. Both of the distance measures we applied were shown to be improvements on the simpler Heterogeneous Euclidean Overlap Metric when used in this ecological assessment setting.

An instance-based learning approach based on grey relational structure

In instance-based learning, the `nearness' between two instances--used for pattern classification--is generally determined by some similarity functions, such as the Euclidean or Value Difference Metric (VDM). However, Euclidean-like similarity functions are normally only suitable for domains with numeric attributes. The VDM metrics are mainly applicable to domains with symbolic attributes, and their complexity increases with the number of classes in a specific application domain. This paper proposes an instance-based learning approach to alleviate these shortcomings. Grey relational analysis is used to precisely describe the entire relational structure of all instances in a specific domain. By using the grey relational structure, new instances can be classified with high accuracy. Moreover, the total number of classes in a specific domain does not affect the complexity of the proposed approach. Forty classification problems are used for performance comparison. Experimental results show that the proposed approach yields higher performance over other methods that adopt one of the above similarity functions or both. Meanwhile, the proposed method can yield higher performance, compared to some other classification algorithms.

Network intrusion detection method based on RS-MSVM

A new method called RS-MSVM (Rough Set and Multi-class Support Vector Machine) is proposed for network intrusion detection. This method is based on rough set followed by MSVM for attribute reduction and classification respectively. The number of attributes of the network data used in this paper is reduced from 41 to 30 using rough set theory. The kernel function of HVDM-RBF (Heterogeneous Value Difference Metric Radial Basis Function), based on the heterogeneous value difference metric of heterogeneous datasets, is constructed for the heterogeneous network data. HVDM-RBF and one-against-one method are applied to build MSVM. DARPA (Defense Advanced Research Projects Agency) intrusion detection evaluating data were used in the experiment. The testing results show that our method outperforms other methods mentioned in this paper on six aspects: detection accuracy, number of support vectors, false positive rate, false negative rate, training time and testing time.

Case-based reasoning and neural network based expert system for personalization

Abstract We suggest a hybrid expert system of case-based reasoning (CBR) and neural network (NN) for symbolic domain. In previous research, we proposed a hybrid system of memory and neural network based learning. In the system, the feature weights are extracted from the trained neural network, and used to improve retrieval accuracy of case-based reasoning. However, this system has worked best in domains in which all features had numeric values. When the feature values are symbolic, nearest neighbor methods typically resort to much simpler metrics, such as counting the features that match. A more sophisticated treatment of the feature space is required in symbolic domains. We propose feature-weighted CBR with neural network, which uses value difference metric (VDM) as distance function for symbolic features. In our system, the feature weight set calculated from the trained neural network plays the core role in connecting both the learning strategies. Moreover, the explanation on prediction can be given by presenting the most similar cases from the case base. To validate our system, illustrative experimental results are presented. We use datasets from the UCI machine learning archive for experiments. Finally, we present an application with a personalized counseling system for cosmetic industry whose questionnaires have symbolic features. Feature-weighted CBR with neural network predicts the five elements, which show customers’ character and physical constitution, with relatively high accuracy and expert system for personalization recommends personalized make-up style, color, life style and products.

Dissimilarity learning for nominal data

Defining a good distance (dissimilarity) measure between patterns is of crucial importance in many classification and clustering algorithms. While a lot of work has been performed on continuous attributes, nominal attributes are more difficult to handle. A popular approach is to use the value difference metric (VDM) to define a real-valued distance measure on nominal values. However, VDM treats the attributes separately and ignores any possible interactions among attributes. In this paper, we propose the use of adaptive dissimilarity matrices for measuring the dissimilarities between nominal values. These matrices are learned via optimizing an error function on the training samples. Experimental results show that this approach leads to better classification performance. Moreover, it also allows easier interpretation of (dis)similarity between different nominal values.