Performance Of Case-based Reasoning Research Articles

RetrieveDroid: k-NN Performance Evaluation of Distance Measurement Schemes for Android Malware Detection Using Case-Based Retrieval

One of the big challenges in cybersecurity is the detection of Android attacks since Android is the most popular mobile operating system. Within this system, applications require certain permissions to access critical resources. Investigation of the use of permissions is a concern to check whether an application is not mislead to divulge sensitive information. This work aims to determine whose distance schemes offers the best malware detection based on permission similarities. Case based reasoning (CBR) is a concept which aims to find a solution based on historical experiences. CBR performance relies on finding similarities between actual cases and stored cases and then to deduce solutions. This paper proposes to transform app data as vector of appearance of dangerous permissions and to store such vectors based on CBR structure. Then we investigate k-NN classification performance related to five distance-based metrics such as Euclidean, Cosine, Manhattan, Minkowski, and Mahalanobis. Experiments were carried out with a set of 419 applications, including 203 malicious and 216 benign samples. The whole dataset has been split in training set of 291 samples with 162 benign and 129 malicious, and the testing set of 128 samples with 54 benign and 74 malicious samples. k-Nearest Neighbor (k-NN) are used as the similarity algorithm in which the distance model is varied in each of the five distance models. Results reveal that Minkowski and Manhattan models provide the best overall performance to detect Android malware based on permissions, in terms of accuracy (99.21%) and precision (97%). This work is a good start to recommend to researchers distance metrics exploitable when performing permission similarity-based detection.

An artificial immune systems approach to Case-based Reasoning applied to fault detection and diagnosis

This work presents a hybrid model of Case-based Reasoning (CBR) and artificial immune systems (AIS), which is able to manage the processes of recovery, adaptation (reuse and revision) and retention of cases. The developed model also provides an alternative way of clustering cases, identifying high density areas, improve search efficiency in the case space and store relationships among similar cases. The proposed model is applied to a fault detection and diagnosis problem of direct currnet motor nenchmark and the obtained results are compared using specific CBR performance metrics showing promising perspectives for the proposed model.

Multiple retrieval case-based reasoning for incomplete datasets.

The performance of case-based reasoning (CBR) depends on an accurate ranking of similar cases in the retrieval phase that affects all subsequent phases and profits from the potential of large databases. Unfortunately, growing databases come along with a rising amount of missing data that reduces the stability of the ranking since incomplete cases cannot be ranked as reliable as complete ones. In context of CBR hardly any work was done so far to rigorously analyze the impact of missing data and solutions to tackle this issue. In particular, a generalized solution which is able to process data under different missingness conditions for different variable types is missing. In this paper we present a multiple retrieval case-based reasoning (MRCBR) framework for incomplete databases that provides a statistically accurate ranking for similar cases. It unifies the advantages of multiple imputation and CBR while it preserves both the data distribution and database structure. Built as generalized CBR system, MRCBR was optimized and tested for medical decision support but can be extended to any CBR requirement as well. It is suitable for numerical and categorical variables and all sorts of missingness conditions. The approach was compared to eight competing methods applicable to handle incomplete databases in context of CBR. The comparison to the true ranking was based on two various error measures. In the evaluation we tested four representative scenarios that considered different conditions for missing data analysis. The outcome for every method in each scenario resulted in 200 miscellaneous setups. MRCBR outperforms all compared CBR methods in presence of missing data and shows reliable and stable results in every scenario. Especially with larger databases and rising number of incomplete variables it enlarges its lead to all other methods. Our study demonstrates that missing data must not be ignored when a correct CBR outcome is required.

A Case-Based Reasoning Method with Rank Aggregation

In order to improve the accuracy of case-based reasoning (CBR), this paper addresses a new CBR framework with the basic principle of rank aggregation. First, the ranking methods are put forward in each attribute subspace of case. The ordering relation between cases on each attribute is got between cases. Then, a sorting matrix is got. Second, the similar case retrieval process from ranking matrix is transformed into a rank aggregation optimal problem, which uses the Kemeny optimal. On the basis, a rank aggregation case-based reasoning algorithm, named RA-CBR, is designed. The experiment result on UCI data sets shows that case retrieval accuracy of RA-CBR algorithm is higher than euclidean distance CBR and mahalanobis distance CBR testing.So we can get the conclusion that RA-CBR method can increase the performance and efficiency of CBR.

Memory and forgetting: An improved dynamic maintenance method for case-based reasoning

The problem solving performance of a case-based reasoning (CBR) system is closely related to the quantity and quality of the cases stored in the case base. With the continuous growth of the size of the case base, the so called “swamping problem” may occur when the time cost of retrieval exceeds the benefit of the accuracy. From the perspective of cognitive science, a dynamic maintenance method improved by selective memory and intentional forgetting for CBR is proposed, which can imitate the memory function of the human brain to selectively save new cases, update the forgotten values and intentionally delete the old cases. The experiments show the effectiveness of the proposed method. The selective memory and international forgetting policy can significantly reduce the time and space complexity, and retain or improve the accuracy of the CBR classifier, thus improving the performance of CBR.

사례기반추론의 유사 임계치 및 커버리지 최적화

사례기반추론(CBR)은 많은 장점으로 인해 지금까지 의료진단, 생산계획, 고객분류 등 다양한 분야의 의사결정 지원에 적용되어 왔다. 그러나, 효과적인 CBR 시스템을 설계, 구축하기 위해서는 연구자가 직관적으로 설정해야 할 많은 설계요소들이 존재한다. 본 연구에서는 이러한 CBR의 여러 설계요소들 중 사례 검색 단계에서 결합할 이웃 사례들을 보다 효과적으로 선정할 수 있는 새로운 모형을 제시한다. 기존 연구에서는 결합할 이웃 사례를 선정하는 방법으로 사전에 정해진 이웃사례의 수(k-NN의 k)를 적용하든가, 혹은 최대 유사도의 상대적 비율을 임계치로 사용하는 방식을 적용해 왔다. 하지만, 본 연구에서는 결합할 유사사례를 선택하는 새로운 기준으로 0에서 1사이의 값을 갖는 절대적 유사 임계치를 사용할 것을 제안한다. 이 경우, 임계치 값이 과도하게 작아지게 되면, 예측결과의 생성이 잘 이루어지지 않을 수 있는 문제가 발생할 수 있다. 이에, 전체 학습사례들 중에서 예측결과가 생성된 사례의 비중을 커버리지(coverage)로 정의하고, 이를 유사 임계치 최적화 시 제약조건으로 설정함으로서, 사용자가 원하는 수준의 커버리지는 유지한 상태에서 가장 효과적인 유사 사례를 찾아 추론할 수 있도록 모형을 설계하였다. 제안 모형의 유용성을 검증하기 위해, 본 연구에서는 이 모형을 실존하는 국내 한 온라인 쇼핑몰의 표적 마케팅 사례에 적용하였다. 그 결과, 제안 모형이 CBR의 예측 성과를 유의미하게 개선시킬 수 있음을 확인할 수 있었다. Since case-based reasoning(CBR) has many advantages, it has been used for supporting decision making in various areas including medical checkup, production planning, customer classification, and so on. However, there are several factors to be set by heuristics when designing effective CBR systems. Among these factors, this study addresses the issue of selecting appropriate neighbors in case retrieval step. As the criterion for selecting appropriate neighbors, conventional studies have used the preset number of neighbors to combine(i.e. k of k-nearest neighbor), or the relative portion of the maximum similarity. However, this study proposes to use the absolute similarity threshold varying from 0 to 1, as the criterion for selecting appropriate neighbors to combine. In this case, too small similarity threshold value may make the model rarely produce the solution. To avoid this, we propose to adopt the coverage, which implies the ratio of the cases in which solutions are produced over the total number of the training cases, and to set it as the constraint when optimizing the similarity threshold. To validate the usefulness of the proposed model, we applied it to a real-world target marketing case of an online shopping mall in Korea. As a result, we found that the proposed model might significantly improve the performance of CBR.

FORECASTING FIRM RISK IN THE EMERGING MARKET OF CHINA WITH SEQUENTIAL OPTIMIZATION OF INFLUENCE FACTORS ON PERFORMANCE OF CASE‐BASED REASONING: AN EMPIRICAL STUDY WITH IMBALANCED SAMPLES

SUMMARYWith the development of the Chinese economy, how to make the right decision regarding firms’ risk is becoming more and more important. Case‐based reasoning (CBR) is a potential method that can help forecast business risk status in advance; it is easy to apply and is able to provide good explanations of output. In order to obtain more accurate prediction with CBR, it is essential to investigate factors that influence CBR's performance, and to optimize these factors sequentially for the improvement of CBR's performance in firm risk prediction in emerging markets under a more practicable assumption. We verified that sequential optimization of feature selection, feature weighting, instance selection and the number of nearest neighbours is a possible alternative for improving predictive performance of CBR forecasting under the assumption that the number of failed samples is smaller than that of nonfailed samples. The detailed implementation includes: (1) selecting significant features through a correlation matrix and reducing feature dimensions with factor analysis; (2) using variance contribution ratios of features from factor analysis as feature weights; (3) eliminating noisy cases via a state matrix; and (4) obtaining the optimal number of nearest neighbours from empirical results among different numbers of nearest neighbours. To validate the usefulness of the sequential optimization approach, we applied it to a real‐world case: firm risk prediction with imbalanced data from the emerging market of China. Experimental results show that predictive accuracy of CBR applied in the emerging market was improved with the sequential optimization approach. Insightful thoughts from the results of the sequential optimization of the CBR forecasting system on modelling social tasks are also provided. Copyright © 2013 John Wiley & Sons, Ltd.

On Performance of Feature Normalization in Classification with Distance-Based Case-Based Reasoning

Distance-based k-nearest-neighbors (KNN) retrieval is usually adopted to find nearest neighbors for the target case in case-based reasoning (CBR). However, in practical application, the similarity metrics calculated by this means often estimate the actual situation with bias because different features have different units and magnitudes. To avoid this problem, method of feature normalization is usually employed before the implementation of case retrieval. In this study on performance of CBR in classification, distance-based case retrieval was, respectively, implemented on eight binary classification data sets, including two credit scoring datasets and one bankruptcy prediction dataset, after, respectively, implementing seven feature normalized methods on the datasets. The results show that: 1) The two data normalization methods of T3, namely: scaling feature values into the range of 0 to 1 by revised minimum-maximum normalization, and T7, namely: the use of normalization utility function, helped CBR perform the best on 3 out of 8 data sets; 2) the data normalization method of T4, namely: dividing by the maximum value of the feature or maximum normalization, made CBR perform better on 7 out of 8 data sets than the other methods; 3) The data normalization method of T7 helped CBR improve classification performance on 6 out of 8 data sets. As a whole, T7 and T4, followed by T3/T2(minimum-maximum normalization) and T1(Z-score)/T6(Standard deviation normalization), are the best feature normalization methods for distance- based CBR in classification (including bankruptcy prediction and credit scoring problems), as they obtain the top ranking scores in the experiment. In this patent paper, we contribute the current literature on the understanding of how should we choose the most adequate normalization method that could be applied on the data collected in a particular system. Keywords: Bankruptcy prediction, Case-Based Reasoning(CBR), credit scoring, distance-based case retrieval, Feature normalization, K-Nearest Neighbors, Feature normalization methods, Data sets information, Distance-based KNN

Case Adaptation Method of Case-Based Reasoning for Construction Cost Estimation in Korea

Case-based reasoning (CBR) utilizes old solutions to solve a present problem. However, as no past situation is ever exactly the same as a new one, old solutions should be fitted for new circumstances, which is called case adaptation. Thus, the success of CBR depends on the ability of case adaptation. Generally, the case adaptation method can be categorized by two methods: decreasing the need for adaptation and increasing the capability. Nevertheless, previous research has focused on applying the method itself and rarely examined the adaptation or candidates selection. As an effort to enhance the performance of CBR, this research develops a case adaptation method that is balanced for both methods. To validate the method, a CBR cost model was developed that has an adaptation function using 129 military barrack projects in Korea, and then the method was tested by using 13 test cases. Furthermore, an applicability test was conducted based on 164 Korean public apartment projects.

Case-based reasoning and data mining integrated method for the diagnosis of some neuromuscular disease

Case-based reasoning (CBR) method has been deployed in the diagnosis of neuromuscular diseases (NMD) based upon EMG signals and also data mining (DM) methods have been used for the same purpose. Mostly EMG-based parameters and certain non-signal parameters have been considered for the same. We have performed an integrated CBR and DM method for the diagnosis of EMG-based NM diseases. We have taken psychological and cognitive parameters apart from the EMG-based parameters. Computation cost is taken as the metric to evaluate the performance of CBR and CBR-DM integrated model. There is a saving of computation cost, speed and storage in the integrated model.

On performance of case-based reasoning in Chinese business failure prediction from sensitivity, specificity, positive and negative values

Case-based reasoning (CBR) is a machine learning technique of high performance in classification problems, and it is also a chief method in predicting business failure. Recently, several techniques have been introduced into the life-cycle of CBR for business failure prediction (BFP). The drawback of former researches on CBR-based BFP is that they only use total predictive accuracy when assessing CBR. In this research, we provide evidence on performance of CBR in Chinese BFP from various views of sensitivity, specificity, positive and negative values. Data are collected from Shanghai Stock Exchange and Shenzhen Stock Exchange in China. And we present how data are preprocessed from the view of data mining. The classical CBR model on the base of Euclidean metric, the grey CBR model on the base of grey coefficient metric, and the pseudo CBR model on the base of pseudo outranking relations are employed to make a comparative study on CBR's predictive performance in BFP. Meanwhile, support vector machine (SVM) is employed to be a baseline model for comparison. The results indicate that pseudo CBR produces better performance in Chinese BFP than classical CBR and grey CBR significantly on the whole, and it outperforms SVM marginally by total predictive accuracy and sensitivity, while it is not significantly worse than SVM by specificity.

Predicting the Outcome of Construction Litigation Using an Integrated Artificial Intelligence Model

An integrated prediction model (IPM) was used to predict the outcome of construction litigation with the hope that it will generate a higher prediction rate than the rates obtained by using artificial neural networks (ANN), case-based reasoning (CBR), and boosted decision trees (BDTs) in earlier studies. These studies were conducted by using the same 132 Illinois circuit court cases (with slight variations in the case of the BDT and IPM studies) filed between 1992 and 2000. A prediction rate of 91.15% was obtained, higher than the 66.67% in the ANN study, 83.33% in the CBR study, and 89.59% in the BDT study. IPM involves four processes, namely, data consolidation, attribute selection, prediction using hybrid classifiers, and assessment. The performance of IPM is compared with the performance of ANN, CBR, and BDT. If the outcome of construction litigation can be predicted with reasonable accuracy and reliability, all parties involved in the construction process could save considerable money and time.

Bankruptcy prediction modeling with hybrid case-based reasoning and genetic algorithms approach

One of the most important research issues in finance is building effective corporate bankruptcy prediction models because they are essential for the risk management of financial institutions. Researchers have applied various data-driven approaches to enhance prediction performance including statistical and artificial intelligence techniques, and many of them have been proved to be useful. Case-based reasoning (CBR) is one of the most popular data-driven approaches because it is easy to apply, has no possibility of overfitting, and provides good explanation for the output. However, it has a critical limitation—its prediction performance is generally low. In this study, we propose a novel approach to enhance the prediction performance of CBR for the prediction of corporate bankruptcies. Our suggestion is the simultaneous optimization of feature weighting and the instance selection for CBR by using genetic algorithms (GAs). Our model can improve the prediction performance by referencing more relevant cases and eliminating noises. We apply our model to a real-world case. Experimental results show that the prediction accuracy of conventional CBR may be improved significantly by using our model. Our study suggests ways for financial institutions to build a bankruptcy prediction model which produces accurate results as well as good explanations for these results.

Global optimization of case-based reasoning for breast cytology diagnosis

Case-based reasoning (CBR) is one of the most popular prediction techniques in medical domains because it is easy to apply, has no possibility of overfitting, and provides a good explanation for the output. However, it has a critical limitation – its prediction performance is generally lower than other AI techniques like artificial neural networks (ANN). In order to obtain accurate results from CBR, effective retrieval and matching of useful prior cases for the problem is essential, but it is still a controversial issue to design a good matching and retrieval mechanism for CBR systems. In this study, we propose a novel approach to enhance the prediction performance of CBR. Our suggestion is the simultaneous optimization of feature weights, instance selection, and the number of neighbors that combine using genetic algorithms (GA). Our model improves the prediction performance in three ways – (1) measuring similarity between cases more accurately by considering relative importance of each feature, (2) eliminating useless or erroneous reference cases, and (3) combining several similar cases represent significant patterns. To validate the usefulness of our model, this study applied it to a real-world case for evaluating cytological features derived directly from a digital scan of breast fine needle aspirate (FNA) slides. Experimental results showed that the prediction accuracy of conventional CBR may be improved significantly by using our model. We also found that our proposed model outperformed all the other optimized models for CBR using GA.

Global optimization of feature weights and the number of neighbors that combine in a case‐based reasoning system

Abstract: Case‐based reasoning (CBR) often shows significant promise for improving the effectiveness of complex and unstructured decision‐making. Consequently, it has been applied to various problem‐solving areas including manufacturing, finance and marketing. However, the design of appropriate case indexing and retrieval mechanisms to improve the performance of CBR is still a challenging issue. Most previous studies on improving the effectiveness of CBR have focused on the similarity function aspect or optimization of case features and their weights. However, according to some of the prior research, finding the optimal k parameter for the k‐nearest neighbor is also crucial for improving the performance of the CBR system. Nonetheless, there have been few attempts to optimize the number of neighbors, especially using artificial intelligence techniques. In this study, we introduce a genetic algorithm to optimize the number of neighbors that combine, as well as the weight of each feature. The new model is applied to the real‐world case of a major telecommunication company in Korea in order to build a prediction model for customer profitability level. Experimental results show that our genetic‐algorithm‐optimized CBR approach outperforms other artificial intelligence techniques for this multi‐class classification problem.

Detection of Welding Flaws With MLP Neural Network and Case Based Reasoning

The correct detection of welding flaws is important to the successful development of an automated weld inspection system. As a continuation of our previous efforts, this study investigates the performance of multi-layer perceptron (MLP) neural networks and case based reasoning (CBR) individually as well as their combined use. It is found that better performance is attained by all the methods tested in this study than that was obtained by the fuzzy clustering methods employed before. For each method, the effect of using different parameters is also investigated and discussed. An improvement of CBR performance is not guaranteed when the MLP NN based attribute weighting is used. In addition, none of the three combination-of-multiple-classifiers methods (majority voting, Borda count, and arithmetic averaging) tested improve the performance of the best individual classifiers.