Sparse Rule Research Articles

Rule-Ranking-Based Approximate Knowledge Interpolation With Directional Monotonicity.

Fuzzy rule interpolation (FRI) empowers fuzzy rule-based systems (FRBSs) with the ability to infer, even when presented with a sparse rule base where no direct rules are applicable to a given observation. The core principle lies in creating an intermediate fuzzy rule-either interpolated or extrapolated-derived from rules neighboring the observation. Conventionally, the selection of these rules hinges upon distance metrics. While this approach is easy to grasp and has been instrumental in the evolution of various FRI methods, it is burdened by the necessity of extensive distance calculations. This becomes particularly cumbersome when swift responses are imperative or when dealing with large datasets. This article introduces a groundbreaking rule-ranking-based FRI method, termed RT-FRI, which overcomes the constraints of the longstanding distance-centric FRI approach. Instead of relying on distances, RT-FRI harnesses ranking scores for rules and unmatched observation. These scores are produced by amalgamating the antecedent attributes using aggregation functions, thus streamlining the rule selection procedure. Recognizing the rigid monotonicity demands of aggregation functions, a variant-DMRT-FRI-has been introduced to ensure directional monotonicity. Experimental results indicate that RT-FRI emerges as a highly efficient technique, with DMRT-FRI exemplifying a notable balance of accuracy and efficiency.

Interval type-2 fuzzy stochastic configuration networks for soft sensor modeling of industrial processes

Soft sensors have been widely applied to predict key variables that are difficult to measure for industrial process modeling. In this paper, a novel randomized interval type-2 fuzzy neural network with parallel learning, called IT2F-PSCN, is presented for soft sensor modeling of industrial processes. It trains the upper and lower bounds of the interval type-2 fuzzy logic system separately to facilitate type reduction, thereby integrating the fuzzy logic system with stochastic configuration networks. To achieve the appropriate structure and parameters of the model, we develop a two-phase training scheme. In the first phase, a sparse rule interpolation method with stochastic configuration is applied to generate new fuzzy rules. In the second phase, the hidden layer is constructed through parallel stochastic configuration to enhance the nonlinear representational capacity. The validity of IT2F-PSCN is confirmed by a series of experiments, including four benchmark data modelings, simulation on the Tennessee Eastman process, and soft sensor modeling for the slurry grade of the first rougher in a zinc flotation process. The experimental results indicate that the proposed IT2F-PSCN performs favorably compared with other methods.

Embedded feature selection approach based on TSK fuzzy system with sparse rule base for high-dimensional classification problems

In high-dimensional problems of fuzzy rule-based embedded feature selection, the challenges include loss of interpretability, curse of dimensionality, and arithmetic underflow, among others. The primary reason for these problems is the exponential increase in the number of fuzzy rules with an increase in input dimension. In this study, an embedded feature selection approach, the Takagi–Sugeno–Kang (TSK) fuzzy system with sparse rule base (TSK-SRB), is proposed for high-dimensional data. Based on clustering, a broader initial rule base is designed with a suitable number of rules. In the rule layer, refined softmin (Ref-softmin) is introduced to calculate the firing strength, which can approximate the minimum T-norm while avoiding arithmetic underflow. Two Group Lasso regularization terms are used to realize feature and rule selection. In addition, an automatic threshold segmentation is introduced to determine the appropriate number of selected features/rules. Extensive experiments on 17 classification datasets showed that TSK-SRB is effective and competitive in high-dimensional feature selection.

Multi-Label Contrastive Learning for Abstract Visual Reasoning.

For a long time, the ability to solve abstract reasoning tasks was considered one of the hallmarks of human intelligence. Recent advances in the application of deep learning (DL) methods led to surpassing human abstract reasoning performance, specifically in the most popular type of such problems-Raven's progressive matrices (RPMs). While the efficacy of DL systems is indeed impressive, the way they approach the RPMs is very different from that of humans. State-of-the-art systems solving RPMs rely on massive pattern-based training and sometimes on exploiting biases in the dataset, whereas humans concentrate on the identification of the rules/concepts underlying the RPM to be solved. Motivated by this cognitive difference, this work aims at combining DL with the human way of solving RPMs. Specifically, we cast the problem of solving RPMs into a multilabel classification framework where each RPM is viewed as a multilabel data point, with labels determined by the set of abstract rules underlying the RPM. For efficient training of the system, we present a generalization of the noise contrastive estimation algorithm to the case of multilabel samples and a new sparse rule encoding scheme for RPMs. The proposed approach is evaluated on the two most popular benchmark datasets [I-RAVEN and procedurally generated matrices (PGM)] and on both of them demonstrate an advantage over the state-of-the-art results.

Towards Dynamic Fuzzy Rule Interpolation via Density-Based Rule Promotion from Interpolated Outcomes

Traditional fuzzy rule-based systems struggle with scenarios where knowledge gaps exist in the problem domain, due to limited data or experience. Fuzzy rule interpolation (FRI) effectively addresses the challenge of inference in fuzzy systems when faced with unmatched observations, due to the employment of an incomplete or sparse rule base. It generates temporary, interpolated rules for the unmatched observations, ensuring continued inference capability. However, the resultant valuable interpolated rules are conventionally discarded. This paper introduces a formal approach for dynamic fuzzy rule interpolation (D-FRI), based on the concept of density-based rule promotion and assisted by the use of the OPTICS clustering algorithm, through exploiting frequently appearing interpolated rules on the fly. This enhances the system’s knowledge coverage, efficiency, and robustness over time. An implementation of such a D-FRI system is presented, which combines transformation-based fuzzy rule interpolation (T-FRI) with OPTICS clustering. This offers an effective mechanism for evaluating and subsequently selecting potentially powerful interpolated rules for the system to dynamically enrich its knowledge base. The implemented system is verified by experimental investigations.

Prediction of MAG Welding Formation Based on Fuzzy Interpolation Reasoning

Welding is an important production and processing method in industrial production. Due to the uncertainty of its process and the complexity of its mechanism, it is challenging to construct an accurate welding process parameter model to describe it. Fuzzy interpolation reasoning is a valuable tool in the realm of intelligent systems. It helps to reduce the complexity of fuzzy models and can be used for reasoning in fuzzy systems with sparse rule bases. This paper provides an interpolation reasoning method that combines fuzzy set distance and similarity. The proposed method is applied to welding forming predictions, and welding penetration is predicted by the given welding current and speed. The results show that the algorithm has high calculation accuracy.

Density-based approach for fuzzy rule interpolation

Sparse rule base is one of the common problems in fuzzy rule-based systems, and fuzzy rule interpolation (FRI) could derive interpolated results for the input based on the neighbor fuzzy rules when the input is not matched by any of the fuzzy rules. The core idea of FRI is that similar inputs would lead to similar results, and several FRI methods that use a pre-defined number of closest rules to obtain the interpolated results have been presented. However, this could lead to the loss of some information as selecting a given number of rules without considering the exact distance between them and the input could lead to the selection of unwanted rules or the ignoring of similar rules. This paper presents a density-based fuzzy rule interpolation method that uses a density-based approach to search and select the closest rules for unmatched inputs. Instead of selecting a given number of rules, the proposed method adaptively selects the closest rules that are within a certain range of the unmatched inputs, thus assuring the selected rules are with high similarity to the inputs. The performance of the proposed method is verified through fifteen classification benchmarks, showing the effectiveness and efficiency of the proposed method.

Transformation-Based Fuzzy Rule Interpolation With Mahalanobis Distance Measures Supported by Choquet Integral

Fuzzy rule interpolation (FRI) strongly supports approximate inference when a new observation matches no rules, through selecting and subsequently interpolating appropriate rules close to the observation from the given (sparse) rule base. Traditional ways of implementing the critical rule selection process are typically based on the exploitation of Euclidean distances between the observation and rules. It is conceptually straightforward for implementation but applying this distance metric may systematically lead to inferior results because it fails to reflect the variations of the relevance or significance levels among different domain features. To address this important issue, a novel transformation-based FRI approach is presented, on the basis of utilizing the Mahalanobis distance metric. The new FRI method works by transforming a given sparse rule base into a coordinates system where the distance between instances of the same category becomes closer while that between different categories becomes further apart. In so doing, when an observation is present that matches no rules, the most relevant neighboring rules to implement the required interpolation are more likely to be selected. Following this, the scale and move factors within the classical transformation-based FRI procedure are also modified by Choquet integral. Systematic experimental investigation over a range of classification problems demonstrates that the proposed approach remarkably outperforms the existing state-of-the-art FRI methods in both accuracy and efficiency.

Safe RuleFit: Learning Optimal Sparse Rule Model by Meta Safe Screening.

We consider the problem of learning a sparse rule model, a prediction model in the form of a sparse linear combination of rules, where a rule is an indicator function defined over a hyper-rectangle in the input space. Since the number of all possible such rules is extremely large, it has been computationally intractable to select the optimal set of active rules. In this paper, to solve this difficulty for learning the optimal sparse rule model, we propose Safe RuleFit (SRF). Our basic idea is to develop meta safe screening (mSS), which is a non-trivial extension of well-known safe screening (SS) techniques. While SS is used for screening out one feature, mSS can be used for screening out multiple features by exploiting the inclusion-relations of hyper-rectangles in the input space. SRF provides a general framework for fitting sparse rule models for regression and classification, and it can be extended to handle more general sparse regularizations such as group regularization. We demonstrate the advantages of SRF through intensive numerical experiments.

Towards utilization of rule base structure to support fuzzy rule interpolation

AbstractFuzzy rule interpolation (FRI) offers a reliable approach for providing an interpretable approximate decision with a sparse rule base, when a new observation does not match any existing rules. As the mainstream application of a fuzzy rule base is to extract valuable approximate information from each individual rules, existing FRI methods typically work by postulating that the more rules used to implement the interpolation the better the reasoning outcomes. Yet, empirical results have shown that using a large number of rules in an FRI process may adversely lead to worsening the accuracy of the inference outcomes, not just degrading efficiency. The objective of this work is to set a firm theoretical foundation for the eventual establishment of a novel FRI approach. It achieves this goal by mapping the structural patterns within a given fuzzy rule base onto a mathematically isomorphic data space, such that the essential information embedded in the original rule base can be effectively captured, represented and analysed. The resulting mathematically mapped patterns enable the production of a theorem that determines the upper limit of the number of rules required to effectively and efficiently perform FRI. The experimental investigations reported herein demonstrate that the number of required rules to perform FRI obeys the theorem discovered in this work.

An Accurate Detection Approach for IoT Botnet Attacks Using Interpolation Reasoning Method

Nowadays, the rapid growth of technology delivers many new concepts and notations that aim to increase the efficiency and comfort of human life. One of these techniques is the Internet of Things (IoT). The IoT has been used to achieve efficient operation management, cost-effective operations, better business opportunities, etc. However, there are many challenges facing implementing an IoT smart environment. The most critical challenge is protecting the IoT smart environment from different attacks. The IoT Botnet attacks are considered a serious challenge. The danger of this attack lies in that it could be used for several threatening commands. Therefore, the Botnet attacks could be implemented to perform the DDoS attacks, phishing attacks, spamming, and other attack scenarios. This paper has introduced a detection approach against the IoT Botnet attacks using the interpolation reasoning method. The suggested detection approach was implemented using the interpolation reasoning method instead of the classical reasoning methods to handle the knowledge base issues and reduce the size of the detection fuzzy rules. The suggested detection approach was designed, tested, and evaluated using an open-source benchmark IoT Botnet attacks dataset. The implemented experiments show that the suggested detection approach was able to detect the IoT Botnet attacks effectively with a 96.4% detection rate. Furthermore, the obtained results were compared with other literature results; the accomplished comparison showed that the suggested method is a rivalry with other methods, and it effectively reduced the false positive rate and interpolated the IoT Botnet attacks alerts even in case of a sparse rule base.

Cyber-Phishing Website Detection Using Fuzzy Rule Interpolation

This paper introduces a novel detection method for phishing website attacks while avoiding the issues associated with the deficiencies of the knowledge-based representation and the binary decision. The suggested detection method was performed using Fuzzy Rule Interpolation (FRI). The FRI reasoning methods added the benefit of enhancing the robustness of fuzzy systems and effectively reducing the system’s complexity. These benefits help the Intrusion Detection System (IDS) to generate more realistic and comprehensive alerts in case of phishing attacks. The proposed method was applied to an open-source benchmark phishing website dataset. The results show that the proposed detection method obtained a 97.58% detection rate and effectively reduced the false alerts. Moreover, it effectively smooths the boundary between normal and phishing attack traffic because of its fuzzy nature. It has the ability to generate the required security alert in case of deficiencies in the knowledge-based representation. In addition, the results obtained from the proposed detection method were compared with other literature results. The results showed that the accuracy rate of this work is competitive with other methods. In addition, the proposed detection method can generate the required anti-phishing alerts even if one of the anti-phishing sparse rules does not cover some input parameters (observations).

Information theoretic limits of learning a sparse rule

We consider generalized linear models in regimes where the number of nonzero components of the signal and accessible data points are sublinear with respect to the size of the signal. We prove a variational formula for the asymptotic mutual information per sample when the system size grows to infinity. This result allows us to derive an expression for the minimum mean-square error (MMSE) of the Bayesian estimator when the signal entries have a discrete distribution with finite support. We find that, for such signals and suitable vanishing scalings of the sparsity and sampling rate, the MMSE is nonincreasing piecewise constant. In specific instances the MMSE even displays an all-or-nothing phase transition, that is, the MMSE sharply jumps from its maximum value to zero at a critical sampling rate. The all-or-nothing phenomenon has previously been shown to occur in high-dimensional linear regression. Our analysis goes beyond the linear case and applies to learning the weights of a perceptron with general activation function in a teacher-student scenario. In particular, we discuss an all-or-nothing phenomenon for the generalization error with a sublinear set of training examples.

Approximate reasoning with fuzzy rule interpolation: background and recent advances

Approximate reasoning systems facilitate fuzzy inference through activating fuzzy if–then rules in which attribute values are imprecisely described. Fuzzy rule interpolation (FRI) supports such reasoning with sparse rule bases where certain observations may not match any existing fuzzy rules, through manipulation of rules that bear similarity with an unmatched observation. This differs from classical rule-based inference that requires direct pattern matching between observations and the given rules. FRI techniques have been continuously investigated for decades, resulting in various types of approach. Traditionally, it is typically assumed that all antecedent attributes in the rules are of equal significance in deriving the consequents. Recent studies have shown significant interest in developing enhanced FRI mechanisms where the rule antecedent attributes are associated with relative weights, signifying their different importance levels in influencing the generation of the conclusion, thereby improving the interpolation performance. This survey presents a systematic review of both traditional and recently developed FRI methodologies, categorised accordingly into two major groups: FRI with non-weighted rules and FRI with weighted rules. It introduces, and analyses, a range of commonly used representatives chosen from each of the two categories, offering a comprehensive tutorial for this important soft computing approach to rule-based inference. A comparative analysis of different FRI techniques is provided both within each category and between the two, highlighting the main strengths and limitations while applying such FRI mechanisms to different problems. Furthermore, commonly adopted criteria for FRI algorithm evaluation are outlined, and recent developments on weighted FRI methods are presented in a unified pseudo-code form, easing their understanding and facilitating their comparisons.

SURVFIT: Doubly sparse rule learning for survival data

Survival data analysis has been leveraged in medical research to study disease morbidity and mortality, and to discover significant bio-markers affecting them. A crucial objective in studying high dimensional medical data is the development of inherently interpretable models that can efficiently capture sparse underlying signals while retaining a high predictive accuracy. Recently developed rule ensemble models have been shown to effectively accomplish this objective; however, they are computationally expensive when applied to survival data and do not account for sparsity in the number of variables included in the generated rules. To address these gaps, we present SURVFIT, a “doubly sparse” rule extraction formulation for survival data. This doubly sparse method can induce sparsity both in the number of rules and in the number of variables involved in the rules. Our method has the computational efficiency needed to realistically solve the problem of rule-extraction from survival data if we consider both rule sparsity and variable sparsity, by adopting a quadratic loss function with an overlapping group regularization. Further, a systematic rule evaluation framework that includes statistical testing, decomposition analysis and sensitivity analysis is provided. We demonstrate the utility of SURVFIT via experiments carried out on a synthetic dataset and a sepsis survival dataset from MIMIC-III.

A New Approach for Transformation-Based Fuzzy Rule Interpolation

Fuzzy rule interpolation (FRI) is of particular significance for reasoning in the presence of insufficient knowledge or sparse rule bases. As one of the most popular FRI methods, transformation-based fuzzy rule interpolation (TFRI) works by constructing an intermediate fuzzy rule, followed by running scale and move transformations. The process of intermediate rule construction selects a user-defined number of rules closest to an observation that does not match any existing rule, using a distance metric. It relies upon heuristically computed weights to assess the contribution of individual selected rules. This process requires a move operation in an effort to force the intermediate rule to overlap with an unmatched observation, regardless of what rules are selected and how much contribution they may each make. It is, therefore, desirable to avoid this problem and also to improve the automation of rule interpolation without resorting to the user's intervention for fixing the number of closest rules. This article proposes such a novel approach to selecting a subset of rules from the sparse rule base with an embedded rule weighting scheme for the automatic assembling of the intermediate rule. Systematic comparative experimental results are provided on a range of benchmark datasets to demonstrate statistically significant improvement in the performance achieved by the proposed approach over that obtainable using conventional TFRI.

Building a cognizant honeypot for detecting active fingerprinting attacks using dynamic fuzzy rule interpolation

AbstractDynamic fuzzy rule interpolation (D‐FRI) technique delivers a dynamic rule base through the utilisation of fuzzy rule interpolation to infer more accurate results for a given application problem. D‐FRI offered dynamic rule base is very useful in security areas where network conditions are always volatile and require the most updated rule base. A honeypot is a vital part of any security infrastructure for directly investigating attacks and attackers in real‐time to strengthen the overall security of the network. However, a honeypot as a concealed system can only function successfully while its identity is not revealed to any attackers. Attackers always attempt to uncover such honeypots for avoiding any trap and strengthening their attacks. Active fingerprinting attack is used to detect these honeypots by injecting purposefully designed traffic to a network. Such an attack can be prevented by controlling the traffic but this will make honeypot unusable system if its interaction with the outside world is limited. Alternatively, it is practically more useful if this fingerprinting attack is detected in real‐time to manage its immediate consequences and preventing the honeypot. This article offers an approach to building a cognizant honeypot for detecting active fingerprinting attacks through the utilisation of the established D‐FRI technique. It is based on the use of just a sparse rule base while remaining capable of detecting active fingerprinting attacks when the system does not find any matching rules. Also, it learns from current network conditions and offers a dynamic rule base to facilitate more accurate and efficient detection.

D-FRI-Honeypot: A Secure Sting Operation for Hacking the Hackers Using Dynamic Fuzzy Rule Interpolation

As active network defence systems, honeypots are commonly used as a decoy to inspect attackers and their attack tactics in order to improve the cybersecurity infrastructure of an organisation. A honeypot may be successful provided that it disguises its identity. However, cyberattackers continuously endeavour to discover honeypots for evading any deception and bolstering their attacks. Active fingerprinting attack is one such technique that may be used to discover honeypots by sending specially designed traffic. Preventing a fingerprinting attack is possible but doing that may hinder the process of dealing with the attackers, counteracting the purpose of a honeypot. Instead, detecting an attempted fingerprinting attack in real-time can enhance a honeypot’s capability, uninterruptedly managing any immediate consequences and preventing the honeypot being identified. Nevertheless, it is difficult to detect and predict an attempted fingerprinting attack due to the challenge of isolating it from other similar attacks, particularly when imprecise observations are involved in the monitoring of the traffic. Dynamic fuzzy rule interpolation (D-FRI) enables an adaptive approach for effective reasoning with such situations by exploiting the best of both inference and interpolation. The dynamic rules produced by D-FRI facilitate approximate reasoning with perpetual changes that often occur in this type of application, where dynamic rules are required to cover new network conditions. This paper proposes a D-FRI-Honeypot, an enhanced honeypot running D-FRI framework in conjunction with Principal Component Analysis, to detect and predict an attempted fingerprinting attack on honeypots. This D-FRI-Honeypot works with a sparse rule base but is able to detect active fingerprinting attacks when it does not find any matching rules. Also, it learns from current network conditions and offers a dynamically enriched rule base to support more precise detection. This D-FRI-Honeypot is tested against five popular fingerprinting tools (namely, Nmap, Xprobe2, NetScanTools Pro, SinFP3 and Nessus), to demonstrate its successful applications.

A Bidirectional Reasoning Based on Fuzzy Interpolation

In order to deal with both the “curse of dimensionality” and the “sparse rule base” simultaneously, an initial idea of hierarchical bidirectional fuzzy interpolation is presented in this article, combining hierarchical fuzzy systems and forward/backward fuzzy rule interpolation. In particular, backward fuzzy interpolation can be employed to allow interpolation to be carried out when certain antecedents of observation variables are absent, whereas conventional methods do not work. Hierarchical bidirectional fuzzy interpolation is applicable to situations where a multiple multi-antecedent rules system needs to be reconstructed to a multi-layer fuzzy system and any sub-layer rule base is sparse. The implementation of this approach is based on fuzzy rule interpolative reasoning that utilities scale and move transformation. An illustrative example and application scenario are provided to demonstrate the efficacy of this proposed approach.

Bidirectional approximate reasoning-based approach for decision support

Fuzzy rule-based systems are widely applied for real-world decision support, such as policy formation, public health analysis, medical diagnosis, and risk assessment. However, they face significant challenges when the application problem at hand suffers from the “curse of dimensionality” or “sparse knowledge base”. Combination of hierarchical fuzzy rule models and fuzzy rule interpolation offers a potentially efficient and effective approach to dealing with both of these issues simultaneously. In particular, backward fuzzy rule interpolation (B-FRI) facilitates approximate reasoning to be performed given a sparse rule base where rules do not fully cover all observations or the observations are not complete, missing antecedent values in certain available rules. This paper presents a hierarchical bidirectional fuzzy reasoning mechanism by integrating hierarchical rule structures and forward/backward rule interpolation. A computational method is proposed, building on the resulting hierarchical bidirectional fuzzy interpolation to maintain consistency in sparse fuzzy rule bases. The proposed techniques are utilised to address a range of decision support problems, successfully demonstrating their efficacy.