Blackbox Observability of Features and Feature Interactions

Predictive case-based feature importance and interaction

This paper deals with a complicated Feature Interactions (FI) problem that results from new emerging telecommunications architectures, IP telephony for example. That is because these architectures introduce new complications since service providers and end users have the possibility to express their preferences in more sophisticated ways and non telephony applications are combined with telephony services. The article provides examples of new FI that have not been reported before. The main contribution of this work is that it proposes new FI management methods that are required to solve the new FI complications. Old management methods are also considered and some of them are enhanced in order to adapt to the new problem context. Intelligence requirements necessary to implement the proposed management model are finally underlined.

Feature interaction is a newly proposed feature relevance relationship, but the unintentional removal of interactive features can result in poor classification performance for this relationship. However, traditional feature selection algorithms mainly focus on detecting relevant and redundant features while interactive features are usually ignored. To deal with this problem, feature relevance, feature redundancy and feature interaction are redefined based on information theory. Then a new feature selection algorithm named CMIFSI (Conditional Mutual Information based Feature Selection considering Interaction) is proposed in this paper, which makes use of conditional mutual information to estimate feature redundancy and interaction, respectively. To verify the effectiveness of our algorithm, empirical experiments are conducted to compare it with other several representative feature selection algorithms. The results on both synthetic and benchmark datasets indicate that our algorithm achieves better results than other methods in most cases. Further, it highlights the necessity of dealing with feature interaction.

Reengineering a legacy product line has been addressed very little by current product line research activities. This paper introduces a method to investigate feature dependencies and interactions, which restricts the variants that can be derived from the legacy product line assets. Reorganizing the product line assets with respect to new requirements requires more knowledge than what is easily provided by the classical feature-modeling approaches. Hence, adding all the feature dependencies and interactions into the feature tree results in unreadable and unmanageable feature models that fail to achieve their original goals.We therefore propose two complementary views to represent the feature model. One view shows the hierarchical refinement of features similar to common feature-modeling approaches in a feature tree. The second view describes what kind of dependencies and interactions there are between various features.We show two examples of feature dependencies and interactions in the context of an engine-control software product line, and we demonstrate how our approach helps to define correct product configurations from product line variants.

Feature interaction tends to have a wide range of consequences and effects on a feature model and its applications. While these may often be intended, it is also true that feature validity can be violated, one way or another, by feature interactions (Shah & Mäntylä, 1995, Gao & Shah, 1998, Lee & Kim, 1998). They may affect the semantics of a feature, ranging from slight changes in actual parameter values, to some substantial alterations to both geometry and topology or even complete suppression of its contribution to the model shape. To certain extent, successful applications of feature recognition and feature-based techniques have been hindered by interactions among the features. Feature interaction was first studied in relation to feature recognition systems. As an alternative to feature recognition, feature-based design methodology has also become prevalent in recent years. Although a number of successful and commercially available feature-based design systems have been reported, current CAD technology is still unable to provide an effective solution for fully handling the complexity of feature interactions. Very often in a feature-based design system, the interaction between two features gives rise to an unintended feature, nullifying the one-to-one mapping from design features to manufacturing features. The resulting manufacturing feature is usually of a form that the system cannot handle or represent. Thus feature interaction resolution is equally essential for a feature-based design system (Dereli & Baykasoglu, 2004). As discussed in Chapter IV, features can be represented either as a set of faces or as a volume. The interactions between surface features are different from those occurring between volumetric features. This chapter discusses different types of interactions that arise from these two feature representation schemes and uses the interacting entities to classify them. There are two types of surface feature interactions, basic feature interaction and complex feature interaction. Three types of basic feature interactions are discussed. They are nested, overlapping, and intersecting types. Interacting patches are used to classify volumetric feature interactions. These interacting patches can be of a containing, contained, or overlapping type. The significance of feature interactions lies in their effect on the machining sequence of the features involved. This is also discussed in this chapter. When features are close to each other but do not share any geometric entities, interactions may also happen for structural reasons. This type of feature interaction can be called interaction by vicinity. The main aim of this chapter is to take a holistic approach toward feature interaction solutions. The example parts used are from the “Catalogue of the NIST (National Institute of Standards and Technology) Design, Planning and Assembly Repository” (Regli & Gaines, 1996). A case study is provided in the end of the chapter.

The evolving and adapting capabilities of robust intelligence are best manifested in its ability to learn. Machine learning enables computer systems to learn, and improve performance. Feature selection facilitates machine learning (e.g., classification) by aiming to remove irrelevant features. Feature (attribute) interaction presents a challenge to feature subset selection for classification. This is because a feature by itself might have little correlation with the target concept, but when it is combined with some other features, they can be strongly correlated with the target concept. Thus, the unintentional removal of these features may result in poor classification performance. It is computationally intractable to handle feature interactions in general. However, the presence of feature interaction in a wide range of real-world applications demands practical solutions that can reduce high-dimensional data while preserving feature interactions. In this paper, we take up the challenge to design a special data structure for feature quality evaluation, and to employ an information-theoretic feature ranking mechanism to efficiently handle feature interaction in subset selection. We conduct experiments to evaluate our approach by comparing with some representative methods, perform a lesion study to examine the critical components of the proposed algorithm to gain insights, and investigate related issues such as data structure, ranking, time complexity, and scalability in search of interacting features.

In this work we make a distinction between feature interactions and manufacturing interactions. These two terms are usually used interchangeably because feature and manufacturing interactions often occur together, but not always. However, we feel that it is important to make a distinction between feature interactions which result from volumetric intersections of features presenting difficulties for features extractors, and manufacturing interactions which occur when two manufacturing operations interfere with each other’s execution, and present a problem to the process planner. By separating these definitions it allows us to focus separately on each phenomena. In this paper our focus is on manufacturing interactions. We present a non-exhaustive catalog of common manufacturing interaction types in CNC machining, and discuss how they result in precedence constraints in the manufacturing plan.

This paper demonstrates capability of detecting strong synthetic benchmark feature interactions in a set of mixed categorical and continuous variables using a modified version of Monte Carlo Feature Selection algorithm. MCFS’s original way of detecting feature interactions relying on the analysis of structure of trained decision trees is compared with our modified approach consisting of a series of variable permutations combined with a decomposition of feature total effect to main effect and interaction effects. A comparison with unmodified MCFS, which by default handles only classification problems using C4.5 decision trees, shows that the new approach is slightly more robust. Furthermore, the decomposition approach is flexible by allowing to plug in different types of models to MCFS. This opens a way to handle high-throughput supervised feature selection and interaction mining problems for classification, regression and censored survival decision vector.

This paper demonstrates capability of detecting strong synthetic benchmark feature interactions in a set of mixed categorical and continuous variables using a modified version of Monte Carlo Feature Selection algorithm. MCFS’s original way of detecting feature interactions relying on the analysis of structure of trained decision trees is compared with our modified approach consisting of a series of variable permutations combined with a decomposition of feature total effect to main effect and interaction effects. A comparison with unmodified MCFS, which by default handles only classification problems using C4.5 decision trees, shows that the new approach is slightly more robust. Furthermore, the decomposition approach is flexible by allowing to plug in different types of models to MCFS. This opens a way to handle high-throughput supervised feature selection and interaction mining problems for classification, regression and censored survival decision vector.

A new approach in feature interaction testing

Feature interactions (FIs) occur when features of different communication services interfere with each other FI filtering is a pre-processing step before the FI detection, which roughly identifies FI-prone service combinations based on simple indications of the FIs. We have previously (Proc. 6th Intl. Workshop on Feature Interactions in Telecom. Networks and Distributed Systems, pp. 163-178, 2000) proposed an FI filtering method at the requirements stage using use case maps (UCMs). This method identifies FI-prone service combinations by focusing on changes in the user's scenarios before/after the service composition, but it does not tell which of the scenarios in the compound services has the potential for FIs. In this paper, as an extension of the previous method, we propose a new method to derive FI-prone scenarios from the FI-prone combinations obtained by the previous method. From many practical FIs, we first make the following observations: (a) FI tends to occur in scenarios where both services are activated, and (b) FI tends to occur in scenarios where a service by-passes a feature of the other service. Then, based on these observations, we propose heuristics on the UCM scenario paths to derive FI-prone scenarios. An experimental evaluation demonstrates that the derived scenarios successfully cover all scenarios that lead to actual FIs.

Both crosscutting concerns and feature interactions are phenomena that may impair modularity. Crosscutting concerns have been in the center of interest in aspect-oriented software development. Much research in this direction—including my own—is aimed at developing mechanisms to avoid and manage code scattering and tangling, with a strong focus on source-code organization. Feature interactions have been studied even earlier. Research on feature interactions always emphasized behavioral aspects that arise when two features interact, regardless of the implementation. Both phenomena are related, but not as closely as one may think. In this talk, I will tell my personal story on how I started my research career with investigating crosscutting concerns and—after a long journey and many misunderstanding and enlightments—arrived at being interested in feature interactions. In some sense this journey is also a journey of a whole community that emerged from different research areas, and I will share my experience gathered from this journey and synthesize lessons learned. In particular, I will highlight the (historical) differences of research on crosscutting concerns and feature interactions. Furthermore, by means of recent experimental results, I will emphasize that there are different types of feature interactions and that these type are related (some to crosscutting concerns). I will close the talk with a call to action to accept and approach the New Feature Interaction Challenge, which is to understand and exploit the different types of feature interactions to build better software (e.g., to develop better tools for feature-interaction testing).

Feature interaction modeling, which exploits interactive information between features, has been widely explored in various applications. Recently, many graph neural networks (GNNs) based models are proposed to model feature interactions by predicting the existence of edges between pairwise nodes that represent features. However, these models can only directly model two-order feature interactions. Although stacking multiple GNN layers can implicitly capture the arbitrary high-order feature interactions, it may lead to the over-smoothing problem. To this end, we propose DHFM, D iversity enhanced H ypergraph F actorization M achines that incorporate hypergraphs into feature interaction modeling, which can model the diverse feature interactions of different orders explicitly. Specifically, order-wise hyperedge predictors are proposed to discover beneficial feature interactions and explicitly model the feature interactions of different orders. In addition, diversity measures are introduced in hyperedge predictors and in the results of feature interactions to make discovered feature interactions as diverse as possible and avoid generating correlated errors. Extensive experiments on four real-world datasets demonstrate the superiority of the proposed model. In addition, the case study is conducted to further justify the effectiveness of the proposed model.

The classification of a hyperspectral image (HSI) plays a critical role and serves as the foundation for many related applications. The combination of convolutional neural networks (CNNs) and transformers has shown promising performance in HSI classification by using the advantages of the two networks. However, existing hybrid models often offer limited feature interaction and fusion between the two branches. Here, a novel interactive convolution and transformer network (ICTNet) for HSI classification is proposed. Specifically, raw HSI is first fed into a multi-scale feature-enhancement module, where convolution operations with varying kernel sizes are used to extract multi-scale features, and shuffle attention is used to further enhance feature representation. Enhanced features are then processed by a dual-branch CNN and transformer fusion module to leverage local information and long-range dependencies. Additionally, the feature interaction and fusion module is proposed to facilitate dynamic interaction and fusion of features during extraction and propagation between the two branches, enhancing the diversity and interactivity of the features. Extensive experimental results on three real HSI data sets demonstrate that the proposed ICTNet outperforms state-of-the-art HSI classification methods.

Click-through rate (CTR) prediction is crucial for computing advertisement and recommender systems. The key challenge of CTR prediction is to accurately capture user interests and deliver suitable advertisements to the right people. However, there are an immense number of features in CTR prediction datasets, which hardly fit when only using an individual feature. To solve this problem, feature interaction that combines several features via an operation is introduced to enhance prediction performance. Many factorizations machine-based models and deep learning methods have been proposed to capture feature interaction for CTR prediction. They follow an enumeration-filter pattern that could not determine the appropriate order of feature interaction and useful feature interaction. The attention logarithmic network (ALN) is presented in this paper, which uses logarithmic neural networks (LNN) to model feature interactions, and the squeeze excitation (SE) mechanism to adaptively model the importance of higher-order feature interactions. At first, the embedding vector of the input was absolutized and a very small positive number was added to the zeros of the embedding vector, which made the LNN input positive. Then, the adaptive-order feature interactions were learned by logarithmic transformation and exponential transformation in the LNN. Finally, SE was applied to model the importance of high-order feature interactions adaptively for enhancing CTR performance. Based on this, the attention logarithmic interaction network (ALIN) was proposed for the effectiveness and accuracy of CTR, which integrated Newton’s identity into ALN. ALIN supplements the loss of information, which is caused by the operation becoming positive and by adding a small positive value to the embedding vector. Experiments are conducted on two datasets, and the results prove that ALIN is efficient and effective.