Information Gain Theory Research Articles

A machine learning-based credit risk prediction engine system using a stacked classifier and a filter-based feature selection method

Credit risk prediction is a crucial task for financial institutions. The technological advancements in machine learning, coupled with the availability of data and computing power, has given rise to more credit risk prediction models in financial institutions. In this paper, we propose a stacked classifier approach coupled with a filter-based feature selection (FS) technique to achieve efficient credit risk prediction using multiple datasets. The proposed stacked model includes the following base estimators: Random Forest (RF), Gradient Boosting (GB), and Extreme Gradient Boosting (XGB). Furthermore, the estimators in the Stacked architecture were linked sequentially to extract the best performance. The filter- based FS method that is used in this research is based on information gain (IG) theory. The proposed algorithm was evaluated using the accuracy, the F1-Score and the Area Under the Curve (AUC). Furthermore, the Stacked algorithm was compared to the following methods: Artificial Neural Network (ANN), Decision Tree (DT), and k-Nearest Neighbour (KNN). The experimental results show that stacked model obtained AUCs of 0.934, 0.944 and 0.870 on the Australian, German and Taiwan datasets, respectively. These results, in conjunction with the accuracy and F1-score metrics, demonstrated that the proposed stacked classifier outperforms the individual estimators and other existing methods.

Zoom behavior during visual search modulates pupil diameter and reflects adaptive control states.

Adaptive gain theory proposes that the dynamic shifts between exploration and exploitation control states are modulated by the locus coeruleus-norepinephrine system and reflected in tonic and phasic pupil diameter. This study tested predictions of this theory in the context of a societally important visual search task: the review and interpretation of digital whole slide images of breast biopsies by physicians (pathologists). As these medical images are searched, pathologists encounter difficult visual features and intermittently zoom in to examine features of interest. We propose that tonic and phasic pupil diameter changes during image review may correspond to perceived difficulty and dynamic shifts between exploration and exploitation control states. To examine this possibility, we monitored visual search behavior and tonic and phasic pupil diameter while pathologists (N = 89) interpreted 14 digital images of breast biopsy tissue (1,246 total images reviewed). After viewing the images, pathologists provided a diagnosis and rated the level of difficulty of the image. Analyses of tonic pupil diameter examined whether pupil dilation was associated with pathologists' difficulty ratings, diagnostic accuracy, and experience level. To examine phasic pupil diameter, we parsed continuous visual search data into discrete zoom-in and zoom-out events, including shifts from low to high magnification (e.g., 1× to 10×) and the reverse. Analyses examined whether zoom-in and zoom-out events were associated with phasic pupil diameter change. Results demonstrated that tonic pupil diameter was associated with image difficulty ratings and zoom level, and phasic pupil diameter showed constriction upon zoom-in events, and dilation immediately preceding a zoom-out event. Results are interpreted in the context of adaptive gain theory, information gain theory, and the monitoring and assessment of physicians' diagnostic interpretive processes.

Co-attentive representation learning for web services classification

The rapid adoption of services-related technologies, such as cloud computing, has lead to the explosive growth of web services. Automated service classification that groups web services by similar functionality is a widely used technique to facilitate the management and discovery of web services within a large-scale repository. The existing service classification approaches primarily focus on learning the isolated representations of service features but ignored their internal semantic correlations. To address the aforementioned issue, we propose a novel deep neural network with the Co-Attentive Representation Learning (CARL-Net) mechanism for effectively classifying services by learning interdependent characteristics of service without feature engineering. Specifically, we propose a service data augmentation mechanism by extracting informative words from the service description using information gain theory. Such a mechanism can learn a correlation matrix among embedded augmented data and description, thereby obtaining their interdependent semantic correlation representations for service classification. We evaluate the effectiveness of CARL-Net by comprehensive experiments based on a real-world dataset collected from ProgrammableWeb, which includes 10,943 web services. Compared with seven web service classification baselines based on CNN, LSTM, Recurrent-CNN, C-LSTM, BLSTM, ServeNet and ServeNet-BERT, the CARL-Net can achieve an improvement of 5.66%–172.21% in the F-measure of web service classification.

Visual affects: Linking curiosity, Aha-Erlebnis, and memory through information gain

Current theories propose that our sense of curiosity is determined by the learning progress or information gain that our cognitive system expects to make. However, few studies have explicitly tried to quantify subjective information gain and link it to measures of curiosity. Here, we asked people to report their curiosity about the intrinsically engaging perceptual ‘puzzles’ known as Mooney images, and to report on the strength of their aha experience upon revealing the solution image (curiosity relief). We also asked our participants (279) to make a guess concerning the solution of the image, and used the distribution of these guesses to compute the crowdsourced semantic entropy (or ambiguity) of the images, as a measure of the potential for information gain. Our results confirm that curiosity and, even more so, aha experience is substantially associated with this semantic information gain measure. These findings support the expected information gain theory of curiosity and suggest that the aha experience or intrinsic reward is driven by the actual information gain. In an unannounced memory part, we also established that the often reported influence of curiosity on memory is fully mediated by the aha experience or curiosity relief. We discuss the implications of our results for the burgeoning fields of curiosity and psychoaesthetics.

Web service classification based on information gain theory and bidirectional long short‐term memory with attention mechanism

SummaryWith the increasing number of Web services, Web service discovery for service‐oriented application development has become more important. Clustering or classifying Web services according to their functionalities is an effective way for Web service discovery. Extracting latent topic features from service description by exploiting topic model can improve the accuracy of service classification. However, most of them simply treat the description document as a set of flat word features without considering the varying importance of different features as well as sequential relations between features. In this article, we proposed a Web service classification approach based on information gain theory and bidirectional long short‐term memory with attention mechanism for accuracy Web service classification by considering fine‐grained factors implicit in Web service description. The comparative experiments are performed on ProgrammableWeb dataset, and show that the proposed method achieves a significant improvement compared with baseline methods.

Exploration for Object Mapping Guided by Environmental Semantics using UAVs

This paper presents a strategy to autonomously explore unknown indoor environments, focusing on 3D mapping of the environment and performing grid level semantic labeling to identify all available objects. Unlike conventional exploration techniques that utilize geometric heuristics and information gain theory on an occupancy grid map, the work presented in this paper considers semantic information, such as the class of objects, in order to gear the exploration towards environmental segmentation and object labeling. The proposed approach utilizes deep learning to map 2D semantically segmented images into 3D semantic point clouds that encapsulate both occupancy and semantic annotations. A next-best-view exploration algorithm is employed to iteratively explore and label all the objects in the environment using a novel utility function that balances exploration and semantic object labeling. The proposed strategy was evaluated in a realistically simulated indoor environment, and results were benchmarked against other exploration strategies.

User identification by keystroke dynamics using improved binary particle swarm optimisation

As a kind of behavioural characteristic, keystroke features are crucial to the accuracy of user identification system using shallow machine learning algorithms. Filter and wrapper feature selection algorithms are the two most important methods. The information gain and particle swarm optimisation algorithm represent the two feature optimisation methods, respectively. In this paper, new hybrid binary particle swarm optimisation methods combined with information gain theory are proposed in association with opposite-based learning and distributed techniques. The converted information gain values act as weight coefficients to adaptively adjust the flight speed of particles. The support vector machine (SVM) algorithm is applied to evaluate the performance of feature optimisation in terms of user identification accuracy and feature reduction rate. Experimental results of three public keystroke datasets show that the proposed optimisation methods achieve better classification accuracy with fewer features than four existing optimisation methods.

User identification by keystroke dynamics using improved binary particle swarm optimisation

As a kind of behavioural characteristic, keystroke features are crucial to the accuracy of user identification system using shallow machine learning algorithms. Filter and wrapper feature selection algorithms are the two most important methods. The information gain and particle swarm optimisation algorithm represent the two feature optimisation methods, respectively. In this paper, new hybrid binary particle swarm optimisation methods combined with information gain theory are proposed in association with opposite-based learning and distributed techniques. The converted information gain values act as weight coefficients to adaptively adjust the flight speed of particles. The support vector machine (SVM) algorithm is applied to evaluate the performance of feature optimisation in terms of user identification accuracy and feature reduction rate. Experimental results of three public keystroke datasets show that the proposed optimisation methods achieve better classification accuracy with fewer features than four existing optimisation methods.

Deep-Structured Machine Learning Model for the Recognition of Mixed-Defect Patterns in Semiconductor Fabrication Processes

Semiconductor manufacturers aim to fabricate defect-free wafers in order to improve product quality, increase yields, and reduce costs. Typically, wafer defects form spatial patterns that provide useful information, helping to identify problems and faults during the fabrication process. Machine learning (ML) methods have been used to classify these defects in order to locate the root causes of failure. This paper proposes a novel deep-structured ML approach as an extension of our previous randomized general regression network (RGRN) model, to identify and classify both single-defect and mixed-defect patterns. The principal motivation for this paper is that a shallow-structured RGRN performs well on single-pattern defects, achieving an accuracy of 99.8%, but performs poorly when a wafer has mixed-defect patterns. The proposed approach improves RGRN performance, particularly on mixed-pattern defects, by incorporating a novel information gain (IG)-based splitter as well as deep-structured ML. A spatial filter is applied to remove random noise and reduce model bias during training. During the first detection stage, the splitter generates unique rules that are built using the IG theory and splits the defects data into single-defect and mixed-defect patterns. Single-defect patterns are then classified by RGRN, whereas mixed-defect patterns are fed into the deep-structured ML model for further classification. This combination improves the ability of the proposed approach to classify diverse defect patterns and achieve a better overall performance. Our experimental results demonstrate that the proposed approach achieves an overall detection accuracy of 86.17% on a dataset that contains real data representing both single-defect and mixed-defect patterns, as commonly found in real manufacturing scenarios, outperforming existing ML-based models.

IgS-wBSRM: A time-aware Web Service QoS monitoring approach in dynamic environments

ContextQuality of Service(QoS) is an important criterion to measure the quality of third-party web services. However, it is always affected by different environmental factors. Consequently, how to monitor web service QoS timely and accurately in dynamic environments is an important problem. ObjectiveOur article aims to design a novel Web service QoS monitoring approach which can be used in dynamic environments. MethodTo achieve the above objective, we propose a novel weighted naïve Bayesian runtime monitoring approach based on information gain theory and sliding window mechanism, called IgS-wBSRM. IgS-wBSRM initializes the weights of different environmental factors according to training samples collected. Then, according to information entropy and information gain theory, IgS-wBSRM reads the sample data flow in sequence, and calculates the information gain of each environmental impact factor after the arrival of new sample. It updates the initialized weights with TF-IDF algorithm in dynamic environments. In this way, IgS-wBSRM can correct the delay judgement, jitter noise and off-line constant problems in traditional monitoring approaches. Furthermore, considering the timeliness of the training samples, IgS-wBSRM combines a sliding window mechanism to update the weights of each environmental impact factor, and it can eliminate the impact of the recent service state in the accumulated historical data. ResultsA set of dedicated experiments based on a real world data set and a simulated data set demonstrates that IgS-wBSRM can abandon the expiration information of historical data effectively, and can monitor QoS more accurately. ConclusionsThe overall effect of IgS-wBSRM is better than other QoS monitoring approaches. We suggest directions for follow-up work, e.g., exploring the influence of the size of the sliding window, considering multiple QoS attributes combined with data integration theory and applying IgS-wBSRM in other QoS areas.

Radio map updated method based on subscriber locations in indoor WLAN localization

With the rapid development of wireless local area network(WLAN) technology, an important target of indoor positioning systems is to improve the positioning accuracy while reducing the online calibration effort to overcome signal time-varying. A novel fingerprint positioning algorithm, known as the adaptive radio map with updated method based on hidden Markov model(HMM), is proposed. It is shown that by using a collection of user traces that can be cheaply obtained, the proposed algorithm can take advantage of these data to update the labeled calibration data to further improve the position estimation accuracy. This algorithm is a combination of machine learning, information gain theory and fingerprinting. By collecting data and testing the algorithm in a realistic indoor WLAN environment, the experiment results indicate that, compared with the widely used K nearest neighbor algorithm,the proposed algorithm can improve the positioning accuracy while greatly reduce the calibration effort.

Towards Crafting a Smooth and Accurate Functional Link Artificial Neural Networks Based on Differential Evolution and Feature Selection for Noisy Database

AbstractThis work presents an accurate and smooth functional link artificial neural network (FLANN) for classification of noisy database. The accuracy and smoothness of the network is taken birth by suitably tuning the parameters of FLANN using differential evolution and filter based feature selection. We use Qclean algorithm for identification of noise, information gain theory for filtering irrelevant features, and then supplied the remaining relevant attributes to the functional expansion unit of FLANN, which in turn map lower to higher dimensional feature space for constructing a smooth and accurate classifier. In specific, the differential evolution is used to fine tune the weight vector of this network and some trigonometric functions are used in functional expansion unit. The proposed approach is validated with a few benchmarking highly skewed and balanced dataset retrieved from University of California, Irvine (UCI) repository with a range of 5-20% noise. The insightful experimental study signifies...

Fuzzy unordered rule induction algorithm in text categorization on top of geometric particle swarm optimization term selection

Rapid growth of digital information requires automated handling and organization of documents. The two main stages in automated document categorization are (i) term reduction and (ii) classification. In this paper, we present a novel two-stage term reduction strategy based on Information Gain (IG) theory and Geometric Particle Swarm Optimization (GPSO) search. We evaluate performance of the proposed term reduction approach with use of a new classifier, fuzzy unordered rule induction algorithm (FURIA) to categorize multi-label texts. In order to evaluate the performance of FURIA quantitatively, we compared it against two widely used algorithms, Naïve Bayes and Support Vector Machine (SVM). Text Categorization (TC) performance of the proposed term reduction strategy is validated with use of Reuters-21578 and OHSUMED text collection datasets. The experimental results show that performance of the proposed term reduction method is efficient for document organization tasks.

DE+RBFNs based classification: A special attention to removal of inconsistency and irrelevant features

A novel approach for the classification of both balanced and imbalanced dataset is developed in this paper by integrating the best attributes of radial basis function networks and differential evolution. In addition, a special attention is given to handle the problem of inconsistency and removal of irrelevant features. Removing data inconsistency and inputting optimal and relevant set of features to a radial basis function network may greatly enhance the network efficiency (in terms of accuracy), at the same time compact its size. We use Bayesian statistics for making the dataset consistent, information gain theory (a kind of filter approach) for reducing the features, and differential evolution for tuning center, spread and bias of radial basis function networks. The proposed approach is validated with a few benchmarked highly skewed and balanced dataset retrieved from University of California, Irvine (UCI) repository. Our experimental result demonstrates promising classification accuracy, when data inconsistency and feature selection are considered to design this classifier.

Feature Selection for Designing a Novel Differential Evolution Trained Radial Basis Function Network for Classification

This work presents a novel approach for classification of both balanced and unbalanced dataset by suitably tuning the parameters of radial basis function networks with an additional cost of feature selection. Inputting optimal and relevant set of features to a radial basis function may greatly enhance the network efficiency (in terms of accuracy) at the same time compact it size. In this paper, the authors use information gain theory (a kind of filter approach) for reducing the features and differential evolution for tuning center and spread of radial basis functions. The proposed approach is validated with a few benchmarking highly skewed and balanced dataset retrieved from University of California, Irvine (UCI) repository. The experimental study is encouraging to pursue further extensive research in highly skewed data.

The Adaptability of Decision Tree Method in Mining Industry Safety Data

Mining nowadays is extensively applied in researches and business fields. The choice of data mining tools is practically dependent on the nature of the data to be mined. Studies have shown that certain tools perform better in certain types of dataset. This paper x-rays a typical real life incident dataset common to oil and gas industries and how decision tree algorithm fits into mining it. The application of Decision Tree mining tool on a real-world safety records perfectly reveals useful information that are subsumed in the volume and nature of the data. The concept of Entropy and Information Gain theory are used in building the decision tree model and an accuracy of 71.4% arrived at, indicating a good performance of decision tree induction on the dataset. Further areas of research on the use of decision tree method in data mining are recommended.

Detecting single-nucleotide polymorphism by single-nucleotide polymorphism interactions in rheumatoid arthritis using a two-step approach with machine learning and a Bayesian threshold least absolute shrinkage and selection operator (LASSO) model

The objective of this study was to detect interactions between relevant single-nucleotide polymorphisms (SNPs) associated with rheumatoid arthritis (RA). Data from Problem 1 of the Genetic Analysis Workshop 16 were used. These data consisted of 868 cases and 1,194 controls genotyped with the 500 k Illumina chip. First, machine learning methods were applied for preselecting SNPs. One hundred SNPs outside the HLA region and 1,500 SNPs in the HLA region were preselected using information-gain theory. The software weka was used to reduce colinearity and redundancy in the HLA region, resulting in a subset of 6 SNPs out of 1,500. In a second step, a parametric approach to account for interactions between SNPs in the HLA region, as well as HLA-nonHLA interactions was conducted using a Bayesian threshold least absolute shrinkage and selection operator (LASSO) model incorporating 2,560 covariates. This approach detected some main and interaction effects for SNPs in genes that have previously been associated with RA (e.g., rs2395175, rs660895, rs10484560, and rs2476601). Further, some other SNPs detected in this study may be considered in candidate gene studies.

Does Reasoning Occur on the Selection Task? A Comparison of Relevance-based Theories

Does reasoning occur on the Wason selection task, or are card selections determined purely on the basis of heuristic processes? To answer this question two relevance-based theories of reasoning are compared: (1) the theory of Evans (1984, 1989; Evans & Over, 1996), which takes the heuristic viewpoint, and (2) the theory of Sperber, Cara, and Girotto (1995), which takes the reasoning viewpoint. In three experiments, the effect of removing matching cards from the selection task array is examined. It is argued that the Sperber et al. theory makes clearer predictions about the results of these manipulations, which are confirmed, and that the Evans theory can only accommodate them if it allows the operation of reasoning processes. The results are also discussed in relation to Roth's (1979) account of the selection task, mental models theory, and information gain theory.

Combined experimental design and information theory for the optimization of supercritical fluid extraction of organic priority pollutants from sediment

Optimization of the supercritical extraction (SFE) of incurred n-alkanes, PAHs and PCBs from a marine sediment has been carried out by a central composite experimental design maximizing the content of extracted analytes and minimizing the information gain compared with Soxhlet extraction (MeOH:CH 2Cl 2). Distributions of individual compounds found in the SFE optimized by the information gain theory were closer to the Soxhlet results. Furthermore, only the n-alkanes exhibited concurrent concentration maxima and minimum information gain. PCBs and PAHs exhibited optima at different extraction conditions.