Heuristic Selection Research Articles

Improving probability selection based weights for satisfiability problems

Boolean Satisfiability problem (SAT) plays a prominent role in many domains of computer science and artificial intelligence due to its significant importance in both theory and applications. Algorithms for solving SAT problems can be categorized into two main classes: complete algorithms and incomplete algorithms (typically stochastic local search (SLS) algorithms). SLS algorithms are among the most effective for solving uniform random SAT problems, while hybrid algorithms achieved great breakthroughs for solving hard random SAT (HRS) problem recently. However, there is a lack of algorithms that can effectively solve both uniform random SAT and HRS problems. In this paper, a new SLS algorithm named SelectNTS is proposed aiming at solving both uniform random SAT and HRS problem effectively. SelectNTS is essentially an improved probability selection based local search algorithm, the core of which includes new clause and variable selection heuristics: a new clause weighting scheme and a biased random walk strategy are utilized to select a clause, while a new probability selection strategy with the variation of configuration checking strategy is used to select a variable. Extensive experimental results show that SelectNTS outperforms the state-of-the-art random SAT algorithms and hybrid algorithms in solving both uniform random SAT and HRS problems effectively.

Роль эвристического потенциала в реализации технической образности в зарубежной научной фантастике

The paper aims to identify the main features and the focus of imagery realization of heuristic potential by the example of technical vocabulary in the screenplays of English science fiction films. The article clarifies the role of heuristic potential in solving screenwriting problems, identifies the mechanism of heuristic metaphorical selection in the screenplays of science fiction films in the English language, proves that the characteristics of the technical sphere can be used as the primary source of units during the figurative reinterpretation of problematic screenwriting situations. The scientific originality of the research lies in the fact that it studied technical imagery based on the screenplay text for the first time which expands the understanding of an image during its amplification (development, completion) in order to seek understanding between the characters. As a result, it has been proved that heuristic potential uses technical imagery to solve important screenwriting problems, and that leads to the moments of inspiration and discovery of new solutions on the basis of technical imagery schemes.

Selecting and combining complementary feature representations and classifiers for hate speech detection

Hate speech is a major issue in social networks due to the high volume of data generated daily. Recent works demonstrate the usefulness of machine learning (ML) in dealing with the nuances required to distinguish between hateful posts from just sarcasm or offensive language. Many ML solutions for hate speech detection have been proposed by either changing how features are extracted from the text or the classification algorithm employed. However, most works consider only one type of feature extraction and classification algorithm. This work argues that a combination of multiple feature extraction techniques and different classification models is needed. We propose a framework to analyze the relationship between multiple feature extraction and classification techniques to understand how they complement each other. The framework is used to select a subset of complementary techniques to compose a robust multiple classifiers system (MCS) for hate speech detection. The experimental study considering four hate speech classification datasets demonstrates that the proposed framework is a promising methodology for analyzing and designing high-performing MCS for this task. MCS system obtained using the proposed framework significantly outperforms the combination of all models and the homogeneous and heterogeneous selection heuristics, demonstrating the importance of having a proper selection scheme. Source code, figures and dataset splits can be found in the GitHub repository: https://github.com/Menelau/Hate-Speech-MCS.

A Heuristic Feature Selection in Logistic Regression Modeling with Newton Raphson and Gradient Descent Algorithm

Binary choices, such as success or failure, acceptance or rejection, high or low, heavy or light, and so on, can always be used to express decision-making. Based on the known predictor feature values, a classification model can be used to predict an unknown categorical value. The logistic regression model is a commonly used classification approach in a variety of scientific domains. The goal of this research is to create a logistic regression model with a heuristic approach for selecting input characteristics and to compare the Newton Raphson and gradient descent (GD) algorithms for estimating parameters. Among predictor traits, there were four that met the criterion for being both dependent on the target and independent of one another. Also, optional features In Malang, Indonesia, researchers used the Chi-square test to find four significant characteristics that increase the incidence of pregnant women developing preeclampsia: age (X1), parity (X2), history of hypertension (X3) and salty food consumption (X6). In the above work author proposed, the logistic regression model developed using the gradient descent approach had a lower risk of error than the logistic regression model generated using the Newton Raphson algorithm. The model with the gradient descent approach has a precision of 98.54 percent and an F1 score of 97.64 percent, while the model with the Newton Raphson algorithm has a precision of 86.34 percent and an F1 score of 72.55 percent.

Combining max–min ant system with effective local search for solving the maximum set k-covering problem

The maximum set k-covering problem (MKCP) is a well-known combinatorial NP-hard problem with rich application scenarios, with the objective of covering as many elements as possible with a limited number of candidate sets. In this paper, an effective ant colony optimization (ACO) algorithm, namely MMAS-ML, is proposed to solve this important problem. Specifically, the double-layer selection heuristic is presented to obtain a high-quality solution, which is used in the initialization process to accelerate the convergence of the algorithm. Moreover, a customized local search is carefully designed, empowering ants with exploitation capacity around the food sources. For such local search procedures, some useful techniques, that is, the scoring function, delayed configuration checking, oblivious row weighting, and best from multiple selections, have been developed to enhance the performance. In addition, the max–min ant system with memory ants is further designed to confine the upper and lower bounds of the pheromone to avoid ACO stagnation. A detailed experimental evaluation of various instances reveals that the newly proposed algorithm outperforms other state-of-the-art heuristics for the MKCP, and the developed components contribute to the entire framework.

Cost-sensitive learning strategies for high-dimensional and imbalanced data: a comparative study.

High dimensionality and class imbalance have been largely recognized as important issues in machine learning. A vast amount of literature has indeed investigated suitable approaches to address the multiple challenges that arise when dealing with high-dimensional feature spaces (where each problem instance is described by a large number of features). As well, several learning strategies have been devised to cope with the adverse effects of imbalanced class distributions, which may severely impact on the generalization ability of the induced models. Nevertheless, although both the issues have been largely studied for several years, they have mostly been addressed separately, and their combined effects are yet to be fully understood. Indeed, little research has been so far conducted to investigate which approaches might be best suited to deal with datasets that are, at the same time, high-dimensional and class-imbalanced. To make a contribution in this direction, our work presents a comparative study among different learning strategies that leverage both feature selection, to cope with high dimensionality, as well as cost-sensitive learning methods, to cope with class imbalance. Specifically, different ways of incorporating misclassification costs into the learning process have been explored. Also different feature selection heuristics have been considered, both univariate and multivariate, to comparatively evaluate their effectiveness on imbalanced data. The experiments have been conducted on three challenging benchmarks from the genomic domain, gaining interesting insight into the beneficial impact of combining feature selection and cost-sensitive learning, especially in the presence of highly skewed data distributions.

Effective feature learning of multi‐modal genetic and neuroimaging data for prediction of future conversion to Alzheimer’s disease: A machine learning based study

AbstractBackgroundDue to the high dimensionality of Single Nucleotide Polymorphism (SNP) data, current imaging genetic studies of Dementia of Alzheimer’s Type (DAT) usually only select a limited number of candidate SNPs that are reported in the literature, while there might be potentially important AD‐related risk factors in the remaining genes. In this study, we harness a robust feature selection technique that utilizes all available SNPs in the human genome to identify the relevant MRI and genetic features while overcoming feature redundancy.MethodA total of 543 subjects from the Alzheimer's Disease Neuroimaging Initiative (ADNI) that had both MRI and genetic data available are included in the study. These subjects are divided into seven novel stratified groups [Table 1] based on their longitudinal clinical diagnosis. A subject is categorized as DAT+ if it has a follow‐up diagnosis of DAT, and is categorized as DAT‐ otherwise. We utilize a 2‐stage probabilistic multi‐kernel classifier. In the first stage, using 10‐fold cross‐validation: In each fold we select discriminative features by applying Fisher's Exact test on SNP data and Welch's t‐test on MRI data using subjects in the subgroups with the most certain longitudinal diagnosis (sNC in DAT‐ and sDAT in DAT+). In the second stage, using only the most frequently selected features above, we retrain on 80% of sNC and sDAT, validate on the remaining 20%, and test on subjects in the other subgroups with milder longitudinal diagnosis results.ResultOur study showed that although genetic features have a lower prediction power than MRI features, combining both modalities can improve the prediction of future conversion to AD. For subjects in pNC group, MRI features fail to indicate potential risk of developing AD while genetic data can accurately identify the risk [Figure 1]. Our feature selection method has successfully identified significant AD risk factors for both modalities.ConclusionOur study demonstrated that using effective feature selection methods eliminate the need for heuristic selection of AD‐related genes used for machine‐learning studies for image genetics. Our proposed method reveals genetic risk factors already mentioned in literature as well as some novel risk factors that could advance clinical diagnostics in the future.

New Heuristics to Stratify Applicants: Predictors of General Surgery Residency Applicant Step 1 Scores

In 2022, United States Medical Licensing Examination (USMLE) Step 1 scores will become pass/fail. This may be problematic, as residency programs heavily rely on USMLE Step 1 scores as a metric when determining interview invitations. This study aimed to assess candidate application metrics associated with USMLE Step 1 scores to offer programs new cues for stratifying applicants. Retrospective cohort study analyzing interviewed applicants to one general surgery residency program in 2019 and 2020. Applicant data analyzed included USMLE Step 1 scores, number of publications, clerkship scores, letter of recommendation scores (out of 2, scored by 0.25 interval), interview overall score (out of 5, scored by integer level), and standardized question score (out of 10). Each year, applicant's answers to one standardized behavioral question during their interview were scored by interviewers. Tertiary medical center, academic general surgery residency program. Interviewed applicants at one general surgery residency program whose applications contained complete demographic information (203 out of 247). Multiple Linear Regression revealed that higher surgical clerkship (β = 0.19, p = 0.006) and higher standardized interview question (β = 0.32, p < 0.001) scores were positively associated with applicant USMLE Step 1 score (F[7, 195] = 6.61, p < 0.001, R2 = 0.19). Letter of recommendation score, number of peer reviewed publications, gender, race, and applicant type (preliminary/categorical) were not associated with USMLE Step 1 scores. With USMLE Step 1 scores transitioning to pass/fail, surgical residency programs need new selection heuristics. Surgery clerkship scores and standardized behavioral questions answered by applicants prior to the interview could provide a holistic view of applicants and help programs better stratify candidates without USMLE Step 1 scores.

New Heuristics to Stratify Applicants: Predictors of General Surgery Residency Applicant Step 1 Scores

OBJECTIVE In 2022, United States Medical Licensing Examination (USMLE) Step 1 scores will become pass/fail. This may be problematic, as residency programs heavily rely on USMLE Step 1 scores as a metric when determining interview invitations. This study aimed to assess candidate application metrics associated with USMLE Step 1 scores to offer programs new cues for stratifying applicants. DESIGN Retrospective cohort study analyzing interviewed applicants to one general surgery residency program in 2019 and 2020. Applicant data analyzed included USMLE Step 1 scores, number of publications, clerkship scores, letter of recommendation scores (out of 2, scored by 0.25 interval), interview overall score (out of 5, scored by integer level), and standardized question score (out of 10). Each year, applicant's answers to one standardized behavioral question during their interview were scored by interviewers. SETTING Tertiary medical center, academic general surgery residency program. PARTICIPANTS Interviewed applicants at one general surgery residency program whose applications contained complete demographic information (203 out of 247). RESULTS Multiple Linear Regression revealed that higher surgical clerkship (β = 0.19, p = 0.006) and higher standardized interview question (β = 0.32, p CONCLUSIONS With USMLE Step 1 scores transitioning to pass/fail, surgical residency programs need new selection heuristics. Surgery clerkship scores and standardized behavioral questions answered by applicants prior to the interview could provide a holistic view of applicants and help programs better stratify candidates without USMLE Step 1 scores.

A deep reinforcement learning based hyper-heuristic for combinatorial optimisation with uncertainties

In the past decade, considerable advances have been made in the field of computational intelligence and operations research. However, the majority of these optimisation approaches have been developed for deterministically formulated problems, the parameters of which are often assumed perfectly predictable prior to problem-solving. In practice, this strong assumption unfortunately contradicts the reality of many real-world problems which are subject to different levels of uncertainties. The solutions derived from these deterministic approaches can rapidly deteriorate during execution due to the over-optimisation without explicit consideration of the uncertainties. To address this research gap, a deep reinforcement learning based hyper-heuristic framework is proposed in this paper. The proposed approach enhances the existing hyper-heuristics with a powerful data-driven heuristic selection module in the form of deep reinforcement learning on parameter-controlled low-level heuristics, to substantially improve their handling of uncertainties while optimising across various problems. The performance and practicality of the proposed hyper-heuristic approach have been assessed on two combinatorial optimisation problems: a real-world container terminal truck routing problem with uncertain service times and the well-known online 2D strip packing problem. The experimental results demonstrate its superior performance compared to existing solution methods for these problems. Finally, the increased interpretability of the proposed deep reinforcement learning hyper-heuristic has been exhibited in comparison with the conventional deep reinforcement learning methods.

Convex combinations in judgment aggregation

Judgments are the basis for almost all decisions. They often come from different models and multiple experts. This information is typically aggregated using simple averages, which leads to the well-known shared information problem. A weighted average of the individual judgments based on empirically estimated sophisticated weights is commonly discarded in practice, because the sophisticated weights have large estimation errors. In this paper, I explore mixture weights, which are convex combinations of sophisticated and naïve weights. I show analytically that if the data generation process is stable, there always exists a mixture weight which aggregates judgments better than the naïve weights. I thus offer a path to alleviate the shared information problem. In contrast to other proposed solutions, it does not require any control over the judgment process. I demonstrate the utility of mixture weights in numerical analyses and in two empirical applications. I also offer heuristic selection algorithms for the correct mixture weight and analyze them in my numerical and empirical settings.

Learning to select cuts for efficient mixed-integer programming

Cutting plane methods play a significant role in modern solvers for tackling mixed-integer programming (MIP) problems. Proper selection of cuts would remove infeasible solutions in the early stage, thus largely reducing the computational burden without hurting the solution accuracy. However, the major cut selection approaches heavily rely on heuristics, which strongly depend on the specific problem at hand and thus limit their generalization capability. In this paper, we propose a data-driven and generalizable cut selection approach, named Cut Ranking, in the settings of multiple instance learning. To measure the quality of the candidate cuts, a scoring function, which takes the instance-specific cut features as inputs, is trained and applied in cut ranking and selection. In order to evaluate our method, we conduct extensive experiments on both synthetic datasets and real-world datasets. Compared with commonly used heuristics for cut selection, the learning-based policy has shown to be more effective, and is capable of generalizing over multiple problems with different properties. Cut Ranking has been deployed in an industrial solver for large-scale MIPs. In the online A/B testing of the product planning problems with more than 107 variables and constraints daily, Cut Ranking has achieved the average speedup ratio of 12.42% over the production solver without any accuracy loss of solution.

Heuristic Feature Selection for Wind Power Anomaly Events Study

Wind power ramp events are typical harmful anomaly events in wind engineering, which bring new threat to the safety operation of power systems. To in-depth understand ramps and mitigate their harms, suitable ramp characteristics are crucial in many studies, e.g., ramp definition, classification, prediction and so on. However, due to ramps’ specificity on event feature, more profound characteristics are needed besides basic ramp morphological characteristics. In this paper, an approach for extracting and selecting ramp characteristics is proposed for ramp study. First, according to ramps’ causation on energy change, wavelet transformation is introduced to analyze ramp categories, and used to extract ramp energy characteristics. Then, heuristic feature selection methods are proposed to select ramp characteristics based on specific ramp application contexts. The objective of feature selection is to remove redundant characteristics, and to improve ramp studies’ performance. Finally, combining basic ramp characteristics and wavelet characteristics, ramp studies on category classification and prediction of appointed characteristics are implemented on industrial data. The computational results validate the usefulness of wavelet characteristics, the feasibility of the proposed approach, and that performance of ramp study could be improved by using ramp characteristics in this paper.

Attribute‐scale selection for hybrid data with test cost constraint: The approach and uncertainty measures

Recently several novel cost-sensitive attribute-scale selection approaches have been proposed based on measurement errors. They are significant because they can simultaneously select attributes and scale combination to minimize the cost consumed in data processing. However, these approaches cannot deal with hybrid data with test cost constraint, and most of them do not consider the scale diversity between different attributes; and these approaches do not touch the uncertainty measurement, all of which are important issues in real applications. To address this situation, in this paper an effective cost-sensitive attribute-scale selection approach is presented based on the rough set theory, and multiple relevant uncertainty measures are developed. The main contributions of the paper are threefold. First, a generalized confidence level vector-based neighborhood rough set model is constructed. It takes into account the scale diversity between different attributes of hybrid data. Then, multiple uncertainty measures are developed. They consider both attributes and scales, thus are more general than existing ones which consider only attributes or only scales. Finally, an efficient heuristic attribute-scale selection algorithm is designed, which can select attributes and their respective scales to minimize the consumed total cost of hybrid data under any rational value of test cost upper bound. Detailed experiments thoroughly confirm the effectiveness of the proposed cost-sensitive attribute-scale selection approach. The experiments also reveal the influences of different test cost upper bounds to the attribute-scale selection and some related quantities including the uncertainty measures. This study would enrich the rough set theory to some extent, and provide an effective support for some test cost-constrained decision makings.

A Hybrid Quantum-Classical Approach to Solving Scheduling Problems

An effective approach to solving complex problems is to decompose them and integrate dedicated solvers for those subproblems. We introduce a hybrid decomposition that incorporates: (1) a quantum annealer that samples from the configuration space of a relaxed problem to obtain strong candidate solutions, and (2) a classical processor that maintains a global search tree and enforces constraints on the relaxed components of the problem. Our framework is the first to use quantum annealing as part of a complete search. We consider variants of our approach with differing amounts of guidance from the quantum annealer. We empirically test our algorithm and compare the variants on problems from three scheduling domains: graph-coloring-type scheduling, simplified Mars Lander task scheduling, and airport runway scheduling. While we were only able to test on problems of small sizes, due to the limitation of currently available quantum annealing hardware, the empirical results show that results obtained from the quantum annealer can be used for more effective search node pruning and to improve node selection heuristics when compared to a standard classical approach.

Partner selection heuristics of a chip-interleaved vMISO cooperation in WSNs subject to frequency-selective fading

Partner selection heuristics of a chip-interleaved vMISO cooperation in WSNs subject to frequency-selective fading

Hybrid K-Nearest Neighbour and Particle Swarm Optimization Technique for Divorce Classification

Judgment is the ability to make a considered decision by an evolution of knowledge. With the increasing trend of applications of artificial intelligence in law, divorce prediction has become the centre of research. Divorce classification and divorce factors determination are two of the most important matters in societies. Developing an effective technique is essential to prevent communities from collapsing. The traditional techniques of artificial intelligence play a major role in classifying divorce cases. Feature selection is a powerful pre-processing method used for data classification problems. Most previous studies on divorce classification focused on heuristic feature selection methods to determine the main factors behind divorce. These heuristic methods are considered the greedy strategy which does not produce an optimal solution. In this research, a new hybrid swarm intelligence technique was proposed using particle swarm optimisation for feature selection and the K-nearest neighbour algorithm for classification. Specifically, the proposed hybrid classifier can be used in real divorce applications where judges in their investigations can identify the factors that lead to the applications. For the experiment, five classifiers were used for performance analysis. The proposed technique was successfully applied and showed that the performance is better than the existing classifiers, namely naive Bayes, support vector machine, artificial neural network, repeated incremental pruning to pro-duce error reduction, and decision stump. Therefore, the proposed classification model is a more suitable technique for divorce classification than other artificial intelligence techniques.

Heuristic Methodology for Planning AC Rural Medium-Voltage Distribution Grids

The optimal expansion of AC medium-voltage distribution grids for rural applications is addressed in this study from a heuristic perspective. The optimal routes of a distribution feeder are selected by applying the concept of a minimum spanning tree by limiting the number of branches that are connected to a substation (mixed-integer linear programming formulation). In order to choose the caliber of the conductors for the selected feeder routes, the maximum expected current that is absorbed by the loads is calculated, thereby defining the minimum thermal bound of the conductor caliber. With the topology and the initial selection of the conductors, a tabu search algorithm (TSA) is implemented to refine the solution with the help of a three-phase power flow simulation in MATLAB for three different load conditions, i.e., maximum, medium, and minimum consumption with values of 100%, 60%, and 30%, respectively. This helps in calculating the annual costs of the energy losses that will be summed with the investment cost in conductors for determining the final costs of the planning project. Numerical simulations in two test feeders comprising 9 and 25 nodes with one substation show the effectiveness of the proposed methodology regarding the final grid planning cost; in addition, the heuristic selection of the calibers using the minimum expected current absorbed by the loads provides at least 70% of the calibers that are contained in the final solution of the problem. This demonstrates the importance of using adequate starting points to potentiate metaheuristic optimizers such as the TSA.

Feature selection using Fisher score and multilabel neighborhood rough sets for multilabel classification

In recent years, feature selection for multilabel classification has attracted attention in machine learning and data mining. However, some feature selection methods ignore the correlations among labels, resulting in low performance, and most of them face challenges in determining an appropriate neighborhood radius for neighborhood systems and suffer from expensive time cost. To overcome the issues, we propose a novel feature selection method using Fisher score and multilabel neighborhood rough sets (MNRS) in multilabel neighborhood decision systems. First, to identify the correlations between labels under a binary distribution, two types of new mutual information between labels are considered, and their balance coefficients are defined. By enhancing strong correlations and weakening weak correlations between labels, a mutual information-based Fisher score model with a second-order correlation between labels is designed to fit multilabel data. Second, to address the problem of automatically choosing a neighborhood radius, a subset of heterogeneous and homogeneous samples is employed to develop a new classification margin as a neighborhood radius, and some concepts of neighborhood, neighborhood class, and upper and lower approximations are formulated for multilabel neighborhood decision systems. The weight and dependency degree are presented to effectively measure the uncertainty of samples in multilabel neighborhood decision systems. Thus, we further present a new classification margin-based MNRS model. Finally, a filter-wrapper preprocessing algorithm for feature selection using the improved Fisher score model is proposed to decrease the spatiotemporal complexity of multilabel data, and a heuristic feature selection algorithm is designed for improve classification performance on multilabel datasets. Experimental results on thirteen multilabel datasets show that the proposed algorithm is effective in selecting significant features, demonstrating its excellent classification ability in multilabel datasets.

BiLabel-Specific Features for Multi-Label Classification

In multi-label classification, the task is to induce predictive models which can assign a set of relevant labels for the unseen instance. The strategy of label-specific features has been widely employed in learning from multi-label examples, where the classification model for predicting the relevancy of each class label is induced based on its tailored features rather than the original features. Existing approaches work by generating a group of tailored features for each class label independently, where label correlations are not fully considered in the label-specific features generation process. In this article, we extend existing strategy by proposing a simple yet effective approach based on BiLabel-specific features. Specifically, a group of tailored features is generated for a pair of class labels with heuristic prototype selection and embedding. Thereafter, predictions of classifiers induced by BiLabel-specific features are ensembled to determine the relevancy of each class label for unseen instance. To thoroughly evaluate the BiLabel-specific features strategy, extensive experiments are conducted over a total of 35 benchmark datasets. Comparative studies against state-of-the-art label-specific features techniques clearly validate the superiority of utilizing BiLabel-specific features to yield stronger generalization performance for multi-label classification.