Hard Problem Research Articles

Alternative Ruleset Discovery to Support Black-Box Model Predictions

The increasing attention to the interpretability of machine learning models has led to the development of methods to explain the behavior of black-box models in a post-hoc manner. However, such post-hoc approaches generate a new explanation for every new input, and these explanations cannot be checked by humans in advance. A method that selects decision rules from a finite ruleset as explanation for neural networks has been proposed, but it cannot be used for other models. In this paper, we propose a model-agnostic explanation method to find a pre-verifiable finite ruleset from which a decision rule is selected to support every prediction made by a given black-box model. First, we define an explanation model that selects the rule, from a ruleset, that gives the closest prediction; this rule works as an alternative explanation or supportive evidence for the prediction of a black-box model. The ruleset should have high coverage to give close predictions for future inputs, but it should also be small enough to be checkable by humans in advance. However, minimizing the ruleset while keeping high coverage leads to a computationally hard combinatorial problem. Hence, we show that this problem can be reduced to a weighted MaxSAT problem composed only of Horn clauses, which can be efficiently solved with modern solvers. Experimental results showed that our method found small rulesets such that the rules selected from them can achieve higher accuracy for structured data as compared to the existing method using rulesets of almost the same size. We also experimentally compared the proposed method with two purely rule-based models, CORELS and defragTrees. Furthermore, we examine rulesets constructed for real datasets and discuss the characteristics of the proposed method from different viewpoints including interpretability, limitation, and possible use cases.

Mobility Assisted Adaptive Clustering Hierarchy for IoT Based Sensor Networks in 5G and Beyond

One of the massive machine type communication (mMTC) applications for monitoring and sensing in 5G cellular network is the Internet of Things (IoT) based wireless sensor network (WSN). Non uniform battery usage by the nodes in these networks often results in creating energy holes or voids in the network making the network disconnected. An effective solution is to deploy multiple mobile nodes throughout the network, however finding optimal path for these mobile nodes is reported to be an NP hard problem. This paper proposes MAACH (Mobility Assisted Adaptive Clustering Hierarchy), an efficient mobility assisted clustering and routing framework for IoT based sensor network in 5G and beyond. Also, an elaborate method to calculate the exact path optimally for multiple mobile nodes is presented to alleviate non uniform energy dissipation of the sensing nodes. Simulation results show that our algorithm effectively finds the optimal trajectory of multiple mobile nodes in a distribute manner and also improves network stability period by 60-70% and the network lifetime by 70-90% across multiple network deployments.

Improving Performance, Reliability, and Feasibility in Multimodal Multitask Traffic Classification with XAI

The promise of Deep Learning (DL) in solving hard problems such as network Traffic Classification (TC) is being held back by the severe lack of transparency and explainability of this kind of approaches. To cope with this strongly felt issue, the field of eXplainable Artificial Intelligence (XAI) has been recently founded, and is providing effective techniques and approaches. Accordingly, in this work we investigate interpretability via XAIbased techniques to understand and improve the behavior of state-of-the-art multimodal and multitask DL traffic classifiers. Using a publicly available security-related dataset (ISCX VPNNONVPN), we explore and exploit XAI techniques to characterize the considered classifiers providing global interpretations (rather than sample-based ones), and define a novel classifier, DISTILLER-EVOLVED, optimized along three objectives: performance, reliability, feasibility. The proposed methodology proves as highly appealing, allowing to much simplify the architecture to get faster training time and shorter classification time, as fewer packets must be collected. This is at the expenses of negligible (or even positive) impact on classification performance, while understanding and controlling the interplay between inputs, model complexity, performance, and reliability.

A Novel Nature-Inspired Routing Scheme for Improving Routing Quality of Service in Power Grid Monitoring Systems

In the process of power grid monitoring, image data are mostly transmitted in segments by wireless image sensor networks (WISNs), and the external attacks or interference on a communication link easily causes the data to be retransmitted. Therefore, high quality of service (QoS) routing is essential to the secure and robust operation of the WISNs. Specifically, the QoS routing optimization in WISN is a nondeterministic polynomial hard problem, which usually cannot be solved by ordinary algorithms. Based on this motivation, a novel heuristic trust scheme (i.e., multi-objective trust routing scheme based on cloned elite slime mould algorithm (CESMA-MTRS)) is proposed to greatly improve the security and QoS of WISN. The scheme designs a novel trust difference test mechanism to enhance the identification of malicious nodes. Furthermore, a new elite archiving strategy and clone optimization strategy are designed to improve the convergence speed of grid monitoring data transmission routing search. The simulation results show that when the network scale is 100 and the attacked node scale is 30%, CESMA-MTRS reduces the delay by 8% and 15% and the packet loss rate by 24% and 27%, respectively, compared with trust-based routing protocol called the On-demand trust-based multicast routing protocol (ODTMRP) and evolutionary selfcooperative trust approach.

Synchronization and Control for Multiweighted and Directed Complex Networks.

The study of complex networks with multiweights (CNMWs) has been a hot topic recently. For a network with a single weight, previous studies have shown that they can promote synchronization, but for CNMWs, there are no rigorous analyses about the role of coupling matrices. In this brief, the complex network is allowed to be directed, which is the main difference with previous studies and may make the synchronization analysis difficult for multiple couplings. At first, we prove that if the inner coupling matrices are all diagonal, then synchronization can be realized only if the weighted sum (or union) of multiple coupling matrices is strongly connected, which bridges the gap between single-weighted and multiweighted networks. Moreover, we also consider the case that inner coupling matrices are positive definite but not diagonal. We design two techniques for this hard problem. One technique is to decompose inner coupling matrices into diagonal matrices and residual matrices. The other one is to measure the similarity between outer coupling matrices. In virtue of the normalized left eigenvectors (NLEVecs) corresponding to the zero eigenvalue of coupling matrices, we prove that if the Chebyshev distance between NLEVec is less than some value, defined as the allowable deviation bound, then the synchronization and control will be realized with sufficiently large coupling strengths. Furthermore, adaptive rules are also designed for coupling strength.

An energy efficient fog level resource management scheme for software defined cities

The proliferation of Internet of Things (IoT) and smart city services have paved the path for complex network architectures including edge and fog layer resources in order to meet such services’ Quality of Service (QoS). Such services should also be energy efficient for sustainable cities. Though energy efficiency has been widely studied over the past decade, yet most such solutions have focused on device or sensor level solutions or alternatively on cloud/datacenter level solutions. With the prominence of Software Defined Networks (SDNs) in recent times owing to its flexible and scalable network management features, SDN based fog level resource management has become an attractive alternative. This paper addresses fog level energy-aware resource management with the goal of minimizing energy consumption while QoS fulfillment by means of Virtual Machine (VM) consolidation through live migrations. Since any decent sized smart city would have millions of devices and large numbers of edge/fog instances, it would lead any solution mechanism to a NP hard problem. This paper proposes a novel set based Differential Evolutionary (S-DE) algorithm. Simulations have been carried out to test the efficacy of the proposed S-DE solution and the results demonstrate the efficiency of the proposed method.

An Approach for the Classification of Rock Types Using Machine Learning of Core and Log Data

Classifying rocks based on core data is the most common method used by geologists. However, due to factors such as drilling costs, it is impossible to obtain core samples from all wells, which poses challenges for the accurate identification of rocks. In this study, the authors demonstrated the application of an explainable machine-learning workflow using core and log data to identify rock types. The rock type is determined utilizing the flow zone index (FZI) method using core data first, and then based on the collection, collation, and cleaning of well log data, four supervised learning techniques were used to correlate well log data with rock types, and learning and prediction models were constructed. The optimal machine learning algorithm for the classification of rocks is selected based on a 10-fold cross-test and a comparison of AUC (area under curve) values. The accuracy rate of the results indicates that the proposed method can greatly improve the accuracy of the classification of rocks. SHapley Additive exPlanations (SHAP) was used to rank the importance of the various well logs used as input variables for the prediction of rock types and provides both local and global sensitivities, enabling the interpretation of prediction models and solving the “black box” problem with associated machine learning algorithms. The results of this study demonstrated that the proposed method can reliably predict rock types based on well log data and can solve hard problems in geological research. Furthermore, the method can provide consistent well log interpretation arising from the lack of core data while providing a powerful tool for well trajectory optimization. Finally, the system can aid with the selection of intervals to be completed and/or perforated.

The hard problem of ‘pure’ consciousness: Sāṃkhya dualist ontology

AbstractThis article addresses the theme of ‘death and immortality’ from the perspective of consciousness, and takes as its starting point a root text of Hindu philosophy, the Sāṃkhyakārikā by Īśvarakṛṣṇa (c. fourth century ce). The text posits a dualist ontology in which consciousness is separate and autonomous from a material reality that includes body and mind. The goal is to be ontologically situated in a ‘pure’ consciousness (non-objective), which signifies existential liberation. There are mundane ways to understand this claim, such as referring to cognitive states that produce affective dissociation, or more radical interpretations, such as a post-death state. This article explores the question of what Sāṃkhya's consciousness is like: it is said to be immortal, plural, individuated, and contentless. What is the motivation for and implication of engagement with a system that describes an existential freedom that may only be known in a dualist reality or after death? And how can Sāṃkhya's concepts be brought into conversation with contemporary investigations into mind–body questions? Sāṃkhya rationality counters the argument of eternal oblivion or of consciousness as an illusion confined to the brain. Yet there are resonances with Chalmers's notion of consciousness as fundamental. This article concludes that contemporary Anglo-American philosophy of religion can be enhanced by adding Sāṃkhya thought to its purview.

Nearest rank deficient matrix polynomials

The distance from a given n×n regular matrix polynomial to a nearest matrix polynomial of normal rank at most r for a specified positive integer r⩽n−1, is considered in two different norm settings. Particular emphasis is given on the distance to nearest singular matrix polynomials. The problem is shown to be equivalent to computing smallest structure preserving perturbations such that certain convolution matrices associated with the matrix polynomial become suitably rank deficient. In particular, the distance to singularity is seen to be equivalent to the rank deficiency of two different types of block Toeplitz matrices. This leads to new characterizations of the distance. Upper and lower bounds as well as information about the minimal indices of nearest singular matrix polynomials follow from these characterizations. The distances are also established to be reciprocals of certain generalized structured singular values, thus showing that computing the distance to singularity may be an NP hard problem. Based on the results, a strategy to compute the distance to singularity is devised and implemented via numerical algorithm based on BFGS and Matlab's globalsearch.m. Numerical experiments show that the computed distances compare favourably with values obtained in the literature and the bounds are tight.

The use of oral questioning to improve students’ reasoning skills in primary school mathematics learning

Reasoning skills are essential mathematical skills that need to be taught to students starting in primary school and have even become the main domain in global assessments such as TIMSS and PISA. Therefore, this study aimed to explore the implementation of the use of oral questioning in improving students’ reasoning skills in mathematics at the primary school level. Data for the qualitative case study were collected through semi-structured interviews, observations, and field notes. Six mathematics teachers from six different primary schools were selected as participants in the study using the purposive sampling method. The data were then analyzed using a constant comparative method to identify the patterns and themes that emerged. The study found six types of oral questions that are identified as being used by mathematics teachers to help students improve their mathematical reasoning skills and thinking, namely, provocative mathematical questions, puzzle-shaped questions, breaking down hard problems into easier parts, contextual questions, questions to explain the mistakes, and questions asking for clarification. The findings showed that primary mathematics teachers used a variety of oral questions to help students develop their mathematical reasoning skills and, at the same time, assist them in developing higher order thinking skills. This research has implications for expanding the literature and understanding of how primary school teachers perceive that using certain oral questions can help improve students’ mathematical reasoning. In addition, the study’s results revealed the importance of oral questioning in teaching mathematical reasoning skills.

The Silent (R)evolution of SAT

Today's powerful, robust SAT solvers have become primary tools for solving hard computational problems.

Ab initio quantum simulation of strongly correlated materials with quantum embedding

Quantum computing has shown great potential in various quantum chemical applications such as drug discovery, material design, and catalyst optimization. Although significant progress has been made in the quantum simulation of simple molecules, ab initio simulation of solid-state materials on quantum computers is still in its early stage, mostly owing to the fact that the system size quickly becomes prohibitively large when approaching the thermodynamic limit. In this work, we introduce an orbital-based multifragment approach on top of the periodic density matrix embedding theory, resulting in a significantly smaller problem size for the current near-term quantum computer. We demonstrate the accuracy and efficiency of our method compared with the conventional methodologies and experiments on solid-state systems with complex electronic structures. These include spin-polarized states of a hydrogen chain (1D-H), the equation of state of a boron nitride layer (h-BN) as well as the magnetic ordering in nickel oxide (NiO), a prototypical strongly correlated solid. Our results suggest that quantum embedding combined with a chemically intuitive fragmentation can greatly advance quantum simulation of realistic materials, thereby paving the way for solving important yet classically hard industrial problems on near-term quantum devices.

Cutting and packing problems under uncertainty: literature review and classification framework

AbstractCutting and packing problems are hard combinatorial optimization problems that arise in several manufacturing and process industries or in their supply chains. The solution of these problems is not only a scientific challenge but also has a large economic impact, as it contributes to the reduction of one of the major cost factors for many production sectors, namely raw materials, together with a positive environmental impact. The explicit consideration of uncertainty when solving cutting and packing problems with optimization techniques is crucial for a wider adoption of research results by companies. However, current research has paid little attention to the role of uncertainty in these problems. In this paper, we review the existing literature on uncertainty in cutting and packing problems, propose a classification framework, and highlight the many research gaps and opportunities for scientific contributions.

A blockchain‐based framework to optimize shipping container flows in the hinterland

AbstractWe address two interrelated issues affecting the hinterland portion of the maritime container supply chain: reducing the movement of empty containers and reducing empty trips by trucks carrying these containers. In this paper, we show that empty container flow optimization can be implemented via a blockchain based on the proof‐of‐useful‐work concept where the proof of work requires the solution of an ‐hard optimization problem whose solution benefits the blockchain participants. Accordingly, we propose that anonymous miners compete to solve the container truck routing problem, which seeks to find the most efficient routes for trucks. We show that this problem is ‐hard. Miners must also solve the problem of optimally matching consignees and shippers, which will reduce transportation and storage costs for empty containers. In essence, the proposed framework turns blockchain into a massive optimization engine that directly benefits the hinterland container supply chain ecosystem.

An effective hybrid algorithm for joint scheduling of machines and AGVs in flexible job shop

Flexible job shops motivated by small batches and multiple orders require the collaboration of machines and automated guided vehicles (AGVs) scheduling to boost shop floor flexibility and productivity. The joint scheduling of machines and AGVs can better achieve global optimization. However, joint scheduling requires two NP hard problems to be solved simultaneously. Therefore, this paper employs a multi-AGV flexible job shop scheduling problem (MA-FJSP) with an effective hybrid algorithm. First of all, a model is established with the objectives of minimizing the makespan, the total AGV running time and the total machine load. To solve the MA-FJSP, high-quality initialization methods and improved elite strategies are designed to improve global convergence in the proposed algorithm. In addition, a problem-knowledge-based neighborhood search is integrated to improve its exploitation capability. At last, a series of comparative experimental studies were performed to exam the effectiveness of the improved algorithm. The results demonstrate that the solutions gained by the proposed algorithm perform well in respect of convergence, diversity and distribution.

Intelligent performance inference: A graph neural network approach to modeling maximum achievable throughput in optical networks

One of the key performance metrics for optical networks is the maximum achievable throughput for a given network. Determining it, however, is a nondeterministic polynomial time (NP) hard optimization problem, often solved via computationally expensive integer linear programming (ILP) formulations. These are infeasible to implement as objectives, even on very small node scales of a few tens of nodes. Alternatively, heuristics are used although these, too, require considerable computation time for a large number of networks. There is, thus, a need for an ultra-fast and accurate performance evaluation of optical networks. For the first time, we propose the use of a geometric deep learning model, message passing neural networks (MPNNs), to learn the relationship between node and edge features, the network structure, and the maximum achievable network throughput. We demonstrate that MPNNs can accurately predict the maximum achievable throughput while reducing the computational time by up to five-orders of magnitude compared to the ILP for small networks (10–15 nodes) and compared to a heuristic for large networks (25–100 nodes)—proving their suitability for the design and optimization of optical networks on different time- and distance-scales.

Bounding real tensor optimizations via the numerical range

A new method of using the numerical range of a matrix to bound the optimal value of certain optimization problems over real tensor product vectors is presented. This bound is stronger than the trivial bounds based on eigenvalues and can be computed significantly faster than bounds provided by semidefinite programming relaxations. Numerous applications to other hard linear algebra problems are discussed, such as showing that a real subspace of matrices contains no rank-one matrix, and showing that a linear map acting on matrices is positive.

Binary arithmetic optimization algorithm for feature selection.

Feature selection, widely used in data preprocessing, is a challenging problem as it involves hard combinatorial optimization. So far some meta-heuristic algorithms have shown effectiveness in solving hard combinatorial optimization problems. As the arithmetic optimization algorithm only performs well in dealing with continuous optimization problems, multiple binary arithmetic optimization algorithms (BAOAs) utilizing different strategies are proposed to perform feature selection. First, six algorithms are formed based on six different transfer functions by converting the continuous search space to the discrete search space. Second, in order to enhance the speed of searching and the ability of escaping from the local optima, six other algorithms are further developed by integrating the transfer functions and Lévy flight. Based on 20 common University of California Irvine (UCI) datasets, the performance of our proposed algorithms in feature selection is evaluated, and the results demonstrate that BAOA_S1LF is the most superior among all the proposed algorithms. Moreover, the performance of BAOA_S1LF is compared with other meta-heuristic algorithms on 26 UCI datasets, and the corresponding results show the superiority of BAOA_S1LF in feature selection. Source codes of BAOA_S1LF are publicly available at: https://www.mathworks.com/matlabcentral/fileexchange/124545-binary-arithmetic-optimization-algorithm.

An Efficient Metaheuristic Algorithm for Job Shop Scheduling in a Dynamic Environment

This paper proposes an Improved Multi-phase Particle Swarm Optimization (IMPPSO) to solve a Dynamic Job Shop Scheduling Problem (DJSSP) known as an non-deterministic polynomial-time hard (NP-hard) problem. A cellular neighbor network, a velocity reinitialization strategy, a randomly select sub-dimension strategy, and a constraint handling function are introduced in the IMPPSO. The IMPPSO is used to solve the Kundakcı and Kulak problem set and is compared with the original Multi-phase Particle Swarm Optimization (MPPSO) and Heuristic Kalman Algorithm (HKA). The results show that the IMPPSO has better global exploration capability and convergence. The IMPPSO has improved fitness for most of the benchmark instances of the Kundakcı and Kulak problem set, with an average improvement rate of 5.16% compared to the Genetic Algorithm-Mixed (GAM) and of 0.74% compared to HKA. The performance of the IMPPSO for solving real-world problems is verified by a case study. The high level of operational efficiency is also evaluated and demonstrated by proposing a simulation model capable of using the decision-making algorithm in a real-world environment.

A Bayesian method for concurrently designing molecules and synthetic reaction networks

In the last few years, de novo molecular design using machine learning has made great technical progress but there are still challenges to be overcome in their practical applications. This is mostly owing to the cost and technical difficulty of synthesizing such computationally designed molecules. To overcome such barriers, various methods for synthetic route design using deep neural networks have been studied intensively in recent years. However, little progress has been made in designing molecules and their synthetic routes simultaneously. Here, we formulate the problem of simultaneously designing molecules with the desired set of properties and their synthetic routes within the framework of Bayesian inference. The design variables consist of a set of reactants in a reaction network and its network topology. The design space is extremely large because it consists of all combinations of purchasable reactants, often in the order of millions or more. In addition, the designed reaction networks can adopt any topology beyond simple multistep linear reaction routes. To solve this hard combinatorial problem, we present a powerful sequential Monte Carlo algorithm that recursively designs a synthetic reaction network by sequentially building up single-step reactions. In a case study of designing drug-like molecules based on commercially available compounds, compared with heuristic combinatorial search methods, the proposed method showed overwhelming performance in terms of computational efficiency, coverage, and novelty with respect to existing compounds. We also provide the Python library “Seq-Stack-Reaction” with its illustrative example of designing highly viscous lubricant molecules.