Search Space Partitioning Research Articles

FastTuning: Enabling Fast and Efficient Hyper-Parameter Tuning With Partitioning and Parallelism of Search Space

FastTuning: Enabling Fast and Efficient Hyper-Parameter Tuning With Partitioning and Parallelism of Search Space

Quasi-random Fractal Search (QRFS): A dynamic metaheuristic with sigmoid population decrement for global optimization

Global optimization of complex and high-dimensional functions remains a central challenge with broad applications in science and engineering. This study introduces a new optimization approach called quasi-random metaheuristic based on fractal search (QRFS), which harnesses the power of fractal geometry, low discrepancy sequences, and intelligent search space partitioning techniques. The QRFS uses fractals’ inherent self-similarity and intricate structure to guide the solution space exploration. For the proposal, a deterministic but quasi-random element is used in the search process using low discrepancy sequences, such as Sobol, Halton, Hammersley, and Latin Hypercube. This integration allows the algorithm to systematically cover the search space while maintaining the level of diversity necessary for efficient exploration. The QRFS employs a dynamic strategy of partitioning the search space and reducing the population of solutions to optimize the use of function accesses, which causes it to adapt well to the characteristics of the problem. The algorithm intelligently identifies and prioritizes promising regions within the fractal-based representation, allocating computational resources where they are most likely to yield optimal solutions. Experimental evaluations on several benchmark problems demonstrate that QRFS consistently outperforms modern, canonical metaheuristics and variants of algorithms such as differential evolution (DE), particle swarm optimization (PSO), covariance matrix adaptive evolution strategy (CMA-ES), regarding solution quality. Besides, the algorithm shows remarkable scalability, which makes it suitable for high-dimensional optimization tasks. Overall, QRFS offers a robust and efficient approach to solving complex optimization problems in various domains, paving the way for improved decision-making in real-world applications.

Balanced Clique Computation in Signed Networks: Concepts and Algorithms

Clique is one of the most fundamental models for cohesive subgraph mining in network analysis. Existing clique model mainly focuses on unsigned networks. However, in real world, many applications are modeled as signed networks with positive and negative edges. As the signed networks hold their own properties different from the unsigned networks, the existing clique model is inapplicable for the signed networks. Motivated by this, we propose the balanced clique model that considers the most fundamental and dominant theory, structural balance theory, for signed networks. Following the balanced clique model, we study the <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</u> aximal <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</u> alanced <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</u> lique <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e</u> numeration problem (MBCE) which computes all the maximal balanced cliques in a given signed network. Moreover, in some applications, users prefer a unique and representative balanced clique with maximum size rather than all balanced cliques. Thus, we also study the <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</u> aximum <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</u> alanced <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</u> lique <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s</u> earch problem (MBCS) which computes the balanced clique with maximum size. We show that MBCE problem and MBCS problem are both NP-Hard. For the MBCE problem, a straightforward solution is to treat the signed network as two unsigned networks and leverage the off-the-shelf techniques for unsigned networks. However, such a solution is inefficient for large signed networks. To address this problem, in this paper, we first propose a new maximal balanced clique enumeration algorithm by exploiting the unique properties of signed networks. Based on the new proposed algorithm, we devise two optimization strategies to further improve the efficiency of the enumeration. For the MBCS problem, we first propose a baseline solution. To overcome the huge search space problem of the baseline solution, we propose a new search framework based on search space partition. To further improve the efficiency of the new framework, we propose multiple optimization strategies regarding to redundant search branches and invalid candidates. We conduct extensive experiments on large real datasets. The experimental results demonstrate the efficiency, effectiveness and scalability of our proposed algorithms for MBCE problem and MBCS problem.

Fireworks algorithm based on search space partition

Fireworks algorithm is a novel swarm intelligence optimization framework which focuses on the potential of collaboration among multiple subpopulations with independent search ability. Although it has been proved to perform excellently in many tasks, the collaborative mechanism of fireworks is still quite undeveloped. In this paper, a theoretical model of fireworks algorithm based on search space partition is proposed, analyzed and implemented. The local search of each firework is replaced by covariance matrix adaptation evolution strategy for efficient exploitation. A coordination strategy inspired from Voronoi diagram is proposed to approximate the theoretical model for stable global exploration. Experimental results show that the proposed algorithm not only outperforms previous variants of fireworks algorithm significantly, but also achieves competitive results compared with state-of-the-art evolutionary algorithms, which are intensively fine-tuned on the objective functions.

Hierarchical Collaborated Fireworks Algorithm

The fireworks algorithm (FWA) has achieved significant global optimization ability by organizing multiple simultaneous local searches. By dynamically decomposing the target problem and handling each one with a sub-population, it has presented distinct property and applicability compared with traditional evolutionary algorithms. In this paper, we extend the theoretical model of fireworks algorithm based on search space partition to obtain a hierarchical collaboration model. It maintains both multiple local fireworks for local exploitation and one global firework for overall population distribution control. The implemented hierarchical collaborated fireworks algorithm is able to combine the advantages of both classic evolutionary algorithms and fireworks algorithms. Several experiments are provided for in-depth analysis and discussion on the proposed algorithm. The effectiveness of proposed strategy is demonstrated on the benchmark test suite from CEC 2020. Experimental results validate that the hierarchical collaborated fireworks algorithm outperforms former fireworks algorithms significantly and achieves similar results compared with state-of-the-art evolutionary algorithms.

A Detailed Study of the Distributed Rough Set Based Locality Sensitive Hashing Feature Selection Technique

In the context of big data, granular computing has recently been implemented by some mathematical tools, especially Rough Set Theory (RST). As a key topic of rough set theory, feature selection has been investigated to adapt the related granular concepts of RST to deal with large amounts of data, leading to the development of the distributed RST version. However, despite of its scalability, the distributed RST version faces a key challenge tied to the partitioning of the feature search space in the distributed environment while guaranteeing data dependency. Therefore, in this manuscript, we propose a new distributed RST version based on Locality Sensitive Hashing (LSH), named LSH-dRST, for big data feature selection. LSH-dRST uses LSH to match similar features into the same bucket and maps the generated buckets into partitions to enable the splitting of the universe in a more efficient way. More precisely, in this paper, we perform a detailed analysis of the performance of LSH-dRST by comparing it to the standard distributed RST version, which is based on a random partitioning of the universe. We demonstrate that our LSH-dRST is scalable when dealing with large amounts of data. We also demonstrate that LSH-dRST ensures the partitioning of the high dimensional feature search space in a more reliable way; hence better preserving data dependency in the distributed environment and ensuring a lower computational cost.

Diversity Control in Evolutionary Computation using Asynchronous Dual-Populations with Search Space Partitioning

Diversity control is vital for effective global optimization using evolutionary computation (EC) techniques. This paper classifies the various diversity control policies in the EC literature. Many research works have attributed the high risk of premature convergence to sub-optimal solutions to the poor exploration capabilities resulting from diversity collapse. Also, excessive cost of convergence to optimal solution has been linked to the poor exploitation capabilities necessary to focus the search. To address this exploration-exploitation trade-off, this paper deploys diversity control policies that ensure sustained exploration of the search space without compromising effective exploitation of its promising regions. First, a dual-pool EC algorithm that facilitates a temporal evolution-diversification strategy is proposed. Then a quasi-random heuristic initialisation based on search space partitioning (SSP) is introduced to ensure uniform sampling of the initial search space. Second, for the diversity measurement, a robust convergence detection mechanism that combines a spatial diversity measure; and a population evolvability measure is utilised. It was found that the proposed algorithm needed a pool size of only 50 samples to converge to optimal solutions of a variety of global optimization benchmarks. Overall, the proposed algorithm yields a 33.34% reduction in the cost incurred by a standard EC algorithm. The outcome justifies the efficacy of effective diversity control on solving complex global optimization landscapes.&#x0D; Keywords: Diversity, exploration-exploitation tradeoff, evolutionary algorithms, heuristic initialisation, taxonomy.

Parallel algorithms for the automated discovery of declarative process models

The aim of process discovery is to build a process model from an event log without prior information about the process. The discovery of declarative process models is useful when a process works in an unpredictable and unstable environment since several allowed paths can be represented as a compact set of rules. One of the tools available in the literature for discovering declarative models from logs is the Declare Miner, a plug-in of the process mining tool ProM. Using this plug-in, the discovered models are represented using Declare, a declarative process modeling language based on ltl for finite traces. However, the high execution times of the Declare Miner when processing large sets of data hampers the applicability of the tool to real-life settings. Therefore, in this paper, we propose a new approach for the discovery of Declare models based on the combination of an Apriori algorithm and a group of algorithms for Sequence Analysis to enhance the time performance of the plug-in. The approach has been developed in a way that it is easy to be parallelized using two different partitioning methods: the search space partitioning, in which different groups of candidate constraints are processed in parallel, and the database partitioning, in which different chunks of the log are processed at the same time. The approach has been implemented in ProM in its sequential version and in two multi-threading implementations leveraging these two partitioning methods. All the new variants of the plug-in have been evaluated using a large set of synthetic and real-life event logs.

Marker-based human pose tracking using adaptive annealed particle swarm optimization with search space partitioning

Pose estimation and tracking of an articulated structure based on data from multiple cameras has seen numerous applications in recent years. In this paper, a marker-based human pose tracking algorithm from multi view video sequences is proposed. The purpose of the proposed algorithm is to present a low cost motion capture system that can be used as an alternative to high cost available commercial human motion capture systems. The problem is defined as the optimization of 45 parameters which define body pose model and is solved using a modified version of particle swarm optimization (PSO) algorithm. The objective of this optimization is to maximize a fitness function which formulates how much the body model matches with 2D marker coordinates in video frames. A sampling covariance matrix is used in the first part of the velocity equation of PSO and is annealed with iterations. The sampling covariance matrix is computed adaptively, based on variance of parameters in the swarm. One of the concerns in this algorithm is the high number of parameters to define the model of body pose. To tackle this problem, we partition the optimization state space into six stages that exploit the hierarchical structure of the skeletal model. The first stage optimizes the six parameters that define the global orientation and position of the body. Other stages relate to optimization of right and left hand, right and left leg and head orientation. In the proposed partitioning method previously optimized parameters are allowed some variation in each step that is called soft partitioning. Experimental results on Pose Estimation and Action Recognition (PEAR) database indicate that the proposed algorithm achieves lower estimation error in tracking human motion compared with Annealed Particle Filter (APF) and Parametric Annealing (PA) methods.

An Effective Simulated Annealing for Influence Maximization Problem of Online Social Networks

The influence maximization problem (IMP) is one of the most well-known problems in the research domain of online social networks (OSN) that has attracted the attention of many researchers from different disciplines in recent years. One of the reasons is that the speed of information propagation in the OSN can be increased if we can find out users that have maximum influence on other users. The traditional rule-based and heuristic algorithms may not be able to find useful information out of these data because the data are generally large and complex. Although metaheuristic algorithms can be used to solve the IMP, there is still plenty of room for improvement. That is why an effective and efficient algorithm is presented in this paper. The proposed algorithm, called simulated annealing with search partition (SASP), is based on a search space partitioning mechanism to enhance the search performance of simulated annealing for the IMP. The experimental results show that the proposed algorithm outperforms the other state-of-the-art influence maximization problem algorithms compared in this paper in terms of the quality of the end result and the number of objective function evaluations.

Feasibility of rigorous multi-objective optimization of wastewater management and treatment plants

The present paper describes a study on the feasibility of coupling rigorous dynamic modelling (DM) and its extended boundaries through Life Cycle Assessment (LCA) with an Efficient Multi-Objective Optimization (EMOO) tool. The combined framework (DM-LCA-EMOO) was then applied to a real-world dynamic system: the wastewater treatment. To give a global view of all environmental, economic and technological performance, three objectives were considered: Effluent Quality Index (EQI), Operational Cost Index (OCI) and environmental impacts quantified through Life Cycle Impact Assessment (LCIA). Legally imposed constraints, including total nitrogen, total phosphorus, total chemical oxygen demand, total suspended solids and ammonium ion were also taken into account. Given the contradictory nature of objectives, the presence of constraints and the time-consuming simulation-based calculations, an Efficient Multi-Objective Optimization framework, namely Archive-based Multi-Objective Evolutionary Algorithm with Memory-based Adaptive Partitioning of search space (AMOEA-MAP) was used. The practicality of such a combined DM-LCA-EMOO tool for the evaluation of wastewater management and treatment was then addressed and demonstrated through a case application.

Weakest-Link Coalition: Further Investigation on Cooperative Interference-Aware Spectrum Sensing and Access

Interference-aware spectrum sensing extends conventional interference-agnostic sensing by exploring the fact that missed detection does not necessarily result in outage or intolerable interference on the primary user. While related work has motivated and demonstrated the benefits of interference-aware spectrum sensing for spectrum hole discovery, the inherent conflict of interests among the set of cooperative users with interference-aware spectrum detectors has not been properly formulated and addressed. In this paper, we aim to bridge the gap by considering a coalition-based model, where the set of secondary users is to be partitioned into multiple coalitions for cooperative spectrum sensing and access. The goal is to maximize the utility sum of all secondary users while observing the protection requirement of the primary user. Since the user with the largest interference probability in a coalition critically limits the detection performance of the coalition, there is a trade-off in determining the optimal coalition structure in the network. To proceed, we first formulate a joint threshold detection and coalition formation problem under the target cooperative model, and then explore important properties of the target problem. Our proposed algorithms based on polyblock approximation and search space partition can effectively solve the formulated problem with significant performance gains and complexity reduction against baseline approaches proposed in related work.

An archive-based multi-objective evolutionary algorithm with adaptive search space partitioning to deal with expensive optimization problems: Application to process eco-design

In eco-design, the integration of environmental aspects into the earliest stage of design is considered with the aim of reducing adverse environmental impacts throughout a product's life cycle. An eco-design problem is therefore multi-objective, where several objectives (environmental, economic, and technological) are to be simultaneously optimized.The optimization of industrial processes usually requires solving expensive multi-objective optimization problems (MOPs). Aiming to solve efficiently MOPs, with a limited computational budget, this paper proposes a new framework called AMOEA-MAP. The framework relies on the structure of the NSGAII algorithm and possesses two novel operators: a memory-based adaptive partitioning strategy, which provides an adaptive reticulation of the search space for a quick identification of optimal zones with less computational effort; and a bi-population evolutionary algorithm, tailored for expensive optimization problems.To ascertain its generality, the framework is first tested on several tough benchmarks. Its performance is subsequently validated on a real-world eco-design problem.

Memory-based adaptive partitioning (MAP) of search space for the enhancement of convergence in Pareto-based multi-objective evolutionary algorithms

A new algorithm, dubbed memory-based adaptive partitioning (MAP) of search space, which is intended to provide a better accuracy/speed ratio in the convergence of multi-objective evolutionary algorithms (MOEAs) is presented in this work. This algorithm works by performing an adaptive-probabilistic refinement of the search space, with no aggregation in objective space. This work investigated the integration of MAP within the state-of-the-art fast and elitist non-dominated sorting genetic algorithm (NSGAII). Considerable improvements in convergence were achieved, in terms of both speed and accuracy. Results are provided for several commonly used constrained and unconstrained benchmark problems, and comparisons are made with standalone NSGAII and hybrid NSGAII-efficient local search (eLS).

Solving multimodal optimization problems based on efficient partitioning of genotypic search space

Enhancing the exploration and exploitation of multimodal optimization is still an interesting and challenging problem in optimization. Here we present a new population-based evolutionary algorithm for multiple optimal determinations. The approach is simply based on partitioning the genotypic search space and running a simple genetic algorithm with a small population within each partition. To increase the efficiency of the algorithm, the first-order discrete derivative of the fitness function for elite solutions was used to omit extra solutions. If the derivative of the fitness function is larger than a specified gradient threshold, no optimum exists within a given partition, and no population is created there after the first iteration. Except the adjusted gradient threshold, the proposed algorithm requires no other parameters rather than those of classical genetic algorithms. Moreover, unlike the other well-known algorithms, the proposed algorithm is not sensitive to the niche radius, and it needs no prior knowledge about the fitness function. The algorithm was tested for 10 multimodal benchmark functions, and its results were compared with the other related algorithms considering 4 commonly performance criteria. Our results show that the proposed algorithm is not only acceptable in terms of diversification and function evaluation number, but also has improved efficiency as compared to the others.

Towards Next Generation Sequential and Parallel SAT Solvers

This thesis focuses on improving the SAT solving technology. The improvements focus on two major subjects: sequential SAT solving and parallel SAT solving. To better understand sequential SAT algorithms, the abstract reduction system Generic CDCL is introduced. With Generic CDCL, the soundness of solving techniques can be modeled. Next, the conflict driven clause learning algorithm is extended with the three techniques local look-ahead, local probing and all UIP learning that allow more global reasoning during search. These techniques improve the performance of the sequential SAT solver Riss. Then, the formula simplification techniques bounded variable addition, covered literal elimination and an advanced cardinality constraint extraction are introduced. By using these techniques, the reasoning of the overall SAT solving tool chain becomes stronger than plain resolution. When using these three techniques in the formula simplification tool Coprocessor before using Riss to solve a formula, the performance can be improved further. Due to the increasing number of cores in CPUs, the scalable parallel SAT solving approach iterative partitioning has been implemented in Pcasso for the multi-core architecture. Related work on parallel SAT solving has been studied to extract main ideas that can improve Pcasso. Besides parallel formula simplification with bounded variable elimination, the major extension is the extended clause sharing level based clause tagging, which builds the basis for conflict driven node killing. The latter allows to better identify unsatisfiable search space partitions. Another improvement is to combine scattering and look-ahead as a superior search space partitioning function. In combination with Coprocessor, the introduced extensions increase the performance of the parallel solver Pcasso. The implemented system turns out to be scalable for the multi-core architecture. Hence iterative partitioning is interesting for future parallel SAT solvers. Constraints (2015) 20:504–505 DOI 10.1007/s10601-015-9226-6 * Norbert Manthey norbert.manthey@tu-dresden.de 1 TU Dresden, Dresden, Germany The implemented solvers participated in international SAT competitions. In 2013 and 2014 Pcasso showed a good performance. Riss in combination with Coprocessor won several first, second and third prices, including two Kurt-Godel-Medals. Hence, the introduced algorithms improved modern SAT solving technology. School: TU Dresden Supervisors: Prof. Steffen Holldober Graduated: Monday, December 1, 2014 Link to full text: http://nbn-resolving.de/urn:nbn:de:bsz:14-qucosa-158672 Notes: Nominated for ACP Doctoral Research Award, and GI-Dissertationspreis 2014 Constraints (2015) 20:504–505 505

Towards next generation sequential and parallel SAT solvers

This thesis focuses on improving the SAT solving technology. The improvements focus on two major subjects: sequential SAT solving and parallel SAT solving. To better understand sequential SAT algorithms, the abstract reduction system Generic CDCL is introduced. With Generic CDCL, the soundness of solving techniques can be modeled. Next, the conflict driven clause learning algorithm is extended with the three techniques local look-ahead, local probing and all UIP learning that allow more global reasoning during search. These techniques improve the performance of the sequential SAT solver Riss. Then, the formula simplification techniques bounded variable addition, covered literal elimination and an advanced cardinality constraint extraction are introduced. By using these techniques, the reasoning of the overall SAT solving tool chain becomes stronger than plain resolution. When using these three techniques in the formula simplification tool Coprocessor before using Riss to solve a formula, the performance can be improved further. Due to the increasing number of cores in CPUs, the scalable parallel SAT solving approach iterative partitioning has been implemented in Pcasso for the multi-core architecture. Related work on parallel SAT solving has been studied to extract main ideas that can improve Pcasso. Besides parallel formula simplification with bounded variable elimination, the major extension is the extended clause sharing level based clause tagging, which builds the basis for conflict driven node killing. The latter allows to better identify unsatisfiable search space partitions. Another improvement is to combine scattering and look-ahead as a superior search space partitioning function. In combination with Coprocessor, the introduced extensions increase the performance of the parallel solver Pcasso. The implemented system turns out to be scalable for the multi-core architecture. Hence iterative partitioning is interesting for future parallel SAT solvers. Constraints (2015) 20:504–505 DOI 10.1007/s10601-015-9226-6 * Norbert Manthey norbert.manthey@tu-dresden.de 1 TU Dresden, Dresden, Germany The implemented solvers participated in international SAT competitions. In 2013 and 2014 Pcasso showed a good performance. Riss in combination with Coprocessor won several first, second and third prices, including two Kurt-Godel-Medals. Hence, the introduced algorithms improved modern SAT solving technology. School: TU Dresden Supervisors: Prof. Steffen Holldober Graduated: Monday, December 1, 2014 Link to full text: http://nbn-resolving.de/urn:nbn:de:bsz:14-qucosa-158672 Notes: Nominated for ACP Doctoral Research Award, and GI-Dissertationspreis 2014 Constraints (2015) 20:504–505 505

An empirical evaluation of a three‐tier conduit framework for multifaceted test case classification and selection using fuzzy‐ant colony optimisation approach

SummaryThe test case optimisation is an NP‐complete, knowledge‐driven, data‐driven, and multidimensional search space partitioning and dimension reduction problem. In the multifaceted test case classification, partitioning and reducing the multidimensional test case fitness search space is the critical problem. The vague nature of fitness parameters, conflicting nature objectives, and ambiguity in the test case fitness evaluation have created and increased the uncertainty, the imprecision, and the incompleteness in the test case classification and selection. Because of the increasing ambiguity, the complexity, and the cost of software testing, automated test case classification and selection has emerged as an appropriate tool to classify test cases into predefined categories using the multifaceted concept. Most of the test cases affecting the performance of the classifier are irrelevant and redundant. A strong need therefore exists to devise an intelligent technique to identify and remove test cases affecting the performance of the classifier. For increasing the performance of classifier, multifaceted test case selection is used to reduce fitness search space to be searched. In this paper, a three‐tier sequential framework is proposed for a multifaceted test case classification and selection. The first stage of the proposed framework is the fuzzy synthesis‐based filtration approach for multifaceted test case fitness evaluation and classification. The second stage of the proposed framework is the fuzzy entropy‐based filtration technique with a backward search strategy, used for estimating and reducing the ambiguity in test case fitness evaluation, classification, and selection. The third stage of the proposed framework is the ant colony optimisation‐based wrapper technique with a forward search strategy, employed to select test cases from the output (reduced) test suite by the second stage. The proposed framework is tested on artefacts of benchmark applications. The results of the empirical study clearly show that the third stage of our proposed method outperforms the second and first stages, and the performance of the algorithms used in all three stages increases on average as the stages are escalating. The classification accuracy is enhanced by reducing the ambiguity in fitness and the classification of test cases, increasing the number of test cases accurately classified, and reducing the number in the test case pool to be exercised. Copyright © 2014 John Wiley &amp; Sons, Ltd.

A Quantum Genetic Algorithm for Continuous Space Optimization based on Search Space Partition

A Quantum Genetic Algorithm for Continuous Space Optimization based on Search Space Partition

Partitioning Search Spaces of a Randomized Search

This paper studies the following question: given an instance of the propositional satisfiability problem, a randomized satisfiability solver, and a cluster of n computers, what is the best way to use the computers to solve the instance? Two approaches, simple distribution and search space partitioning as well as their combinations are investigated both analytically and empirically. It is shown that the results depend heavily on the type of the problem (unsatisfiable, satisfiable with few solutions, and satisfiable with many solutions) as well as on how good the search space partitioning function is. In addition, the behavior of a real search space partitioning function is evaluated in the same framework. The results suggest that in practice one should combine the simple distribution and search space partitioning approaches.