Tree Search Research Articles

Intelligent sequential multi-impulse collision avoidance method for non-cooperative spacecraft based on an improved search tree algorithm

The problem of collision avoidance for non-cooperative targets has received significant attention from researchers in recent years. Non-cooperative targets exhibit uncertain states and unpredictable behaviors, making collision avoidance significantly more challenging than that for space debris. Much existing research focuses on the continuous thrust model, whereas the impulsive maneuver model is more appropriate for long-duration and long-distance avoidance missions. Additionally, it is important to minimize the impact on the original mission while avoiding non-cooperative targets. On the other hand, the existing avoidance algorithms are computationally complex and time-consuming especially with the limited computing capability of the on-board computer, posing challenges for practical engineering applications. To conquer these difficulties, this paper makes the following key contributions: (A) a turn-based (sequential decision-making) limited-area impulsive collision avoidance model considering the time delay of precision orbit determination is established for the first time; (B) a novel Selection Probability Learning Adaptive Search-depth Search Tree (SPL-ASST) algorithm is proposed for non-cooperative target avoidance, which improves the decision-making efficiency by introducing an adaptive-search-depth mechanism and a neural network into the traditional Monte Carlo Tree Search (MCTS). Numerical simulations confirm the effectiveness and efficiency of the proposed method.

On prime scenarios in qualitative spatial and temporal reasoning

The concept of prime implicant is a fundamental tool in Boolean algebra, which is used in Boolean circuit design and, recently, in explainable AI. This study investigates an analogous concept in qualitative spatial and temporal reasoning, called prime scenario. Specifically, we define a prime scenario of a qualitative constraint network (QCN) as a minimal set of decisions that can uniquely determine solutions of this QCN. We propose in this paper a collection of algorithms designed to address various problems related to prime scenarios, and also show how certain results can be useful for measuring the robustness of a QCN. In addition, we study the relationship between our notions in this paper and the notion of prime sub-QCN in the literature, and establish theoretical results in the process. Further, we devise a language based on the notion of prime scenario for knowledge compilation. Finally, an experimental evaluation is performed with instances of Allen's Interval Algebra and RCC8 to assess the efficiency of our algorithms and, hence, also the difficulty of the newly introduced problems here.

Monte Carlo tree search applied to design of wireless power transfer system

Monte Carlo tree search applied to design of wireless power transfer system

A crossword solving system based on Monte Carlo tree search

Although the development of AI in games is remarkable, intelligent machines still lag behind humans in games that require the ability of language understanding. In this paper, we focus on the crossword puzzle resolution task. Solving crossword puzzles is a challenging task since it requires the ability to answer natural language questions with knowledge and the ability to execute a search over possible answers to find an optimal set of solutions for the grid. Previous solutions are devoted to exploiting heuristic strategies in search to find solutions while having limited ability to explore the search space. We build a comprehensive system for crossword puzzle resolution based on Monte Carlo Tree Search (MCTS). As far as we know, we are the first to model the crossword puzzle resolution problem as a Markov Decision Process and apply the MCTS to solve it. We construct a dataset for crossword puzzle resolution based on daily puzzles from The New York Times with detailed specifications of both the puzzle and clue database selection. Our method achieves state-of-the-art performance on the dataset. The code of the system and experiments in this paper is publicly available: https://www.github.com/lhlclhl/CP.

Dynamic compact data structure for temporal reachability with unsorted contact insertions

Abstract Temporal graphs represent interactions between entities over time. Deciding whether entities can reach each other through temporal paths is useful for various applications such as in communication networks and epidemiology. Previous works have studied the scenario in which addition of new interactions can happen at any point in time. A known strategy maintains, incrementally, a Timed Transitive Closure using a dynamic data structure composed of $O(n^{2})$ binary search trees containing non-nested time intervals. However, space usage for storing these trees grows rapidly as more interactions are inserted. In this paper, we introduce a compact data structure that represents each tree as two dynamic bit-vectors. Furthermore, we present two variants of this data structure: one representing bits in dynamic bit-vectors explicitly and the other representing only the active bits by encoding their consecutive distances. In our experiments, we observed that our data structure improves space usage while having similar time performance for incremental updates when comparing with the previous strategy. The first variant of our data structure gives the best space improvement when constructing Time Transitive Closures for temporally dense temporal graphs, and the second variant uses less space for temporally sparse temporal graphs.

Bi-HS-RRTX\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$^\ ext {X}$$\\end{document}: an efficient sampling-based motion planning algorithm for unknown dynamic environments

In the field of autonomous mobile robots, sampling-based motion planning methods have demonstrated their efficiency in complex environments. Although the Rapidly-exploring Random Tree (RRT) algorithm and its variants have achieved significant success in known static environment, it is still challenging in achieving optimal motion planning in unknown dynamic environments. To address this issue, this paper proposes a novel motion planning algorithm Bi-HS-RRTX\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$^\ ext {X}$$\\end{document}, which facilitates asymptotically optimal real-time planning in continuously changing unknown environments. The algorithm swiftly determines an initial feasible path by employing the bidirectional search. When dynamic obstacles render the planned path infeasible, the bidirectional search is reactivated promptly to reconstruct the search tree in a local area, thereby significantly reducing the search planning time. Additionally, this paper adopts a hybrid heuristic sampling strategy to optimize the planned path quality and search efficiency. The convergence of the proposed algorithm is accelerated by merging local biased sampling with nominal path and global heuristic sampling in hyper-ellipsoid region. To verify the effectiveness and efficiency of the proposed algorithm in unknown dynamic environments, numerous comparative experiments with existing algorithms were conducted. The experimental results indicate that the proposed planning algorithm has significant advantages in planned path length and planning time.

Monte Carlo tree search for dynamic shortest‐path interdiction

AbstractWe present a reinforcement learning‐based heuristic for a two‐player interdiction game called the dynamic shortest path interdiction problem (DSPI). The DSPI involves an evader and an interdictor who take turns in the problem, with the interdictor selecting a set of arcs to attack and the evader choosing an arc to traverse at each step of the game. Our model employs the Monte Carlo tree search framework to learn a policy for the players using randomized roll‐outs. This policy is stored as an asymmetric game tree and can be further refined as the game unfolds. We leverage alpha–beta pruning and existing bounding schemes in the literature to prune suboptimal branches. Our numerical experiments demonstrate that the prescribed approach yields near‐optimal solutions in many cases and allows for flexibility in balancing solution quality and computational effort.

Domain independent heuristics for online stochastic contingent planning

AbstractPartially observable Markov decision processes (POMDP) are a useful model for decision-making under partial observability and stochastic actions. Partially Observable Monte-Carlo Planning (POMCP) is an online algorithm for deciding on the next action to perform, using a Monte-Carlo tree search approach, based on the UCT algorithm for fully observable Markov-decision processes. POMCP develops an action-observation tree, and at the leaves, uses a rollout policy to provide a value estimate for the leaf. As such, POMCP is highly dependent on the rollout policy to compute good estimates, and hence identify good actions. Thus, many practitioners who use POMCP are required to create strong, domain-specific heuristics. In this paper, we model POMDPs as stochastic contingent planning problems. This allows us to leverage domain-independent heuristics that were developed in the planning community. We suggest two heuristics, the first is based on the well-known $$h_{add}$$ h add heuristic from classical planning, and the second is computed in belief space, taking the value of information into account.

Effectiveness of removable expander appliances in posterior crossbite: A review of the literature

Objective: To evaluate the effectiveness of removable expander appliances in the treatment of posterior crossbite by reviewing and analyzing clinical studies, scientific articles and published case reports. Background: Posterior crossbite is a dental malocclusion characterized by a transverse misalignment of the upper and lower teeth, where the upper teeth bite into the lower teeth. This condition can affect one or more dental organs and if left untreated, can lead to functional and esthetic problems, as well as imbalances in maxillofacial growth. Early correction of posterior crossbite is crucial to avoid future complications such as uneven tooth wear, temporomandibular dysfunction and facial asymmetries, and different treatment methods have been used, including removable appliances. Methodology: The research was structured from a search tree (PubMed, Scielo, Google Scholar, Dialnet, Scopus), with keywords such as: removable expandable appliance; posterior crossbite; removable orthodontics; treatment efficacy AND posterior crossbite; posterior crossbite NOT fixed appliance. Conclusions: Removable expander appliances are an effective option for treating posterior crossbite in children. Studies show that these devices correct malocclusion, promote maxillary growth and improve dental alignment. The effectiveness of the treatment depends on the cooperation of the child and parents. The choice of appliance should be based on a careful evaluation of the patient's needs, considering cooperation, severity of the malocclusion and the preferences of the patient and the orthodontist.

Model inductive bias enhanced deep reinforcement learning for robot navigation in crowded environments

Navigating mobile robots in crowded environments poses a significant challenge and is essential for the coexistence of robots and humans in future intelligent societies. As a pragmatic data-driven approach, deep reinforcement learning (DRL) holds promise for addressing this challenge. However, current DRL-based navigation methods have possible improvements in understanding agent interactions, feedback mechanism design, and decision foresight in dynamic environments. This paper introduces the model inductive bias enhanced deep reinforcement learning (MIBE-DRL) method, drawing inspiration from a fusion of data-driven and model-driven techniques. MIBE-DRL extensively incorporates model inductive bias into the deep reinforcement learning framework, enhancing the efficiency and safety of robot navigation. The proposed approach entails a multi-interaction network featuring three modules designed to comprehensively understand potential agent interactions in dynamic environments. The pedestrian interaction module can model interactions among humans, while the temporal and spatial interaction modules consider agent interactions in both temporal and spatial dimensions. Additionally, the paper constructs a reward system that fully accounts for the robot’s direction and position factors. This system's directional and positional reward functions are built based on artificial potential fields (APF) and navigation rules, respectively, which can provide reasoned evaluations for the robot's motion direction and position during training, enabling it to receive comprehensive feedback. Furthermore, the incorporation of Monte-Carlo tree search (MCTS) facilitates the development of a foresighted action strategy, enabling robots to execute actions with long-term planning considerations. Experimental results demonstrate that integrating model inductive bias significantly enhances the navigation performance of MIBE-DRL. Compared to state-of-the-art methods, MIBE-DRL achieves the highest success rate in crowded environments and demonstrates advantages in navigation time and maintaining a safe social distance from humans.

Optimizing Harmonic Mitigation for Smooth Integration of Renewable Energy: A Novel Approach Using Atomic Orbital Search and Feedback Artificial Tree Control

Optimizing Harmonic Mitigation for Smooth Integration of Renewable Energy: A Novel Approach Using Atomic Orbital Search and Feedback Artificial Tree Control

The Influence of the Number of Tree Searches on Maximum Likelihood Inference in Phylogenomics.

Maximum likelihood (ML) phylogenetic inference is widely used in phylogenomics. As heuristic searches most likely find suboptimal trees, it is recommended to conduct multiple (e.g., 10) tree searches in phylogenetic analyses. However, beyond its positive role, how and to what extent multiple tree searches aid ML phylogenetic inference remains poorly explored. Here, we found that a random starting tree was not as effective as the BioNJ and parsimony starting trees in inferring the ML gene tree and that RAxML-NG and PhyML were less sensitive to different starting trees than IQ-TREE. We then examined the effect of the number of tree searches on ML tree inference with IQ-TREE and RAxML-NG, by running 100 tree searches on 19,414 gene alignments from 15 animal, plant, and fungal phylogenomic datasets. We found that the number of tree searches substantially impacted the recovery of the best-of-100 ML gene tree topology among 100 searches for a given ML program. In addition, all of the concatenation-based trees were topologically identical if the number of tree searches was ≥10. Quartet-based ASTRAL trees inferred from 1 to 80 tree searches differed topologically from those inferred from 100 tree searches for 6/15 phylogenomic datasets. Finally, our simulations showed that gene alignments with lower difficulty scores had a higher chance of finding the best-of-100 gene tree topology and were more likely to yield the correct trees.

Phylo2Vec: a vector representation for binary trees

Abstract Binary phylogenetic trees inferred from biological data are central to understanding the shared history among evolutionary units. However, inferring the placement of latent nodes in a tree is computationally expensive. State-of-the-art methods rely on carefully designed heuristics for tree search, using different data structures for easy manipulation (e.g., classes in object-oriented programming languages) and readable representation of trees (e.g., Newick-format strings). Here, we present Phylo2Vec, a parsimonious encoding for phylogenetic trees that serves as a unified approach for both manipulating and representing phylogenetic trees. Phylo2Vec maps any binary tree with n leaves to a unique integer vector of length n − 1. The advantages of Phylo2Vec are fourfold: i) fast tree sampling, (ii) compressed tree representation compared to a Newick string, iii) quick and unambiguous verification if two binary trees are identical topologically, and iv) systematic ability to traverse tree space in very large or small jumps. As a proof of concept, we use Phylo2Vec for maximum likelihood inference on five real-world datasets and show that a simple hill-climbing-based optimisation scheme can efficiently traverse the vastness of tree space from a random to an optimal tree.

PPB-MCTS: A novel distributed-memory parallel partial-backpropagation Monte Carlo tree search algorithm

Monte-Carlo Tree Search (MCTS) is an adaptive and heuristic tree-search algorithm designed to uncover sub-optimal actions at each decision-making point. This method progressively constructs a search tree by gathering samples throughout its execution. Predominantly applied within the realm of gaming, MCTS has exhibited exceptional achievements. Additionally, it has displayed promising outcomes when employed to solve NP-hard combinatorial optimization problems. MCTS has been adapted for distributed-memory parallel platforms. The primary challenges associated with distributed-memory parallel MCTS are the substantial communication overhead and the necessity to balance the computational load among various processes. In this work, we introduce a novel distributed-memory parallel MCTS algorithm with partial backpropagations, referred to as Parallel Partial-Backpropagation MCTS (PPB-MCTS). Our design approach aims to significantly reduce the communication overhead while maintaining, or even slightly improving, the performance in the context of combinatorial optimization problems. To address the communication overhead challenge, we propose a strategy involving transmitting an additional backpropagation message. This strategy avoids attaching an information table to the communication messages exchanged by the processes, thus reducing the communication overhead. Furthermore, this approach contributes to enhancing the decision-making accuracy during the selection phase. The load balancing issue is also effectively addressed by implementing a shared transposition table among the parallel processes. Furthermore, we introduce two primary methods for managing duplicate states within distributed-memory parallel MCTS, drawing upon techniques utilized in addressing duplicate states within sequential MCTS. Duplicate states can transform the conventional search tree into a Directed Acyclic Graph (DAG). To evaluate the performance of our proposed parallel algorithm, we conduct an extensive series of experiments on solving instances of the Job-Shop Scheduling Problem (JSSP) and the Weighted Set-Cover Problem (WSCP). These problems are recognized for their complexity and classified as NP-hard combinatorial optimization problems with considerable relevance within industrial applications. The experiments are performed on a cluster of computers with many cores. The empirical results highlight the enhanced scalability of our algorithm compared to that of the existing distributed-memory parallel MCTS algorithms. As the number of processes increases, our algorithm demonstrates increased rollout efficiency while maintaining an improved load balance across processes.

Distant retrograde orbit baseline generation considering solar eclipse mitigation

Mitigating solar eclipses presents a formidable challenge in the design of spacecraft orbits, particularly in the case of a distant retrograde orbit (DRO), which is susceptible to significant solar eclipse threats. This paper undertakes a comprehensive analysis of the occurrence characteristics and patterns of both Moon and Earth eclipses on a DRO. The proposed approach involves a two-stage design procedure, encompassing initial orbit selection and orbit extension. The initial orbit selection method primarily focuses on mitigating Earth eclipses by employing the out-of-plane motion inherent in a DRO. The study meticulously explores the influence of three pivotal elements — amplitude, period, and initial phase — on the efficacy of Earth eclipse avoidance. Concurrently, when faced with unavoidable solar eclipses of the initial orbit, a series of tiny maneuvers are systematically applied to extend the baseline orbit. The parameter design space is discretized, and a tree search method is employed to identify viable maneuver sequences. The research successfully achieves the long-term stabilization of the baseline DRO, ensuring that solar eclipses do not exceed 2 h in 10 years. The proposed method’s robustness and applicability are substantiated through validation under the ephemeris model.

The Functional Essence of Imperative Binary Search Trees

The Functional Essence of Imperative Binary Search Trees

Tackling rolling security-constrained unit commitment under uncertainty with Monte Carlo value search and empirical knowledge

Recently, the growing uncertainty in the power system has created an increasing need for security-constrained unit commitment (SCUC) solved on a rolling perspective (i.e., RSCUC). This puts a high requirement on the decision efficiency of the solution algorithm. To overcome the challenge, this paper proposes a POML-MCVS algorithm, which integrates a probability-oriented machine learning (POML) technique with Monte-Carlo value search (MCVS) algorithm to improve overall performance. First, POML is employed to identify the decision probabilities of units through leveraging empirical knowledge, and determine the on/off states of units with high decision probabilities. This facilitates improving the decision efficiency while maintaining the decision accuracy as much as possible. Then, MCVS is employed to optimize the rest variables and make the rolling decisions. As a modified version of Monte Carlo tree search, MCVS is able to detect the promising decisions, and evaluate their values efficiently through constructing the search tree asymmetrically, which further improves the decision efficiency. Numerical simulations conducted on a modified IEEE 39-bus system and a real 2778-bus system demonstrate the effectiveness of the proposed algorithm.

Stateful black-box fuzzing for encryption protocols and its application in IPsec

Owing to the rapid development of information security technology, the security analysis of encryption protocols has received widespread attention. In this paper, we propose a stateful black-box encryption protocol fuzzing method to analyze the security of real-world black-box encryption protocol programs and devices. This method does not rely on the source code but uses captured packets as input, performs state selection based on a Monte Carlo tree search algorithm, and processes the encryption/decryption conversion of mutant test cases based on the intermediate mapper. It sends test cases and collects responses in interactive communication with the tested program and dynamically optimizes the corpus based on the collected state information. Based on this method, we develop SBEPFuzz and primarily analyze IPsec. We evaluate SBEPFuzz on six widely used IPsec implementations. The experimental results show that SBEPFuzz achieves higher code coverage, and can discover more protocol state sequences and vulnerabilities. Furthermore, we discover four anomalies, including malformed packets triggering service crashes and abnormal interactions leading to plaintext ID leakage, which also reflect the differences in details among different implementations.

Online model adaptation in Monte Carlo tree search planning

AbstractWe propose a model-based reinforcement learning method using Monte Carlo Tree Search planning. The approach assumes a black-box approximated model of the environment developed by an expert using any kind of modeling framework and it improves the model as new information from the environment is collected. This is crucial in real-world applications, since having a complete knowledge of complex environments is impractical. The expert’s model is first translated into a neural network and then it is updated periodically using data, i.e., state-action-next-state triplets, collected from the real environment. We propose three different methods to integrate data acquired from the environment with prior knowledge provided by the expert and we evaluate our approach on a domain concerning air quality and thermal comfort control in smart buildings. We compare the three proposed versions with standard Monte Carlo Tree Search planning using the expert’s model (without adaptation), Proximal Policy Optimization (a popular model-free DRL approach) and Stochastic Lower Bounds Optimization (a popular model-based DRL approach). Results show that our approach achieves the best results, outperforming all analyzed competitors.

Comparing search algorithms on the retrosynthesis problem.

In this article we try different algorithms, namely Nested Monte Carlo Search and Greedy Best First Search, on AstraZeneca's open source retrosynthetic tool : AiZynthFinder. We compare these algorithms to AiZynthFinder's base Monte Carlo Tree Search on a benchmark selected from the PubChem database and by Bayer's chemists. We show that both Nested Monte Carlo Search and Greedy Best First Search outperform AstraZeneca's Monte Carlo Tree Search, with a slight advantage for Nested Monte Carlo Search while experimenting on a playout heuristic. We also show how the search algorithms are bounded by the quality of the policy network, in order to improve our results the next step is to improve the policy network.