Tree Search Research Articles

‘Journeys in the Dark’ - Towards Game Master AI in Complex Board Games

The Game Master is a player role synonymous with many tabletop games. The asymmetric gameplay of the role provides different opportunities compared to other players, and can be both cooperative and competitive with the other players in the same game. Though complex environments for exploring human and Artificial Intelligence collaboration exist, few focus on the Game Master role's semi-cooperative play. Here, we propose a new complex environment based on the board game `Descent: Journeys in the Dark (Second Edition)', as part of the Tabletop Games Framework, showcasing one-versus-many play, tactical combat, and large, dynamic action and state spaces. We include baseline AI player performance of Monte Carlo Tree Search agents in this game, finding them to be well-adept at considering multiple possible end-game conditions compared to the greedy One Step Look Ahead agents. In-depth analysis reveals interesting behaviours and Hero synergies, with the aim of informing the design of games and AI models to enhance human experience in semi-cooperative environments.

Mining contextually meaningful subgraphs from a vertex-attributed graph.

Networks have emerged as a natural data structure to represent relations among entities. Proteins interact to carry out cellular functions and protein-Protein interaction network analysis has been employed for understanding the cellular machinery. Advances in genomics technologies enabled the collection of large data that annotate proteins in interaction networks. Integrative analysis of interaction networks with gene expression and annotations enables the discovery of context-specific complexes and improves the identification of functional modules and pathways. Extracting subnetworks whose vertices are connected and have high attribute similarity have applications in diverse domains. We present an enumeration approach for mining sets of connected and cohesive subgraphs, where vertices in the subgraphs have similar attribute profile. Due to the large number of cohesive connected subgraphs and to overcome the overlap among these subgraphs, we propose an algorithm for enumerating a set of representative subgraphs, the set of all closed subgraphs. We propose pruning strategies for efficiently enumerating the search tree without missing any pattern or reporting duplicate subgraphs. On a real protein-protein interaction network with attributes representing the dysregulation profile of genes in multiple cancers, we mine closed cohesive connected subnetworks and show their biological significance. Moreover, we conduct a runtime comparison with existing algorithms to show the efficiency of our proposed algorithm.

Spectrum Sensing and Resource Allocation for Frequency Hopping Based CRN Using Hazelnut Tree Search Algorithm

This paper presents a novel spectrum sensing and resource allocation method for Cognitive Radio Networks (CRNs) using the Hazelnut Tree Search (HTS) algorithm in a frequency-hopping framework. CRNs enable secondary users (SUs) to utilize unused licensed spectrum without causing interference to primary users (PUs). The proposed Spectrum Sensing and Resource Allocation using Hazelnut Tree Search in CRN (SS-RA-HTS-CRN) method detects PU presence, prompting SUs to switch channels to avoid interference, guided by a designated guide node (GN). The HTS algorithm selects optimal sensing nodes and improves spectrum sensing, enhancing resource allocation reliability. Simulations in NS-3.26 show that SS-RA-HTS-CRN achieves delay reductions of 28.93%, 31.9%, and 22.45%, delivery ratio improvements of 28.21%, 17.41%, and 11.565%, and packet drop rate reductions of 35.4%, 44.3%, and 23.12% compared with Energy-Aware Resource Allocation with Backtracking Search Optimization (SS-RA-BSO-CRN), Spectrum Sensing with Hybrid Particle Swarm and Gravitational Search (SS-hybrid PSOGSA-CRN), and Spectrum Sensing using Modified Spider Monkey Optimization (SS-MSMO-CRN). The proposed SS-RA-HTS-CRN method significantly enhances efficiency, reliability, and adaptability in CRNs, making it a promising solution for dynamic spectrum management.

A Unified Perspective on Value Backup and Exploration in Monte-Carlo Tree Search

Monte-Carlo Tree Search (MCTS) is a class of methods for solving complex decisionmaking problems through the synergy of Monte-Carlo planning and Reinforcement Learning (RL). The highly combinatorial nature of the problems commonly addressed by MCTS requires the use of efficient exploration strategies for navigating the planning tree and quickly convergent value backup methods. These crucial problems are particularly evident in recent advances that combine MCTS with deep neural networks for function approximation. In this work, we propose two methods for improving the convergence rate and exploration based on a newly introduced backup operator and entropy regularization. We provide strong theoretical guarantees to bound convergence rate, approximation error, and regret of our methods. Moreover, we introduce a mathematical framework based on the use of the α-divergence for backup and exploration in MCTS. We show that this theoretical formulation unifies different approaches, including our newly introduced ones, under the same mathematical framework, allowing to obtain different methods by simply changing the value of α. In practice, our unified perspective offers a flexible way to balance between exploration and exploitation by tuning the single α parameter according to the problem at hand. We validate our methods through a rigorous empirical study from basic toy problems to the complex Atari games, and including both MDP and POMDP problems.

A novel reinforcement learning-based method for structure optimization

With the rapid development of deep learning technology, Reinforcement Learning (RL) has garnered considerable acclaim within the realm of structural optimization owing to its excellent exploration mechanism. However, the widespread application of RL in this field is limited owing to the excessive number of iterations required to converge and the expensive computational cost it brings. To address these challenges, this article presents a novel RL framework for structural optimization, combining Monte Carlo tree search with the proximal policy optimization method, called LMPOM. The key contributions of LMPOM encompass: (1) an enhanced Monte Carlo tree search strategy for partitioning the hybrid design space; (2) a strategy for adaptively updating surrogate models to reduce simulation costs; and (3) the introduction of a novel termination condition for the RL algorithms. Through tests on three benchmark problems, compared with previous RL algorithms, LMPOM consistently shows fewer iterations and better optimization results.

Time-lapsed in-situ monitoring of mechanical property in deep soil mixing with surface wave

Evaluating the mechanical properties of deep soil mixing (DSM) requires destructive borehole coring because they are mainly situated underground. Although surface wave method offers potential for quality assurance, its complexity arises from multi-mode phenomena and the need to re-evaluate inversion results, often necessitating manual interpretation. This paper presents a data-driven surface wave framework to retrieve field DSM profile Vs over time, incorporating a mode-free forward operator and the Monte Carlo Tree Search (MCTS) inversion. Validations using synthetic data affirmed the framework’s accuracy and efficiency in tracking subsurface shear wave velocity. In a real-world DSM site, the proposed method successfully captures the mechanical properties evolution across two DSM layers over the curing period, aligning well with site investigations and borehole coring. This pioneering monitoring framework integrates geotechnical engineering with geophysics expertise, underscoring the value of non-destructive seismic methods for measuring subsurface property evolution.

Terraces in species tree inference from gene trees

A terrace in a phylogenetic tree space is a region where all trees contain the same set of subtrees, due to certain patterns of missing data among the taxa sampled, resulting in an identical optimality score for a given data set. This was first investigated in the context of phylogenetic tree estimation from sequence alignments using maximum likelihood (ML) and maximum parsimony (MP). It was later extended to the species tree inference problem from a collection of gene trees, where a set of equally optimal species trees was referred to as a “pseudo” species tree terrace which does not consider the topological proximity of the trees in terms of the induced subtrees resulting from certain patterns of missing data. In this study, we mathematically characterize species tree terraces and investigate the mathematical properties and conditions that lead multiple species trees to induce/display an identical set of locus-specific subtrees owing to missing data. We report that species tree terraces are agnostic to gene tree heterogeneity. Therefore, we introduce and characterize a special type of gene tree topology-aware terrace which we call “peak terrace”. Moreover, we empirically investigated various challenges and opportunities related to species tree terraces through extensive empirical studies using simulated and real biological data. We demonstrate the prevalence of species tree terraces and the resulting ambiguity created for tree search algorithms. Remarkably, our findings indicate that the identification of terraces could potentially lead to advances that enhance the accuracy of summary methods and provide reasonably accurate branch support.

GREMI: An Explainable Multi-Omics Integration Framework for Enhanced Disease Prediction and Module Identification.

Multi-omics integration has demonstrated promising performance in complex disease prediction. However, existing research typically focuses on maximizing prediction accuracy, while often neglecting the essential task of discovering meaningful biomarkers. This issue is particularly important in biomedicine, as molecules often interact rather than function individually to influence disease outcomes. To this end, we propose a two-phase framework named GREMI to assist multi-omics classification and explanation. In the prediction phase, we propose to improve prediction performance by employing a graph attention architecture on sample-wise co-functional networks to incorporate biomolecular interaction information for enhanced feature representation, followed by the integration of a joint-late mixed strategy and the true-class-probability block to adaptively evaluate classification confidence at both feature and omics levels. In the interpretation phase, we propose a multi-view approach to explain disease outcomes from the interaction module perspective, providing a more intuitive understanding and biomedical rationale. We incorporate Monte Carlo tree search (MCTS) to explore local-view subgraphs and pinpoint modules that highly contribute to disease characterization from the global-view. Extensive experiments demonstrate that the proposed framework outperforms state-of-the-art methods in seven different classification tasks, and our model effectively addresses data mutual interference when the number of omics types increases. We further illustrate the functional- and disease-relevance of the identified modules, as well as validate the classification performance of discovered modules using an independent cohort.

Analysis of the Application of Artificial Intelligence in Mahjong

Over the past few years, AI-based models for Mahjong have received a substantial amount of contribution due to the advancements in machine learning and game theory. In this review, these state-of-the-art AI models are considered, including Tjong, Kanachan, Suphx, and Zhejiang University’s developed model. The deployed approaches are disparately represented by reinforcement learning, Monte Carlo tree search (MCTS), and deep neural networks, all of which aim at solving the issues that come with the game’s structure, such as incomplete and overburdening information. The Tjong model complements the power of the transformer architecture with hierarchical decision-making and in turn, brings strategic depth to the gameplay. Kanachan employs Q-learning and MCTS to optimize decision-making, while Suphx combines these methods with the novel ideas of Double Q-learning and Thompson Sampling to achieve higher performance than seen before. The integration of reinforcement learning and a new evaluation model imbues the model with the unique properties of both learning machines and human expertise, namely depth and intelligence. As a result of all these exploits, obstacles still exist, the most notable among them are the management of completely unknown information, handling long-distance games, and the strategic balance between offense and defense. In the future, determinants of probabilistic reasoning, the improvement of deep learning systems, and the prowess of reward shaping will result in AI improvement in Mahjong.

Coordination of NPCs in multi-agent systems based on behavior trees

Abstract. This paper addresses the design and coordination challenges of controlling non-player characters' (NPCs) behaviors in multi-agent systems (MAS) using behavior trees (BTs), which are preferred over finite state machines (FSMs) due to their hierarchical structure and ease of maintenance. While BTs effectively resolve the question of "What to do next?" for individual NPCs, their application in MAS, particularly with the integration of a blackboard system for central control, reveals limitations in efficiency, robustness, and heuristic capacity as system complexity increases. To explore solutions to these challenges, this study analyzes various algorithms that enhance the functionality of behavior trees within MAS. The research focuses on three primary areas: the optimization of behavior trees, the development of behavior tree search algorithms, and the improvement of communication algorithms within BTs. Methods involve a comparative analysis of existing and new algorithmic approaches to identify and address inefficiencies in NPC coordination. The findings indicate that advanced behavior tree configurations, when combined with innovative search and communication strategies, significantly improve the coordination, robustness, and operational efficiency of MAS. These enhancements allow for more dynamic and responsive NPC interactions in complex gaming environments.

Optimization of Truck–Cargo Matching for the LTL Logistics Hub Based on Three-Dimensional Pallet Loading

This study investigates the truck–cargo matching problem in less-than-truckload (LTL) logistics hubs, focusing on optimizing the three-dimensional loading of goods onto standardized pallets and assigning these loaded pallets to a fleet of heterogeneous vehicles. A two-stage hybrid heuristic algorithm is proposed to solve this complex logistics challenge. In the first stage, a tree search algorithm based on residual space is developed to determine the optimal layout for the 3D loading of cargo onto pallets. In the second stage, a heuristic online truck–cargo matching algorithm is introduced to allocate loaded pallets to trucks while optimizing the number of trucks used and minimizing transportation costs. The algorithm operates within a rolling time horizon, allowing it to dynamically handle real-time order arrivals and time window constraints. Numerical experiments demonstrate that the proposed method achieves high pallet loading efficiency (close to 90%), near-optimal truck utilization (nearly 95%), and significant cost reductions, making it a practical solution for real-world LTL logistics operations.

Improved Monte Carlo tree search formulation with multiple root nodes for discrete sizing optimization of truss structures

This article proposes a novel reinforcement learning algorithm using an improved Monte Carlo tree search (IMCTS) formulation for the discrete optimum design of truss structures. IMCTS with multiple root nodes includes the update process, best reward, accelerating technique and terminal condition. The update process means that once a final solution is found, it is used as the initial solution for the next search tree. The best reward is used in the backpropagation step. The accelerating technique is introduced by decreasing the width of the search tree and reducing the maximum number of iterations. The agent is trained to minimize the total structural weight under various constraints until the terminal condition is satisfied. The optimal solution is the minimum value of all solutions found by the search trees. Numerical examples show that the agent can find the optimal solution with low computational cost, stably produces an optimal design, and is suitable for multi-objective structural optimization and large-scale structures.

Uncertainty Qualification for Deep Learning-Based Elementary Reaction Property Prediction.

The prediction of the thermodynamic and kinetic properties of elementary reactions has shown rapid improvement due to the implementation of deep learning (DL) methods. While various studies have reported the success in predicting reaction properties, the quantification of prediction uncertainty has seldom been investigated, thus compromising the confidence in using these predicted properties in practical applications. Here, we integrated graph convolutional neural networks (GCNN) with three uncertainty prediction techniques, including deep ensemble, Monte Carlo (MC)-dropout, and evidential learning, to provide insights into the uncertainty quantification and utility. The deep ensemble model outperforms others in accuracy and shows the highest reliability in estimating prediction uncertainty across all elementary reaction property data sets. We also verified that the deep ensemble model showed a satisfactory capability in recognizing epistemic and aleatoric uncertainties. Additionally, we adopted a Monte Carlo Tree Search method for extracting the explainable reaction substructures, providing a chemical explanation for DL predicted properties and corresponding uncertainties. Finally, to demonstrate the utility of uncertainty qualification in practical applications, we performed an uncertainty-guided calibration of the DL-constructed kinetic model, which achieved a 25% higher hit ratio in identifying dominant reaction pathways compared to that of the calibration without uncertainty guidance.

Tiny Pointers

This paper introduces a new data-structural object that we call the tiny pointer. In many applications, traditional \(\log n\) -bit pointers can be replaced with \(o(\log n)\) -bit tiny pointers at the cost of only a constant-factor time overhead and a small probability of failure. We develop a comprehensive theory of tiny pointers, and give optimal constructions for both fixed-size tiny pointers (i.e., settings in which all of the tiny pointers must be the same size) and variable-size tiny pointers (i.e., settings in which the average tiny-pointer size must be small, but some tiny pointers can be larger). If a tiny pointer references an item in an array filled to load factor \(1-\delta\) , then the optimal tiny-pointer size is \(\Theta(\log\log\log n+\log\delta^{-1})\) bits in the fixed-size case, and \(\Theta(\log\delta^{-1})\) expected bits in the variable-size case. Our tiny-pointer constructions also require us to revisit several classic problems having to do with balls and bins; these results may be of independent interest. Using tiny pointers, we apply tiny pointers to five classic data-structure problems. We show that: A data structure storing \(n\) \(v\) -bit values for \(n\) keys with constant-factor time modifications/queries can be implemented to take space \(nv+O(n\log^{(r)}n)\) bits, for any constant \(r\gt0\) , as long as the user stores a tiny pointer of expected size \(O(1)\) with each key—here, \(\log^{(r)}n\) is the \(r\) -th iterated logarithm. Any binary search tree can be made succinct, meaning that it achieves \((1+o(1))\) times the optimal space, with constant-factor time overhead, and can even be made to be within \(O(n)\) bits of optimal if we allow for \(O(\log^{*}n)\) -time modifications—this holds even for rotation-based trees such as the splay tree and the red-black tree. Any fixed-capacity key-value dictionary can be made stable (i.e., items do not move once inserted) with constant-factor time overhead and \((1+o(1))\) -factor space overhead. Any key-value dictionary that requires uniform-size values can be made to support arbitrary-size values with constant-factor time overhead and with an additional space consumption of \(\log^{(r)}n+O(\log j)\) bits per \(j\) -bit value for an arbitrary constant \(r\gt0\) of our choice. Given an external-memory array \(A\) of size \((1+\varepsilon)n\) containing a dynamic set of up to \(n\) key-value pairs, it is possible to maintain an internal-memory stash of size \(O(n\log\varepsilon^{-1})\) bits so that the location of any key-value pair in \(A\) can be computed in constant time (and with no IOs). In each case tiny pointers allow for us to take a natural space-inefficient solution that uses pointers and make it space-efficient for free.

Optimizing a linear function over the efficient set of a multiple objective integer quadratic program

In this paper, we investigate the optimization of a function over the efficient set of a Multiple Objective Integer Quadratic Program (MOIQP). The method described employs a branch-and-cut approach to address the problem. At every node within the search tree, a profound cut is derived by considering the ascending trends of the criteria. This cut enables the removal of branches where the associated sub-problems don’t contain efficient solutions. The proposed methodology is illustrated in an example. Extensive numerical experiments are conducted to demonstrate the method’s effectiveness against state-of-the-art methods.

Real-time reactive task allocation and planning of large heterogeneous multi-robot systems with temporal logic specifications

Existing methods for the task allocation and planning (TAP) of multi-robot systems with temporal logic specifications mainly rely on optimization-based approaches or graph search techniques applied to the product automaton. However, these methods suffer from high computational cost and scale poorly with the number of robots and the complexity of temporal logic tasks, thus limiting the applicability in real-time implementation, especially for large multi-robot systems. To address these challenges, this work develops a novel TAP framework that can solve reactive temporal logic planning problems for large-scale heterogeneous multi-robot systems (HMRS) in real time. Specifically, we develop a planning decision tree (PDT) to represent the task progression and task allocation specialized for HMRS with temporal logic specifications. Based on the PDT, we develop two key search algorithms—the planning decision tree search (PDTS) and the interactive planning decision tree search (IPDTS)—where PDTS generates an offline plan which will be modified online by PDTS and IPDTS jointly to enable fast reactive planning if environmental changes or temporary tasks occur. Such a design can generate satisfying plan for HMRS with multiple orders of magnitude more robots than those that existing methods can manipulate. Rigorous analysis shows that the PDT-based planning is feasible (i.e., the generated plan is applicable) and complete (i.e., a feasible plan, if exits, is guaranteed to be found). The algorithm complexity further indicates that the solution time is only linearly proportional to the robot numbers and types. Simulation and experiment results demonstrate that reactive plan can be generated for large HMRS in real-time, which outperforms the state-of-the-art methods.

Deconstructing Deep Active Inference: A Contrarian Information Gatherer.

Active inference is a theory of perception, learning, and decision making that can be applied to neuroscience, robotics, psychology, and machine learning. Recently, intensive research has been taking place to scale up this framework using Monte Carlo tree search and deep learning. The goal of this activity is to solve more complicated tasks using deep active inference. First, we review the existing literature and then progressively build a deep active inference agent as follows: we (1) implement a variational autoencoder (VAE), (2) implement a deep hidden Markov model (HMM), and (3) implement a deep critical hidden Markov model (CHMM). For the CHMM, we implemented two versions, one minimizing expected free energy, CHMM[EFE] and one maximizing rewards, CHMM[reward]. Then we experimented with three different action selection strategies: the ε-greedy algorithm as well as softmax and best action selection. According to our experiments, the models able to solve the dSprites environment are the ones that maximize rewards. On further inspection, we found that the CHMM minimizing expected free energy almost always picks the same action, which makes it unable to solve the dSprites environment. In contrast, the CHMM maximizing reward keeps on selecting all the actions, enabling it to successfully solve the task. The only difference between those two CHMMs is the epistemic value, which aims to make the outputs of the transition and encoder networks as close as possible. Thus, the CHMM minimizing expected free energy repeatedly picks a single action and becomes an expert at predicting the future when selecting this action. This effectively makes the KL divergence between the output of the transition and encoder networks small. Additionally, when selecting the action down the average reward is zero, while for all the other actions, the expected reward will be negative. Therefore, if the CHMM has to stick to a single action to keep the KL divergence small, then the action down is the most rewarding. We also show in simulation that the epistemic value used in deep active inference can behave degenerately and in certain circumstances effectively lose, rather than gain, information. As the agent minimizing EFE is not able to explore its environment, the appropriate formulation of the epistemic value in deep active inference remains an open question.

An exact branch-and-price-and-cut algorithm for a practical and large-scale dial-a-ride problem

Vehicle routing problems are encountered in various practical transportation scenarios. One such problem is the dial-a-ride problem (DARP), which involves transporting efficiently passengers from their origin to their destination and whose objective is to design vehicle routes minimizing total costs and accommodating all pickup-and-delivery requests. The solution must also adhere to capacity limitations, time windows, and a maximum ride time for each passenger. This paper proposes an exact branch-price-and-cut algorithm to solve a practical variant of the DARP, where both the passengers and vehicle fleet are heterogeneous, and other practical constraints such as break requirements and maximum route duration are imposed. In this algorithm, column generation which alternates between solving a master problem and subproblems, is used to compute lower bounds in the search tree. We develop a labeling algorithm to efficiently handle these subproblems. The effectiveness of the algorithm is evaluated on a real-world case study involving 849 heterogeneous passengers and more than 70 available vehicles, as well as on 10 smaller instances (with 300 to 820 trips) extracted from another very large real-world dataset. To the best of our knowledge, this 849-trip instance is the largest DARP instance reported to be solved to optimality in the literature.

Separated spacecraft attitude algorithm research based on star sensor multi-information fusion

Faced with the problem of poor anti-interference ability of star map recognition technology, in order to improve its anti-interference ability and achieve higher attitude measurement accuracy, this article proposes a star map recognition method based on binary search trees. Star sensors and gyroscopes are combined to solve the problem of the poor real-time performance of star sensors, optimizing the Extended Kalman Filtering (EKF) algorithm. This improves attitude output accuracy and formulates corresponding attitude measurement schemes considering the situation where the star sensor is not visible. The results showed that the improved star pattern recognition algorithm had better adaptability and robustness than the rasterization algorithm, with a high recognition success rate of 95.6% when the star sensor field of view was 3°*3°, while the rasterization algorithm cannot recognize small fields of view. The recognition success rate of the algorithm was 98.6% when the standard deviation of magnitude noise was 2.0 pixels, which was 15.4% higher than the rasterization algorithm. Unlike the EKF algorithm, the optimized EKF algorithm had a higher attitude output accuracy, with an average error of about 0.000 211°; compared to before optimization, it has increased by 93.43%. In the attitude measurement scheme, the relative attitude angle measurement error was small, and the maximum attitude measurement error in the rolling angle direction was 1.8 × 10−3°. This paper adopts a method to achieve high-precision satellite attitude measurement with good adaptability, which can be applied in complex space missions.

Bounds and Algorithms for Alphabetic Codes and Binary Search Trees

Bounds and Algorithms for Alphabetic Codes and Binary Search Trees