Sampling-based Methods Research Articles

A Sampling-Based Approach to Solve Difficult Path Planning Queries Efficiently in Narrow Environments for Autonomous Ground Vehicles

Path planning is an essential subproblem of autonomous robots’ navigation. Reaching a given goal pose or covering the available space are typical navigation missions, that require different planning approaches. We focus on such problems in this paper, where a goal pose must be reached by a wheeled autonomous ground vehicle in challenging situations, i.e. in complex environments with limited free space. Many path-planning methods are available, from which the sampling-based approaches gained the highest interest due to their computational efficiency. However, the performance of such methods degrades if the free space is limited and narrow passages have to be crossed on the way to the goal. Finding real-time planning methods to deliver high-quality paths in such situations is challenging. This paper aims to take steps toward solving this problem. On the one hand, an approach is presented to characterize free space narrowness and the difficulty of planning tasks. This can be used as a tool to compare planning queries and evaluate the performance of planning methods from the perspective of their sensitivity to environmental narrowness. On the other hand, an improved variant of our previously proposed RTR planner, an incremental sampling-based path-planning method, is introduced that exhibits good performance even in narrow and difficult planning situations. It is shown by simulations that it outperforms the popular RRT and RRT* planners in terms of running time and path quality, and that it is less sensitive to the narrowness of the environment where the planning task has to be solved.

Rapid Approximation of Low-Thrust Spacecraft Reachable Sets within Complex Two-Body and Cislunar Dynamics

The reachable set of controlled dynamical systems is the set of all reachable states from an initial condition over a certain time horizon, subject to operational constraints and exogenous disturbances. In astrodynamics, rapid approximation of reachable sets is invaluable for trajectory planning, collision avoidance, and ensuring safe and optimal performance in complex dynamics. Leveraging the connection between minimum-time trajectories and the boundary of reachable sets, we propose a sampling-based method for rapid and efficient approximation of reachable sets for finite- and low-thrust spacecraft. The proposed method combines a minimum-time multi-stage indirect formulation with the celebrated primer vector theory. Reachable sets are generated under two-body and circular restricted three-body (CR3B) dynamics. For the two-body dynamics, reachable sets are generated for (1) the heliocentric phase of a benchmark Earth-to-Mars problem, (2) two scenarios with uncertainties in the initial position and velocity of the spacecraft at the time of departure from Earth, and (3) a scenario with a bounded single impulse at the time of departure from Earth. For the CR3B dynamics, several cislunar applications are considered, including L1 Halo orbit, L2 Halo orbit, and Lunar Gateway 9:2 NRHO. The results indicate that low-thrust spacecraft reachable sets coincide with invariant manifolds existing in multi-body dynamical environments. The proposed method serves as a valuable tool for qualitatively analyzing the evolution of reachable sets under complex dynamics, which would otherwise be either incoherent with existing grid-based reachability approaches or computationally intractable with a complete Hamilton–Jacobi–Bellman method.

Sampling-based novel heterogeneous multi-layer stacking ensemble method for telecom customer churn prediction

In recent times, customer churn has become one of the most significant issues in business-oriented sectors with telecommunication being no exception. Maintaining current customers is particularly valuable due to the high degree of rivalry among telecommunication companies and the costs of acquiring new ones. The early prediction of churned customers may help telecommunication companies to identify the causes of churn and design industrial tactics to address or mitigate the churn problem. Controlling customer churn by developing efficient and reliable customer churn prediction (CCP) solutions is essential to achieving this objective. Findings from existing CCP studies have shown that numerous methods, such as rule-based and machine-learning (ML) mechanisms, have been devised to solve the CCP problem. Nonetheless, the problems of adaptability and the resilience of rule-based CCP solutions are its major weaknesses, and the skewed pattern of churn datasets (class imbalance) is detrimental to the prediction performances of conventional ML models in CCP. Hence, this research developed a robust heterogeneous multi-layer stacking ensemble method (HMSE) for effective CCP. Specifically, in the HMSE method, the prediction prowess of five ML classifiers (Random Forest (RF), Bayesian network (BN), Support Vector Machine (SVM), K-Nearest Neighbour (KNN), and Repeated Incremental Pruning to Produce Error Reduction (RIPPER)) with distinct computational characteristics are ensembled based on stacking and the resulting model is further enhanced using a forest penalizing attribute (FPA) model. The synthetic minority oversampling technique (SMOTE) is integrated with the proposed HMSE to balance the skewed class label present in the original experimental datasets. Extensive tests were carried out to determine the effectiveness of the proposed HMSE and S-HMSE on standard telecom CCP datasets. Observed findings from the experimental results showed that HMSE and S-HMSE can effectively predict churners even with the class imbalance (skewed datasets) problem. In addition, comparison studies demonstrated that the suggested S-HMSE offered improved prediction performance and optimum solutions for CCP in the telecom sector in comparison with baseline classifiers, homogeneous ensemble methods, and current CCP approaches.

Sampling Methods for Inner Product Sketching

Recently, Bessa et al. (PODS 2023) showed that sketches based on coordinated weighted sampling theoretically and empirically outperform popular linear sketching methods like Johnson-Lindentrauss projection and CountSketch for the ubiquitous problem of inner product estimation. We further develop this finding by introducing and analyzing two alternative sampling-based methods. In contrast to the computationally expensive algorithm in Bessa et al., our methods run in linear time (to compute the sketch) and perform better in practice, significantly beating linear sketching on a variety of tasks. For example, they provide state-of-the-art results for estimating the correlation between columns in unjoined tables, a problem that we show how to reduce to inner product estimation in a black-box way. While based on known sampling techniques (threshold and priority sampling) we introduce significant new theoretical analysis to prove approximation guarantees for our methods.

Smooth-Guided Implicit Data Augmentation for Domain Generalization.

The training process of a domain generalization (DG) model involves utilizing one or more interrelated source domains to attain optimal performance on an unseen target domain. Existing DG methods often use auxiliary networks or require high computational costs to improve the model's generalization ability by incorporating a diverse set of source domains. In contrast, this work proposes a method called Smooth-Guided Implicit Data Augmentation (SGIDA) that operates in the feature space to capture the diversity of source domains. To amplify the model's generalization capacity, a distance metric learning (DML) loss function is incorporated. Additionally, rather than depending on deep features, the suggested approach employs logits produced from cross entropy (CE) losses with infinite augmentations. A theoretical analysis shows that logits are effective in estimating distances defined on original features, and the proposed approach is thoroughly analyzed to provide a better understanding of why logits are beneficial for DG. Moreover, to increase the diversity of the source domain, a sampling-based method called smooth is introduced to obtain semantic directions from interclass relations. The effectiveness of the proposed approach is demonstrated through extensive experiments on widely used DG, object detection, and remote sensing datasets, where it achieves significant improvements over existing state-of-the-art methods across various backbone networks.

CDFRS: A scalable sampling approach for efficient big data analysis

The sampling-based approximation method has demonstrated its potential in various domains such as machine learning, query processing, and data analysis. Most preceding sampling algorithms generate samples at the record level, making it impractical to apply them to very large datasets using a single machine. Even distributed solutions encounter efficiency issues when dealing with terabyte-scale datasets. In this paper, we introduce a scalable sampling approach named CDFRS, which can generate samples with a distribution-preserving guarantee from extensive datasets. CDFRS exhibits significantly improved speed compared to existing sampling algorithms when dealing with terabyte-scale datasets. We provide theoretical guarantees and empirical justifications, demonstrating that samples generated by the CDFRS approach maintain the distribution characteristics of the original dataset. Additionally, we propose a sample size determination algorithm, denoted as A2. Experiment results indicate that the running time of CDFRS shows at least an order of magnitude improvement over other distributed sampling methods. Notably, sampling a 10TB dataset using CDFRS only takes hundreds of seconds, while the compared method requires more than ten thousand seconds. In the context of big data analysis, including tasks such as classification and clustering, models trained with samples generated by CDFRS closely match those trained with the entire training set. Furthermore, the proposed A2 algorithm efficiently determines an appropriate sample size compared with traditional methods.

Position Checking-Based Sampling Approach Combined with Attraction Point Local Optimization for Safe Flight of UAVs.

Trading off the allocation of limited computational resources between front-end path generation and back-end trajectory optimization plays a key role in improving the efficiency of unmanned aerial vehicle (UAV) motion planning. In this paper, a sampling-based kinodynamic planning method that can reduce the computational cost as well as the risks of UAV flight is proposed. Firstly, an initial trajectory connecting the start and end points without considering obstacles is generated. Then, a spherical space is constructed around the topological vertices of the environment, based on the intersections of the trajectory with the obstacles. Next, some unnecessary sampling points, as well as node rewiring, are discarded by the designed position-checking strategy to minimize the computational cost and reduce the risks of UAV flight. Finally, in order to make the planning framework adaptable to complex scenarios, the strategies for selecting different attraction points according to the environment are designed, which further ensures the safe flight of the UAV while improving the success rate of the front-end trajectory. Simulations and real-world experiment comparisons are conducted on a vision-based platform to verify the performance of the proposed method.

A Unified Approach for the Calculation of Different Sample-Based Measures with the Single Sampling Method

This paper explores two sample-based methods for analysing multistable systems: basin stability and basin entropy. Both methods rely on many numerical integration trials conducted with diverse initial conditions. The collected data is categorised and used to compute metrics that characterise solution stability, phase space structure, and system dynamics predictability. Basin stability assesses the overall likelihood of reaching specific solutions, while the basin entropy measure aims to capture the structure of attraction basins and the complexity of their boundaries. Although these two metrics complement each other effectively, their original procedures for computation differ significantly. This paper introduces a universal approach and algorithm for calculating basin stability and entropy measures. The suitability of these procedures is demonstrated through the analysis of two non-linear systems.

SHAP@k: Efficient and Probably Approximately Correct (PAC) Identification of Top-K Features

The SHAP framework provides a principled method to explain the predictions of a model by computing feature importance. Motivated by applications in finance, we introduce the Top-k Identification Problem (TkIP) (and its ordered variant TkIP- O), where the objective is to identify the subset (or ordered subset for TkIP-O) of k features corresponding to the highest SHAP values with PAC guarantees. While any sampling-based method that estimates SHAP values (such as KernelSHAP and SamplingSHAP) can be trivially adapted to solve TkIP, doing so is highly sample inefficient. Instead, we leverage the connection between SHAP values and multi-armed bandits (MAB) to show that both TkIP and TkIP-O can be reduced to variants of problems in MAB literature. This reduction allows us to use insights from the MAB literature to develop sample-efficient variants of KernelSHAP and SamplingSHAP. We propose KernelSHAP@k and SamplingSHAP@k for solving TkIP; along with KernelSHAP-O and SamplingSHAP-O to solve the ordering problem in TkIP-O. We perform extensive experiments using several credit-related datasets to show that our methods offer significant improvements of up to 40× in sample efficiency and 39× in runtime.

Piecewise Linear Transformation – Propagating Aleatoric Uncertainty in Neural Networks

Real-world data typically exhibit aleatoric uncertainty which has to be considered during data-driven decision-making to assess the confidence of the decision provided by machine learning models. To propagate aleatoric uncertainty represented by probability distributions (PDs) through neural networks (NNs), both sampling-based and function approximation-based methods have been proposed. However, these methods suffer from significant approximation errors and are not able to accurately represent predictive uncertainty in the NN output. In this paper, we present a novel method, Piecewise Linear Transformation (PLT), for propagating PDs through NNs with piecewise linear activation functions (e.g., ReLU NNs). PLT does not require sampling or specific assumptions about the PDs. Instead, it harnesses the piecewise linear structure of such NNs to determine the propagated PD in the output space. In this way, PLT supports the accurate quantification of predictive uncertainty based on the criterion exactness of the propagated PD. We assess this exactness in theory by showing error bounds for our propagated PD. Further, our experimental evaluation validates that PLT outperforms competing methods on publicly available real-world classification and regression datasets regarding exactness. Thus, the PDs propagated by PLT allow to assess the uncertainty of the provided decisions, offering valuable support.

Sample-and-Bound for Non-convex Optimization

Standard approaches for global optimization of non-convex functions, such as branch-and-bound, maintain partition trees to systematically prune the domain. The tree size grows exponentially in the number of dimensions. We propose new sampling-based methods for non-convex optimization that adapts Monte Carlo Tree Search (MCTS) to improve efficiency. Instead of the standard use of visitation count in Upper Confidence Bounds, we utilize numerical overapproximations of the objective as an uncertainty metric, and also take into account of sampled estimates of first-order and second-order information. The Monte Carlo tree in our approach avoids the usual fixed combinatorial patterns in growing the tree, and aggressively zooms into the promising regions, while still balancing exploration and exploitation. We evaluate the proposed algorithms on high-dimensional non-convex optimization benchmarks against competitive baselines and analyze the effects of the hyper parameters.

Global sensitivity analysis for degraded braided composite with interval process

Global sensitivity analysis has predominantly focused on examining the effect of time-invariant parameters on the mechanical properties of fiber-reinforced composites, while ignoring degradation. However, fiber-reinforced composites are subject to degradation, which means that the physical system is dynamic. It is important to consider this dynamic behavior in sensitivity analysis. This paper presents a set of sensitivity indices based on the interval method to measure the influence of the time-invariant and time-variant parameters on the synthetic uncertainty of output in global sensitivity analysis for 2D triaxially braided composites (2DTBCs) considering degradation. To achieve this purpose, we represent time-variant uncertain parameter using an interval process and describe them with multiple uncorrelated interval variables using the interval Karhunen-Loève explansion method. We then introduce a two-dimensional interval process and similarity degree to define a new sensitivity index for time-invariant uncertain parameters. For time-variant uncertain parameter, a novel sensitivity index combining multi-dimensional interval process and similarity degree is proposed which considers the synthetic influence of the group of multiple uncorrelated interval variables. Additionally, an efficient sampling-based method is presented to estimate the global sensitivity analysis indices using the Monte Carlo method and Newton-Cotes formula. Finally, the feasibility and potential benefits of the proposed sensitivity indices are demonstrated by applying them to a 2DTBC made of carbon fiber reinforced polyether ether ketone with thermal ageing consideration.

Entropy-based concept drift detection in information systems

As time passes, the data within information systems may continuously evolve, causing the target concept to drift. To ensure the effectiveness of data-driven decision making, it is crucial to detect drift in a timely manner and gather relevant information. In this paper, we introduce two methods that can directly detect concept drift in the provided information system, by considering a new perspective on uncertainty. First, using entropy under a single attribute constraint, we define the uncertainty of the target concept in an information system. By integrating the uncertainty of each attribute, the overall uncertainty of the target concept in the information system is obtained. Subsequently, two concept drift detection methods are proposed, namely EBTBM (Entropy-Based Threshold-Based Method) and EBSBM (Entropy-Based Sampling-Based Method). These methods utilize the defined uncertainty of the target concept as a statistical measure of the difference between two data blocks. Finally, extensive experiments on artificial and real-world data sets are conducted to validate the effectiveness of the proposed concept drift detection methods.

A heuristic autonomous exploration method based on environmental information gain during quadrotor flight

Autonomous exploration is a widely studied fundamental application in the field of quadrotor, which requires them to automatically explore unknown space to obtain complete information about the environment. The frontier-based method, one of the representative works on autonomous exploration, drives autonomous determination by the definition of frontier information so that complete information about the environment is available to the quadrotor. However, existing frontier-based methods are able to accomplish the task but still suffer from inefficient exploration, and how to improve the efficiency of autonomous exploration is the focus of research nowadays. Slow frontier generation affecting real-time viewpoint determination and insufficient determination methods affecting the quality of viewpoints are typical of these problems. Therefore, to overcome the aforementioned problems, this article proposes a two-level viewpoint determination method for frontier-based autonomous exploration. First, a sampling-based frontier detection method is presented for faster frontier generation, improving the immediacy of environmental representation compared to traditional traversal-based methods. Second, the access to environmental information during flight is considered for the first time, and an innovative heuristic evaluation function is designed to decide on high-quality viewpoint as the next local navigation target in each exploration iteration. Extensive benchmark and real-world tests are conducted to validate our method. The results confirm that our method optimizes the frontier search time by 85%, the exploration time by around 20%–30%, and the exploration path by 25%–35%.

Machine learning aided uncertainty quantification for engineering structures involving material-geometric randomness and data imperfection

In real-world engineering, uncertainty is ubiquitous within material properties, structural geometry, load conditions, and the like. These uncertainties have substantial impacts on the estimation of structural performance. Furthermore, information or datasets in real life commonly contain imperfections, e.g., noise, outliers, or missing data. To quantify these impacts induced by uncertainties on structural behaviours and reduce the effects of data imperfections simultaneously, a machine learning-aided stochastic analysis framework is proposed. A novel supervised machine learning technique, namely the Capped Extended Support Vector Regression (CX-SVR) technique, is developed to effectively suppress the effects of outliers and noise in datasets. Its inherent convexity in optimization and capped strategy theoretically supports the accuracy of CX-SVR, especially in handling datasets with imperfections. Once the effective surrogate model is established, subsequent analyses, like sampling-based methods, can circumvent the cumbersome physical model, which is potentially the nest of computational burden and errors in engineering applications. The high robustness of the proposed approach can be summarized in four main aspects: unrestrictive selection of the system inputs and their statistical information, ‘perfect’ or ‘imperfect’ data, enough statistical information (including statistical moments, probability density functions, and cumulative distribution functions) of the system outputs, and physical problems from various engineering fields.

Fast autonomous exploration with sparse topological graphs in large-scale environments

Exploring large-scale environments autonomously poses a significant challenge. As the size of environments increases, the computational cost becomes a hindrance to real-time operation. Additionally, while frontier-based exploration planning provides convenient access to environment frontiers, it suffers from slow global exploration speed. On the other hand, sampling-based methods can effectively explore individual regions but fail to cover the entire environment. To overcome these limitations, we present a hierarchical exploration approach that integrates frontier-based and sampling-based methods. It assesses the informational gain of sampling points by considering the quantity of frontiers in the vicinity, and effectively enhances exploration efficiency by utilizing a utility function that takes account of the direction of advancement for the purpose of selecting targets. To improve the search speed of global topological graph in large-scale environments, this paper introduces a method for constructing a sparse topological graph. It incrementally constructs a three-dimensional sparse topological graph by dynamically capturing the spatial structure of free space through uniform sampling. In various challenging simulated environments, the proposed approach demonstrates comparable exploration performance in comparison with the state-of-the-art approaches. Notably, in terms of computational efficiency, the single iteration time of our approach is less than one-tenth of that required by the recent advances in autonomous exploration.

Improved nonlinear model predictive control with inequality constraints using particle filtering for nonlinear and highly coupled dynamical systems

Abstract Motion planning and controller design are challenging tasks for highly coupled and nonlinear dynamical systems such as autonomous vehicles and robotic applications. Nonlinear model predictive control (NMPC) is an emerging technique in which sampling-based methods are used to synthesize the control and trajectories for complex systems. In this study, we have developed the sampling-based motion planning algorithm with NMPC through Bayesian estimation to solve the online nonlinear constrained optimization problem. In the literature, different filtration techniques have been applied to extract knowledge of states in the presence of noise. Due to the detrimental effects of linearization, the Kalman filter with NMPC only achieves modest effectiveness. Moving horizon estimation (MHE), on the other hand, frequently relies on simplifying assumptions and lacks an effective recursive construction. Additionally, it adds another optimization challenge to the regulation problem that has to be solved online. To address this problem, particle filtering is implemented for Bayesian filtering in nonlinear and highly coupled dynamical systems. It is a sequential Monte Carlo method that involves representing the posterior distribution of the state of the system using a set of weighted particles that are propagated through time using a recursive algorithm. For nonlinear and strongly coupled dynamical systems, the novel sampling-based NMPC technique is effective and simple to use. The efficiency of the suggested method has been assessed using simulated studies.

P969 Unveiling the molecular mechanisms of the combination of vedolizumab with JAK inhibitors in Crohn’s Disease through a systems biology and artificial intelligence-based approach

Abstract Background Clinical evidence supports the combination therapy of vedolizumab (a gut-selective, anti-lymphocyte trafficking drug inhibiting α4β7-integrin) with JAK 1, 2, 3 inhibitors (JAKi) for Inflammatory Bowel Disease. However, the mechanistic explanation for the improved outcomes in Ulcerative Colitis patients treated with vedolizumab plus JAKi remains unknown and even more in Crohn's Disease (CD). In this study we used an unbiased in silico systems biology and artificial intelligence approach to gain a holistic view of the mechanisms that may underpin the potential benefits of the mentioned combination therapy in the treatment of CD. Methods The drugs under study -vedolizumab and JAKi- and CD were characterized at protein level by compiling data from extensive literature search. Through it, human protein interaction networks focused on CD protein effectors were generated and used to construct mathematical models using Therapeutic Performance Mapping System (TPMS) technology (Anaxomics Biotech, Barcelona, Spain). Sampling-based methods were employed to assess the impact on CD of the combined therapy vedolizumab plus JAKi and to describe underlying molecular mechanisms. Results Characterization of CD allowed identifying 4 pathological processes (referred to as motives-M) underlying the manifestation of the disease: intestinal barrier disruption (M1), increased innate immune response (M2), chronic inflammation and Th1/Th17 adaptive immune response (M3) and tissue remodeling (M4) and the molecular effectors or proteins involved in each motive. The combination of vedolizumab and JAKi induced a high degree of reversion of the effectors altered in CD (54.76%), higher than the percentage reverted by the individual drugs. Combined vedolizumab plus JAKi therapy mainly modulated M3, where both drugs mostly converge although by distinct mechanisms, thus providing additional individual benefits. The combination therapy reverted a great percentage of M3 effectors, for example, CCR9 and FASLG which were increased in CD, were reverted (or downregulated). In addition to M3 modulation, the combination therapy also modulated M1 and M4. In M1, vedolizumab is the one that would provide more complementary mechanisms apart from those that already converge in both drugs. Regarding M4, it seems that main contribution comes from complementarity pathways of the different mechanisms of action of each individual drug. Conclusion The constructed in silico models reveal that the combined vedolizumab plus JAKi therapy could modulate a high array of effectors altered in CD, more than any of the therapies alone, by enhancing the effects of the individual drugs, and provide a mechanistic rationale for employing the combination of these drugs as potential therapy for CD.

A sequential sampling-based Bayesian numerical method for reliability-based design optimization

For efficiently solving the Reliability-Based Design Optimization (RBDO) problem with multi-modal, highly nonlinear and expensive-to-evaluate limit state functions (LSFs), a sequential sampling-based Bayesian active learning method is developed in this work. The penalty function method is embedded to transform the constrained optimization problem into a non-constrained one to reduce the model complexity. The proposed method for solving RBDO problems starts by training a Gaussian process (GP) model, in the augmented space of random and design variables. It is then based on an efficient sampling scheme for simulating the GP model, the adaptive Bayesian optimization (BO) and Bayesian reliability analysis (BRA) procedures are combined in a collaborative way for sequentially producing the joint training points. BO driven by expected improvement (EI) function is used for inferring the global optimum in the design space with global convergence, and the BRA equipped with U function is implemented for inferring the failure probabilities at the identified design points with the desired accuracy. The superiority of the proposed method is demonstrated with two numerical and two real-world engineering examples.

A Sampling-Based Method for Detecting Data Poisoning Attacks in Recommendation Systems

The recommendation algorithm based on collaborative filtering is vulnerable to data poisoning attacks, wherein attackers can manipulate system output by injecting a large volume of fake rating data. To address this issue, it is essential to investigate methods for detecting systematically injected poisoning data within the rating matrix. Since attackers often inject a significant quantity of poisoning data in a short period to achieve their desired impact, these data may exhibit spatial proximity. In other words, poisoning data may be concentrated in adjacent rows of the rating matrix. This paper capitalizes on the proximity characteristics of poisoning data in the rating matrix and introduces a sampling-based method for detecting data poisoning attacks. First, we designed a rating matrix sampling method specifically for detecting poisoning data. By sampling differences obtained from the original rating matrix, it is possible to infer the presence of poisoning attacks and effectively discard poisoning data. Second, we developed a method for pinpointing malicious data based on the distance of rating vectors. Through distance calculations, we can accurately identify the positions of malicious data. After that, we validated the method on three real-world datasets. The results demonstrate the effectiveness of our method in identifying malicious data within the rating matrix.