High-dimensional Space Research Articles

Referring Image Segmentation With Fine-Grained Semantic Funneling Infusion.

Recently, referring image segmentation has attracted wide attention given its huge potential in human-robot interaction. Networks to identify the referred region must have a deep understanding of both the image and language semantics. To do so, existing works tend to design various mechanisms to achieve cross-modality fusion, for example, tile and concatenation and vanilla nonlocal manipulation. However, the plain fusion usually is either coarse or constrained by the exorbitant computation overhead, finally causing not enough understanding of the referent. In this work, we propose a fine-grained semantic funneling infusion (FSFI) mechanism to solve the problem. The FSFI introduces a constant spatial constraint on the querying entities from different encoding stages and dynamically infuses the gleaned language semantic into the vision branch. Moreover, it decomposes the features from different modalities into more delicate components, allowing the fusion to happen in multiple low-dimensional spaces. The fusion is more effective than the one only happening in one high-dimensional space, given its ability to sink more representative information along the channel dimension. Another problem haunting the task is that the instilling of high-abstract semantic will blur the details of the referent. Targetedly, we propose a multiscale attention-enhanced decoder (MAED) to alleviate the problem. We design a detail enhancement operator (DeEh) and apply it in a multiscale and progressive way. Features from the higher level are used to generate attention guidance to enlighten the lower-level features to more attend to the detail regions. Extensive results on the challenging benchmarks show that our network performs favorably against the state-of-the-arts (SOTAs).

From human explanations to explainable AI: Insights from constrained optimization

Many complex decision-making scenarios encountered in the real-world, including energy systems and infrastructure planning, can be formulated as constrained optimization problems. Solutions for these problems are often obtained using white-box solvers based on linear program representations. Even though these algorithms are well understood and the optimality of the solution is guaranteed, explanations for the solutions are still necessary to build trust and ensure the implementation of policies. Solution algorithms represent the problem in a high-dimensional abstract space, which does not translate well to intuitive explanations for lay people. Here, we report three studies in which we pose constrained optimization problems in the form of a computer game to participants. In the game, called Furniture Factory, participants manage a company that produces furniture. In two qualitative studies, we first elicit representations and heuristics with concurrent explanations and validate their use in post-hoc explanations. We analyze the complexity of the explanations given by participants to gain a deeper understanding of how complex cognitively adequate explanations should be. Based on insights from the analysis of the two qualitative studies, we formalize strategies that in combination can act as descriptors for participants’ behavior and optimal solutions. We match the strategies to decisions in a large behavioral dataset (>150 participants) gathered in a third study, and compare the complexity of strategy combinations to the complexity featured in participants’ explanations. Based on the analyses from these three studies, we discuss how these insights can inform the automatic generation of cognitively adequate explanations in future AI systems.

A short-term wind power forecasting model based on CUR

Abstract Wind power forecasting plays a crucial role in the contemporary renewable energy system. During the process of forecasting wind power, the establishment of LSTM models requires a lot of time and effort, and the interpretability of prediction results is poor, making it difficult to understand and verify the results. To accomplish interpretable and precise wind power predictions, this paper introduces a wind power prediction algorithm model leveraging CUR matrix decomposition. The CUR matrix decomposition method first obtains the original matrix A (wind power data matrix). The statistical influence scores of rows and columns in A are calculated, and several columns and rows with higher scores are selected to form a low dimensional matrix C and R. Matrix C contains the main characteristic factors that affect wind power, while matrix R contains time series features. Then, the matrix U is approximated by A, C, and R to transform the preference feature extraction problem in high-dimensional space into a matrix analysis problem in low-dimensional space, making it more interpretable and accurate. The efficacy of the wind power forecasting approach utilizing CUR matrix decomposition is assessed and confirmed on openly accessible datasets. The results indicate that the CUR matrix decomposition method has good prediction accuracy and interpretability.

Integrating granular computing with density estimation for anomaly detection in high-dimensional heterogeneous data

Detecting anomalies in complex data is crucial for knowledge discovery and data mining across a wide range of applications. While density-based methods are effective for handling varying data densities and diverse distributions, they often struggle with accurately estimating densities in heterogeneous, uncertain data and capturing interdependencies among features in high-dimensional spaces. This paper proposes a fuzzy granule density-based anomaly detection algorithm (GDAD) for heterogeneous data. Specifically, GDAD first partitions high-dimensional attributes into subspaces based on their interdependencies and employs fuzzy information granules to represent data. The core of the method is the definition of fuzzy granule density, which leverages local neighborhood information alongside global density patterns and effectively characterizes anomalies in data. Each object is then assigned a fuzzy granule density-based anomaly factor, reflecting its likelihood of being anomalous. Through extensive experimentation on various real-world datasets, GDAD has demonstrated superior performance, matching or surpassing existing state-of-the-art methods. GDAD's integration of granular computing with density estimation provides a practical framework for anomaly detection in high-dimensional heterogeneous data.

Research on dangerous flight weather prediction based on machine learning

Abstract With the continuous expansion of the scale of air transport, the demand for aviation meteorological support also continues to grow. The impact of hazardous weather on flight safety is critical. How to effectively use meteorological data to improve the early warning capability of flight dangerous weather and ensure the safe flight of aircraft is the primary task of aviation meteorological services. In this work, support vector machine (SVM) models are used to predict hazardous flight weather, especially for meteorological conditions with high uncertainty such as storms and turbulence. SVM is a supervised learning method that distinguishes between different classes of data by finding optimal decision boundaries in a high-dimensional space. To meet the needs of this study, we chose the radial basis function (RBF) as the kernel function, which helps to deal with nonlinear problems and enables the model to better capture complex meteorological data structures. During the model training phase, we used historical meteorological observations from multiple weather stations, including temperature, humidity, wind speed, wind direction, and other meteorological indicators closely related to flight safety. From this data, the SVM model learns how to distinguish between normal and dangerous flight weather conditions.

Learning on manifolds without manifold learning

Function approximation based on data drawn randomly from an unknown distribution is an important problem in machine learning. The manifold hypothesis assumes that the data is sampled from an unknown submanifold of a high dimensional Euclidean space. A great deal of research deals with obtaining information about this manifold, such as the eigendecomposition of the Laplace–Beltrami operator or coordinate charts, and using this information for function approximation. This two-step approach implies some extra errors in the approximation stemming from estimating the basic quantities of the data manifold in addition to the errors inherent in function approximation. In this paper, we project the unknown manifold as a submanifold of an ambient hypersphere and study the question of constructing a one-shot approximation using a specially designed sequence of localized spherical polynomial kernels on the hypersphere. Our approach does not require preprocessing of the data to obtain information about the manifold other than its dimension. We give optimal rates of approximation for relatively “rough” functions.

Accelerating material discovery with a threshold-driven hybrid acquisition policy-based Bayesian optimization

Advancements in materials play a crucial role in technological progress. However, the process of discovering and developing materials with desired properties is often impeded by substantial experimental costs, extensive resource utilization, and lengthy development periods. To address these challenges, modern approaches often employ machine learning (ML) techniques such as Bayesian Optimization (BO), which streamline the search for optimal materials by iteratively selecting experiments that are most likely to yield beneficial results. However, traditional BO methods, while beneficial, often struggle with balancing the trade-off between exploration and exploitation, leading to sub-optimal performance in accelerated material discovery processes. This paper introduces a novel Threshold-Driven UCB-EI Bayesian Optimization (TDUE-BO) method, which dynamically integrates the strengths of Upper Confidence Bound (UCB) and Expected Improvement (EI) acquisition functions to optimize the material discovery process. Unlike the classical BO, our method focuses on efficiently navigating the high-dimensional material design space (MDS). TDUE-BO begins with an exploration-focused UCB approach, ensuring a comprehensive initial sweep of the MDS. As the model gains confidence, indicated by reduced uncertainty, it transitions to the more exploitative EI method, focusing on promising areas identified earlier. The UCB-to-EI switching policy, which is steered by ongoing monitoring of model uncertainty at each stage of sequential sampling, enables more efficient navigation through the MDS while guaranteeing quicker convergence. The effectiveness of TDUE-BO is demonstrated through its application on three different material science datasets, showing significantly better approximation and optimization performance over traditional EI and UCB-based BO methods in terms of the RMSE scores and convergence efficiency, respectively.

Heuristic Optimization Algorithm of Black-Winged Kite Fused with Osprey and Its Engineering Application.

Swarm intelligence optimization methods have steadily gained popularity as a solution to multi-objective optimization issues in recent years. Their study has garnered a lot of attention since multi-objective optimization problems have a hard high-dimensional goal space. The black-winged kite optimization algorithm still suffers from the imbalance between global search and local development capabilities, and it is prone to local optimization even though it combines Cauchy mutation to enhance the algorithm's optimization ability. The heuristic optimization algorithm of the black-winged kite fused with osprey (OCBKA), which initializes the population by logistic chaotic mapping and fuses the osprey optimization algorithm to improve the search performance of the algorithm, is proposed as a means of enhancing the search ability of the black-winged kite algorithm (BKA). By using numerical comparisons between the CEC2005 and CEC2021 benchmark functions, along with other swarm intelligence optimization methods and the solutions to three engineering optimization problems, the upgraded strategy's efficacy is confirmed. Based on numerical experiment findings, the revised OCBKA is very competitive because it can handle complicated engineering optimization problems with a high convergence accuracy and quick convergence time when compared to other comparable algorithms.

Efficient and Robust Point Cloud Registration via Heuristics-Guided Parameter Search.

Estimating the rigid transformation with 6 degrees of freedom based on a putative 3D correspondence set is a crucial procedure in point cloud registration. Existing correspondence identification methods usually lead to large outlier ratios (>95% is common), underscoring the significance of robust registration methods. Many researchers turn to parameter search-based strategies (e.g., Branch-and-Bround) for robust registration. Although related methods show high robustness, their efficiency is limited to the high-dimensional search space. This paper proposes a heuristics-guided parameter search strategy to accelerate the search while maintaining high robustness. We first sample some correspondences (i.e., heuristics) and then just need to sequentially search the feasible regions that make each sample an inlier. Our strategy largely reduces the search space and can guarantee accuracy with only a few inlier samples, therefore enjoying an excellent trade-off between efficiency and robustness. Since directly parameterizing the 6-dimensional nonlinear feasible region for efficient search is intractable, we construct a three-stage decomposition pipeline to reparameterize the feasible region, resulting in three lower-dimensional sub-problems that are easily solvable via our strategy. Besides reducing the searching dimension, our decomposition enables the leverage of 1-dimensional interval stabbing at all three stages for searching acceleration. Moreover, we propose a valid sampling strategy to guarantee our sampling effectiveness, and a compatibility verification setup to further accelerate our search. Extensive experiments on both simulated and real-world datasets demonstrate that our approach exhibits comparable robustness with state-of-the-art methods while achieving a significant efficiency boost.

Research on Supply Chain Optimization and Management Based on Deep Reinforcement Learning

This research introduces a groundbreaking approach to supply chain optimization and management, termed as Deep Reinforcement Learning based Supply Chain Optimization and Management (DRL-SCOM). At the core of this approach is the utilization of advancements in Deep Reinforcement Learning (DRL), specifically through the integration of Randomized Ensembled Double Q-learning (REDQ) and Trust Region Policy Optimization (TRPO). DRL-SCOM is designed to effectively tackle the inherent complexities and dynamic challenges that are characteristic of supply chain management. One of the key strengths of DRL-SCOM lies in its use of REDQ, which plays a crucial role in mitigating the overestimation bias commonly associated with traditional Q-learning methods. This results in more accurate value estimation and policy improvement, a critical factor in the effective management of supply chains. Additionally, the integration of TRPO into the framework brings the advantage of safe and stable policy updates. Such stability is vital for maintaining the robustness required in the fluctuating environment of supply chain operations. The combination of REDQ and TRPO in DRL-SCOM creates a powerful synergy. REDQ’s ensembled learning approach, when fused with TRPO’s trust-region method, enables the framework to efficiently navigate the complex and high-dimensional decision space typical of supply chains. This allows for real-time optimization of decisions while staying within operational constraints. The DRL-SCOM methodology shows significant potential in addressing various aspects of supply chain management, from demand forecasting and inventory management to logistics, adeptly handling the nonlinearities and uncertainties that are prevalent in these areas. Thus, the DRL-SCOM framework emerges as an innovative solution, pushing the frontiers of traditional supply chain management. It paves the way for a more agile, responsive, and intelligent system, equipped to adapt to changing market demands and operational challenges. This approach represents a significant stride towards transforming supply chain management into a more advanced, data-driven, and adaptive field.

Seismic reliability analysis using Subset Simulation enhanced with an explorative adaptive conditional sampling algorithm

Reliability analysis of structures under earthquake loading represents a significant engineering challenge. This is due to the required and rather numerically involving non-linear dynamic analysis, the large computational burden when targeting small failure probabilities, and the synthetic earthquake model representation that may contain thousands of random variables. Subset Simulation is an efficient reliability analysis technique that can handle the challenge of a high-dimensional space with a reduced number of structural analysis calls compared to crude Monte Carlo Simulation. In this contribution, firstly, we investigate the conditions for which Subset Simulation performs efficiently. Thereafter we propose an enhancement to the existing Subset Simulation schemes that shows significant potentials for enhancing the strategy for the starting of the Markov Chain Monte Carlo simulations whenever a new level is reached in the Subset Simulation. Finally, the information gathered from the simulations is investigated to verify that Subset Simulation provides meaningful results from a physical point of view.

RandONets: Shallow networks with random projections for learning linear and nonlinear operators

Deep neural networks have been extensively used for the solution of both the forward and the inverse problem for dynamical systems. However, their implementation necessitates optimizing a high-dimensional space of parameters and hyperparameters. This fact, along with the requirement of substantial computational resources, pose a barrier to achieving high numerical accuracy, but also interpretability. Here, to address the above challenges, we present Random Projection-based Operator Networks (RandONets): shallow networks with random projections and tailor-made numerical analysis methods that learn accurately and fast linear and nonlinear operators. Building on previous works, we prove that RandOnets are universal approximators of linear and nonlinear operators. Due to their simplicity, RandONets provide a one-step transformation of the input space, facilitating interpretability. For the evaluation of their performance, we focus on operators of PDEs. We show, that RandONets outperform by several orders of magnitude, both in terms of numerical approximation accuracy and computational cost, the “vanilla” DeepONets. Hence, we believe that our method will trigger further developments in the field of scientific machine learning, for the development of new ‘’light”schemes that will provide high accuracy while reducing dramatically the computational cost. A MATLAB toolbox for RandONets, including demos, is available on GitHub at https://github.com/GianlucaFabiani/RandONets.

Inverse design of high-strength medium-Mn steel using a machine learning-aided genetic algorithm approach

To develop medium-Mn steels with an ultimate tensile strength (UTS) exceeding 2 GPa and excellent ductility, we created a highly accurate UTS prediction machine learning (ML) model using a boosted decision tree model and 1520 dataset of tensile properties of medium-Mn steels with micro-alloying elements. We also optimized the hyper-parameters of a genetic algorithm (GA) using the Shannon diversity index to enhance search efficiency while retaining diversity. In a high-dimensional search space with millions of potential combinations, the ML-GA approach efficiently identified diverse chemical compositions and austenitizing conditions to achieve UTSs above 2 GPa. The k-means clustering method then grouped them into five distinct specimens based on similarities. These five specimens, fabricated using inversly designed chemical compositions and austenitizing temperatures, successfully exhibited UTSs exceeding 2 GPa and greater ductility compared to hot-stamped C steels. These excellent tensile properties were attributed to grain refinement resulting from the low austenitizing temperature and the pinning effect of micro-alloying element carbides, such as TiC and VC.

Neural representations of concreteness and concrete concepts are specific to the individual.

Different people listening to the same story may converge upon a largely shared interpretation while still developing idiosyncratic experiences atop that shared foundation. What linguistic properties support this individualized experience of natural language? Here, we investigate how the "concrete-abstract" axis - i.e., the extent to which a word is grounded in sensory experience - relates to within- and across-subject variability in the neural representations of language. Leveraging a dataset of human participants of both sexes who each listened to four auditory stories while undergoing functional MRI, we demonstrate that neural representations of "concreteness" are both reliable across stories and relatively unique to individuals, while neural representations of "abstractness" are variable both within individuals and across the population. Using natural language processing tools, we show that concrete words exhibit similar neural representations despite spanning larger distances within a high-dimensional semantic space, which potentially reflects an underlying representational signature of sensory experience - namely, imageability - shared by concrete words but absent from abstract words. Our findings situate the concrete-abstract axis as a core dimension that supports both shared and individualized representations of natural language.Significance Statement The meaning of spoken language is often ambiguous. As a result, people may form different interpretations despite being presented with the same information. What properties of language does the brain leverage to form this diverse, individual experience? Analyses of functional MRI data demonstrated that "concreteness", the extent to which language is related to sensory experience, evoked reliable neural patterns that were unique to individual subjects and allowed us to identify individuals solely based on their neural data. Application of machine learning methods showed that sets of concrete concepts, but not abstract concepts, show stable neural patterns, potentially due to a sensory signature: imageability. Overall, this study characterizes concreteness as a central property supporting the individualized experience of real-world language.

Dynamical Sphere Regrouping Particle Swarm Optimization Programming: An Automatic Programming Algorithm Avoiding Premature Convergence

Symbolic regression plays a crucial role in machine learning and data science by allowing the extraction of meaningful mathematical models directly from data without imposing a specific structure. This level of adaptability is especially beneficial in scientific and engineering fields, where comprehending and articulating the underlying data relationships is just as important as making accurate predictions. Genetic Programming (GP) has been extensively utilized for symbolic regression and has demonstrated remarkable success in diverse domains. However, GP’s heavy reliance on evolutionary mechanisms makes it computationally intensive and challenging to handle. On the other hand, Particle Swarm Optimization (PSO) has demonstrated remarkable performance in numerical optimization with parallelism, simplicity, and rapid convergence. These attributes position PSO as a compelling option for Automatic Programming (AP), which focuses on the automatic generation of programs or mathematical models. Particle Swarm Programming (PSP) has emerged as an alternative to Genetic Programming (GP), with a specific emphasis on harnessing the efficiency of PSO for symbolic regression. However, PSP remains unsolved due to the high-dimensional search spaces and local optimal regions in AP, where traditional PSO can encounter issues such as premature convergence and stagnation. To tackle these challenges, we introduce Dynamical Sphere Regrouping PSO Programming (DSRegPSOP), an innovative PSP implementation that integrates DSRegPSO’s dynamical sphere regrouping and momentum conservation mechanisms. DSRegPSOP is specifically developed to deal with large-scale, high-dimensional search spaces featuring numerous local optima, thus proving effective behavior for symbolic regression tasks. We assess DSRegPSOP by generating 10 mathematical expressions for mapping points from functions with varying complexity, including noise in position and cost evaluation. Moreover, we also evaluate its performance using real-world datasets. Our results show that DSRegPSOP effectively addresses the shortcomings of PSO in PSP by producing mathematical models entirely generated by AP that achieve accuracy similar to other machine learning algorithms optimized for regression tasks involving numerical structures. Additionally, DSRegPSOP combines the benefits of symbolic regression with the efficiency of PSO.

Navigating the complexity of hybrid materials without structural dependency: PerovGNN as a map

Hybrid organic-inorganic materials, spearheaded by perovskites, have showcased exceptional potential in optical and electrical applications. However, finding suitable compositions with desired properties is challenging due to the vast, high-dimensional search space. Traditional machine learning techniques often fall short in capturing crucial interactions within the composites of hybrid organic-inorganic materials, while others necessitate structural information that is often unavailable prior to material synthesis. To address this, we propose PerovGNN, a structure-agnostic graph representation learning method for property prediction. It transforms the chemical formula of each mixed perovskite into a weighted graph of atoms, with weights representing the relative frequency of interactions between neighboring atoms. By embedding both fractional ratios and atomic features, this graph formulation effectively retains crucial mixture information and learns a superior representation required for a specific targeted property. We applied PerovGNN to experimental datasets for mixed HOIPs bandgap prediction, achieving a lower mean absolute error (MAE) of 0.0483 eV compared to other methods. New perovskites with bandgaps ranging from 1.3 to 2.0 eV were experimentally characterized, yielding a consistent MAE of 0.043 eV. Our work presents a unified model for predicting properties of inorganic, organic, and hybrid crystals, serving as a map for researchers navigating vast design spaces.

Maximum subspace transferability discriminant analysis: A new cross-domain similarity measure for wind-turbine fault transfer diagnosis

In the field of fault transfer diagnosis, many approaches only focus on the distribution alignment and knowledge transfer between the source domain and target domain. However, most of these approaches ignore the precondition of whether this transfer task is transferable. Current mainstream transferability discrimination methods heavily depend on expert knowledge and are extremely vulnerable to the noise interference and variations in feature scale. This limits their applicability due to the intelligent requirements and complex industrial environment. To address the challenges mentioned previously, this paper introduces a novel cross-domain similarity measure called maximum subspace transferability discriminant analysis (MSTDA) with zero-label prior knowledge. MSTDA is comprised of a maximum subspace representation and a similarity measurement criterion. During the phase of maximum subspace representation, a new kernel-induced Hilbert space is designed to map the low-dimensional original samples into the high-dimensional space to maximize the separability of different faults and then solve the separable intrinsic fault features. Following that, a novel similarity measurement criterion that is resistant to variations in feature scale is developed. This criterion is based on the orthogonal bases of intrinsic feature subspaces. The mini-batch sampling strategy is used to ensure the timeliness of MSTDA. Finally, the experimental results on three cases, particularly in the actual wind turbine dataset, confirm that the proposed MSTDA outperforms other well-known similarity measure methods in terms of transferability evaluation. The related code can be downloaded from https://qinyi-team.github.io/2024/09/Maximum-subspace-transferability-discriminant-analysis.

The genetic architecture of protein stability

There are more ways to synthesize a 100-amino acid (aa) protein (20100) than there are atoms in the universe. Only a very small fraction of such a vast sequence space can ever be experimentally or computationally surveyed. Deep neural networks are increasingly being used to navigate high-dimensional sequence spaces1. However, these models are extremely complicated. Here, by experimentally sampling from sequence spaces larger than 1010, we show that the genetic architecture of at least some proteins is remarkably simple, allowing accurate genetic prediction in high-dimensional sequence spaces with fully interpretable energy models. These models capture the nonlinear relationships between free energies and phenotypes but otherwise consist of additive free energy changes with a small contribution from pairwise energetic couplings. These energetic couplings are sparse and associated with structural contacts and backbone proximity. Our results indicate that protein genetics is actually both rather simple and intelligible.

Nonparametric and Continuous Variable-Based Stratigraphic Modelling from Sparse Boreholes using Signed Distance Function and Bayesian Compressive Sensing

An accurate stochastic interpretation of subsurface stratigraphy with quantified uncertainty can benefit the subsequent risk management of geotechnical infrastructure. Traditional approaches to developing geological cross-sections from sparse boreholes typically require the calibration or definition of empirical model parameters and functions, which may introduce subjectivity and bias. In this study, a non-parametric and continuous variable-based spatial predictor that leverages the signed distance function and Bayesian compressive sensing (BCS) is proposed for subsurface stratigraphic modelling. The proposed method transforms sparse categorical borehole data from a low-dimensional space into continuous variables in a high-dimensional space, enabling a comprehensive representation of more implicit characteristics of intricate geological patterns. This transformation facilitates the use of the continuous-variable-based BCS for non-parametric spatial prediction. The most probable geological cross-section and uncertainty qualification plot are derived after transforming spatially interpreted fields of continuous variables back into soil types. The performance of the proposed method is demonstrated using synthetic and real-world cases. Results indicate that the proposed approach can handle intricate stratigraphic scenarios characterized by complex geological structures, such as crossed-inclined, folded, inclined-folded, and interbedded strata, in a data-driven and non-parametric manner. The advantages of the proposed method over existing spatial predictors for developing geological cross-sections are also demonstrated.

Hyperdimensional computing: A fast, robust, and interpretable paradigm for biological data.

Advances in bioinformatics are primarily due to new algorithms for processing diverse biological data sources. While sophisticated alignment algorithms have been pivotal in analyzing biological sequences, deep learning has substantially transformed bioinformatics, addressing sequence, structure, and functional analyses. However, these methods are incredibly data-hungry, compute-intensive, and hard to interpret. Hyperdimensional computing (HDC) has recently emerged as an exciting alternative. The key idea is that random vectors of high dimensionality can represent concepts such as sequence identity or phylogeny. These vectors can then be combined using simple operators for learning, reasoning, or querying by exploiting the peculiar properties of high-dimensional spaces. Our work reviews and explores HDC's potential for bioinformatics, emphasizing its efficiency, interpretability, and adeptness in handling multimodal and structured data. HDC holds great potential for various omics data searching, biosignal analysis, and health applications.