High-dimensional Vectors Research Articles

An adaptive test of the independence of high-dimensional data based on Kendall rank correlation coefficient

The paper considers the test on the components independence of high-dimensional continuous random vectors based on the Kendall rank correlation coefficient. Under the null hypothesis, it proves that the sum-of-powers type statistics constructed by the Kendall rank correlation coefficient are asymptotically normally distributed and independent when the sum of the orders is odd, and they are also independent with the extreme-value-type test statistics of the Kendall rank correlation coefficient. Based on the asymptotic independence property, it proposes an adaptive test procedure which combines p-values from the sum-of-powers-type test and extreme-value-type test. Some simulations reveal that the proposed test method can significantly enhance the statistical power of the test while keep the type I error well controlled, and it is powerful against a wide range of alternatives (including dense, sparse or moderately sparse signals). To illustrate the use of the proposed adaptive test method, two real data sets are also analysed.

TMS: Ensemble Deep Learning Model for Accurate Classification of Monkeypox Lesions Based on Transformer Models with SVM

Background/Objectives:The emergence of monkeypox outside its endemic region in Africa has raised significant concerns within the public health community due to its rapid global dissemination. Early clinical differentiation of monkeypox from similar diseases, such as chickenpox and measles, presents a challenge. The Monkeypox Skin Lesion Dataset (MSLD) used in this study comprises monkeypox skin lesions, which were collected primarily from publicly accessible sources. The dataset contains 770 original images captured from 162 unique patients. The MSLD includes four distinct class labels: monkeypox, measles, chickenpox, and normal. Methods: This paper presents an ensemble model for classifying the monkeypox dataset, which includes transformer models and support vector machine (SVM). The model development process begins with an evaluation of seven convolutional neural network (CNN) architectures. The proposed model is developed by selecting the top four models based on evaluation metrics for performance. The top four CNN architectures, namely EfficientNetB0, ResNet50, MobileNet, and Xception, are used for feature extraction. The high-dimensional feature vectors extracted from each network are then concatenated and optimized before being inputted into the SVM classifier. Results: The proposed ensemble model, in conjunction with the SVM classifier, achieves an accuracy of 95.45b%. Furthermore, the model demonstrates high precision (95.51%), recall (95.45%), and F1 score (95.46%), indicating its effectiveness in identifying monkeypox lesions. Conclusions: The results of the study show that the proposed hybrid framework achieves robust diagnostic performance in monkeypox detection, offering potential utility for enhanced disease monitoring and outbreak management. The model’s high diagnostic accuracy and computational efficiency indicate that it can be used as an additional tool for clinical decision support.

Federated Learning-Assisted Coati Deep Learning-Based Model for Intrusion Detection in MANET

MANET is a set of self-arranged, wirelessly connected nodes. Each mobile ad hoc network node acts as a router to send the packet from the source node to the destination node. MANET nodes’ random movements and decentralized architecture pose security challenges, making them vulnerable to various attacks like node selfishness, network partition, black hole, and DoS due to limited hardware resources. In this paper, a novel Hybrid Intrusion DEtection for MANet (HIDE-MAN) technique has been proposed to detect intrusion like DDoS and MitM attacks in MANET. The proposed HIDE-MAN framework initiates by preprocessing malicious data packets through data cleaning and data transformation resulting in the creation of high-dimensional vectors. The intrusion detection system then makes use of the CO-BiLSTM model, which is based on the actions of Coati and BiLSTM. It categorizes outputs into DDoS attacks, MitM attacks, or the absence of any attacks. Federated learning with GAN networks allows for the aggregation of updates from multiple local models distributed across MANET. Assessment metrics such as accuracy, precision, F1 score, detection rate, recall, and security rate have been utilized to assess the efficacy of the proposed HIDE-MAN method. The comparative analysis shows that the detection rate of the proposed HIDE-MAN is greater by 18.9%, 18.07%, and 4.03% than that of the current KBIDS, WOA-DNN, and MSA-GCNN techniques, respectively.

An Innovative Neighbor Attention Mechanism Based on Coordinates for the Recognition of Facial Expressions.

For solving the facial expression recognition (FER) problem, we introduce a novel feature extractor called the coordinate-based neighborhood attention mechanism (CNAM), which uses the coordinate attention (CA) method to capture the directional relationships in separate horizontal and vertical directions, the input features from a preprocessing unit, and then passes this to two residual blocks, one consisting of the neighborhood attention (NA) mechanism, which captures the local interaction of features within the neighborhood of a feature vector, while the other one contains a channel attention implemented by a multilayer perceptron (MLP). We apply the feature extractor, the CNAM module, to four FER benchmark datasets, namely, RAF-DB, AffectNet(7cls), AffectNet(8cls), and CK+, and through qualitative and quantitative analysis techniques, we conclude that the insertion of the CNAM module could decrease the intra-cluster distances and increase the inter-cluster distances among the high-dimensional feature vectors. The CNAM compares well with other state-of-the-art (SOTA) methods, being the best-performing method for the AffectNet(7cls) and CK+ datasets, while for the RAF-DB and AffectNet(8cls) datasets, its performance is among the top-performing SOTA methods.

Computing With Residue Numbers in High-Dimensional Representation.

We introduce residue hyperdimensional computing, a computing framework that unifies residue number systems with an algebra defined over random, high-dimensional vectors. We show how residue numbers can be represented as high-dimensional vectors in a manner that allows algebraic operations to be performed with component-wise, parallelizable operations on the vector elements. The resulting framework, when combined with an efficient method for factorizing high-dimensional vectors, can represent and operate on numerical values over a large dynamic range using resources that scale only logarithmically with the range, a vast improvement over previous methods. It also exhibits impressive robustness to noise. We demonstrate the potential for this framework to solve computationally difficult problems in visual perception and combinatorial optimization, showing improvement over baseline methods. More broadly, the framework provides a possible account for the computational operations of grid cells in the brain, and it suggests new machine learning architectures for representing and manipulating numerical data.

Near-optimal quantum kernel principal component analysis

Abstract Kernel principal component analysis (kernel PCA) is a nonlinear dimensionality reduction technique that employs kernel functions to map data into a high-dimensional feature space, thereby extending the applicability of linear PCA to nonlinear data and facilitating the extraction of informative principal components. However, kernel PCA necessitates the manipulation of large-scale matrices, leading to high computational complexity and posing challenges for efficient implementation in big data environments. Quantum computing has recently been integrated with kernel methods in machine learning, enabling effective analysis of input data within intractable feature spaces. Although existing quantum kernel PCA proposals promise exponential speedups, they impose stringent requirements on quantum hardware that are challenging to fulfill. In this work, we propose a quantum algorithm for kernel PCA by establishing a connection between quantum kernel methods and block encoding, thereby diagonalizing the centralized kernel matrix on a quantum computer. The query complexity is logarithmic with respect to the size of the data vector, $D$, and linear with respect to the size of the dataset. An exponential speedup could be achieved when the dataset consists of a few high-dimensional vectors, wherein the dataset size is polynomial in $\log(D)$, with $D$ being significantly large. In contrast to existing work, our algorithm enhances the efficiency of quantum kernel PCA and reduces the requirements for quantum hardware. Furthermore, we have also demonstrated that the algorithm based on block encoding matches the lower bound of query complexity, indicating that our algorithm is nearly optimal. Our research has laid down new pathways for developing quantum machine learning algorithms aimed at addressing tangible real-world problems and demonstrating quantum advantages within machine learning.

Data-driven inference of high-dimensional spatiotemporal state of plasma systems

Many plasma systems and technologies, such as Hall thrusters for spacecraft propulsion, exhibit complex underlying physics that affect the global operation. When characterizing such systems in an experiment, obtaining full spatiotemporal maps of the involved state variables can be, thus, highly informative. However, this goal is not practically realizable because of various experimental limitations, e.g., finite spatial resolution of the diagnostics and geometrical accessibility constraints. Therefore, having the capability to reconstruct the full high-dimensional states of plasma systems from low-dimensional time-history measurements is greatly desirable. Compressed sensing is a signal processing technique that can answer this crucial need. However, existing compressed sensing approaches have several limitations that restrict their effectiveness for complex physical systems like plasma technologies. These include the need for abundant sensor measurements and a principled sensor placement. In this paper, we demonstrate the capabilities of Shallow Recurrent Decoder (SHRED) architecture for compressed sensing. We show in several plasma test cases that SHRED can robustly infer full high-dimensional spatiotemporal state vectors of these systems (i.e., all macroscopic plasma properties) from minimal system information. This minimal information can consist of three finite time-history measurements of either local values of a plasma property or the global plasma properties (spatially averaged or performance parameters). An application of SHRED's inference capability in the numerical plasma simulation context is “super-resolution” enhancement. We will discuss this application by presenting how SHRED can effectively establish mappings between a low-resolution and a high-resolution simulation, recovering detailed spatial plasma features that are below the simulation's grid size.

Stochastic Black-Box Optimization using Multi-Fidelity Score Function Estimator

Abstract Optimizing parameters of physics-based simulators is crucial in the design pro-
cess of engineering and scientific systems. This becomes particularly challenging
when the simulator is stochastic, computationally expensive, black-box and when a
high-dimensional vector of parameters needs to be optimized, as e.g. is the case
in complex climate models that involve numerous interdependent variables and
uncertain parameters. Many traditional optimization methods rely on gradient
information, which is frequently unavailable in legacy black-box codes. To address
these challenges, we present SCOUT-Nd (Stochastic Constrained Optimization for
N dimensions), a gradient-based algorithm that can be used on non-differentiable
objectives. It can be combined with natural gradients in order to further enhance
convergence properties. and it also incorporates multi-fidelity schemes and an
adaptive selection of samples in order to minimize computational effort. We vali-
date our approach using standard, benchmark problems, demonstrating its superior
performance in parameter optimization compared to existing methods. Addition-
ally, we showcase the algorithm’s efficacy in a complex real-world application, i.e.
the optimization of a wind farm layout.

Improving information theory of context-aware phrase embeddings in HR domain

The object of the study is context-aware phrase representations. The growing need to automate candidate recruitment and job recommendation processes has paved the way for the utilization of text embeddings. These embeddings involve translating the semantic essence of text into a continuous, high-dimensional vector space. By learning context-aware rich and meaningful representations of phrases within the human resource domain, the efficacy of similarity searches and matching procedures is enhanced, which contributes to a more streamlined and effective recruitment process. However, existing approaches do not take into account the context when modeling phrases. This necessitates the improvement of information technology analysis in this area. In this paper, it is proposed to mark the beginning and end of phrases in the text using special tokens. This made it possible to reduce the requirements for computing power by calculating all phrase representations present in the text simultaneously. The effectiveness of the improvement was tested on a new dataset to compare and evaluate the models in the task of modeling phrases in the field of human resources management. The proposed approach to modeling phrase representations with regard to context in the field of human resources management leads to an improvement in computational efficiency by up to 50 % and an increase in accuracy by up to 10 %. The architecture of the machine learning model for creating context-aware phrase representations is developed, which is characterized by the presence of blocks for taking into account phrase boundaries. Experiments and comparisons with existing approaches have confirmed the effectiveness of the proposed solution. In practice, the proposed information analysis technology can be used to automate the process of identifying and normalizing candidates' skills in online recruiting

Hyperdimensional computing: a framework for stochastic computation and symbolic AI

Hyperdimensional Computing (HDC), also known as Vector Symbolic Architectures (VSA), is a neuro-inspired computing framework that exploits high-dimensional random vector spaces. HDC uses extremely parallelizable arithmetic to provide computational solutions that balance accuracy, efficiency and robustness. The majority of current HDC research focuses on the learning capabilities of these high-dimensional spaces. However, a tangential research direction investigates the properties of these high-dimensional spaces more generally as a probabilistic model for computation. In this manuscript, we provide an approachable, yet thorough, survey of the components of HDC. To highlight the dual use of HDC, we provide an in-depth analysis of two vastly different applications. The first uses HDC in a learning setting to classify graphs. Graphs are among the most important forms of information representation, and graph learning in IoT and sensor networks introduces challenges because of the limited compute capabilities. Compared to the state-of-the-art Graph Neural Networks, our proposed method achieves comparable accuracy, while training and inference times are on average 14.6× and 2.0× faster, respectively. Secondly, we analyse a dynamic hash table that uses a novel hypervector type called circular-hypervectors to map requests to a dynamic set of resources. The proposed hyperdimensional hashing method has the efficiency to be deployed in large systems. Moreover, our approach remains unaffected by a realistic level of memory errors which causes significant mismatches for existing methods.

Foundation model for efficient biological discovery in single-molecule data.

Modern data-intensive techniques offer ever deeper insights into biology, but render the process of discovery increasingly complex. For example, exploiting the unique ability of single-molecule fluorescence microscopy (SMFM)1-5. to uncover rare but critical intermediates often demands manual inspection of time traces and iterative ad hoc approaches that are difficult to systematize. To facilitate systematic and efficient discovery from SMFM data, we introduce META-SiM, a transformer-based foundation model pre-trained on diverse SMFM analysis tasks. META-SiM achieves high performance-rivaling best-in-class algorithms-on a broad range of analysis tasks including trace selection, classification, segmentation, idealization, and stepwise photobleaching analysis. Additionally, the model produces high-dimensional embedding vectors that encapsulate detailed information about each trace, which the web-based META-SiM Projector (https://www.simol-projector.org) casts into lower-dimensional space for efficient whole-dataset visualization, labeling, comparison, and sharing. Combining this Projector with the objective metric of Local Shannon Entropy enables rapid identification of condition-specific behaviors, even if rare or subtle. As a result, by applying META-SiM to an existing single-molecule Förster resonance energy transfer (smFRET) dataset6, we discover a previously unobserved intermediate state in pre-mRNA splicing. META-SiM thus removes bottlenecks, improves objectivity, and both systematizes and accelerates biological discovery in complex single-molecule data.

A flexible and efficient algorithm for high dimensional support vector regression

In high dimensional statistical learning, variable selection and handling highly correlated phenomena are two crucial topics. Elastic-net regularization can automatically perform variable selection and tends to either simultaneously select or remove highly correlated variables. Consequently, it has been widely applied in machine learning. In this paper, we incorporate elastic-net regularization into the support vector regression model, introducing the Elastic-net Support Vector Regression (En-SVR) model. Due to the inclusion of elastic-net regularization, the En-SVR model possesses the capability of variable selection, addressing high dimensional and highly correlated statistical learning problems. However, the optimization problem for the En-SVR model is rather complex, and common methods for solving the En-SVR model are challenging. Nevertheless, we observe that the optimization problem for the En-SVR model can be reformulated as a convex optimization problem where the objective function is separable into multiple blocks and connected by an inequality constraint. Therefore, we employ a novel and efficient Alternating Direction Method of Multipliers (ADMM) algorithm to solve the En-SVR model, and provide a complexity analysis as well as convergence analysis for the algorithm. Furthermore, extensive numerical experiments validate the outstanding performance of the En-SVR model in high dimensional statistical learning and the efficiency of this novel ADMM algorithm.

The classification of Boolean degree 1 functions in high-dimensional finite vector spaces

We classify the Boolean degree 1 1 functions of k k -spaces in a vector space of dimension n n (also known as Cameron-Liebler classes) over the field with q q elements for n ≥ n 0 ( k , q ) n \geq n_0(k, q) . This also implies that two-intersecting sets with respect to k k -spaces do not exist for n ≥ n 0 ( k , q ) n \geq n_0(k, q) . Our main ingredient is the Ramsey theory for geometric lattices.

A wee exploration of techniques for risk assessments of extreme events

AbstractAssessing the behaviour of extreme events in univariate and multivariate settings entails many challenges, from the need to capture different sources of non-stationarity to adequately extrapolate into the tail of the distribution and compute probabilities of extreme events associated with high-dimensional vectors. Motivated by these common issues, we use a combination of methods from extreme-value theory, dimensionality reduction, non-parametric statistics, copula theory, and bootstrap model averaging to provide estimates of risk measures associated with environmental extremes. The work is tailored to the four data challenges presented in the EVA (2023) Conference Data Challenge competition, and the methods introduced here represent the approach taken by the Wee Extremes group.

Vector Databases

Vector databases represent a significant advancement in data management, tailored to efficiently handle high-dimensional vector embeddings. Unlike traditional databases, which face challenges with the complexity and scale of high-dimensional data, vector databases are engineered for optimal storage, indexing, and retrieval of vector data. This work explores the challenges posed by high-dimensional data, including the curse of dimensionality, and examines how vector databases address these issues through advanced indexing techniques such as Inverted File (IVF), Product Quantization (PQ), and Locality Sensitive Hashing (LSH). It highlights the importance of vector databases in facilitating rapid similarity searches, which are crucial for applications such as recommendation systems and search engines. The discussion also covers the evolution of vector databases and their impact on AI and machine learning.

The air transport research society world conference: A data science-based literature review on the years 2014–2024

The Air Transport Research Society (ATRS) World Conference is one of the major venues for air transport research. The conference covers a wide range of research talks, practice/industrial sessions, and research workshop activities. In this paper, we perform a data-driven analysis of the research abstracts that have been accepted and presented at the conference since 2014. We have grouped the abstracts from the ten annual conferences using t-distributed stochastic neighbor embedding to map high-dimensional keyword vectors into a two-dimensional plane for clustering, analysis, and visualization. The major focus of our study concerns three directions. First, we provide a formal description of the actual research presented at the ATRS World Conference series by using methods from natural language processing and machine learning, leading to a data-driven classification consisting of 35 major subject categories. Second, we analyze the origin of main authors/presenters and their background, including their institutions and countries of origin. Third, we perform a network-driven analysis of co-authorships across abstracts to identify the role and importance of key researchers in the community. Finally, we provide an analysis of popular research topics indicated by authors when submitting their abstracts, and a set of major recommendations for future work, based on the insights obtained from our study.

Photonic modes prediction via multi-modal diffusion model

Abstract The concept of photonic modes is the cornerstone in optics and photonics, which can describe the propagation of the light. The Maxwell's equations play the role in calculating the mode field based on the structure information, while this process needs a great deal of computations, especially in the handle with a three-dimensional model. To overcome this obstacle, we introduce the Multi-Modal Diffusion model to predict the photonic modes in one certain structure. The Contrastive Language–Image Pre-training (CLIP) model is used to build the connections between photonic structures and the corresponding modes. Then we exemplify Stable Diffusion (SD) model to realize the function of optical fields generation from structure information. Our work introduces Multi-Modal deep learning to construct complex mapping between structural information and light field as high-dimensional vectors, and generates light field images based on this mapping.

Bayesian Optimal Two-Sample Tests for High-Dimensional Gaussian Populations

We propose minimax optimal Bayesian two-sample tests for testing equality of high-dimensional mean vectors and covariance matrices between two populations. In many applications including genomics and medical imaging, it is natural to assume that only a few entries of two mean vectors or covariance matrices are different. Many existing tests that rely on aggregating the difference between empirical means or covariance matrices are not optimal or yield low power under such setups. Motivated by this, we develop Bayesian two-sample tests employing a divide-and-conquer idea, which is powerful especially when the differences between two populations are rare but large. The proposed two-sample tests manifest closed forms of Bayes factors and allow scalable computations even in high-dimensions. We prove that the proposed tests are consistent under relatively mild conditions compared to existing tests in the literature. Furthermore, the testable regions from the proposed tests turn out to be minimax optimal in terms of rates. Simulation studies show clear advantages of the proposed tests over other state-of-the-art methods in various scenarios. Our tests are also applied to the analysis of the gene expression data of two cancer data sets.

Adjoint-based sensitivity analysis and assimilation of multi-source data for the inference of spatio-temporal parameters in a 2D urban flood hydraulic model

This contribution presents a novel approach for the calibration of distributed parameters in a 2D urban flood hydraulic model. It focuses on the challenging issue of inferring distributed friction parameters from multi-source heterogeneous spatio-temporal observations of their hydraulic signatures in the context of an urban flash flood in a complex street network. A variational data assimilation algorithm is used to infer high-dimensional multi-variate parameters (spatialized friction and inflow discharge time series) using multi-source observations. This method relies on a differentiable 2D shallow water hydraulic model which enables to generate high-resolution sensitivity maps of local gradients and Derivative-based Global Sensitivity Measures (DGSM), enabling to guide adequate definition of parameter spatialization for the data assimilation process. Assimilated data include real local limnigraphic measurements and high-water marks collected after a major flood event, as well as modeled flow velocity used in twin experiments setups. This study is the first to leverage high-water marks with a variational method for the calibration of distributed parameters in an urban flood model. The multi-source data is used to infer inflow hydrographs and distributed friction parameters in setups of varying complexity. In the main setup, the complex structure of the street network, along with the sensitivity maps and hydraulics expertise led to define a model configuration with 45 friction patches. A high-dimensional parameter vector composed of these friction values and an upstream inflow is inferred simultaneously by assimilating real limnigraphic data and high-water marks. This leads to an increase in model fit to observations and satisfying parameter estimates.

Foundation model for efficient biological discovery in single-molecule data.

Modern data-intensive techniques offer ever deeper insights into biology, but render the process of discovery increasingly complex. For example, exploiting the unique ability of single-molecule fluorescence microscopy (SMFM)1-5. to uncover rare but critical intermediates often demands manual inspection of time traces and iterative ad hoc approaches that are difficult to systematize. To facilitate systematic and efficient discovery from SMFM data, we introduce META-SiM, a transformer-based foundation model pre-trained on diverse SMFM analysis tasks. META-SiM achieves high performance-rivaling best-in-class algorithms-on a broad range of analysis tasks including trace selection, classification, segmentation, idealization, and stepwise photobleaching analysis. Additionally, the model produces high-dimensional embedding vectors that encapsulate detailed information about each trace, which the web-based META-SiM Projector (https://www.simol-projector.org) casts into lower-dimensional space for efficient whole-dataset visualization, labeling, comparison, and sharing. Combining this Projector with the objective metric of Local Shannon Entropy enables rapid identification of condition-specific behaviors, even if rare or subtle. As a result, by applying META-SiM to an existing single-molecule Förster resonance energy transfer (smFRET) dataset6, we discover a previously unobserved intermediate state in pre-mRNA splicing. META-SiM thus removes bottlenecks, improves objectivity, and both systematizes and accelerates biological discovery in complex single-molecule data.