Hypothesis Space Research Articles

Enhancing Unsupervised Anomaly Detection With Score-Guided Network.

Anomaly detection plays a crucial role in various real-world applications, including healthcare and finance systems. Owing to the limited number of anomaly labels in these complex systems, unsupervised anomaly detection methods have attracted great attention in recent years. Two major challenges faced by the existing unsupervised methods are as follows: 1) distinguishing between normal and abnormal data when they are highly mixed together and 2) defining an effective metric to maximize the gap between normal and abnormal data in a hypothesis space, which is built by a representation learner. To that end, this work proposes a novel scoring network with a score-guided regularization to learn and enlarge the anomaly score disparities between normal and abnormal data, enhancing the capability of anomaly detection. With such score-guided strategy, the representation learner can gradually learn more informative representation during the model training stage, especially for the samples in the transition field. Moreover, the scoring network can be incorporated into most of the deep unsupervised representation learning (URL)-based anomaly detection models and enhances them as a plug-in component. We next integrate the scoring network into an autoencoder (AE) and four state-of-the-art models to demonstrate the effectiveness and transferability of the design. These score-guided models are collectively called SG-Models. Extensive experiments on both synthetic and real-world datasets confirm the state-of-the-art performance of SG-Models.

A multi-layer nesting and integration approach for predicting groundwater levels in agriculturally intensive areas using data-driven models

Agricultural water demand, groundwater extraction, surface water delivery and climate exhibit complex nonlinear relationships with groundwater storage in agricultural regions. As alternatives to computationally intensive physical models, data-driven machine learning methods are frequently employed as surrogates to capture these complex relationships, owing to their high computational efficiency. Inevitably, reliance on a single machine learning model may lead to underestimation of prediction uncertainty and potentially result in reduced accuracy. This paper presents a multi-layer nesting and integration approach for predicting groundwater levels in agriculturally intensive areas using data-driven models. The key contributions of the research are threefold: 1) the development of a comprehensive input variable selection process considering the lag effects and driving mechanisms of groundwater level variations; 2) the implementation of an optimization-based hyperparameter tuning method to enhance the performance of individual machine learning models; and 3) the establishment of a multi-model integration framework based on a multi-layer nesting technique. This approach combines the outputs of multiple machine learning models to consolidate predictions and expand the hypothesis space. The effectiveness of the proposed framework is demonstrated through case studies in the Huaihe River Basin, a major agricultural region in China. The results show that the multi-layer nesting and integration approach outperforms the use of individual machine learning models, providing more accurate and reliable groundwater level predictions. This framework offers valuable insights for decision-makers and water resource managers, supporting sustainable groundwater management and addressing the challenges faced in agriculturally intensive areas.

Introducing Mr. Three: Attention, Perception, and Meaning Selection in the Acquisition of Number and Color Words.

Children learn their first number words gradually over the course of many months, which is surprising given their ability to discriminate small numerosities. One potential explanation for this is that children are sensitive to the numerical features of stimuli, but don't consider exact cardinality as a primary hypothesis for novel word meanings. To test this, we trained 144 children on a number word they hadn't yet learned, and contrasted this with a condition in which they were merely required to attend to number to identify the word's referent, without encoding number as its meaning. In the first condition, children were trained to find a "giraffe with three spots." In the second condition, children were instead trained to find "Mr. Three", which also named a giraffe with three spots. In both conditions, children had to attend to number to identify the target giraffe, but, because proper nouns refer to individuals rather than their properties, the second condition did not require children to encode number as the meaning of the expression. We found that children were significantly better at identifying the giraffe when it had been labeled with the proper noun than with the number word. This finding contrasted with a second experiment involving color words, in which children (n = 56) were equally successful with a proper noun ("Mr. Purple") and an adjective ("the giraffe with purple spots"). Together, these findings suggest that, for number, but not for color, children's difficulty acquiring new words cannot be solely attributed to problems with attention or perception, but instead may be due to difficulty selecting the correct meaning from their hypothesis space for learning unknown words.

Quantum Machine Learning: Leveraging Quantum Computing for Enhanced Learning Algorithms

The paper "Quantum Machine Learning: Leveraging Quantum Computing for Enhanced Learning Algorithms" explores the integration of quantum computing principles into classical machine learning techniques, aiming to address limitations such as scalability and computational inefficiency. It presents the foundational concepts of quantum computing, including superposition and entanglement, and their application in accelerating machine learning processes. The study emphasizes the potential for quantum algorithms to significantly improve the performance of machine learning tasks by processing large datasets more efficiently and exploring larger hypothesis spaces. Key quantum machine learning algorithms discussed include Quantum Support Vector Machines (QSVM), Quantum Principal Component Analysis (QPCA), and Quantum Neural Networks (QNN), each of which leverages quantum mechanics to overcome the computational barriers faced by classical algorithms. The Quantum Approximate Optimization Algorithm (QAOA) is also highlighted for its ability to optimize machine learning models more effectively. While the theoretical benefits of Quantum Machine Learning (QML) are promising, the practical application of these techniques is currently limited by the constraints of existing quantum hardware. This research contributes to the emerging field of QML by examining its potential advantages and future implications in addressing complex data processing challenges.

Neural Solving Uninterpreted Predicates with Abstract Gradient Descent

Uninterpreted predicate solving is a fundamental problem in formal verification, including loop invariant and Constrained Horn Clauses predicate solving. Existing approaches have been mostly in symbolic ways. While achieving sustainable progress, they still suffer from inefficiency and seem unable to leverage the ever-increasing computility such as GPU. Recently, Neural Relaxation has been proposed to tackle this problem. They treat the uninterpreted predicate-solving task as an optimization problem by relaxing the discrete search process into a learning process of neural networks. However, two bottlenecks keep them from being valid. First, relaxed neural networks cannot match the original semantics rigorously; second, the neural networks are difficult to train to reach global optimization. Therefore, this paper presents a novel discrete neural architecture with the Abstract Gradient Decent (AGD) algorithm to directly solve uninterpreted predicates in the discrete hypothesis space. The abstract gradient is for discrete neurons whose calculation rules are designed in an abstract domain. Our approach conforms to the original semantics, and the proposed AGD algorithm can achieve global optimization satisfactorily. We implement Dasp in the Boxes Abstract Domain to solve uninterpreted predicates in the QF-NIA SMT theory. In the experiments, Dasp has outperformed 7 state-of-the-art tools across three predicate synthesis tasks.

Investigating the Model Hypothesis Space: Benchmarking Automatic Model Structure Identification With a Large Model Ensemble

AbstractSelecting an appropriate model for a catchment is challenging, and choosing an inappropriate model can yield unreliable results. The Automatic Model Structure Identification (AMSI) method simultaneously calibrates model structural choices and model parameters, which reduces the workload of comparing different models. In this study we benchmark AMSI's capabilities in two ways, using 12 hydro‐climatically diverse Model Parameter Estimation Experiment catchments. First, we calibrate parameter values for 7,488 different model structures and test AMSI's ability to find the best‐performing models in this set. Second, we compare the performance of these 7,488 models and AMSI's selection to the performance of 45 commonly used, structurally more diverse, conceptual models. In both cases, we quantify model accuracy (through the Kling‐Gupta Efficiency) and model adequacy (through various hydrologic signatures). AMSI effectively identifies high‐accuracy models among the 7,488 options, with Kling‐Gupta‐Efficiency scores comparable to the best among the 45 models. However, model adequacy remains poor for the accurate models, regardless of the selection method. In nine of the tested catchments, none of the most accurate models replicate observed signatures with less than 50% errors; in the remaining three catchments, only a handful of models do so. This paper thus provides strong empirical evidence that relying on aggregated efficiency metrics is unlikely to result in hydrologically adequate models, no matter how the models themselves are selected. Nevertheless, AMSI has been shown to effectively search the model hypothesis space it was given. Combined with an improved calibration approach it can therefore offer new ways to address the challenges of model structure selection.

Towards zero-shot object counting via deep spatial prior cross-modality fusion

Existing counting models predominantly operate on a specific category of objects, such as crowds and vehicles. The recent emergence of multi-modal foundational models, e.g., Contrastive Language-Image Pre-training (CLIP), has facilitated class-agnostic counting. This involves counting objects of any given class from a single image based on textual instructions. However, CLIP-based class-agnostic counting models face two primary challenges. Firstly, the CLIP model lacks sensitivity to location information. It generally considers global content rather than the fine-grain location of objects. Therefore, adapting the CLIP model directly is suboptimal. Secondly, these models generally freeze pre-trained vision and language encoders, while neglecting the potential misalignment in the constructed hypothesis space. In this paper, we address these two issues in a unified framework termed Deep Spatial Prior Interaction (DSPI) network. The DSPI leverages the spatial-awareness ability of large-scale pre-trained object grounding models, i.e., Grounding DINO, to incorporate spatial location as an additional prior for a specific query class. This enables the network to be more specifically focused on the precise location of the objects. Additionally, to align the feature space across different modalities, we tailor a meta adapter that extracts textual information into an object query. This serves as an instruction for cross-modality matching. These two modules collaboratively ensure the alignment of multi-modal representations while preserving their discriminative nature. Comprehensive experiments conducted on a diverse set of benchmarks verify the superiority of the proposed model. The code is available at https://github.com/jinyongch/DSPI.

Exploiting indirect linear correlation for label distribution learning

Label distribution learning represents the relevance of labels to samples using description degree, which can provide richer semantic information, thus finding wider applications. Exploiting label correlations is an effective approach to narrow down the hypothesis space of label distribution learning models. In existing works that utilize low-rank assumptions or label linear dependence to mine correlations, it is assumed that a label can be linearly expressed by other labels. However, this assumption can only be satisfied when there are linear dependency relationships between labels, thus the label correlation obtained by such methods is subject to certain distortion. To address this issue, this paper assumes that labels can be linearly represented by the same set of bases. The correlation between labels is represented by sharing common bases. Specifically, the paper employs matrix factorization to extract bases that can be used to represent all labels. And then designs a label distribution learning algorithm based on the property of sharing the same set of bases of the ground truth label distribution and predict label distribution. The effectiveness of the algorithm is verified through experimental validation. Generally speaking, the algorithm presented in this paper achieves optimal performance at 73.15% of the cases, with the best average ranking. In the two-tailed t-test, the algorithm in this paper exhibits statistical superiority compared to all comparison algorithms.

Spectrum sensing in uncalibrated MIMO-based cognitive radios

This paper addresses the problem of spectrum sensing (SS) in cognitive radio networks utilizing multiple-input multiple-output (MIMO) technology, even when the receivers are not calibrated. The study develops a framework by formulating various composite hypothesis testing SS problems based on different assumptions about the parameter space of the alternative hypothesis. To accomplish this, four new detectors are designed using the likelihood ratio test (LRT), Rao, and matrix information geometry (MIG) principles. We demonstrate analytically that three of the proposed detectors maintain a constant false alarm rate (CFAR) despite noise covariance matrix uncertainty. For the non-CFAR detectors, we introduce a novel analytical threshold-setting method to ensure that their false alarm probabilities remain below a predetermined level, thereby making them useful for practical situations. Finally, extensive simulations are conducted to verify the analytical results, assess false alarm regulation, and compare the proposed SS algorithms with existing approaches in terms of detection probability.

Simulating biosignatures from pre-oxygen photosynthesizing life on TRAPPIST-1e

ABSTRACT In order to assess observational evidence for potential atmospheric biosignatures on exoplanets, it will be essential to test whether spectral fingerprints from multiple gases can be explained by abiotic or biotic-only processes. Here, we develop and apply a coupled 1D atmosphere-ocean-ecosystem model to understand how primitive biospheres, which exploit abiotic sources of H$_2$, CO, and O$_2$, could influence the atmospheric composition of rocky terrestrial exoplanets. We apply this to the Earth at 3.8 Ga and to TRAPPIST-1e. We focus on metabolisms that evolved before the evolution of oxygenic photosynthesis, which consume H$_2$ and CO and produce potentially detectable levels of CH$_4$. O$_2$-consuming metabolisms are also considered for TRAPPIST-1e, as abiotic O$_2$ production is predicted on M-dwarf orbiting planets. We show that these biospheres can lead to high levels of surface O$_2$ (approximately 1–5 per cent) as a result of CO consumption, which could allow high O$_2$ scenarios, by removing the main loss mechanisms of atomic oxygen. Increasing stratospheric temperatures, which increases atmospheric OH can reduce the likelihood of such a state forming. O$_2$-consuming metabolisms could also lower O$_2$ levels to around 10 ppm and support a productive biosphere at low reductant inputs. Using predicted transmission spectral features from CH$_4$, CO, O$_2$/O$_3$, and CO$_2$ across the hypothesis space for tectonic reductant input, we show that biotically produced CH$_4$ may only be detectable at high reductant inputs. CO is also likely to be a dominant feature in transmission spectra for planets orbiting M-dwarfs, which could reduce the confidence in any potential biosignature observations linked to these biospheres.

Cognitive Classifier of Hand Gesture Images for Automated Sign Language Recognition: Soft Robot Assistance Based on Neutrosophic Markov Chain Paradigm

In recent years, Sign Language Recognition (SLR) has become an additional topic of discussion in the human–computer interface (HCI) field. The most significant difficulty confronting SLR recognition is finding algorithms that will scale effectively with a growing vocabulary size and a limited supply of training data for signer-independent applications. Due to its sensitivity to shape information, automated SLR based on hidden Markov models (HMMs) cannot characterize the confusing distributions of the observations in gesture features with sufficiently precise parameters. In order to simulate uncertainty in hypothesis spaces, many scholars provide an extension of the HMMs, utilizing higher-order fuzzy sets to generate interval-type-2 fuzzy HMMs. This expansion is helpful because it brings the uncertainty and fuzziness of conventional HMM mapping under control. The neutrosophic sets are used in this work to deal with indeterminacy in a practical SLR setting. Existing interval-type-2 fuzzy HMMs cannot consider uncertain information that includes indeterminacy. However, the neutrosophic hidden Markov model successfully identifies the best route between states when there is vagueness. This expansion is helpful because it brings the uncertainty and fuzziness of conventional HMM mapping under control. The neutrosophic three membership functions (truth, indeterminate, and falsity grades) provide more layers of autonomy for assessing HMM’s uncertainty. This approach could be helpful for an extensive vocabulary and hence seeks to solve the scalability issue. In addition, it may function independently of the signer, without needing data gloves or any other input devices. The experimental results demonstrate that the neutrosophic HMM is nearly as computationally difficult as the fuzzy HMM but has a similar performance and is more robust to gesture variations.

Reproducing property of bounded linear operators and kernel regularized least square regressions

We consider bounded linear operators from the view of functional reproducing property. For some bounded linear operators associated with orthogonal polynomials we define an inner product space associated with a kernel constructed with orthogonal polynomials, show that it is a functional reproducing kernel Hilbert space (FRKHS) associated with these bounded linear operators and give decay rate for the best FRKHS approximation with a [Formula: see text]-functional associated with the FRKHS. On this basis, we provide a learning rate for kernel regularized regression whose hypothesis space is the defined FRKHS. As applications, we define some concrete FRKHSs associated with polynomial operators such as the Bernstein–Durrmeyer operators, the de la Vallée Poussin operators on both the unit sphere [Formula: see text] and the unit ball [Formula: see text]. We show that these polynomial operators have reproducing property with respect to the corresponding concrete FRKHSs and show the learning rate for the kernel regularized regression. In short, we provide a way of constructing FRKHS operators with Fourier multipliers and show a learning framework from the view of operator approximation.

Ahpatron: A New Budgeted Online Kernel Learning Machine with Tighter Mistake Bound

In this paper, we study the mistake bound of online kernel learning on a budget. We propose a new budgeted online kernel learning model, called Ahpatron, which significantly improves the mistake bound of previous work and resolves an open problem related to upper bounds of hypothesis space constraints. We first present an aggressive variant of Perceptron, named AVP, a model without budget, which uses an active updating rule. Then we design a new budget maintenance mechanism, which removes a half of examples, and projects the removed examples onto a hypothesis space spanned by the remaining examples. Ahpatron adopts the above mechanism to approximate AVP. Theoretical analyses prove that Ahpatron has tighter mistake bounds, and experimental results show that Ahpatron outperforms the state-of-the-art algorithms on the same or a smaller budget.

Game-Theoretic Unlearnable Example Generator

Unlearnable example attacks are data poisoning attacks aiming to degrade the clean test accuracy of deep learning by adding imperceptible perturbations to the training samples, which can be formulated as a bi-level optimization problem. However, directly solving this optimization problem is intractable for deep neural networks. In this paper, we investigate unlearnable example attacks from a game-theoretic perspective, by formulating the attack as a nonzero sum Stackelberg game. First, the existence of game equilibria is proved under the normal setting and the adversarial training setting. It is shown that the game equilibrium gives the most powerful poison attack in that the victim has the lowest test accuracy among all networks within the same hypothesis space when certain loss functions are used. Second, we propose a novel attack method, called the Game Unlearnable Example (GUE), which has three main gradients. (1) The poisons are obtained by directly solving the equilibrium of the Stackelberg game with a first-order algorithm. (2) We employ an autoencoder-like generative network model as the poison attacker. (3) A novel payoff function is introduced to evaluate the performance of the poison. Comprehensive experiments demonstrate that GUE can effectively poison the model in various scenarios. Furthermore, the GUE still works by using a relatively small percentage of the training data to train the generator, and the poison generator can generalize to unseen data well. Our implementation code can be found at https://github.com/hong-xian/gue.

Multi-label deep transfer learning method for coupling fault diagnosis

With the development of complexity and integration of machines, the multiple components of the system are prone to simultaneous failures and the multi-fault signals may be measured in limited sensors. This fault mode is defined as coupling fault in this paper and becomes more common and attracts great attention. Current coupling fault diagnosis approaches rely on labeled data under consistent working conditions. However, in practical varying working conditions, the data is difficult to label and the traditional methods perform poorly. To address the challenge problem, we utilize some related annotated data under other working conditions for auxiliary learning and abstract this problem as a multi-label transfer learning problem which is a new research topic in fault diagnosis. We proposed a novel deep transfer coupling fault diagnosis method from the global and local views to achieve the effect of transfer, which is the first attempt to address this practical problem as far as we know. At the global level, we used Maximum Mean Discrepancy (MMD) at multiple stages of the network to reduce the hypothesis space of the model. Furthermore, we design a special manifold framework with multi-level similarity to constrain the hierarchy of the distance of the sample and the hierarchy of the labels being consistent between the two domains, to further improve the effect of local alignment. The proposed method has high accuracy in multiple transfer tasks for both public and laboratory datasets, powerfully demonstrating its effectiveness.

Quantifying robustness of the gap gene network

Early development of Drosophila melanogaster (fruit fly) facilitated by the gap gene network has been shown to be incredibly robust, and the same patterns emerge even when the process is seriously disrupted. We investigate this robustness using a previously developed computational framework called DSGRN (Dynamic Signatures Generated by Regulatory Networks). Our mathematical innovations include the conceptual extension of this established modeling technique to enable modeling of spatially monotone environmental effects, as well as the development of a collection of graph theoretic robustness scores for network models. This allows us to rank order the robustness of network models of cellular systems where each cell contains the same genetic network topology but operates under a parameter regime that changes continuously from cell to cell. We demonstrate the power of this method by comparing the robustness of two previously introduced network models of gap gene expression along the anterior–posterior axis of the fruit fly embryo, both to each other and to a random sample of networks with same number of nodes and edges. We observe that there is a substantial difference in robustness scores between the two models. Our biological insight is that random network topologies are in general capable of reproducing complex patterns of expression, but that using measures of robustness to rank order networks permits a large reduction in hypothesis space for highly conserved systems such as developmental networks.

Sparse Additive Machine With the Correntropy-Induced Loss.

Sparse additive machines (SAMs) have shown competitive performance on variable selection and classification in high-dimensional data due to their representation flexibility and interpretability. However, the existing methods often employ the unbounded or nonsmooth functions as the surrogates of 0-1 classification loss, which may encounter the degraded performance for data with outliers. To alleviate this problem, we propose a robust classification method, named SAM with the correntropy-induced loss (CSAM), by integrating the correntropy-induced loss (C-loss), the data-dependent hypothesis space, and the weighted lq,1 -norm regularizer ( q ≥ 1 ) into additive machines. In theory, the generalization error bound is estimated via a novel error decomposition and the concentration estimation techniques, which shows that the convergence rate O(n-1/4) can be achieved under proper parameter conditions. In addition, the theoretical guarantee on variable selection consistency is analyzed. Experimental evaluations on both synthetic and real-world datasets consistently validate the effectiveness and robustness of the proposed approach.

Meta-learning as a bridge between neural networks and symbolic Bayesian models

Abstract Meta-learning is even more broadly relevant to the study of inductive biases than Binz et al. suggest: Its implications go beyond the extensions to rational analysis that they discuss. One noteworthy example is that meta-learning can act as a bridge between the vector representations of neural networks and the symbolic hypothesis spaces used in many Bayesian models.

The Efficiency of Question-Asking Strategies in a Real-World Visual Search Task.

In recent years, a multitude of datasets of human-human conversations has been released for the main purpose of training conversational agents based on data-hungry artificial neural networks. In this paper, we argue that datasets of this sort represent a useful and underexplored source to validate, complement, and enhance cognitive studies on human behavior and language use. We present a method that leverages the recent development of powerful computational models to obtain the fine-grained annotation required to apply metrics and techniques from Cognitive Science to large datasets. Previous work in Cognitive Science has investigated the question-asking strategies of human participants by employing different variants of the so-called 20-question-game setting and proposing several evaluation methods. In our work, we focus on GuessWhat, a task proposed within the Computer Vision and Natural Language Processing communities that is similar in structure to the 20-question-game setting. Crucially, the GuessWhat dataset contains tens of thousands of dialogues based on real-world images, making it a suitable setting to investigate the question-asking strategies of human players on a large scale and in a natural setting. Our results demonstrate the effectiveness of computational tools to automatically code how the hypothesis space changes throughout the dialogue in complex visual scenes. On the one hand, we confirm findings from previous work on smaller and more controlled settings. On the other hand, our analyses allow us to highlight the presence of "uninformative" questions (in terms of Expected Information Gain) at specific rounds of the dialogue. We hypothesize that these questions fulfill pragmatic constraints that are exploited by human players to solve visual tasks in complex scenes successfully. Our work illustrates a method that brings together efforts and findings from different disciplines to gain a better understanding of human question-asking strategies on large-scale datasets, while at the same time posing new questions about the development of conversationalsystems.

SIN: Semantic Inference Network for Few-Shot Streaming Label Learning.

Streaming label learning aims to model newly emerged labels for multilabel classification systems, which requires plenty of new label data for training. However, in changing environments, only a small amount of new label data can practically be collected. In this work, we formulate and study few-shot streaming label learning (FSLL), which models emerging new labels with only a few annotated examples by utilizing the knowledge learned from past labels. We propose a meta-learning framework, semantic inference network (SIN), which can learn and infer the semantic correlation between new labels and past labels to adapt FSLL tasks from a few examples effectively. SIN leverages label semantic representation to regularize the output space and acquires labelwise meta-knowledge based on gradient-based meta-learning. Moreover, SIN incorporates a novel label decision module with a meta-threshold loss to find the optimal confidence thresholds for each new label. Theoretically, we illustrate that the proposed semantic inference mechanism could constrain the complexity of hypotheses space to reduce the risk of overfitting and achieve better generalizability. Experimentally, extensive empirical results and ablation studies demonstrate the performance of SIN is superior to the prior state-of-the-art methods on FSLL.