Sparse Set Research Articles

Abstract 2024: Elucidating The Role Of Shape In Aortic Disease State Pathology With Machine Learning

Maximum aortic diameter has been the mainstay metric for aortic disease management and surgical intervention for decades, specifically acute aortic dissection and aneurysm. While this size metric has been effective at separating patient populations into groupings associated with required intervention and monitoring for eventual intervention respectively, size alone ignores the role of complex geometric shape and how it may be further indicative of disease state. Analyses of aortic shape in addition to size have been discussed at length in literature, however the representations and methods to convey shape are siloed and often highly subjective. Machine learning methodologies offer a means to explore high dimensional feature spaces and can identify a sparse set of metrics which result in a division of patient responses parametrized by physical and interpretable variables. Additionally, by introducing a modeling workflow which contains engineered parameters designed to describe aortic shape, in addition to traditional size metrics, the exploration of shape dependence on intervention outcomes becomes more comprehensive across size-shape parameter spaces. The engineered parameters presented primarily incorporate estimations of integrated Gaussian surface curvature into features that contain both the magnitude and spatial variation in shape of the aortas. The models discussed accurately classify over two hundred aortic disease patients spanning aneurysm, dissection, and other modalities with accuracies above 90% and performance across multiple modeling methodologies is discussed. The classification success of the geometrically informed feature space suggests the underlying dependence on pathology shape on the success of the intervention outcome. The most clinically effective models will preserve the wealth of information in modern medical diagnostics and imaging into descriptive, and physically interpretable, parameters to aid in clinical decision making.

Perceptual Quality Assessment of NeRF and Neural View Synthesis Methods for Front‐Facing Views

AbstractNeural view synthesis (NVS) is one of the most successful techniques for synthesizing free viewpoint videos, capable of achieving high fidelity from only a sparse set of captured images. This success has led to many variants of the techniques, each evaluated on a set of test views typically using image quality metrics such as PSNR, SSIM, or LPIPS. There has been a lack of research on how NVS methods perform with respect to perceived video quality. We present the first study on perceptual evaluation of NVS and NeRF variants. For this study, we collected two datasets of scenes captured in a controlled lab environment as well as in‐the‐wild. In contrast to existing datasets, these scenes come with reference video sequences, allowing us to test for temporal artifacts and subtle distortions that are easily overlooked when viewing only static images. We measured the quality of videos synthesized by several NVS methods in a well‐controlled perceptual quality assessment experiment as well as with many existing state‐of‐the‐art image/video quality metrics. We present a detailed analysis of the results and recommendations for dataset and metric selection for NVS evaluation.

Development of Novel Methods for QSAR Modeling by Machine Learning Repeatedly: A Case Study on Drug Distribution to Each Tissue.

Artificial intelligence is expected to help identify excellent candidates in drug discovery. However, we face a lack of data, as it is time-consuming and expensive to acquire raw data perfectly for many compounds. Hence, we tried to develop a novel quantitative structure-activity relationship (QSAR) method to predict a parameter more precisely from an incomplete data set via optimizing data handling by making use of predicted explanatory variables. As a case study we focused on the tissue-to-plasma partition coefficient (Kp), which is an important parameter for understanding drug distribution in tissues and building the physiologically based pharmacokinetic model and is a representative of small and sparse data sets. In this study, we predicted the Kp values of 119 compounds in nine tissues (adipose, brain, gut, heart, kidney, liver, lung, muscle, and skin), although some of these were not available. To fill the missing values in Kp for each tissue, first we predicted those Kp values by the nonmissing data set using a random forest (RF) model with in vitro parameters (log P, fu, Drug Class, and fi) like a classical prediction by a QSAR model. Next, to predict the tissue-specific Kp values in a test data set, we constructed a second RF model with not only in vitro parameters but also the Kp values of other tissues (i.e., other than target tissues) predicted by the first RF model as explanatory variables. Furthermore, we tested all possible combinations of explanatory variables and selected the model with the highest predictability from the test data set as the final model. The evaluation of Kp prediction accuracy based on the root-mean-square error and R2 value revealed that the proposed models outperformed other machine learning methods such as the conventional RF and message-passing neural networks. Significant improvements were observed in the Kp values of adipose tissue, brain, kidney, liver, and skin. These improvements indicated that the Kp information on other tissues can be used to predict the same for a specific tissue. Additionally, we found a novel relationship between each tissue by evaluating all combinations of explanatory variables. In conclusion, we developed a novel RF model to predict Kp values. We hope that this method will be applied to various problems in the field of experimental biology which often contains missing values in the near future.

A robust computational framework for variational data assimilation of mean flows with sparse measurements corrupted by strong outliers

A variational framework for the reconstruction of time-averaged mean flows using a sparse set of observations with large magnitudes of noise (referred to as outliers), is presented. The observations constitute a set of point-wise measurements of the mean flow with outliers at certain measurement locations and are incorporated into a numerical simulation governed by the two-dimensional, incompressible Reynolds-averaged Navier-Stokes (RANS) equations with an unknown momentum forcing. This forcing, which corresponds to the divergence of the Reynolds stress tensor, is calculated from a direct-adjoint optimization procedure to reduce the deviation between the measured and estimated mean velocities. ℓ2, ℓ1, Huber, and hybrid loss functions are used to represent the discrepancy in the mean velocity field between the measurements and the predictions. A variety of algorithms are considered to solve the optimization problem with these loss functions and a performance comparison in terms of the quality and physical features of the recovered mean flow is presented. The Huber loss function performed best as it remained robust to strong outliers in the measurements with its ℓ1 contribution and also ensured the uniqueness of the optimal solution with its ℓ2 contribution. Huber loss functions restrict the effect of outliers at the local measurement locations, thereby not affecting the quality of the high-dimensional reconstructed mean flow field. The hybrid loss function, a modified form of the continuous Huber loss function, also recovered the mean flow with high accuracy. We demonstrate the performance of the data assimilation framework for the case of two-dimensional laminar flow around a circular cylinder at Re=100. We then extend the analysis to the case of two-dimensional laminar flow over a backward-facing step at Re=500, to further assess the efficacy and robustness of the data assimilation framework.

Oscillations of Fourier coefficients of product of L-functions at integers in a sparse set

Let [Formula: see text] be a normalized Hecke eigenform of weight [Formula: see text] for the full modular group [Formula: see text]. In this paper, we obtain the asymptotic of higher moments of general divisor functions associated to the Fourier coefficients of Rankin–Selberg [Formula: see text]-functions [Formula: see text], supported at the integers represented by primitive integral positive-definite binary quadratic forms (reduced forms) of a fixed discriminant [Formula: see text] We improve previous results in the case when the reduced form is given by [Formula: see text]

High‐dimensional feature screening for nonlinear associations with survival outcome using restricted mean survival time

SummaryFeature screening is an important tool in analysing ultrahigh‐dimensional data, particularly in the field of Omics and oncology studies. However, most attention has been focused on identifying features that have a linear or monotonic impact on the response variable. Detecting a sparse set of variables that have a nonlinear or nonmonotonic relationship with the response variable is still a challenging task. To fill the gap, this paper proposed a robust model‐free screening approach for right‐censored survival data by providing a new perspective of quantifying the covariate effect on the restricted mean survival time, rather than the routinely used hazard function. The proposed measure, based on the difference between the restricted mean survival time of covariate‐stratified and overall data, is able to identify comprehensive types of associations including linear, nonlinear, nonmonotone and even local dependencies like change points. The sure screening property is established, and a more flexible iterative screening procedure is developed to increase the accuracy of the variable screening. Simulation studies are carried out to demonstrate the superiority of the proposed method in selecting important features with a complex association with the response variable. The potential of applying the proposed method to handle interval‐censored failure time data has also been explored in simulations, and the results have been promising. The method is applied to a breast cancer dataset to identify potential prognostic factors, which reveals potential associations between breast cancer and lymphoma.

DIGCN: A Dynamic Interaction Graph Convolutional Network Based on Learnable Proposals for Object Detection

We propose a Dynamic Interaction Graph Convolutional Network (DIGCN), an image object detection method based on learnable proposals and GCN. Existing object detection methods usually work on dense candidates, resulting in redundant and near-duplicate results. Meanwhile, non-maximum suppression post-processing operations are required to eliminate negative effects, which increases the computational complexity. Although the existing sparse detector avoids cumbersome post-processing operations, it ignores the potential relationship between objects and proposals, which hinders detection accuracy improvement. Therefore, we propose a dynamic interaction GCN module in the DIGCN, which performs dynamic interaction and relational modeling on the proposal boxes and proposal features to improve the object detection accuracy. In addition, we introduce a learnable proposal method with a sparse set of learned object proposals to eliminate a huge number of hand-designed object candidates, avoiding complicated tasks such as object candidate design and many-to-one label assignment, and reducing object detection model complexity to a certain extent. DIGCN demonstrates accuracy and run-time performance on par with the well-established and highly optimized detector baselines on the challenging COCO dataset, e.g. with the ResNet-101FPN as the backbone our method attains the accuracy of 46.5 AP while processing 13 frames per second. Our work provides a new method for object detection research.

Tracking long-term modal behaviour of a footbridge and identifying potential SHM approaches

Numerous studies have investigated the long-term monitoring of natural frequencies, primarily focusing on medium–large highway bridges, using expensive monitoring systems with a large array of sensors. However, this paper addresses the less explored issue of monitoring a footbridge, examining four critical aspects: (i) sensing system, (ii) frequency extraction method, (iii) data modelling techniques, and (iv) damage detection. The paper proposes a low-cost all-in-one sensor/logger unit instead of a conventional sensing system to address the first issue. For the second issue, many studies use natural frequency data extracted from measured acceleration for data modelling, the paper highlights the impact of the input parameters used in the automated frequency extraction process, which affects the number and quality of frequency data points extracted and subsequently influences the data models that can be created. After that, the paper proposes a modified PCA model optimised for computational efficiency, designed explicitly for sparse data from a low-cost monitoring system, and suitable for future on-board computation. It also explores the capabilities and limitations of a data model developed using a limited data set. The paper demonstrates these aspects using data collected from a 108 m cable-stayed footbridge over several months. Finally, the detection of damage is achieved by employing the one-class SVM machine learning technique, which utilises the outcomes obtained from data modelling. In summary, this paper addresses the challenges associated with the long-term monitoring of a footbridge, including selecting a suitable sensing system, automated frequency extraction, data modelling techniques, and damage detection. The proposed solutions offer a cost-effective and efficient approach to monitoring footbridges while considering the challenges of sparse data sets.

On inductive biases for the robust and interpretable prediction of drug concentrations using deep compartment models

Conventional pharmacokinetic (PK) models contain several useful inductive biases guiding model convergence to more realistic predictions of drug concentrations. Implementing similar biases in standard neural networks can be challenging, but might be fundamental for model robustness and predictive performance. In this study, we build on the deep compartment model (DCM) architecture by introducing constraints that guide the model to explore more physiologically realistic solutions. Using a simulation study, we show that constraints improve robustness in sparse data settings. Additionally, predicted concentration–time curves took on more realistic shapes compared to unconstrained models. Next, we propose the use of multi-branch networks, where each covariate can be connected to specific PK parameters, to reduce the propensity of models to learn spurious effects. Another benefit of this architecture is that covariate effects are isolated, enabling model interpretability through the visualization of learned functions. We show that all models were sensitive to learning false effects when trained in the presence of unimportant covariates, indicating the importance of selecting an appropriate set of covariates to link to the PK parameters. Finally, we compared the predictive performance of the constrained models to previous relevant population PK models on a real-world data set of 69 haemophilia A patients. Here, constrained models obtained higher accuracy compared to the standard DCM, with the multi-branch network outperforming previous PK models. We conclude that physiological-based constraints can improve model robustness. We describe an interpretable architecture which aids model trust, which will be key for the adoption of machine learning-based models in clinical practice.

Probabilistic Offline Policy Ranking with Approximate Bayesian Computation

In practice, it is essential to compare and rank candidate policies offline before real-world deployment for safety and reliability. Prior work seeks to solve this offline policy ranking (OPR) problem through value-based methods, such as Off-policy evaluation (OPE). However, they fail to analyze special case performance (e.g., worst or best cases), due to the lack of holistic characterization of policies’ performance. It is even more difficult to estimate precise policy values when the reward is not fully accessible under sparse settings. In this paper, we present Probabilistic Offline Policy Ranking (POPR), a framework to address OPR problems by leveraging expert data to characterize the probability of a candidate policy behaving like experts, and approximating its entire performance posterior distribution to help with ranking. POPR does not rely on value estimation, and the derived performance posterior can be used to distinguish candidates in worst-, best-, and average-cases. To estimate the posterior, we propose POPR-EABC, an Energy-based Approximate Bayesian Computation (ABC) method conducting likelihood-free inference. POPR-EABC reduces the heuristic nature of ABC by a smooth energy function, and improves the sampling efficiency by a pseudo-likelihood. We empirically demonstrate that POPR-EABC is adequate for evaluating policies in both discrete and continuous action spaces across various experiment environments, and facilitates probabilistic comparisons of candidate policies before deployment.

Learning Generalizable and Composable Abstractions for Transfer in Reinforcement Learning

Reinforcement Learning (RL) in complex environments presents many challenges: agents require learning concise representations of both environments and behaviors for efficient reasoning and generalizing experiences to new, unseen situations. However, RL approaches can be sample-inefficient and difficult to scale, especially in long-horizon sparse reward settings. To address these issues, the goal of my doctoral research is to develop methods that automatically construct semantically meaningful state and temporal abstractions for efficient transfer and generalization. In my work, I develop hierarchical approaches for learning transferable, generalizable knowledge in the form of symbolically represented options, as well as for integrating search techniques with RL to solve new problems by efficiently composing the learned options. Empirical results show that the resulting approaches effectively learn and transfer knowledge, achieving superior sample efficiency compared to SOTA methods while also enhancing interpretability.

Pantypes: Diverse Representatives for Self-Explainable Models

Prototypical self-explainable classifiers have emerged to meet the growing demand for interpretable AI systems. These classifiers are designed to incorporate high transparency in their decisions by basing inference on similarity with learned prototypical objects. While these models are designed with diversity in mind, the learned prototypes often do not sufficiently represent all aspects of the input distribution, particularly those in low density regions. Such lack of sufficient data representation, known as representation bias, has been associated with various detrimental properties related to machine learning diversity and fairness. In light of this, we introduce pantypes, a new family of prototypical objects designed to capture the full diversity of the input distribution through a sparse set of objects. We show that pantypes can empower prototypical self-explainable models by occupying divergent regions of the latent space and thus fostering high diversity, interpretability and fairness.

NeuSurf: On-Surface Priors for Neural Surface Reconstruction from Sparse Input Views

Recently, neural implicit functions have demonstrated remarkable results in the field of multi-view reconstruction. However, most existing methods are tailored for dense views and exhibit unsatisfactory performance when dealing with sparse views. Several latest methods have been proposed for generalizing implicit reconstruction to address the sparse view reconstruction task, but they still suffer from high training costs and are merely valid under carefully selected perspectives. In this paper, we propose a novel sparse view reconstruction framework that leverages on-surface priors to achieve highly faithful surface reconstruction. Specifically, we design several constraints on global geometry alignment and local geometry refinement for jointly optimizing coarse shapes and fine details. To achieve this, we train a neural network to learn a global implicit field from the on-surface points obtained from SfM and then leverage it as a coarse geometric constraint. To exploit local geometric consistency, we project on-surface points onto seen and unseen views, treating the consistent loss of projected features as a fine geometric constraint. The experimental results with DTU and BlendedMVS datasets in two prevalent sparse settings demonstrate significant improvements over the state-of-the-art methods.

Synergistic Anchored Contrastive Pre-training for Few-Shot Relation Extraction

Few-shot Relation Extraction (FSRE) aims to extract relational facts from a sparse set of labeled corpora. Recent studies have shown promising results in FSRE by employing Pre-trained Language Models (PLMs) within the framework of supervised contrastive learning, which considers both instances and label facts. However, how to effectively harness massive instance-label pairs to encompass the learned representation with semantic richness in this learning paradigm is not fully explored. To address this gap, we introduce a novel synergistic anchored contrastive pre-training framework. This framework is motivated by the insight that the diverse viewpoints conveyed through instance-label pairs capture incomplete yet complementary intrinsic textual semantics. Specifically, our framework involves a symmetrical contrastive objective that encompasses both sentence-anchored and label-anchored contrastive losses. By combining these two losses, the model establishes a robust and uniform representation space. This space effectively captures the reciprocal alignment of feature distributions among instances and relational facts, simultaneously enhancing the maximization of mutual information across diverse perspectives within the same relation. Experimental results demonstrate that our framework achieves significant performance enhancements compared to baseline models in downstream FSRE tasks. Furthermore, our approach exhibits superior adaptability to handle the challenges of domain shift and zero-shot relation extraction. Our code is available online at https://github.com/AONE-NLP/FSRE-SaCon.

Active self-training for weakly supervised 3D scene semantic segmentation

Since the preparation of labeled data for training semantic segmentation networks of point clouds is a time-consuming process, weakly supervised approaches have been introduced to learn from only a small fraction of data. These methods are typically based on learning with contrastive losses while automatically deriving per-point pseudo-labels from a sparse set of user-annotated labels. In this paper, our key observation is that the selection of which samples to annotate is as important as how these samples are used for training. Thus, we introduce a method for weakly supervised segmentation of 3D scenes that combines self-training with active learning. Active learning selects points for annotation that are likely to result in improvements to the trained model, while self-training makes efficient use of the user-provided labels for learning the model. We demonstrate that our approach leads to an effective method that provides improvements in scene segmentation over previous work and baselines, while requiring only a few user annotations.

Voluntary E-Learning Exercises Support Students in Mastering Statistics

University students often learn statistics in large classes, and in such learning environments, students face an exceptionally high risk of failure. One reason for this is students’ frequent statistics anxiety. This study shows how students can be supported using e-learning exercises with automated knowledge of correct response feedback, supplementing a face-to-face lecture. To this end, we surveyed 67 undergraduate social science students at a German university and observed their weekly e-learning exercises. We aggregated students’ exercise behavior throughout the semester to explain their exam performance. To control for participation bias, we included essential predictors of educational success, such as prior achievement, motivation, personality traits, time preferences, and goals. We applied a double selection procedure based on the machine learning method Elastic Net to include an optimal but sparse set of control variables. The e-learning exercises indirectly promoted the self-regulated learning techniques of retrieval practice and spacing and provided corrective feedback. Working on the e-learning exercises increased students’ performance on the final exam, even after controlling for the rich set of control variables. Two-thirds of students used our designed e-learning exercises; however, only a fraction of students spaced out the exercises, although students who completed the exercises during the semester and were not cramming at the end benefited additionally. Finally, we discuss how the results of our study inform the literature on retrieval practice, spacing, feedback, and e-learning in higher education.

Hair-Derived Exposome Exploration of Cardiometabolic Health: Piloting a Bayesian Multitrait Variable Selection Approach.

Cardiometabolic health is complex and characterized by an ensemble of correlated and/or co-occurring conditions including obesity, dyslipidemia, hypertension, and diabetes mellitus. It is affected by social, lifestyle, and environmental factors, which in-turn exhibit complex correlation patterns. To account for the complexity of (i) exposure profiles and (ii) health outcomes, we propose to use a multitrait Bayesian variable selection approach and identify a sparse set of exposures jointly explanatory of the complex cardiometabolic health status. Using data from a subset (N = 941 participants) of the nutrition, environment, and cardiovascular health (NESCAV) study, we evaluated the link between measurements of the cumulative exposure to (N = 33) pollutants derived from hair and cardiometabolic health as proxied by up to nine measured traits. Our multitrait analysis showed increased statistical power, compared to single-trait analyses, to detect subtle contributions of exposures to a set of clinical phenotypes, while providing parsimonious results with improved interpretability. We identified six exposures that were jointly explanatory of cardiometabolic health as modeled by six complementary traits, of which, we identified strong associations between hexachlorobenzene and trifluralin exposure and adverse cardiometabolic health, including traits of obesity, dyslipidemia, and hypertension. This supports the use of this type of approach for the joint modeling, in an exposome context, of correlated exposures in relation to complex and multifaceted outcomes.

Correlation-informed ordered dictionary learning for imaging in complex media

We propose a method for imaging in scattering media when large and diverse datasets are available. It has two steps. Using a dictionary learning algorithm the first step estimates the true Green's function vectors as columns in an unordered sensing matrix. The array data comes from many sparse sets of sources whose location and strength are not known to us. In the second step, the columns of the estimated sensing matrix are ordered for imaging using the multidimensional scaling algorithm with connectivity information derived from cross-correlations of its columns, as in time reversal. For these two steps to work together, we need data from large arrays of receivers so the columns of the sensing matrix are incoherent for the first step, as well as from sub-arrays so that they are coherent enough to obtain connectivity needed in the second step. Through simulation experiments, we show that the proposed method is able to provide images in complex media whose resolution is that of a homogeneous medium.

Portability of genomic predictions trained on sparse factorial designs across two maize silage breeding cycles

Key messageWe validated the efficiency of genomic predictions calibrated on sparse factorial training sets to predict the next generation of hybrids and tested different strategies for updating predictions along generations.Genomic selection offers new prospects for revisiting hybrid breeding schemes by replacing extensive phenotyping of individuals with genomic predictions. Finding the ideal design for training genomic prediction models is still an open question. Previous studies have shown promising predictive abilities using sparse factorial instead of tester-based training sets to predict single-cross hybrids from the same generation. This study aims to further investigate the use of factorials and their optimization to predict line general combining abilities (GCAs) and hybrid values across breeding cycles. It relies on two breeding cycles of a maize reciprocal genomic selection scheme involving multiparental connected reciprocal populations from flint and dent complementary heterotic groups selected for silage performances. Selection based on genomic predictions trained on a factorial design resulted in a significant genetic gain for dry matter yield in the new generation. Results confirmed the efficiency of sparse factorial training sets to predict candidate line GCAs and hybrid values across breeding cycles. Compared to a previous study based on the first generation, the advantage of factorial over tester training sets appeared lower across generations. Updating factorial training sets by adding single-cross hybrids between selected lines from the previous generation or a random subset of hybrids from the new generation both improved predictive abilities. The CDmean criterion helped determine the set of single-crosses to phenotype to update the training set efficiently. Our results validated the efficiency of sparse factorial designs for calibrating hybrid genomic prediction experimentally and showed the benefit of updating it along generations.

Multi‐Spacecraft Magnetic Field Reconstructions: A Cross‐Scale Comparison of Methods

AbstractSpace plasma studies frequently use in situ magnetic field measurements taken from many spacecraft simultaneously. A useful data product of these measurements is the reconstructed magnetic field in a volume near the spacecraft observatory. We compare a standard Linear method of computing the magnetic field at arbitrary spatial points to two novel approaches: a Radial Basis Function interpolation and a time‐dependent 2D inverse distance weighted interpolation scheme called Timesync. These three methods, which only require in situ measurements of the magnetic fields and bulk plasma velocities at a sparse set of spatial points, are implemented on synthetic data drawn from a time‐evolving numerical simulation of plasma turbulence. We compare both the topology of the reconstructed field to the ground truth of the simulation and the statistics of the fluctuations found in the reconstructed field to those from the simulated turbulence. We conclude that the Radial Basis Function and Timesync methods outperform the Linear method in both topological and statistical comparisons.