Sparse Method Research Articles

Transient Interference Excision and Spectrum Reconstruction with Partial Samples Using Modified Alternating Direction Method of Multipliers-Net for the Over-the-Horizon Radar.

Transient interference often submerges the actual targets when employing over-the-horizon radar (OTHR) to detect targets. In addition, modern OTHR needs to carry out multi-target detection from sea to air, resulting in the sparse sampling of echo data. The sparse OTHR signal will raise serious grating lobes using conventional methods and thus degrade target detection performance. This article proposes a modified Alternating Direction Method of Multipliers (ADMM)-Net to reconstruct the target and clutter spectrum of sparse OTHR signals so that target detection can be performed normally. Firstly, transient interferences are identified based on the sparse basis representation and then excised. Therefore, the processed signal can be seen as a sparse OTHR signal. By solving the Doppler sparsity-constrained optimization with the trained network, the complete Doppler spectrum is reconstructed effectively for target detection. Compared with traditional sparse solution methods, the presented approach can balance the efficiency and accuracy of OTHR signal spectrum reconstruction. Both simulation and real-measured OTHR data proved the proposed approach's performance.

Video velocity measurement: a two-stage flow velocity prediction method based on deep learning

Abstract Due to the uncertainty in output caused by environmental changes, significant discrepancies are expected between the surface flow velocities predicted using deep learning methods and the instantaneous flow velocities. In this paper, a two-stage deep learning flow velocity measurement algorithm is proposed. During the external calibration process, the upper and lower frames of the recorded water flow video are cyclically traversed to acquire predicted flow velocity values using the deep learning velocity measurement algorithm. Meanwhile, the pixel displacement is obtained using the sparse optical flow tracking method and then post-processed to derive the velocity calibration value and pixel calibration value. During the detection process, the deep learning-predicted flow velocity is internally calibrated using the velocity calibration value and the pixel calibration value to adapt to changes in water flows. Compared with the pre-improved algorithm, the method achieves the minimum root mean square error in five different flow velocity videos and maintains high accuracy when the flow velocity changes rapidly. The obtained results are very promising and can help improve the reliability of video flow rate assessment algorithms.

A Sparse SAR Imaging Method for Low-Oversampled Staggered Mode via Compound Regularization

High-resolution wide-swath (HRWS) imaging is the research focus of the modern spaceborne synthetic-aperture radar (SAR) imaging field, with significant relevance and vast application potential. Staggered SAR, as an innovative imaging system, mitigates blind areas across the entire swath by periodically altering the radar pulse repetition interval (PRI), thereby extending the swath width to multiples of that achievable by conventional systems. However, the staggered mode introduces inherent challenges, such as nonuniform azimuth sampling and echo data loss, leading to azimuth ambiguities and substantially impacting image quality. This paper proposes a sparse SAR imaging method for the low-oversampled staggered mode via compound regularization. The proposed method not only effectively suppresses azimuth ambiguities arising from nonuniform sampling without necessitating the restoration of missing echo data, but also incorporates total variation (TV) regularization into the sparse reconstruction model. This enhances the accurate reconstruction of distributed targets within the scene. The efficacy of the proposed method is substantiated through simulations and real data experiments from spaceborne missions.

Dimensional homogeneity constrained gene expression programming for discovering governing equations

Data-driven discovery of governing equations is of great significance for helping us understand intrinsic mechanisms and build physical models. Recently, numerous highly innovative algorithms have emerged, aimed at inversely discovering the underlying governing equations from data, such as sparse regression-based methods and symbolic regression-based methods. Along this direction, a novel dimensional homogeneity constrained gene expression programming (DHC-GEP) method is proposed in this work. The DHC-GEP simultaneously discovers the forms and coefficients of functions using basic mathematical operators and physical variables, without requiring preassumed candidate functions. The constraint of dimensional homogeneity is capable of filtering out the overfitting equations effectively. The key advantages of DHC-GEP compared with the original gene expression programming, including being more robust to hyperparameters, the noise level and the size of datasets, are demonstrated on two benchmark studies. Furthermore, DHC-GEP is employed to discover the unknown constitutive relations of two representative non-equilibrium flows. Galilean invariance and the second law of thermodynamics are imposed as constraints to enhance the reliability of the discovered constitutive relations. Comparisons, both quantitative and qualitative, indicate that the derived constitutive relations are more accurate than the conventional Burnett equations in a wide range of Knudsen numbers and Mach numbers, and are also applicable to the cases beyond the parameter space of the training data.

Bearing Fault Diagnosis via Stepwise Sparse Regularization with an Adaptive Sparse Dictionary.

Vibration monitoring is one of the most effective approaches for bearing fault diagnosis. Within this category of techniques, sparsity constraint-based regularization has received considerable attention for its capability to accurately extract repetitive transients from noisy vibration signals. The optimal solution of a sparse regularization problem is determined by the regularization term and the data fitting term in the cost function according to their weights, so a tradeoff between sparsity and data fidelity has to be made inevitably, which restricts conventional regularization methods from maintaining strong sparsity-promoting capability and high fitting accuracy at the same time. To address the limitation, a stepwise sparse regularization (SSR) method with an adaptive sparse dictionary is proposed. In this method, the bearing fault diagnosis is modeled as a multi-parameter optimization problem, including time indexes of the sparse dictionary and sparse coefficients. Firstly, sparsity-enhanced optimization is conducted by amplifying the regularization parameter, making the time indexes and the number of atoms adaptively converge to the moments when impulses occur and the number of impulses, respectively. Then, fidelity-enhanced optimization is carried out by removing the regularization term, thereby obtaining the high-precision reconstruction amplitudes. Simulations and experiments verify that the reconstruction accuracy of the SSR method outperforms other sparse regularization methods under most noise conditions, and thus the proposed method can provide more accurate results for bearing fault diagnosis.

Using a hyperbolic cross to solve non-linear macroeconomic models

The paper presents a sparse grid approximation method based on the hyperbolic cross and applies it to solve non-linear macroeconomic models. We show how the standard hyperbolic cross can be extended to give greater control over the approximating grid and we discuss how to implement an anisotropic hyperbolic cross. Applying the approximation method to four macroeconomic models, we establish that it delivers a level of accuracy on par or better than Smolyak's method and that it can produce accurate approximations using fewer points than Smolyak's method.

A Bayesian Structural Modal Updating Method Based on Sparse Grid and Ensemble Kalman Filter

This study presents a sparse grid interpolation and ensemble Kalman filter (EnKF)-based Markov Chain Monte Carlo (MCMC) method (SG-EnMCMC). Initiating with the formulation of a recursive equation for the state space vector, derived from the structural dynamic equation, this study adopts a dimensionality reduction strategy. This approach involves a separation of physical parameters and the state space vector. The acquisition of physical parameters is accomplished through sampling, utilizing sample moments to substitute population moments, thereby mitigating the need for computationally high-dimensional covariance matrix calculations. To further streamline the recursive equation of the state space vector, a sparse grid method is employed for interpolation. This step simplifies the process while ensuring superior accuracy compared to the Extended Kalman Filter (EKF) and Unscented Kalman Filter (UKF). Subsequent to this, acceptance rates and the final parameter posterior distribution within the MCMC framework are derived. The efficiency of the proposed method is assessed through validation in two shaking table experiments.

Voxelwise Multivariate Analysis of Brain-Psychosocial Associations in Adolescents Reveals 6 Latent Dimensions of Cognition and Psychopathology

BackgroundAdolescence heralds the onset of considerable psychopathology, which may be conceptualized as an emergence of altered covariation between symptoms and brain measures. Multivariate methods can detect such modes of covariation or latent dimensions, but none specifically relating to psychopathology have yet been found using population-level structural brain data. Using voxelwise (instead of parcellated) brain data may strengthen latent dimensions’ brain-psychosocial relationships, but this creates computational challenges. MethodsWe obtained voxelwise gray matter density and psychosocial variables from the baseline (ages 9–10 years) Adolescent Brain Cognitive Development (ABCD) Study cohort (N = 11,288) and employed a state-of-the-art segmentation method, sparse partial least squares, and a rigorous machine learning framework to prevent overfitting. ResultsWe found 6 latent dimensions, 4 of which pertain specifically to mental health. The mental health dimensions were related to overeating, anorexia/internalizing, oppositional symptoms (all ps < .002) and attention-deficit/hyperactivity disorder symptoms (p = .03). Attention-deficit/hyperactivity disorder was related to increased and internalizing symptoms related to decreased gray matter density in dopaminergic and serotonergic midbrain areas, whereas oppositional symptoms were related to increased gray matter in a noradrenergic nucleus. Internalizing symptoms were related to increased and oppositional symptoms to reduced gray matter density in the insular, cingulate, and auditory cortices. Striatal regions featured strongly, with reduced caudate nucleus gray matter in attention-deficit/hyperactivity disorder and reduced putamen gray matter in oppositional/conduct problems. Voxelwise gray matter density generated stronger brain-psychosocial correlations than brain parcellations. ConclusionsVoxelwise brain data strengthen latent dimensions of brain-psychosocial covariation, and sparse multivariate methods increase their psychopathological specificity. Internalizing and externalizing symptoms are associated with opposite gray matter changes in similar cortical and subcortical areas.

Sparse reconstruction of ultrasonic guided wave signals of fluid-filled pipes by multistrategy hybrid DBO-OMP using dispersive Hanning-windowed chirplet model

Ultrasonic guided waves (UGWs) in fluid-filled pipes are subject to more severe dispersion and coherent noise than those of vacant pipes, posing significant challenges for defect identification and localization. To this end, a novel sparse signal decomposition method called multistrategy hybrid dung beetle optimization based orthogonal matching pursuit (MHDBO-OMP) using dispersive Hanning-windowed chirplet model was proposed, of which the model not only represents the nonlinear frequency and asymmetric envelope of the dispersive UGW, but also contains the mode and propagation distance information, and this model exhibits better matching and interpretability than the other UGW models. For dispersive perturbed UGW signals with multiple modes and features, MHDBO-OMP achieves high reconstruction accuracy in both signal reconstruction (RES < 0.3) and propagation distance estimation (RES < 3 cm). When applied to experimental UGW signals obtained from water/oil-filled pipes containing cracks and corrosion pits, MHDBO-OMP successfully reconstructs dispersion-compensated signals that solely contain feature signals, and the minimum location error is 0.72 cm while the average location error is 2.65 cm. © 2001 Elsevier Science. All rights reserved.

Bayesian hierarchical hyper-Laplacian priors for high-resolution defect imaging in pipe structures

The sparse defect imaging using guided waves for pipe structures has attracted a lot of attention due to its intuitive way of defect localization. However, there exist few attempts to employ the lp norm (p < 1) in sparse imaging to improve imaging performance. In this study, a new sparse imaging method based on Bayesian hierarchical hyper-Laplacian prior is proposed for pipe defect detection. A hyper-Laplacian prior is considered to obtain a sparser resolution, which aims at improving the imaging performance including defect imaging resolution and defect detection accuracy. A solid theoretical framework is constructed and derived to represent the hyper-Laplacian prior and to adaptively estimate all model parameters. Meanwhile, a two-step discretizing strategy is designed to reduce the computational cost. The experimental results demonstrate the superiority of the proposed method.

Sparse Spectral Methods for Solving High-Dimensional and Multiscale Elliptic PDEs

Sparse Spectral Methods for Solving High-Dimensional and Multiscale Elliptic PDEs

PPG heart rate extraction algorithm based on the motion artifact intensity Classification and removal framework

When heart rate is extracted by wearable PPG, motion artifact noise causes serious interference. Since the accuracy of PPG heart rate extraction tends to be positively correlated with the complexity of heart rate extraction algorithm, it is challenging to extract heart rate via wearable PPG throughout the day by taking into account both algorithmic complexity and the accuracy of heart rate extraction. In this paper, a PPG heart rate extraction algorithm is proposed under the motion artifact intensity classification and removal (MAICR) framework. Firstly, it evaluates the motion artifact intensity using acceleration time-domain information and PPG frequency-domain information. Then, different heart rate estimation methods are selected according to motion artifact intensity, in ascending order. This corresponds to the use of the frequency-domain direct extraction method, the MNLMS adaptive filtering method, and the joint sparse spectral reconstruction method. Additionally, the overall framework of the algorithm (MAICR) is developed by considering the performance of various methods. Finally, experiments are conducted on the public dataset to demonstrate that the proposed algorithm is applicable to reduce the algorithmic complexity effectively while maintaining the accuracy in heart rate extraction at a certain degree.

A Python Toolbox for Data-Driven Aerodynamic Modeling Using Sparse Gaussian Processes

In aerodynamics, characterizing the aerodynamic behavior of aircraft typically requires a large number of observation data points. Real experiments can generate thousands of data points with suitable accuracy, but they are time-consuming and resource-intensive. Consequently, conducting real experiments at new input configurations might be impractical. To address this challenge, data-driven surrogate models have emerged as a cost-effective and time-efficient alternative. They provide simplified mathematical representations that approximate the output of interest. Models based on Gaussian Processes (GPs) have gained popularity in aerodynamics due to their ability to provide accurate predictions and quantify uncertainty while maintaining tractable execution times. To handle large datasets, sparse approximations of GPs have been further investigated to reduce the computational complexity of exact inference. In this paper, we revisit and adapt two classic sparse methods for GPs to address the specific requirements frequently encountered in aerodynamic applications. We compare different strategies for choosing the inducing inputs, which significantly impact the complexity reduction. We formally integrate our implementations into the open-source Python toolbox SMT, enabling the use of sparse methods across the GP regression pipeline. We demonstrate the performance of our Sparse GP (SGP) developments in a comprehensive 1D analytic example as well as in a real wind tunnel application with thousands of training data points.

Global probability distribution structure-sparsity filter pruning for edge fault diagnosis in resource constrained wireless sensor networks

In this paper, a global probability distribution structure-sparsity filter pruning is proposed to address the problem of difficult deployment of diagnostic models in resource constrained wireless sensor networks (WSNs) for edge fault diagnosis. Firstly, local and global weight distributions are analyzed. A global probability distribution weight sparsity method is proposed to obtain the global sparse boundary and the important features of the model. Secondly, according to the distribution of weight amplitude, the problem of uneven weight distribution is disclosed. A computational acceleration sensing method is proposed to reduce the high computational cost caused by small weight amplitude with low importance estimation. Moreover, a novel layer filter clustering pruning scheme is proposed. The intra-layer filters are classified and redundant filters are removed by using the clustering idea according to the predetermined pruning rate. Thus, the structure-sparsity filter pruning is realized. Compared with other advanced filter pruning methods, the proposed method analyzes the global and local model weight distribution and the influence of weight importance on the computational complexity of the model. The proposed method then performs filter pruning on the importance of overall filters for each network layer rather than individual filters. The accuracy of the proposed method on experimental dataset for pruned ResNet8 model under pruning rate at 0.95 is 99.12% with 3.668K parameters and 3.636MFLOPs. The experimental results elucidate that the proposed method is more suitable for deployment in resource-constrained WSNs for fault diagnosis of rotating machinery. This provides a potential solution for practical engineering applications.

A three-dimensional left atrial motion estimation from retrospective gated computed tomography: application in heart failure patients with atrial fibrillation.

A reduced left atrial (LA) strain correlates with the presence of atrial fibrillation (AF). Conventional atrial strain analysis uses two-dimensional (2D) imaging, which is, however, limited by atrial foreshortening and an underestimation of through-plane motion. Retrospective gated computed tomography (RGCT) produces high-fidelity three-dimensional (3D) images of the cardiac anatomy throughout the cardiac cycle that can be used for estimating 3D mechanics. Its feasibility for LA strain measurement, however, is understudied. The aim of this study is to develop and apply a novel workflow to estimate 3D LA motion and calculate the strain from RGCT imaging. The utility of global and regional strains to separate heart failure in patients with reduced ejection fraction (HFrEF) with and without AF is investigated. A cohort of 30 HFrEF patients with (n = 9) and without (n = 21) AF underwent RGCT prior to cardiac resynchronisation therapy. The temporal sparse free form deformation image registration method was optimised for LA feature tracking in RGCT images and used to estimate 3D LA endocardial motion. The area and fibre reservoir strains were calculated over the LA body. Universal atrial coordinates and a human atrial fibre atlas enabled the regional strain calculation and the fibre strain calculation along the local myofibre orientation, respectively. It was found that global reservoir strains were significantly reduced in the HFrEF + AF group patients compared with the HFrEF-only group patients (area strain: 11.2 ± 4.8% vs. 25.3 ± 12.6%, P = 0.001; fibre strain: 4.5 ± 2.0% vs. 15.2 ± 8.8%, P = 0.001), with HFrEF + AF patients having a greater regional reservoir strain dyssynchrony. All regional reservoir strains were reduced in the HFrEF + AF patient group, in whom the inferior wall strains exhibited the most significant differences. The global reservoir fibre strain and LA volume + posterior wall reservoir fibre strain exceeded LA volume alone and 2D global longitudinal strain (GLS) for AF classification (area-under-the-curve: global reservoir fibre strain: 0.94 ± 0.02, LA volume + posterior wall reservoir fibre strain: 0.95 ± 0.02, LA volume: 0.89 ± 0.03, 2D GLS: 0.90 ± 0.03). RGCT enables 3D LA motion estimation and strain calculation that outperforms 2D strain metrics and LA enlargement for AF classification. Differences in regional LA strain could reflect regional myocardial properties such as atrial fibrosis burden.

Discriminative sparse subspace learning with manifold regularization

Common subspace learning methods only utilize local or global structure in feature extraction, and cannot obtain the global optimal discriminative projection matrix. For this reason, this paper proposes a discriminative sparse subspace learning method based on the manifold regularization framework (DSSL-MR), which introduces the graph Laplacian matrix that reflects the intrinsic geometric structure of the sample as a penalty term. DSSL-MR simultaneously uses both sub-manifold and multi-manifold information of samples for obtaining optimal projection to enhance the discriminability of different classes in subspace. DSSL-MR uses the sparse property of the L2,1-norm to constrain the projection matrix, which can eliminate redundant features and select features that are significant for classification. It is a linear supervised method, which belongs to the Fisher discriminant analysis framework. Experimental results on multiple real-world datasets show that the algorithm is very effective in classification and has high recognition rates.

Sparse Bayesian Deep Learning for Cross Domain Medical Image Reconstruction

Cross domain medical image reconstruction aims to address the issue that deep learning models trained solely on one source dataset might not generalize effectively to unseen target datasets from different hospitals. Some recent methods achieve satisfactory reconstruction performance, but often at the expense of extensive parameters and time consumption. To strike a balance between cross domain image reconstruction quality and model computational efficiency, we propose a lightweight sparse Bayesian deep learning method. Notably, we apply a fixed-form variational Bayes (FFVB) approach to quantify pixel-wise uncertainty priors derived from degradation distribution of the source domain. Furthermore, by integrating the uncertainty prior into the posterior sampled through stochastic gradient Langevin dynamics (SGLD), we develop a training strategy that dynamically generates and optimizes the prior distribution on the network weights for each unseen domain. This strategy enhances generalizability and ensures robust reconstruction performance. When evaluated on medical image reconstruction tasks, our proposed approach demonstrates impressive performance across various previously unseen domains.

ColNeRF: Collaboration for Generalizable Sparse Input Neural Radiance Field

Neural Radiance Fields (NeRF) have demonstrated impressive potential in synthesizing novel views from dense input, however, their effectiveness is challenged when dealing with sparse input. Existing approaches that incorporate additional depth or semantic supervision can alleviate this issue to an extent. However, the process of supervision collection is not only costly but also potentially inaccurate. In our work, we introduce a novel model: the Collaborative Neural Radiance Fields (ColNeRF) designed to work with sparse input. The collaboration in ColNeRF includes the cooperation among sparse input source images and the cooperation among the output of the NeRF. Through this, we construct a novel collaborative module that aligns information from various views and meanwhile imposes self-supervised constraints to ensure multi-view consistency in both geometry and appearance. A Collaborative Cross-View Volume Integration module (CCVI) is proposed to capture complex occlusions and implicitly infer the spatial location of objects. Moreover, we introduce self-supervision of target rays projected in multiple directions to ensure geometric and color consistency in adjacent regions. Benefiting from the collaboration at the input and output ends, ColNeRF is capable of capturing richer and more generalized scene representation, thereby facilitating higher-quality results of the novel view synthesis. Our extensive experimental results demonstrate that ColNeRF outperforms state-of-the-art sparse input generalizable NeRF methods. Furthermore, our approach exhibits superiority in fine-tuning towards adapting to new scenes, achieving competitive performance compared to per-scene optimized NeRF-based methods while significantly reducing computational costs. Our code is available at: https://github.com/eezkni/ColNeRF.

Learning to Rank in Generative Retrieval

Generative retrieval stands out as a promising new paradigm in text retrieval that aims to generate identifier strings of relevant passages as the retrieval target. This generative paradigm taps into powerful generative language models, distinct from traditional sparse or dense retrieval methods. However, only learning to generate is insufficient for generative retrieval. Generative retrieval learns to generate identifiers of relevant passages as an intermediate goal and then converts predicted identifiers into the final passage rank list. The disconnect between the learning objective of autoregressive models and the desired passage ranking target leads to a learning gap. To bridge this gap, we propose a learning-to-rank framework for generative retrieval, dubbed LTRGR. LTRGR enables generative retrieval to learn to rank passages directly, optimizing the autoregressive model toward the final passage ranking target via a rank loss. This framework only requires an additional learning-to-rank training phase to enhance current generative retrieval systems and does not add any burden to the inference stage. We conducted experiments on three public benchmarks, and the results demonstrate that LTRGR achieves state-of-the-art performance among generative retrieval methods. The code and checkpoints are released at https://github.com/liyongqi67/LTRGR.

Learning Multi-Task Sparse Representation Based on Fisher Information

Multi-task learning deals with multiple related tasks simultaneously by sharing knowledge. In a typical deep multi-task learning model, all tasks use the same feature space and share the latent knowledge. If the tasks are weakly correlated or some features are negatively correlated, sharing all knowledge often leads to negative knowledge transfer among. To overcome this issue, this paper proposes a Fisher sparse multi-task learning method. It can obtain a sparse sharing representation for each task. In such a way, tasks share features on a sparse subspace. Our method can ensure that the knowledge transferred among tasks is beneficial. Specifically, we first propose a sparse deep multi-task learning model, and then introduce Fisher sparse module into traditional deep multi-task learning to learn the sparse variables of task. By alternately updating the neural network parameters and sparse variables, a sparse sharing representation can be learned for each task. In addition, in order to reduce the computational overhead, an heuristic method is used to estimate the Fisher information of neural network parameters. Experimental results show that, comparing with other methods, our proposed method can improve the performance for all tasks, and has high sparsity in multi-task learning.