Adaptive Regularization Research Articles

Tailored Channel Pruning: Achieve Targeted Model Complexity through Adaptive Sparsity Regularization

Tailored Channel Pruning: Achieve Targeted Model Complexity through Adaptive Sparsity Regularization

Robust blind Q-compensated denoising for seismic data with adaptive structural regularization

Robust blind Q-compensated denoising for seismic data with adaptive structural regularization

A Low-Cost 3D Mapping System for Indoor Scenes Based on 2D LiDAR and Monocular Cameras

The cost of indoor mapping methods based on three-dimensional (3D) LiDAR can be relatively high, and they lack environmental color information, thereby limiting their application scenarios. This study presents an innovative, low-cost, omnidirectional 3D color LiDAR mapping system for indoor environments. The system consists of two two-dimensional (2D) LiDARs, six monocular cameras, and a servo motor. The point clouds are fused with imagery using a pixel-spatial dual-constrained depth gradient adaptive regularization (PS-DGAR) algorithm to produce dense 3D color point clouds. During fusion, the point cloud is reconstructed inversely based on the predicted pixel depth values, compensating for areas of sparse spatial features. For indoor scene reconstruction, a globally consistent alignment algorithm based on particle filter and iterative closest point (PF-ICP) is proposed, which incorporates adjacent frame registration and global pose optimization to reduce mapping errors. Experimental results demonstrate that the proposed density enhancement method achieves an average error of 1.5 cm, significantly improving the density and geometric integrity of sparse point clouds. The registration algorithm achieves a root mean square error (RMSE) of 0.0217 and a runtime of less than 4 s, both of which outperform traditional iterative closest point (ICP) variants. Furthermore, the proposed low-cost omnidirectional 3D color LiDAR mapping system demonstrates superior measurement accuracy in indoor environments.

Blind Infrared Remote-Sensing Image Deblurring Algorithm via Edge Composite-Gradient Feature Prior and Detail Maintenance

The problem of blind image deblurring remains a challenging inverse problem, due to the ill-posed nature of estimating unknown blur kernels and latent images within the Maximum A Posteriori (MAP) framework. To address this challenge, traditional methods often rely on sparse regularization priors to mitigate the uncertainty inherent in the problem. In this paper, we propose a novel blind deblurring model based on the MAP framework that leverages Composite-Gradient Feature (CGF) variations in edge regions after image blurring. This prior term is specifically designed to exploit the high sparsity of sharp edge regions in clear images, thereby effectively alleviating the ill-posedness of the problem. Unlike existing methods that focus on local gradient information, our approach focuses on the aggregation of edge regions, enabling better detection of both sharp and smoothed edges in blurred images. In the blur kernel estimation process, we enhance the accuracy of the kernel by assigning effective edge information from the blurred image to the smoothed intermediate latent image, preserving critical structural details lost during the blurring process. To further improve the edge-preserving restoration, we introduce an adaptive regularizer that outperforms traditional total variation regularization by better maintaining edge integrity in both clear and blurred images. The proposed variational model is efficiently implemented using alternating iterative techniques. Extensive numerical experiments and comparisons with state-of-the-art methods demonstrate the superior performance of our approach, highlighting its effectiveness and real-world applicability in diverse image-restoration tasks.

Complexity of linearized quadratic penalty for optimization with nonlinear equality constraints

AbstractIn this paper we consider a nonconvex optimization problem with nonlinear equality constraints. We assume that both, the objective function and the functional constraints, are locally smooth. For solving this problem, we propose a linearized quadratic penalty method, i.e., we linearize the objective function and the functional constraints in the penalty formulation at the current iterate and add a quadratic regularization, thus yielding a subproblem that is easy to solve, and whose solution is the next iterate. Under a new adaptive regularization parameter choice, we provide convergence guarantees for the iterates of this method to an $$\epsilon $$ ϵ first-order optimal solution in $${\mathcal {O}}({\epsilon ^{-2.5}})$$ O ( ϵ - 2.5 ) iterations. Finally, we show that when the problem data satisfy Kurdyka–Lojasiewicz property, e.g., are semialgebraic, the whole sequence generated by the proposed algorithm converges and we derive improved local convergence rates depending on the KL parameter. We validate the theory and the performance of the proposed algorithm by numerically comparing it with some existing methods from the literature.

DebSDF: Delving Into the Details and Bias of Neural Indoor Scene Reconstruction.

In recent years, the neural implicit surface has emerged as a powerful representation for multi-view surface reconstruction due to its simplicity and State-of-the-Art performance. However, reconstructing smooth and detailed surfaces in indoor scenes from multi-view images presents unique challenges. Indoor scenes typically contain large texture-less regions, making the photometric loss unreliable for optimizing the implicit surface. Previous work utilizes monocular geometry priors to improve the reconstruction in indoor scenes. However, monocular priors often contain substantial errors in thin structure regions due to domain gaps and the inherent inconsistencies when derived independently from different views. This paper presents DebSDF to address these challenges, focusing on the utilization of uncertainty in monocular priors and the bias in SDF-based volume rendering. We propose an uncertainty modeling technique that associates larger uncertainties with larger errors in the monocular priors. High-uncertainty priors are then excluded from optimization to prevent bias. This uncertainty measure also informs an importance-guided ray sampling and adaptive smoothness regularization, enhancing the learning of fine structures. We further introduce a bias-aware signed distance function to density transformation that takes into account the curvature and the angle between the view direction and the SDF normals to reconstruct fine details better. Our approach has been validated through extensive experiments on several challenging datasets, demonstrating improved qualitative and quantitative results in reconstructing thin structures in indoor scenes, thereby outperforming previous work.

DcnnGrasp: towards accurate grasp pattern recognition with adaptive regularizer learning

DcnnGrasp: towards accurate grasp pattern recognition with adaptive regularizer learning

3D Investigation of Pt-Based Nanocatalyst Durability on High Surface Area Carbon by Automated Electron Tomography

Proton-exchange membrane fuel cells (PEMFCs) hold potential for hydrogen-powered heavy-duty vehicles, offering high efficiencies and comparable driving ranges and refueling times to internal combustion engines. Yet, durability challenges persist, notably in cathode catalyst degradation [1]. Understanding degradation mechanisms during accelerated stress tests (AST) is crucial. Electron tomography aids in high-resolution analysis of Pt-based nanoparticle distribution, revealing degradation rates relative to their location on or in the carbon support, and providing a possible explanation to the enhanced particles accessibility at low relative humidity after AST [2,3].In this work, we developed an automated electron tomography workflow performed in the scanning transmission electron microscopy (STEM) utilizing SerialEM for data acquisition[4]. Data were reconstructed using a model-based iterative reconstruction with adaptive regularization (MBIR-ARAR) [5], and the segmentation of the nanoparticles and carbon was performed using the commercial software Avizo.We examined Pt catalyst particle size distributions at both the beginning (BOT) and end (EOT) of the test (see Figure 1). Our results revealed a noticeable increase in particle size alongside a reduction in specific surface area (SSA), regardless of nanoparticle location. Moreover, a significant portion of particles remained within the carbon pores at EOT. In addition, there was evidence of an expanded size distribution of pores post AST, potentially due to minor carbon corrosion. These observed degradation mechanisms challenge the assumption that Pt dissolution from the interior of the carbon support and re-deposition on the surface is a predominant factor for the enhanced accessibility after AST [6] but suggests that other factors such as carbon corrosion causing pore opening and increased ionomer intrusion. Additionally, we will discuss the potential of employing low-dose cryo-tomography to mitigate damage to the catalyst, support, and particularly beam sensitive proton conducting ionomer [7,8].

Complexity analysis of inexact cubic-regularized primal-dual methods for finding second-order stationary points

Motivated by recent developments of using cubic regularization to escape saddle points of unconstrained optimization, in this paper we explore its potential in pursuing second-order stationary points of nonconvex constrained optimization whose exact objective function information may be hard to obtain. We first propose an algorithmic framework of inexact cubic-regularized primal-dual methods for equality constrained optimization, named as ICPD. To update the primal variable at each iteration, we construct a cubic regularized model relying on inexact first- and second-order derivatives of the objective function together with information of constraint functions. By allowing an inexact solution to each subproblem under certain conditions, we establish the iteration complexity of ICPD to find an ϵ \epsilon -approximate first- and second-order stationary point, respectively. We then consider a stochastic variant of algorithm, stochastic cubic-regularized primal-dual algorithm (SCPD) for equality constrained optimization whose objective takes an expectation form. Through a proper sampling strategy to calculate stochastic gradients and Hessians, we address the oracle complexity of SCPD to reach approximate stationary points with high probability. We also investigate the behavior of the standard gradient descent when solving each subproblem with a random perturbation. We provide a detailed analysis on how to fulfill the required conditions on an inexact subproblem solution with high probability at each iteration. Additionally, we present an analysis of an adaptive variant of ICPD which updates penalty parameters dynamically and discuss the applicability of adaptive cubic regularization parameters. Finally, preliminary numerical results are reported to showcase the performances of our proposed algorithms.

Flexible Krylov methods for group sparsity regularization

Abstract This paper introduces new solvers for efficiently computing solutions to large-scale inverse problems with group sparsity regularization, including both non-overlapping and overlapping groups. Group sparsity regularization refers to a type of structured sparsity regularization, where the goal is to impose additional structure in the regularization process by assigning variables to predefined groups that may represent graph or network structures. Special cases of group sparsity regularization include ℓ 1 and isotropic total variation regularization. In this work, we develop hybrid projection methods based on flexible Krylov subspaces, where we first recast the group sparsity regularization term as a sequence of 2-norm penalization terms using adaptive regularization matrices in an iterative reweighted norm fashion. Then we exploit flexible preconditioning techniques to efficiently incorporate the weight updates. The main advantages of these methods are that they are computationally efficient (leveraging the advantages of flexible methods), they are general (and therefore very easily adaptable to new regularization term choices), and they are able to select the regularization parameters automatically and adaptively (exploiting the advantages of hybrid methods). Extensions to multiple regularization terms and solution decomposition frameworks (e.g. for anomaly detection) are described, and a variety of numerical examples demonstrate both the efficiency and accuracy of the proposed approaches compared to existing solvers.

Spatially adaptive oscillation total generalized variation for image restoration with structured textures

Cartoon and texture are the main components of an image, and decomposing them has gained much attention in various image restoration tasks. In this paper, we propose a novel infimal convolution type model based on total generalized variation (TGV) and spatially adaptive oscillation TGV to address cartoon-texture restoration problems. The proposed spatially adaptive oscillation TGV regularizer is capable of capturing structured textures with different orientations and frequencies in localized regions. Additionally, we incorporate the second-order tensor TGV to regularize the orientation and frequency parameter. The lower semicontinuity of the new functional is established and the existence of the solutions to the proposed model is analyzed. Furthermore, we discuss suitable discretizations of the proposed model, and introduce the alternating minimization algorithm where each subproblem can be implemented by the primal-dual method. Numerical experiments on image decomposition, denoising and inpainting demonstrate that the proposed model excels in preserving textures and is competitive against existing variational and learning-based models.

Online tree-structure-constrained RPCA for background subtraction of X-ray coronary angiography images

Background and objectiveBackground subtraction of X-ray coronary angiograms (XCA) can significantly improve the diagnosis and treatment of coronary vessel diseases. The XCA background is complex and dynamic due to structures with different intensities and independent motion patterns, making XCA background subtraction challenging. MethodsThe current work proposes an online tree-structure-constrained robust PCA (OTS-RPCA) method to subtract the XCA background. A morphological closing operation is used as a pre-processing step to remove large-scale structures like the spine, chest and diaphragm. In the following, the XCA sequence is decomposed into three different subspaces: low-rank background, residual dynamic background and vascular foreground. A tree-structured norm is introduced and applied to the vascular submatrix to guarantee the vessel spatial coherency. Moreover, the residual dynamic background is separately extracted to remove noise and motion artifacts from the vascular foreground. The proposed algorithm also employs an adaptive regularization coefficient that tracks the vessel area changes in the XCA frames. ResultsThe proposed method is evaluated on two datasets of real clinical and synthetic low-contrast XCA sequences of 38 patients using the global and local contrast-to-noise ratio (CNR) and structural similarity index (SSIM) criteria. For the real XCA dataset, the average values of global CNR, local CNR and SSIM are 6.27, 3.07 and 0.97, while these values over the synthetic low-contrast dataset are obtained as 5.15, 2.69 and 0.94, respectively. The implemented quantitative and qualitative experiments verify the superiority of the proposed method over seven selected state-of-the-art methods in increasing the coronary vessel contrast and preserving the vessel structure. ConclusionsThe proposed OTS-RPCA background subtraction method accurately subtracts backgrounds from XCA images. Our method might provide the basis for reducing the contrast agent dose and the number of needed injections in coronary interventions.

Early screening of miliary tuberculosis with tuberculous meningitis based on few-shot learning with multiple windows and feature granularities

Tuberculous meningitis (TBM) is a fatal tuberculosis caused by a large number of Mycobacterium tuberculosis (M. tuberculosis) spread by blood flow, with a case fatality rate of more than 50%. It is one of the most serious complications of miliary tuberculosis (MT), whose incidence is closely related to MT. If doctors can provide early diagnosis and active treatment for TBM, the case fatality rate will be significantly reduced. At present, there is a lack of methods to predict the progression of MT to TBM in clinic. To explore whether MT cases will experience TBM, we propose an early screening model of miliary tuberculosis with tuberculous meningitis (MT-TBM) based on few-shot learning with multiple windows and feature granularities (MWFG). This model aims to screen potential TBM cases through chest computerized tomography (CT) images of MT cases. Chest CT is a routine examination for MT cases. The MWFG module can extract more comprehensive features from a set of CT images of each MT case. The softmax classifier with adaptive regularization is trained on the cooperation of support set and query set, which can effectively prevent overfitting. Experiments on a dataset of 40 MT cases with chest CT images established by the medical records demonstrate that our proposed model achieves state-of-the-art performance in the early screening of MT-TBM. It can establish the connection between MT and MT-TBM through chest CT images of MT cases. The early screening model of MT-TBM based on few-shot learning with MWFG fills the research gap in computer-aided predicting TBM and has certain clinical effects. This research can provide some reference for clinicians in early diagnosis of MT-TBM and help clinicians in the early prevention and treatment of TBM for MT patients.

Adaptive Attention-Driven Manifold Regularization for Deep Learning Networks: Industrial Predictive Modeling Applications and Beyond

Adaptive Attention-Driven Manifold Regularization for Deep Learning Networks: Industrial Predictive Modeling Applications and Beyond

Adaptive regularization for the Richards equation

Adaptive regularization for the Richards equation

Fast convergence to non-isolated minima: four equivalent conditions for $${\textrm{C}^{2}}$$ functions

AbstractOptimization algorithms can see their local convergence rates deteriorate when the Hessian at the optimum is singular. These singularities are inescapable when the optima are non-isolated. Yet, under the right circumstances, several algorithms preserve their favorable rates even when optima form a continuum (e.g., due to over-parameterization). This has been explained under various structural assumptions, including the Polyak–Łojasiewicz condition, Quadratic Growth and the Error Bound. We show that, for cost functions which are twice continuously differentiable ($$\textrm{C}^2$$ C 2 ), those three (local) properties are equivalent. Moreover, we show they are equivalent to the Morse–Bott property, that is, local minima form differentiable submanifolds, and the Hessian of the cost function is positive definite along its normal directions. We leverage this insight to improve local convergence guarantees for safe-guarded Newton-type methods under any (hence all) of the above assumptions. First, for adaptive cubic regularization, we secure quadratic convergence even with inexact subproblem solvers. Second, for trust-region methods, we argue capture can fail with an exact subproblem solver, then proceed to show linear convergence with an inexact one (Cauchy steps).

Normalizing flow regularization for photoacoustic tomography

Abstract Proper regularization is crucial in inverse problems to achieve high-quality reconstruction, even with an ill-conditioned measurement system. This is particularly true for three-dimensional photoacoustic tomography, which is computationally demanding and requires rapid scanning, often leading to incomplete measurements. Deep neural networks, known for their efficiency in handling big data, are anticipated to be adept at extracting underlying information from images sharing certain characteristics, such as specific types of natural or medical images. We introduce a \ac{NFR} method designed to reconstruct images from incomplete and noisy measurements. The method involves training a normalizing flow network to understand the statistical distribution of sample images by mapping them to Gaussian distributions. This well-trained network then acts as a regularization tool within a Bayesian inversion framework. Additionally, we explore the concept of adaptive regularization selection, providing theoretical proof of its admissibility. A significant challenge in three-dimensional image training is the extensive memory and computation requirements. We address this by training the normalizing flow model using only small-size images and applying a patch-based model for reconstructing larger images. Our approach is model-independent, allowing the reuse of a well-trained network as regularization for various imaging systems. Moreover, as a data-driven prior, \ac{NFR} effectively leverages the available dataset information, outperforming artificial priors. This advantage is demonstrated through numerical simulations of three-dimensional photoacoustic tomography under various conditions of sparsity, noise levels, and limited-view scenarios.

Joint consensus kernel learning and adaptive hypergraph regularization for graph-based clustering

Recent advancements in multiple kernel learning-based graph clustering methods have demonstrated significant promise in effectively learning a consensus kernel matrix from various candidate kernel matrices. This approach enables the creation of a low-dimensional representation in the kernel space of high-dimensional data through self-expressiveness. However, a key challenge remains in capturing the latent geometric properties embedded within different kernel matrices to enhance data representation. In this paper, we propose a novel method called joint consensus kernel learning and adaptive hypergraph regularization for graph-based clustering (JKHR). Our approach integrates an innovative adaptive hypergraph Laplacian regularizer, which is characterized by the fusion of multiple nearest neighbor kernel graphs, into the multiple kernel learning-based graph clustering framework. JKHR jointly and adaptively optimizes both the consensus kernel matrix and the hypergraph Laplacian regularizer to achieve a low-dimensional representation that effectively preserves the intrinsic geometry of the data. Experimental results on both synthetic and real benchmark datasets demonstrate that JKHR outperforms state-of-the-art self-expressiveness-based graph clustering methods as well as traditional clustering techniques.

Pyramid Convolutional Recurrent Network for Serial Medical Image Registration With Adaptive Motion Regularizations

Pyramid Convolutional Recurrent Network for Serial Medical Image Registration With Adaptive Motion Regularizations

Claim Prediction and Premium Pricing for Telematics Auto Insurance Data Using Poisson Regression with Lasso Regularisation

We leverage telematics data on driving behavior variables to assess driver risk and predict future insurance claims in a case study utilising a representative telematics sample. In the study, we aim to categorise drivers according to their driving habits and establish premiums that accurately reflect their driving risk. To accomplish our goal, we employ the two-stage Poisson model, the Poisson mixture model, and the Zero-Inflated Poisson model to analyse the telematics data. These models are further enhanced by incorporating regularisation techniques such as lasso, adaptive lasso, elastic net, and adaptive elastic net. Our empirical findings demonstrate that the Poisson mixture model with the adaptive lasso regularisation outperforms other models. Based on predicted claim frequencies and drivers’ risk groups, we introduce a novel usage-based experience rating premium pricing method. This method enables more frequent premium updates based on recent driving behaviour, providing instant rewards and incentivising responsible driving practices. Consequently, it helps to alleviate cross-subsidization among risky drivers and improves the accuracy of loss reserving for auto insurance companies.