Adaptive Regularization Research Articles

Hyperspectral Image Denoising Using Adaptive Weight Graph Total Variation Regularization and Low-Rank Matrix Recovery

Hyperspectral image (HSI) is often corrupted by various kinds of noises. This letter proposes an innovative HSI denoising approach by leveraging the graph signal processing (GSP) theory and the low-rank (LR) matrix recovery model. With GSP, the piecewise smoothness (PWS) property of the HSI can be efficiently characterized, leading to a new regularization for HSI denoising, termed the adaptive weight graph total variation (AWGTV) regularization. Then, the denoising problem is formulated into a constrained optimization problem that incorporates the AWGTV and the LR property of the HSI. An augmented Lagrange multiplier method is adopted to solve the problem. Numerical experiments conducted on synthetic and real-world datasets and comparisons with existing methods demonstrate the effectiveness of the proposed denoising algorithm.

The Effectiveness and Application of the Gradient Sparse Regularization-Based Deconvolution Method

Sparse constraint-based deconvolution can break through the limitation of the effective frequency band of seismic data and enable the acquisition of higher resolution data. Further, the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{0}$ </tex-math></inline-formula> -norm is the best measure of the sparsity of seismic data, and sparse deconvolution based on the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{0}$ </tex-math></inline-formula> -norm can lead to the acquisition of the sparsest solution. This method, which has the advantage of a simple solution form, has been widely used in iterative hard-threshold algorithms based on <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{0}$ </tex-math></inline-formula> -norm sparse deconvolution. However, with this method, it is difficult to select the appropriate threshold, and it has been demonstrated that selecting an inappropriate threshold significantly influences deconvolution. Therefore, to avoid the selection of an inappropriate threshold, we changed the conventional sparse regularization of the reflection coefficient to the adaptive sparse regularization of the reflection coefficient gradient. Further, via the iterative retention of the gradient of the reflection coefficient, the large gradient was gradually iterated to the small gradient of the weak signal, while avoiding the loss of weak signals and significantly improving the computational efficiency, accuracy, and adaptability of the method. Furthermore, both synthetic and field data indicated that this method is highly adaptable and possesses a good convergence speed. Additionally, the method showed high accuracy in extracting the amplitude and position of the reflection coefficient, and its antinoise performance was also good.

Diagnosis of Non-Small Cell Lung Carcinoma Via Adaptive Group Lasso Regularized Multinomial Regression

Diagnosis of Non-Small Cell Lung Carcinoma Via Adaptive Group Lasso Regularized Multinomial Regression

Regularized Tapered Sample Covariance Matrix

Covariance matrix tapers have a long history in signal processing and related fields. Examples of applications include autoregressive models (promoting a banded structure) or beamforming (widening the spectral null width associated with an interferer). In this paper, the focus is on high-dimensional setting where the dimension <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$p$</tex-math></inline-formula> is high, while the data aspect ratio <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$n/p$</tex-math></inline-formula> is low. We propose an estimator called <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Tabasco</small> (TApered or BAnded Shrinkage COvariance matrix) that shrinks the tapered sample covariance matrix towards a scaled identity matrix. We derive optimal and estimated (data adaptive) regularization parameters that are designed to minimize the mean squared error (MSE) between the proposed shrinkage estimator and the true covariance matrix. These parameters are derived under the general assumption that the data is sampled from an unspecified elliptically symmetric distribution with finite 4th order moments (both real- and complex-valued cases are addressed). Simulation studies show that the proposed <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Tabasco</small> outperforms all competing tapering covariance matrix estimators in diverse setups. An application to space-time adaptive processing (STAP) also illustrates the benefit of the proposed estimator in a practical signal processing setup.

Hyperspectral Image Denoising With Weighted Nonlocal Low-Rank Model and Adaptive Total Variation Regularization

Hyperspectral image (HSI) is always corrupted by various types of noise during image capturing, such as Gaussian noise, stripe noise, deadline noise, impulse noise, and more. Such complicated noise significantly degrades imaging quality and thus limits the performance of downstream vision tasks. Current HSI denoising methods tackle this problem by modeling either the spectral-spatial prior of HSI or the noise characteristic of HSI, and few work consider the two aspects simultaneously. In this paper, we propose a new HSI denoising method by simultaneously modeling the HSI prior and the HSI noise characteristic. Specifically, we firstly utilize the non independent and identically distributed (non i.i.d.) mixture of Gaussian (MoG) assumption to characterize the complex noise, which corresponds to optimize a weighted fidelity function. Secondly, we exploit HSI’s non-local similarity and spatial-spectral correlation priors by applying non-local low rank model. Thirdly, we design an adaptive edge preserving total variation regularization term to characterize the non-local smooth property of HSI. Finally, we propose a new denoising model and develop effective ADMM algorithm to solve it. Extensive experiments on simulated data and real data substantiate the superiority of the proposed method beyond state-of-the-arts.

Adaptive Regularization for Inverse Problems in Imaging

Adaptive Regularization for Inverse Problems in Imaging

Hyperspectral Image Classification Based on Domain Adversarial Broad Adaptation Network

For hyperspectral image (HSI) classification tasks, obtaining sufficient labeled samples is usually difficult, time-consuming, and expensive. To address the aforementioned issue, by transferring the labeled sample information of a relevant source domain to the unlabeled target domain, an HSI classification method based on the domain adversarial broad adaptation network (DABAN) is proposed. First, the bottleneck adaptation module composed of a bottleneck layer and a domain adaptation layer is constructed and introduced to the domain adversarial neural network; thus, the domain adversarial adaptation network (DAAN) is designed. By simultaneously performing domain adversarial learning, reducing both the marginal distribution difference and second-order statistic difference between two domains, the distributions of the source and target domains are aligned. Then, the conditional distribution adaptation regularization term based on the maximum mean discrepancy is embedded into a broad learning system to obtain the conditional adaptation broad network (CABN). On the one hand, CABN can perform the class-level distribution adaptation on the domain-invariant features extracted by DAAN. On the other hand, the representation ability of the domain-invariant features expanded by CABN can be further enhanced. Experimental results on ten real hyperspectral data pairs show that, compared with the existing mainstream methods, DABAN can effectively utilize relevant source-domain information to assist in improving the classification accuracy of the target domain.

Nonuniform B-Spline Lung Image Registration Based on Adaptive Regularization Term

Nonuniform B-Spline Lung Image Registration Based on Adaptive Regularization Term

Continuous Viewpoint Planning in Conjunction with Dynamic Exploration for Active Object Recognition.

Active object recognition (AOR) aims at collecting additional information to improve recognition performance by purposefully adjusting the viewpoint of an agent. How to determine the next best viewpoint of the agent, i.e., viewpoint planning (VP), is a research focus. Most existing VP methods perform viewpoint exploration in the discrete viewpoint space, which have to sample viewpoint space and may bring in significant quantization error. To address this challenge, a continuous VP approach for AOR based on reinforcement learning is proposed. Specifically, we use two separate neural networks to model the VP policy as a parameterized Gaussian distribution and resort the proximal policy optimization framework to learn the policy. Furthermore, an adaptive entropy regularization based dynamic exploration scheme is presented to automatically adjust the viewpoint exploration ability in the learning process. To the end, experimental results on the public dataset GERMS well demonstrate the superiority of our proposed VP method.

An intelligent fault diagnosis method for rolling bearings based on feature transfer with improved DenseNet and joint distribution adaptation

Mechanical intelligent fault diagnosis is an important method to accurately identify the health status of mechanical equipment. Traditional fault diagnosis methods perform poorly in the diagnosis of rolling bearings under complex conditions. In this paper, a feature transfer learning model based on improved DenseNet and joint distribution adaptation (FT-IDJ) is proposed. With this model, we apply it to implement rolling bearing fault diagnosis. A lightweight DenseNet model is firstly proposed to extract the transferable features of the raw vibration signal. Furthermore, the parameters in the DenseNet are constrained by the domain adaptive regularization term and pseudo label learning. The marginal distribution discrepancy and the conditional distribution discrepancy of the learned transferable features are reduced by this way. The proposed method is validated by the diagnosis experiments with CWRU and Jiangnan University rolling bearing datasets. The experimental results showed that the proposed FT-IDJ can classify common rolling bearing faults such as inner ring fault, outer ring fault and rolling element fault. Also, it has higher classification accuracy than DAN and other eight methods, which demonstrates its effectively learning transferable features from auxiliary data.

Sorted L1 regularization method for damage detection based on electrical impedance tomography.

Carbon fiber reinforced polymers (CFRPs) are composite materials in which carbon provides strength and stiffness, whereas polymers provide cohesiveness and toughness. The electrical impedance of CFRP laminates is changed due to different kinds of damages. Electrical impedance tomography (EIT) has significant advantages such as non-intrusion, portability, low cost, and quick response and has widely been used as a nondestructive testing method. Therefore, EIT has great potential in structural health monitoring of CFRPs. Regularization can solve the ill-posed inverse problem of EIT. However, conventional regularization algorithms have their own limitations, such as over-smoothness of reconstructed edges and unstable solution caused by measurement noise. In addition, the anisotropic property of CFRPs also affects the image quality based on traditional methods. In this paper, the sorted L1-norm regularization is proposed. It promotes grouping highly correlated variables while encouraging sparsity by using more effective penalty terms. The sharp edges between different materials can be obtained, and the obtained solution is more stable. The image quality of different objects, especially the image quality of multi-targets, can be significantly improved with this new method. In addition, the sorted L1 norm can generate adaptive regularization parameters without empirical selection. The new regularization problem is solved by the alternating direction method of multipliers. Both experimental and simulation results demonstrate that the sorted L1 norm improves the quality of reconstructed images under various noise levels. The proposed method is comprehensively evaluated with three image quality criteria by numerical simulation quantitatively.

A multiparametric volumetric quantitative ultrasound imaging technique for soft tissue characterization.

Quantitative ultrasound (QUS) offers a non-invasive and objective way to quantify tissue health. We recently presented a spatially adaptive regularization method for reconstruction of a single QUS parameter, limited to a two dimensional region. That proof-of-concept study showed that regularization using homogeneity prior improves the fundamental precision-resolution trade-off in QUS estimation. Based on the weighted regularization scheme, we now present a multiparametric 3D weighted QUS (3D QUS) method, involving the reconstruction of three QUS parameters: attenuation coefficient estimate (ACE), integrated backscatter coefficient (IBC) and effective scatterer diameter (ESD). With the phantom studies, we demonstrate that our proposed method accurately reconstructs QUS parameters, resulting in high reconstruction contrast and therefore improved diagnostic utility. Additionally, the proposed method offers the ability to analyze the spatial distribution of QUS parameters in 3D, which allows for superior tissue characterization. We apply a three-dimensional total variation regularization method for the volumetric QUS reconstruction. The 3D regularization involving N planes results in a high QUS estimation precision, with an improvement of standard deviation over the theoretical 1/N rate achievable by compounding N independent realizations. In the in vivo liver study, we demonstrate the advantage of adopting a multiparametric approach over the single parametric counterpart, where a simple quadratic discriminant classifier using feature combination of three QUS parameters was able to attain a perfect classification performance to distinguish between normal and fatty liver cases.

Disentangling disease heterogeneity from neuroimaging data via adaptive distribution modeling–based collaborative clustering

AbstractBackgroundNeurodegenerative diseases, including Parkinson’s disease (PD), are clinically heterogeneous, with distinct subtypes encompassed within the same clinical diagnosis. Heterogeneity in brain diseases makes it challenging to recruit ideal patients into trials for suitable treatment or develop markers that capture the disease’s signals. Data‐driven clustering coupled with high dimensional neuroimaging data could further our understanding of heterogeneous disease biology and assigning specific patients groups to individualized treatments.MethodWe propose an Adaptive Distribution based COllaborative Clustering (ADCoC) method, which simultaneously clusters subjects and features via nonnegative matrix tri‐factorization. For denoising, we further introduce adaptive regularization based on coefficient distribution modeling. Unlike related sparsity techniques [1], we form the distributions using the data of interest to better fit the coefficients. We studied structural MRI data from 170 PD patients and 77 healthy controls enrolled in the Parkinson Progression Markers Initiative (PPMI). ADCoC partitioned the patients into 2 clusters. We compared the distribution of ICV normalized grey matter volumes in different regions between the two patient clusters and the healthy control group by student’s t‐test. Then we analyzed neuropsychological test data to compare the cognitive performance of the two patient clusters and the control group.ResultTest statistic values of regions with FDR corrected p‐values smaller than 0.05 are plotted in Figure 1. Comparisons with the healthy control group show that PD cluster PD‐Post (54.7% of the PD samples) is characterized by a posterior cortical‐medial temporal atrophy pattern with worse cognition compared to PD cluster PD‐Front which had atrophy circumscribed to the frontal lobe areas. Analysis of neuropsychological test data further demonstrates that these MRI‐defined patient subtypes also show differences in clinical presentation, where PD‐Post show statistically significant lower cognitive performance compared to PD‐Front (Montreal Cognitive Assessment [MoCA] scores: 27.1 ± 2.4 [PD‐Post] vs 27.9 ± 1.7 [PD‐Front], p = 0.014).ConclusionWhen applied to a clinical dataset of MRI data from PD patients, ADCoC identified two stable and highly reproducible patient clusters characterized by frontal and posterior cortical‐medial temporal atrophy patterns.

A Novel Anti-Drift Visual Object Tracking Algorithm Based on Sparse Response and Adaptive Spatial-Temporal Context-Aware

Correlation filter (CF) based trackers have gained significant attention in the field of visual single-object tracking, owing to their favorable performance and high efficiency; however, existing trackers still suffer from model drift caused by boundary effects and filter degradation. In visual tracking, long-term occlusion and large appearance variations easily cause model degradation. To remedy these drawbacks, we propose a sparse adaptive spatial-temporal context-aware method that effectively avoids model drift. Specifically, a global context is explicitly incorporated into the correlation filter to mitigate boundary effects. Subsequently, an adaptive temporal regularization constraint is adopted in the filter training stage to avoid model degradation. Meanwhile, a sparse response constraint is introduced to reduce the risk of further model drift. Furthermore, we apply the alternating direction multiplier method (ADMM) to derive a closed-solution of the object function with a low computational cost. In addition, an updating scheme based on the APCE-pool and Peak-pool is proposed to reveal the tracking condition and ensure updates of the target’s appearance model with high-confidence. The Kalam filter is adopted to track the target when the appearance model is persistently unreliable and abnormality occurs. Finally, extensive experimental results on OTB-2013, OTB-2015 and VOT2018 datasets show that our proposed tracker performs favorably against several state-of-the-art trackers.

Bayesian Regularized Strain Imaging for Assessment of Murine Cardiac Function In vivo.

A cardiac strain imaging framework with adaptive Bayesian regularization (ABR) is proposed for in vivo assessment of murine cardiac function. The framework uses ultrasound (US) radio-frequency data collected with a high frequency (fc = 30MHz) imaging system and a multi-level block matching algorithm with ABR to derive inter-frame cardiac displacements. Lagrangian cardiac strain (radial, er and longitudinal, el) tensors were derived by segmenting the myocardial wall starting at the ECG R-wave and accumulating interframe deformations over a cardiac cycle. In vivo feasibility was investigated through a longitudinal study with two mice (one ischemia-perfusion (IR) injury and one sham) imaged at five sessions (pre-surgery (BL) and 1,2,7 and 14 days post-surgery). End-systole (ES) strain images and segmental strain curves were derived for quantitative evaluation. Both mice showed periodic variation of er and el strain at BL with segmental synchroneity. Infarcted regions of IR mouse at Day 14 were associated with reduced or sign reversed ES er and el values while the sham mouse had similar or higher strain than at BL. Infarcted regions identified in vivo were associated with increased collagen content confirmed with Masson's Trichrome stained ex vivo heart sections.Clinical Relevance-Higher quality cardiac strain images derived with RF data and Bayesian regularization can potentially improve the sensitivity and accuracy of non-invasive assessment of cardiovascular disease models.

Activity recognition from Smartphone data using WSVM-HMM classification

A lot of real-life mobile sensing applications are becoming available nowadays. The traditional approach for activity recognition employs machine learning algorithms to learn from collected data from smartphpne and induce a model. The model generation is usually performed offline on a server system and later deployed to the phone for activity recognition. In this paper, we propose a new hybrid classification model to perform automatic recognition of activities using built-in embedded sensors present in smartphones. The proposed method uses a trick to classify the ongoing activity by combining Weighted Support Vector Machines (WSVM) model and Hidden Markov Model (HMM) model. The sensory inputs to the classifier are reduced with the Linear Discriminant Analysis (LDA). We demonstrate how to train the hybrid approach in this setting, introduce an adaptive regularization parameter for WSVM approach, and illustrate how our proposed method outperforms the state-of-the-art on a large benchmark dataset.

Projected Adaptive Cubic Regularization Algorithm with Derivative-Free Filter Technique for Box Constrained Optimization

An adaptive projected affine scaling algorithm of cubic regularization method using a filter technique for solving box constrained optimization without derivatives is put forward in the passage. The affine scaling interior-point cubic model is based on the quadratic probabilistic interpolation approach on the objective function. The new iterations are obtained by the solutions of the projected adaptive cubic regularization algorithm with filter technique. We prove the convergence of the proposed algorithm under some assumptions. Finally, experiments results showed that the presented algorithm is effective in detail.

Importance-Penalized Joint Graphical Lasso (IPJGL): differential network inference via GGMs.

MotivationDifferential network inference is a fundamental and challenging problem to reveal gene interactions and regulation relationships under different conditions. Many algorithms have been developed for this problem; however, they do not consider the differences between the importance of genes, which may not fit the real-world situation. Different genes have different mutation probabilities, and the vital genes associated with basic life activities have less fault tolerance to mutation. Equally treating all genes may bias the results of differential network inference. Thus, it is necessary to consider the importance of genes in the models of differential network inference.ResultsBased on the Gaussian graphical model with adaptive gene importance regularization, we develop a novel Importance-Penalized Joint Graphical Lasso method (IPJGL) for differential network inference. The presented method is validated by the simulation experiments as well as the real datasets. Furthermore, to precisely evaluate the results of differential network inference, we propose a new metric named APC2 for the differential levels of gene pairs. We apply IPJGL to analyze the TCGA colorectal and breast cancer datasets and find some candidate cancer genes with significant survival analysis results, including SOST for colorectal cancer and RBBP8 for breast cancer. We also conduct further analysis based on the interactions in the Reactome database and confirm the utility of our method.Availability and implementationR source code of Importance-Penalized Joint Graphical Lasso is freely available at https://github.com/Wu-Lab/IPJGL.Supplementary information Supplementary data are available at Bioinformatics online.

Flexible Krylov Methods for Edge Enhancement in Imaging.

Many successful variational regularization methods employed to solve linear inverse problems in imaging applications (such as image deblurring, image inpainting, and computed tomography) aim at enhancing edges in the solution, and often involve non-smooth regularization terms (e.g., total variation). Such regularization methods can be treated as iteratively reweighted least squares problems (IRLS), which are usually solved by the repeated application of a Krylov projection method. This approach gives rise to an inner–outer iterative scheme where the outer iterations update the weights and the inner iterations solve a least squares problem with fixed weights. Recently, flexible or generalized Krylov solvers, which avoid inner–outer iterations by incorporating iteration-dependent weights within a single approximation subspace for the solution, have been devised to efficiently handle IRLS problems. Indeed, substantial computational savings are generally possible by avoiding the repeated application of a traditional Krylov solver. This paper aims to extend the available flexible Krylov algorithms in order to handle a variety of edge-enhancing regularization terms, with computationally convenient adaptive regularization parameter choice. In order to tackle both square and rectangular linear systems, flexible Krylov methods based on the so-called flexible Golub–Kahan decomposition are considered. Some theoretical results are presented (including a convergence proof) and numerical comparisons with other edge-enhancing solvers show that the new methods compute solutions of similar or better quality, with increased speedup.

A soft sensor model of Pichia pastoris cell concentration based on IBDA-RELM

For Pichia pastoris fermentation process with multi-operating conditions, it is difficult to predict the cell concentration under the new operating conditions by the soft sensor model established under the specific operating conditions. Inspired by the idea of transfer learning, a method based on an improved balanced distribution adaptive regularization extreme learning machine (IBDA-RELM) was proposed to solve the problem. The domain adaptation (DA) method in transfer learning is developed to reduce distribution distance by transforming data. However, the joint distribution adaptation (JDA) and the balanced distribution adaptation (BDA) in DA cannot be directly applied to regression problems. The fuzzy sets (FSs) method was proposed to solve this issue. Finally, a soft sensor model of Pichia pastoris cell concentration was realized by inputting the converted data to the RELM model. Simulation verification was carried out with three operating conditions at the scene of fermentation. The transfer effects of three DA methods, including transfer component analysis (TCA), improved joint distribution adaptation (IJDA) as well as IBDA, were compared. The predicted results show that IBDA-RELM had a better performance in the soft sensor of Pichia pastoris cell concentration under multi-operating conditions.