Adaptive Regularization Parameter Research Articles

Learning the Hybrid Nonlocal Self-Similarity Prior for Image Restoration

As an immensely important characteristic of natural images, the nonlocal self-similarity (NSS) prior has demonstrated great promise in a variety of inverse problems. Unfortunately, most current methods utilize either the internal or the external NSS prior learned from the degraded image or training images. The former is inevitably disturbed by degradation, while the latter is not adapted to the image to be restored. To mitigate such problems, this work proposes to learn a hybrid NSS prior from both internal images and external training images and employs it in image restoration tasks. To achieve our aims, we first learn internal and external NSS priors from the measured image and high-quality image sets, respectively. Then, with the learned priors, an efficient method, involving only singular value decomposition (SVD) and a simple weighting method, is developed to learn the HNSS prior for patch groups. Subsequently, taking the learned HNSS prior as the dictionary, we formulate a structural sparse representation model with adaptive regularization parameters called HNSS-SSR for image restoration, and a general and efficient image restoration algorithm is developed via an alternating minimization strategy. The experimental results indicate that the proposed HNSS-SSR-based restoration method exceeds many existing competition algorithms in PSNR and SSIM values.

An adaptive fractional-order regularization primal-dual image denoising algorithm based on non-convex function

In this paper, a novel non-convex fractional-order image denoising model is proposed to suppress the staircase effect produced by the TV model while maintaining a neat contour. The model combines ℓq(0<q<1) quasi-norm and fractional-order regularization, and employs a diffusion coefficient with a faster convergence rate to preserve more image edges and details. Additionally, an adaptive regularization parameter is designed to adjust the denoising performance of the algorithm. To obtain the optimal approximate solution of the model, an enhanced primal-dual algorithm is adopted and the complexity and convergence of the algorithm are theoretically analyzed. Finally, the effectiveness of the proposed method is demonstrated through numerical experiments.

Regularized full waveform inversion for low frequency ultrasound tomography with a structural similarity EIT prior

In previous work by Rueter et al., low frequency (10–750 kHz) ultrasound was shown to penetrate the lung, motivating its use as a nonionizing tomographic technique for pulmonary monitoring. Here, we present a method for regularized full waveform inversion for low-frequency tomographic ultrasound data. A novel structural similarity index metric (SSIM)-based regularization term is introduced that compares the structural correlation of the current sound speed iterate to a prior reconstruction computed by electrical impedance tomography (EIT). Full waveform reconstructions of sound speeds from numerically simulated low frequency ultrasound data with 0.1% additive Gaussian noise are computed, and the results are compared using Tikhonov regularization, total variation regularization, and both terms combined with the SSIM-EIT regularization term. The EIT reconstruction was computed from simulated voltage data on the same phantom using one step of a Newton-Raphson method with a high-pass Gaussian filter as a regularizer. Reconstructions including the SSIM-EIT regularization term converged fastest, and an adaptive regularization parameter provided the most accurate reconstructions. The method is then iterated by computing an EIT reconstruction using the USCT result as a prior, and a subsequent USCT reconstruction is computed using the SSIM-EIT regularization term with the updated EIT reconstruction.

Natureness‐preserving tone mapping operator based on improved guided filter and adaptive Gamma curve

AbstractMapping high dynamic range (HDR) images to low dynamic range (LDR) display devices while performing enhancement processing is a challenging task. To address the issues of chromatic drift, the imbalance between preserving naturalness and enhancing details during the mapping process, this paper proposed a tone mapping operator (TMO) based on an improved guided filter with multi‐layered processing. Firstly, the operator tackled the problem of insufficient extraction of detail layer information in complex scene images by redefining the formula of the guided filter and introducing adaptive regularization parameters. This enhancement enabled more effective extraction of details compared to traditional fixed‐parameter guided filters. Secondly, the algorithm employed a centre‐surround approach to subdivide and processed the detail layers individually, effectively suppressing halos and artefacts. Thirdly, image entropy was utilized to guide the fusion and enhancement processing of the detail layers, preventing excessive or insufficient enhancement. Finally, this paper proposed an adaptive gamma correction algorithm and a chromatic consistency algorithm to improve overall contrast and hue saturation. The results demonstrate convincing performance in terms of subjective visual evaluation and image quality assessment.

Reconstruction Algorithm Optimization Based on Multi-Iteration Adaptive Regularity for Laser Absorption Spectroscopy Tomography

Laser absorption spectroscopy tomography is an effective combustion diagnostic method for obtaining simultaneous two-dimensional distribution measurements of temperature and gas molar concentrations. For the reconstruction process of complex combustion flames, a new algorithm named ‘multi-iterative adaptive optimization regularization’ (MIARO) is proposed. This algorithm is a further development of another algorithm known as the ‘modified adaptive algebraic reconstruction technique’ (MAART) with the improvement of the initial value and adaptive regularization parameter selections. In MIARO, the problem of the MAART’s initial value sensitivity is compensated for, and in addition, reconstruction parameters are also introduced into the regularization so that both the quality of reconstruction and the convergence of regularization are guaranteed. In butane burner experiments, an average relative error of 1.82% was achieved with MIARO, compared to 2.44% with MAART, which is a significant reduction of 25.1%. The simulation and experimental results clearly demonstrate that the MIARO algorithm can be used to reconstruct dynamic combustion fields and eliminate boundary artifacts with improved measurement accuracy and robustness.

Adaptive sparse modeling in spectral & spatial domain for compressed image restoration

Block discrete cosine transform (BDCT) is an indispensable component of modern image and video coding standards, specifically for its decorrelation and superior energy compaction aspects. BDCT typically employs block-specific quantization, which results in unpleasant compression-blocking artifacts which predominates at low bit rates. The proposed method aims to minimize these blocking artifacts to generate high-quality images under the framework of the alternating direction method of multipliers (ADMM) optimization. The proposed method exploits the local structures identified and extracted via wavelet-patch-based sparse representation and non-local self-similarity identified and extracted via group-based sparse representation, which are subsequently combined optimally employing ADMM. Moreover, the method uses a Gaussian quantization noise model, which allows a more precise and reliable assessment. An adaptive regularization parameter is used, which integrates spectral and spatial domain sparse representations with multi-resolution dictionaries and PCA-based dictionary. The proposed algorithm improves the overall practicality of the process and outperforms existing methods in terms of objective measures like structural similarity index measure, peak signal-to-noise ratio and visual perception.

Generalized conditional gradient method with adaptive regularization parameters for fluorescence molecular tomography.

Fluorescence molecular tomography (FMT) is an optical imaging technology with the ability of visualizing the three-dimensional distribution of fluorescently labelled probes in vivo. However, due to the light scattering effect and ill-posed inverse problems, obtaining satisfactory FMT reconstruction is still a challenging problem. In this work, to improve the performance of FMT reconstruction, we proposed a generalized conditional gradient method with adaptive regularization parameters (GCGM-ARP). In order to make a tradeoff between the sparsity and shape preservation of the reconstruction source, and to maintain its robustness, elastic-net (EN) regularization is introduced. EN regularization combines the advantages of L1-norm and L2-norm, and overcomes the shortcomings of traditional Lp-norm regularization, such as over-sparsity, over-smoothness, and non-robustness. Thus, the equivalent optimization formulation of the original problem can be obtained. To further improve the performance of the reconstruction, the L-curve is adopted to adaptively adjust the regularization parameters. Then, the generalized conditional gradient method (GCGM) is used to split the minimization problem based on EN regularization into two simpler sub-problems, which are determining the direction of the gradient and the step size. These sub-problems are addressed efficiently to obtain more sparse solutions. To assess the performance of our proposed method, a series of numerical simulation experiments and in vivo experiments were implemented. The experimental results show that, compared with other mathematical reconstruction methods, GCGM-ARP method has the minimum location error (LE) and relative intensity error (RIE), and the maximum dice coefficient (Dice) in the case of different sources number or shape, or Gaussian noise of 5%-25%. This indicates that GCGM-ARP has superior reconstruction performance in source localization, dual-source resolution, morphology recovery, and robustness. In conclusion, the proposed GCGM-ARP is an effective and robust strategy for FMT reconstruction in biomedical application.

EPLL image denoising with multi-feature dictionaries

The expected patch log likelihood (EPLL) is a patch-prior based denoising model that has become a popular tool in image restoration for its outstanding edge preservation and noise reduction abilities. However, without considering the geometrical feature of patches, the EPLL removes numerous important details and frequently suffers from the staircase effect, residual noise and artifacts. To preserve more detailed information and suppress the generation of artifacts, we propose a novel EPLL denoising model with multi-feature dictionaries (MFDs) and adaptive regularization parameters in this paper. In the proposed method, to exploit the statistical and geometrical characteristics of patches, two feature layers are designed before training the Gaussian mixture model (GMM). In the first layer, the within-class variance minimum (WCVM) method is used to learn the statistical features of patches. In the second layer, two geometrical characteristic dictionaries (GCDs) are trained. In the denoising phase, we select the corresponding dictionary for each patch. Then, each patch is denoised independently. Moreover, the mean localized spectral response (MLSR) based regularization parameters are employed in the proposed method, which is spatial self-adaptive and robust to noise. Extensive experiments demonstrate that our approach achieves desired performance in both detail preservation and artifact suppression compared to many existing denoising methods.

A four-aperture super-resolution camera based on adaptive regularization parameter tuning

Multi-aperture imaging can improve spatial resolution by fusing complementary information from multiple low-resolution (LR) frames using super-resolution (SR) algorithms. The existing SR algorithms are offline used due to the lack of fast regularization parameter update solution. The classical general cross validation (GCV) and the L-curve algorithms search for a suitable solution on each updated logarithmic parameter set for a long time. In this work, we proposed an adaptive regularization parameter tuning (ARPT) method. The regularization parameter is updated by a function of the fidelity term, the standard deviation of the noise, the weighted bilateral total variation and its standard deviation. Simulations demonstrate that the PSNR and SSIM of the ARPT algorithm, are 0.27 dB and 0.002 larger than the GCV algorithm on average, and the computation time is reduced by more than four times. A four-aperture SR camera is built by randomly shifting the sensors on a sub-pixel scale. An SR image was obtained by the ARPT method using 20 LR images. The average consuming time is reduced by 6.17 times compared with the GCV algorithm. Experiments indicate that the proposed four-aperture SR camera can improve the spatial resolution, suppress the noise, and realize a high-quality and more efficient reconstruction.

Regularization Solver Guided FISTA for Electrical Impedance Tomography.

Electrical impedance tomography (EIT) is non-destructive monitoring technology that can visualize the conductivity distribution in the observed area. The inverse problem for imaging is characterized by a serious nonlinear and ill-posed nature, which leads to the low spatial resolution of the reconstructions. The iterative algorithm is an effective method to deal with the imaging inverse problem. However, the existing iterative imaging methods have some drawbacks, such as random and subjective initial parameter setting, very time consuming in vast iterations and shape blurring with less high-order information, etc. To solve these problems, this paper proposes a novel fast convergent iteration method for solving the inverse problem and designs an initial guess method based on an adaptive regularization parameter adjustment. This method is named the Regularization Solver Guided Fast Iterative Shrinkage Threshold Algorithm (RS-FISTA). The iterative solution process under the L1-norm regular constraint is derived in the LASSO problem. Meanwhile, the Nesterov accelerator is introduced to accelerate the gradient optimization race in the ISTA method. In order to make the initial guess contain more prior information and be independent of subjective factors such as human experience, a new adaptive regularization weight coefficient selection method is introduced into the initial conjecture of the FISTA iteration as it contains more accurate prior information of the conductivity distribution. The RS-FISTA method is compared with the methods of Landweber, CG, NOSER, Newton-Raphson, ISTA and FISTA, six different distributions with their optimal parameters. The SSIM, RMSE and PSNR of RS-FISTA methods are 0.7253, 3.44 and 37.55, respectively. In the performance test of convergence, the evaluation metrics of this method are relatively stable at 30 iterations. This shows that the proposed method not only has better visualization, but also has fast convergence. It is verified that the RS-FISTA algorithm is the better algorithm for EIT reconstruction from both simulation and physical experiments.

Image Haze Removal Method Based on Histogram Gradient Feature Guidance.

Optical remote sensing images obtained in haze weather not only have poor quality, but also have the characteristics of gray color, blurred details and low contrast, which seriously affect their visual effect and applications. Therefore, improving the image clarity, reducing the impact of haze and obtaining more valuable information have become the important aims of remote sensing image preprocessing. Based on the characteristics of haze images, combined with the earlier dark channel method and guided filtering theory, this paper proposed a new image haze removal method based on histogram gradient feature guidance (HGFG). In this method, the multidirectional gradient features are obtained, the atmospheric transmittance map is modified using the principle of guided filtering, and the adaptive regularization parameters are designed to achieve the image haze removal. Different types of image data were used to verify the experiment. The experimental result images have high definition and contrast, and maintain significant details and color fidelity. This shows that the new method has a strong ability to remove haze, abundant detail information, wide adaptability and high application value.

A Bayesian Framework of Non-Synchronous Measurements at Coprime Positions for Sound Source Localization With High Resolution

The noise distribution in the mechanical system is tightly connected to the structure design and particular operating conditions. The noise source localization can effectively assist with the operating condition monitoring and noise reduction design of the mechanical device. The performance limitation of the array aperture for lower frequency acoustic localization is broken up by the non-synchronous measurement at coprime positions (CP-NSM), while this method has high uncertainty and requires an efficient adaptive regularization method to solve its corresponding acoustic inverse problem. The algorithms under the Bayesian framework with Student-t priors (variational Bayesian approximation and subspace variational Bayesian) are derived and deployed to solve the acoustic inverse problem in the CP-NSM method, and the results are compared with those obtained by the interior-point method and the alternating direction method of the multipliers algorithm. The proposed Bayesian methods have the advantages of adaptive regularization parameter estimation, which can reduce the influence of various interferences in CP-NSM. At the same time, in addition to using simulations and experiments in the anechoic chambers, the proposed Bayesian algorithms are validated further in real industrial applications. The proposed method is applied to improve the noise reduction design of the centrifugal fan, a mechanical device containing more information from noise sources.

Choosing a Regularization Parameter in the Problem of Analytical Continuation of Gravitational Fields (Separation of Anomalies Generated by Shallow and Deep Sources)

Abstract —We present a filtration tomography technique for isolating components of the gravity field anomalies generated by inhomogeneities of the horizontally layered density model. The filtration algorithm of field separation relies on the solution of the forward and inverse problems of analytical continuation of harmonic functions through the horizontal boundary plane. We applied the regularizing algorithms to analytical continuation of the gravity field “down” to its generating sources. The fields successively recalculated upward and downward relative to preset depths allowed us to partition the initial (total) field as the sum of the fields generated in the layers based on the properly selected adaptive regularization parameter α. For the sake of stability of the inverse problem solution in the analytical continuation of the observed gravity field to a certain depth, we used the Lavrentiev regularization scheme involving the L-curve method (for selecting the adaptive regularization parameter). The smoothing regularization parameter values obtained from the preset successive depth intervals and grid step for the observed field are shown to be optimal for dividing the observed field into components corresponding to different depths. The developed algorithms for massively parallel computing systems and their application to a group of different heights were numerically implemented on the Uran supercomputer.

Image Restoration using Nonlocal Regularized Variational Model with Spatially Adapted Regularization Parameter

Restoration of blurred images corrupted with impulse noise has attracted increasing attention due to its great potential toward practical applications. To guarantee the noisy image deblurring performance, total variation (TV), one of the most popular regularizers, has been extensively exploited in the literature. This local regularizer is known for its capability of preserving edges and penalizing oscillations but not sharp discontinuities. However, TV regularized variational models commonly suffer from undesirable staircase-like artifacts in homogeneous regions. The reason behind this phenomenon is that TV regularizer favors solutions that are piecewise constant in numerical experiments. To overcome the model-dependent deficiency, we proposed to replace the commonly used local regularizer by its nonlocal extension, i.e., nonlocal total variation (NLTV) regularizer. By taking advantage of the high degree of self-similarity within images, the proposed nonlocal variational model is able to remove both blurring and impulse noise while preserving meaningful image details. To guarantee solution stability and efficiency, the resulting nonsmooth image restoration problem was effectively dealt with using an alternating optimization algorithm. In addition, to further enhance the image quality, a local variance estimator-based automatic method was introduced to calculate the spatially adapted regularization parameters during image restoration. Extensive experiments have demonstrated that the proposed method could achieve superior imaging performance compared with several state-of-the-art image restoration methods.

Time-lapse adaptive inversion on resistivity monitoring data with Lp norm regularization and cross-time weight

Time-lapse electrical resistivity tomography (ERT) can describe the dynamic processes occurring underground and has been widely employed in the fields of hydrology, environmental management, and engineering. The time-lapse inversion method is based on the space- and time-dimension regularization technique as it considers complementary information about more than one time interval and reduces the potential causes of the geophysical data inversion of the artifact. However, the L2 norm inversions smear out the models and cannot distinguish between the resistivity interfaces. Moreover, the determination of the two optimal regularization parameters for time-lapse inversion remains challenging, and the fixed-time regularization parameter smooths out the low-sensitivity area and inhibits the real variation on the target body of the deep region. To overcome this problem, we propose a time-lapse ERT inversion method of adaptively imaging the time-lapse model and focusing on the resistivity variation on the time dimension. This method is based on the Lp norm regularization provided by the M-estimator and uses the adaptive regularization parameters based on the model roughness and data misfit. In addition, a cross-time matrix improving the sensitivity in the deep region of model space is added to the time regularization term. We present two synthetic examples and a field dataset to validate the method. Numerical simulation and field dataset inversion results revealed that the proposed time-lapse inversion strategy can avoid the excessive smoothness caused by the low sensitivity in the model deep region and can thus enhance the response from the true ground variation to improve the accuracy of the inversion results. Owing to the adaptive regularization parameters that can preserve the dynamic balance between the objective function components, the inversion iteration progress can remain stable and can maintain the trend toward convergence.

An Adaptive Lp Norm Minimization Algorithm for Direction of Arrival Estimation

In this paper, we propose a new direction of arrival (DOA) estimation algorithm, in which DOA estimation is achieved by finding the sparsest support set of multiple measurement vectors (MMV) in an over-complete dictionary. The proposed algorithm is based on ℓp norm minimization, which belongs to non-convex optimization. Therefore, the quasi-Newton method is used to converge the iterative process. There are two advantages of this algorithm: one is the higher possibility and resolution of distinguishing closely spaced sources, and the other is the adaptive regularization parameter adjustment. Moreover, an accelerating strategy is applied in the computation, and a weighted method of the proposed algorithm is also introduced to improve the accuracy. We conducted experiments to validate the effectiveness of the proposed algorithm. The performance was compared with several popular DOA estimation algorithms and the Cramer–Rao bound (CRB).

GRACGE: Graph Signal Clustering and Multiple Graph Estimation

In graph signal processing (GSP), complex datasets arise from several underlying graphs and in the presence of heterogeneity. Graph learning from heterogeneous graph signals often results in challenging high-dimensional multiple graph estimation problems, and prior information regarding which graph the data was observed is typically unknown. To address the above challenges, we develop a novel framework called GRACGE ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GRAph signal Clustering and multiple Graph Estimation</i> ) to partition the graph signals into clusters and jointly learn the multiple underlying graphs for each of the clusters. GRACGE advocates a regularized EM (rEM) algorithm where a structure fusion penalty with adaptive regularization parameters is imposed on the M-step. Such a penalty can exploit the structural similarities among graphs to overcome the curse of dimensionality. Moreover, we provide a non-asymptotic bound on the estimation error of the GRACGE algorithm, which establishes its computational and statistical guarantees. Furthermore, this theoretical analysis motivates us to adaptively re-weight the regularization parameters. With the adaptive regularization scheme, the final estimates of GRACGE will geometrically converge to the true parameters within statistical precision. Finally, experimental results on both synthetic and real data demonstrate the performance of the proposed GRACGE algorithm.

Regularized Tapered Sample Covariance Matrix

Covariance matrix tapers have a long history in signal processing and related fields. Examples of applications include autoregressive models (promoting a banded structure) or beamforming (widening the spectral null width associated with an interferer). In this paper, the focus is on high-dimensional setting where the dimension <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$p$</tex-math></inline-formula> is high, while the data aspect ratio <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$n/p$</tex-math></inline-formula> is low. We propose an estimator called <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Tabasco</small> (TApered or BAnded Shrinkage COvariance matrix) that shrinks the tapered sample covariance matrix towards a scaled identity matrix. We derive optimal and estimated (data adaptive) regularization parameters that are designed to minimize the mean squared error (MSE) between the proposed shrinkage estimator and the true covariance matrix. These parameters are derived under the general assumption that the data is sampled from an unspecified elliptically symmetric distribution with finite 4th order moments (both real- and complex-valued cases are addressed). Simulation studies show that the proposed <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Tabasco</small> outperforms all competing tapering covariance matrix estimators in diverse setups. An application to space-time adaptive processing (STAP) also illustrates the benefit of the proposed estimator in a practical signal processing setup.

Sorted L1 regularization method for damage detection based on electrical impedance tomography.

Carbon fiber reinforced polymers (CFRPs) are composite materials in which carbon provides strength and stiffness, whereas polymers provide cohesiveness and toughness. The electrical impedance of CFRP laminates is changed due to different kinds of damages. Electrical impedance tomography (EIT) has significant advantages such as non-intrusion, portability, low cost, and quick response and has widely been used as a nondestructive testing method. Therefore, EIT has great potential in structural health monitoring of CFRPs. Regularization can solve the ill-posed inverse problem of EIT. However, conventional regularization algorithms have their own limitations, such as over-smoothness of reconstructed edges and unstable solution caused by measurement noise. In addition, the anisotropic property of CFRPs also affects the image quality based on traditional methods. In this paper, the sorted L1-norm regularization is proposed. It promotes grouping highly correlated variables while encouraging sparsity by using more effective penalty terms. The sharp edges between different materials can be obtained, and the obtained solution is more stable. The image quality of different objects, especially the image quality of multi-targets, can be significantly improved with this new method. In addition, the sorted L1 norm can generate adaptive regularization parameters without empirical selection. The new regularization problem is solved by the alternating direction method of multipliers. Both experimental and simulation results demonstrate that the sorted L1 norm improves the quality of reconstructed images under various noise levels. The proposed method is comprehensively evaluated with three image quality criteria by numerical simulation quantitatively.

Activity recognition from Smartphone data using WSVM-HMM classification

A lot of real-life mobile sensing applications are becoming available nowadays. The traditional approach for activity recognition employs machine learning algorithms to learn from collected data from smartphpne and induce a model. The model generation is usually performed offline on a server system and later deployed to the phone for activity recognition. In this paper, we propose a new hybrid classification model to perform automatic recognition of activities using built-in embedded sensors present in smartphones. The proposed method uses a trick to classify the ongoing activity by combining Weighted Support Vector Machines (WSVM) model and Hidden Markov Model (HMM) model. The sensory inputs to the classifier are reduced with the Linear Discriminant Analysis (LDA). We demonstrate how to train the hybrid approach in this setting, introduce an adaptive regularization parameter for WSVM approach, and illustrate how our proposed method outperforms the state-of-the-art on a large benchmark dataset.