L1-norm Regularization Research Articles

A semi-analytical method for moving force identification of bridge structures based on the discrete cosine transform and FEM

A semi-analytical method is proposed for moving force identification (MFI) based on the finite element model of the bridge structures. The moving force can be expanded by a set of cosine basis functions based on the discrete cosine transform (DCT) in the time domain. By constructing the segmented continuous mode shape of the bridge deck, the analytical relationship between the measured acceleration responses and the basis coefficients is derived using the Duhamel integral, which is not affected by the time step. The weighted l1-norm regularization method is used to solve the basis coefficients to improve the accuracy and noise immunity for identification. For complex bridge structures, the relationship between the responses and the moving forces can be analytically characterized by only knowing node modes of the bridge deck elements. Thus the identification can be carried out effectively. In numerical simulations, the effects of measured locations, noise levels, mode numbers, and moving speeds on the recognition accuracy are discussed under different models. The effectiveness, applicability, and robustness of the proposed method are verified.

Sparse DDK: A Data-Driven Decorrelation Filter for GRACE Level-2 Products

High-frequency and correlated noise filtering is one of the important preprocessing steps for GRACE level-2 products before calculating mass anomaly. Decorrelation and denoising kernel (DDK) filters are usually considered as such optimal filters to solve this problem. In this work, a sparse DDK filter is proposed. This is achieved by replacing Tikhonov regularization in traditional DDK filters with weighted L1 norm regularization. The proposed sparse DDK filter adopts a time-varying error covariance matrix, while the equivalent signal covariance matrix is adaptively determined by the Gravity Recovery and Climate Experiment (GRACE) monthly solution. The covariance matrix of the sparse DDK filtered solution is also developed from the Bayesian and error-propagation perspectives, respectively. Furthermore, we also compare and discuss the properties of different filters. The proposed sparse DDK has all the advantages of traditional filters, such as time-varying, location inhomogeneity, and anisotropy, etc. In addition, the filtered solution is sparse; that is, some high-degree and high-order terms are strictly zeros. This sparsity is beneficial in the following sense: high-degree and high-order sparsity mean that the dominating noise in high-degree and high-order terms is completely suppressed, at a slight cost that the tiny signals of these terms are also discarded. The Center for Space Research (CSR) GRACE monthly solutions and their error covariance matrices, from January 2004 to December 2010, are used to test the performance of the proposed sparse DDK filter. The results show that the sparse DDK can effectively decorrelate and denoise these data.

A new robust fuzzy clustering framework considering different data weights in different clusters

In conventional fuzzy C-means clustering algorithms, each data and each feature are treated equally, the clustering performance is sensitive to the noise points; in existing weighting clustering algorithms, few studies have focus on data weighting and feature weighting simultaneously, besides, the same data in different clusters is treated equally. To address this issue, in this paper, taking the different data weights in different clusters and the different feature weights in different clusters into consideration, we present a new robust fuzzy C-means clustering framework. For the first time, we propose a whole new idea that the same data in different clusters should have different importance, the different data in a cluster should have different importance, as well; By the new data weighting method, the proposed clustering algorithm can weaken the impact of noise points on the formation of each clustering center, which could enhance the robustness of clustering; to stimulate more data and more features to take part in the process of clustering and to avoid overfitting, we add l2-norm regularization of the data weights and l2-norm regularization of the feature weights to the objective function. Then, based on the presented objective function, we get the scientific update rules of the different data weights in different clusters, the different feature weights in different clusters, the membership degrees, and the cluster centers, during each iteration. To assess the performance of the new fuzzy C-means framework, experimental verifications on synthetic dataset and real-world datasets are conducted, experimental results have shown that the new algorithm can achieve better clustering performances in comparison to other related clustering methods.

Variable selection in convex quantile regression: [formula omitted]-norm or [formula omitted]-norm regularization?

The curse of dimensionality is a recognized challenge in nonparametric estimation. This paper develops a new L0-norm regularization approach to the convex quantile and expectile regressions for subset selection. We show how to use mixed-integer programming to solve the proposed L0-norm regularization approach in practice and build a link to the commonly used L1-norm regularization approach. A Monte Carlo study is performed to compare the finite sample performances of the proposed L0-penalized convex quantile and expectile regression approaches with the L1-norm regularization approaches. The proposed approach is further applied to benchmark the sustainable development performance of the OECD countries and empirically analyze the accuracy in the dimensionality reduction of variables. The results from the simulation and application illustrate that the proposed L0-norm regularization approach can more effectively address the curse of dimensionality than the L1-norm regularization approach in multidimensional spaces.

Field road segmentation network based on PraNet

ABSTRACT Nowdays, many methods based on CNN have been proposed for road extraction. However, there are still great challenges. Therefore, according to image characteristics, this paper made corresponding improvements based on medical segmentation network PraNet. First, the reverse attention module (RA) connected at the last layer of PraNet is changed to the positive attention module (PA). Then, the negative matrix L1 norm regularization is added into the loss function. We conducted experiments on a data set made by UAV in the field of Henan Academy of Agricultural Sciences. The results show that the proposed method is better than the comparison models.

Towards a better understanding of the recoverability of physical property relationships from geophysical inversions of multiple potential-field data sets

SUMMRY Collecting multiple potential-field data sets to image the subsurface has been popular in resource explorations. How to effectively integrate multiphysics data and extract useful information about the subsurface geology is an active area of research. Among existing approaches, geophysical inversions followed by geological classification has shown promising results. This procedure contains two steps. First, multiple geophysical data sets are inverted, either separately or jointly, to obtain their corresponding physical property models. Secondly, the inverted values are visualized in a scatterplot and classified into different groups, a process known as geology differentiation. The implicit assumption for geology differentiation is that the recovered physical property relationships are reliable. However, whether this assumption is generally valid or not remains unknown. Moreover, it is well known that the standard smooth inversion would underestimate the physical property values. On the other hand, the sparse inversion is able to recover compact anomalies with elevated magnitudes. However, it remains unclear as to how smooth and sparse inversions affect the recoverability. To understand the recoverability of physical property relationships, we have designed six geological scenarios with three geological anomalies located at different depths and assigned with different physical property values. For each scenario, we performed four inversions: both separate and joint inversions using smooth L2-norm and sparse mixed L12-norm regularizations. Each inversion is followed by geology differentiation. We have found (1) that the recovered physical property relationships from geophysical inversions are not always reliable, (2) the depths of the source bodies and inversion strategies significantly affect the recoverability, and (3) joint sparsity inversion consistently gives rise to the best recoverability.

Spatially constrained inversion of semi-airborne transient electromagnetic data based on a mixed norm

The interpretation of semi-airborne transient electromagnetic data is mainly based on a one-dimensional inversion algorithm. The continuity of the resistivity profile is generally poor when the data noise is strong because the resistivity is laterally unconstrained, and the constraint between the resistivity in the vertical distribution is usually taken as the L2 norm regular term. However, the main disadvantage of this regularization constraint is that it over-smoothens the model and cannot effectively describe the interface information of an abrupt electrical change. To address these issues, this paper proposes a mixed norm spatially constrained inversion (SCI) algorithm that adopts the L1 norm regularization term for the vertical spatial constraint and the L2 norm regularization term for the lateral spatial constraint. The Gauss-Newton iterative method is used to solve the objective function. Fixed and adaptive strategies are proposed for the regularization factors, and a parameter is used to adjust the weights of the two constraints. The influence of different constraint weights on the results is analyzed. The inversion of noisy synthetic data verifies that the proposed SCI algorithm has a higher resolution for electrical interfaces and better noise suppression effect than the L2 norm SCI algorithm. The inversion of field data shows a good effect and practicability.

Parameter tuning in machine learning based on radiomics biomarkers of lung cancer.

Lung cancer is one of the most common cancers, and early diagnosis and intervention can improve cancer cure rate. To improve predictive performance of radiomics features for lung cancer by tuning the machine learning model parameters. Using a dataset involving 263 cases (125 benign and 138 malignant) acquired from our hospital, each classifier model is trained and tested using 237 and 26 cases, respectively. We initially extract 867 radiomics features of CT images for model development and then test 10 feature selections and 7 models to determine the best method. We further tune the parameter of the final model to reach the best performance. The adjusted final model is then validated using 224 cases acquired from Lung Image Database Consortium (LIDC) dataset (64 benign and 160 malignant) with the same set of selected radiomics features. During model development, the feature selection via concave minimization method show the best performance of area under ROC curve (AUC = 0.765), followed by l0-norm regularization (AUC = 0.741) and Fisher discrimination criterion (AUC = 0.734). Support vector machine (SVM) and random forest (RF) are the top two machine learning algorithms showing the best performance (AUC = 0.765 and 0.734, respectively), using by the default parameter. After parameter tuning, SVM with linear kernel achieves the best performance (AUC = 0.837), whereas the best tuned RF with the number of trees is 510 and yields a slightly lower performance (AUC = 0.775) in 26 test samples data. During model validation, the SVM and RF models yield AUC = 0.78 and 0.77, respectively. Appropriate quantitative radiomics features and accurate parameters can improve the model's performance to predict lung cancer.

Iterative weighted fidelity and hybrid regularization-based algorithm for ECT

A new image reconstruction algorithm for electrical capacitance tomography based on iterative weighted fidelity and hybrid regularization is proposed. Specifically, an improved cost function model with the weighted data fidelity term and non-convex regularization term can better describe the sparsity of images and strengthen the anti-noises ability compared with the L1-norm or L2-norm regularization methods. In order to solve the proposed model efficiently, the alternating direction method of multipliers is used to divide the complex optimization problem into several simple iterative sub-problems. Moreover, the iterative shrinkage thresholding algorithm and the iterative p-shrinkage algorithm are also adopted to solve the sub-problems. Besides, simulation and experiments for different permittivity distributions are investigated with noise-free and noise-contaminated cases respectively. The research results verify that reconstruction images with the proposed algorithm have fewer artifacts and deformations, clearer edges, and better noise robustness than the other methods considered.

AVA inversion of pre-stack seismic data with cross-gradient constraints

The ill-posed feature is one basic attribute of geophysical inversion methods. As an example of geophysical inverse problem, AVA (Amplitude variation with incident angle) inversion of pre-stack seismic data is susceptible to noise and uncertainty in the acquisition. To get stable and accurate inversion results, the regularization constraints on model parameter need to be added into the objective function of AVA inversion. In AVA inversion, the most commonly used regularization includes sparsity constraint (e.g. L1-norm regularization, Cauchy regularization) and a priori model parameters constraint, and so forth. However, the existing AVA inversion methods do not consider the structural similarity of different model parameters. All of the different model parameters represent the same underground geological structure, so they should have similar structure. This paper adopts the cross gradient to measure the structural similarity of different model parameters. Next, the cross gradients of different model parameters are added into the objective function of AVA inversion as a regularization term to implement structural similarity constraint. Results of the model numerical tests and real seismic data indicate that the AVA inversion with cross-gradient constraint has higher stability compared to the AVA inversion without structural similarity constraint, especially for the density inversion results.

Second-Order Ultrasound Elastography With L1-Norm Spatial Regularization.

Time delay estimation (TDE) between two radio-frequency (RF) frames is one of the major steps of quasi-static ultrasound elastography, which detects tissue pathology by estimating its mechanical properties. Regularized optimization-based techniques, a prominent class of TDE algorithms, optimize a nonlinear energy functional consisting of data constancy and spatial continuity constraints to obtain the displacement and strain maps between the time-series frames under consideration. The existing optimization-based TDE methods often consider the L2 -norm of displacement derivatives to construct the regularizer. However, such a formulation over-penalizes the displacement irregularity and poses two major issues to the estimated strain field. First, the boundaries between different tissues are blurred. Second, the visual contrast between the target and the background is suboptimal. To resolve these issues, herein, we propose a novel TDE algorithm where instead of L2 -, L1 -norms of both first- and second-order displacement derivatives are taken into account to devise the continuity functional. We handle the non-differentiability of L1 -norm by smoothing the absolute value function's sharp corner and optimize the resulting cost function in an iterative manner. We call our technique Second-Order Ultrasound eLastography (SOUL) with the L1 -norm spatial regularization ( L1 -SOUL). In terms of both sharpness and visual contrast, L1 -SOUL substantially outperforms GLobal Ultrasound Elastography (GLUE), tOtal Variation rEgulaRization and WINDow-based time delay estimation (OVERWIND), and SOUL, three recently published TDE algorithms in all validation experiments performed in this study. In cases of simulated, phantom, and in vivo datasets, respectively, L1 -SOUL achieves 67.8%, 46.81%, and 117.35% improvements of contrast-to-noise ratio (CNR) over SOUL. The L1 -SOUL code can be downloaded from http://code.sonography.ai.

Similarity and sparsity collaborative embedding and its application to robust process monitoring

Multivariate measurements in complex industrial processes are commonly contaminated by a number of outliers. In this context, robustness to the corrupted data is an important problem in process monitoring tasks. This paper proposed a novel approach called similarity and sparsity collaborative embedding (SSCE) for efficient robust process monitoring. The proposed SSCE can learn a sparse coefficient matrix by a ℓ1-norm regularization as the sparse constraint on the reconstruction errors, making it robust to the data contaminated by outliers. The similarity preserving matrix is proposed to capture the local structure of the given data, and then the local information is transferred to the sparse coefficients such that the similarity among data points can be preserved. In this way, the reduced-dimensional representations extracted is capable of containing more informative and discriminating characteristics of the original data, which is beneficial to enhance monitoring performance. Meanwhile, projection learning is integrated into the proposed objective function to learn an explicit projection matrix in an overall optimum way, which enables the SSCE to circumvent the out-of-sample problem and facilitate subsequent process monitoring tasks. Two case studies on a simulated typical chemical process and a practical fractionation process demonstrate the effectiveness of the proposed approach.

Bi-regularization enhanced azimuthal mode analysis method for the aero-engine fan

Azimuthal mode analysis is regarded as an effective tool for aero-engine flow field investigation, where numerous microphones are evenly mounted as a ring to provide sufficient spatial resolution for azimuthal pressure acquisition. The current compressive sampling method enables azimuthal modes to be obtained from much fewer microphones, with dominant and non-dominant modes separately estimated by L1-norm regularization and spatial Fourier transform or least square method. Despite the effort of measurement system simplification, this classical compressive sampling method regularized by L1-norm inherently introduces accuracy loss in amplitude estimation for both dominant and non-dominant modes. With the aim of accuracy promotion and further measurement reduction, the Bi-regularization enhanced azimuthal mode analysis (BRAMA) method is devised to investigate the azimuthal modes of the aero-engine fan via limited acoustic measurements. The BRAMA method substitutes L1-norm regularization with Lp-norm (0<p<1) regularization to reach more accurate reconstruction of dominant modes. Meanwhile, the Tikhonov regularization based on L2-norm is introduced to estimate non-dominant modes. The effectiveness of the BRAMA method is verified in two cases, by operating a 2.5-stage aero-engine fan test rig at 50% and 90% of the nominal speed, respectively. Firstly, acoustic pressure signals are pre-processed to extract the dominant tonal component, which are used as the input of dominant mode estimation. Next, comparative realizations by L1-norm and Lp-norm are conducted to detect the dominant modes, meanwhile the amplitude accuracy with respect to p value and measurement number is discussed in both cases. Finally, non-dominant modes are estimated by using Tikhonov regularization in comparison to the classical approaches. Experimental results of both cases indicate that the BRAMA method outperforms the classical compressive sampling approach in accuracy improving, measurements reducing and interference proof capability.

Design of stabilized fuzzy relation-based neural networks driven to ensemble neurons/layers and multi-optimization

In this study, a design methodology based on fuzzy relation-based neural networks for stabilized network structure is introduced to cope with over-fitting as well as multi-collinearity problems which generally appear in conventional fuzzy neural networks. The design method of the proposed self-organizing network structure provides an efficient solution to construct the stabilized Fuzzy Relation-based Neural Networks (FRNN) through a synergy of multi-techniques such as ensemble neurons/layers, L2-norm regularization, compromise technique, and multi-optimization, in order to generate the structure of the multi-layered self-organizing network designed with the aid of the learning as well as novel structural design.The overall network structure is realized with the aid of parallel network structure with newly added layers as well as effective node selection method through the combination technique of both a sum of squared coefficients (SSC) and performance index (PI) as a node selection criterion for each layer in FRNN. Ensemble neurons consist of the current inputs and the original inputs, and they serve to reduce the bias of the proposed FRNN by maintaining the information of the original inputs. Also, ensemble layers stand for the combination of the current layer and the front layer, and they play a role for control the variance of the model by taking into account the output of the previous layers. The least square error estimation (LSE)-based learning method with L2-norm regularization is used for constructing the stabilized network architecture, and their ensuring design methodologies result in alleviating the overfitting phenomenon and also enhancing the generalization ability. For the performance enhancement of FRNN directly affected by some parameters such as the number of input variables, collocation of the specific subset of input variables, the number of membership functions per each variable, and the order of polynomial in the consequent parts of the fuzzy rules, multi-particle swarm optimization (MPSO) is exploited for the effectively structural as well as parametric optimization of the proposed networks. That is, the multi-optimization helps achieve a compromise between the better generation performance and the alleviated over-fitting leading to the stabilization of the proposed multi-layered self-organizing network structure realized with the aid of synergistic multi-techniques and ensemble structures such as a) L2-norm regularization-based LSE learning, b) combination technique for effective node selection through the combination of SSC and PI, and c) a novel parallel network structure including newly added layers and node selection method. The performance of the proposed network structure is quantified by comprehensive experiments and comparative analysis.

An Efficient Point-Matching Method-of-Moments for 2D and 3D Electrical Impedance Tomography Using Radial Basis Functions.

The inverse problem of computing conductivity distributions in 2D and 3D objects interrogated by low-frequency electrical signals, which is called Electrical Impedance Tomography (EIT), is treated using a Method-of-Moment technique. A Point-Matching-Method-of-Moment technique is used to formulate a global integral equation solver. Radial Basis Functions are adopted to express the conductivity distribution. Single-step quadratic-norm ( L2) and iterative total variation ( L1) regularization techniques are exploited to solve the inverse problem. Simulation and experimental tests on a circular reconstruction domain show satisfactory performance in deriving conductivity distribution, achieving a Correlation Coefficient ( CC) up to 0.863 for 70 dB voltage SNR and 0.842 for 40 dB voltage SNR. The proposed methodology with L2-norm regularization provided better results than traditional iterative Gauss-Newton's approach, whereas with L1-norm regularization it showed promising performance. Moreover, 3D reconstructions on a cylindrical cavity demonstrated superior results near the electrodes' planes compared to those of the conventional linearized approach. Finally, application to EIT medical data for dynamic lung imaging successfully revealed the breath-cycle conductivity changes. The results show that the proposed method can be effective for both 2D and 3D EIT and applicable to many applications. Strong conductivity variations are successfully tackled with a very good Correlation Coefficient. In contrast to conventional EIT solutions based on weak-form and linearization on small conductivity changes, the proposed method requires only one step to converge with L2-norm regularization. The proposed method with L1-norm regularization also achieves good reconstruction quality with a low number of iterations.

L2-Norm Shapelet Dictionary Learning-Based Bearing-Fault Diagnosis in Uncertain Working Conditions

Bearing-fault diagnosis is an important research topic and is crucial in modern industrial systems. Several methods for bearing-fault diagnosis have been developed. However, most existing methods can only detect faults under controlled working conditions or the variation of a single working condition (e.g., a varying load). When the working conditions change or multiple uncertain working conditions are encountered, the model performance is significantly degraded. Therefore, this paper proposes an L2-norm shapelet dictionary learning (SDL)-based bearing diagnosis method that aims to improve the accuracy of fault classification under uncertain working conditions. First, the L1-norm regularization term in the SDL algorithm is replaced with an L2-norm regularization to capture diversified shapelets from multiple varying working conditions. Then, a comprehensive bearing-fault diagnosis method is developed using the shapelet-transformation and support vector machine methods to handle the uncertainty of multiple working conditions. Two case studies using laboratory data and practical experiments were conducted to validate the proposed method. The experimental results for both the case studies indicate the effectiveness of the proposed method with regard to fault classification accuracy and visualization interpretability.

Hierarchically Reorganized Multi-Layer Fuzzy Neural Networks Architecture Driven With the Aid of Node Selection Strategies and Structural Network Optimization

In this study, a design methodology based on fuzzy sets inference and polynomial neural network(PNN) for hierarchically reorganized self-organizing network architecture is introduced to cope with over-fitting as well as multi-collinearity problems which generally appear in a conventional fuzzy neural network. The design method of the proposed self-organizing network structure provides an efficient solution to construct the hierarchically reorganized multi-layer fuzzy neural networks (HRmFNN) architecture through a synergy of multi-techniques such as L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -norm regularization, probability theory, and multi-optimization. The overall network structure is realized with the aid of parallel network structure with newly added inputs as well as effective neuron selection method through the exponential-based roulette selection technique for each layer in HRmFNN, and the least square error estimation (LSE)-based learning method with L2-norm regularization is used for constructing the stabilized network architecture, and their ensuring design methodologies result in alleviating the overfitting phenomenon and also enhancing the generalization ability. For the performance enhancement of HRmFNN directly affected by some parameters such as the number of input variables, collocation of the specific subset of input variables, the number of membership functions per each variable, and the order of polynomial in the consequent parts of the fuzzy rules, multi-particle swarm optimization (MPSO) is exploited for the effectively structural as well as parametric optimization of the proposed network. That is, the multi-optimization helps achieve a compromise between the better generation performance and the alleviated over-fitting leading to the stabilization of the proposed multi-layered self-organizing network structure with the aid of synergistic multi-techniques such as a) L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -norm regularization-based LSE learning, b) probability theory for effective neuron selection, and c) novel parallel network structure including newly added inputs and neuron selection method. The performance of the proposed network structure is quantified by comprehensive experiments and comparative analysis. It is also demonstrated through the application to cement compressive strength.

L2 Regularization Model with Removal of Gaussian Noise

In order to remove Gaussian noise from images, a Gaussian noise image restoration method based on the L2 norm regularization model was proposed. The L2 norm is selected as the data fidelity term and the gradient operator and wavelet frame as the regularization term to suppress the image ladder effect and protect the image edge details. Since the objective function of the model is a large convex function, the solving process is very tedious. The split Bregman iterative algorithm and alternate direction multiplier method are combined to restore the image. The experimental results show that show that the alternate direction multiplier method can effectively reduce the difficulty of solving the restoration model, and the image recovered by using this model has a higher peak signal-to-n ratio and better structural similarity and can get a clearer image.

Non-Iterative Data-Driven Tuning of Model-Free Control Based on an Ultra-Local Model

In this paper, we present a data-driven tuning method for model-free control based on an ultra-local model (MFC-ULM), which is also called intelligent proportional-integral-derivative control. In industries, the control design must be easy, and it is important that the control law can be applied to nonlinear systems. The MFC-ULM has most of these features. However, in practice, trial-and-error tuning of MFC-ULM design parameters is necessary. To address this problem, we adopt a data-driven tuning approach. In the proposed method, the MFC-ULM design parameters can be tuned from single-experiment data without requiring system identification, and optimal parameters for the MFC-ULM are obtained using the least-squares method. Additionally, we adopt L2-norm regularization to avoid overlearning. The effectiveness of this method was examined using simulations of two nonlinear systems. The results revealed that the MFC-ULM design parameters can be obtained directly without knowing the characteristics of the controlled object.

Adaptive adjacent signal difference lasso for bearing fault detection

Extraction of the fault-related periodic impulses from vibration signals with heavy background noise is fundamental but difficult for bearing fault diagnosis. In this paper, a novel fault feature extraction method based on a new sparse representation, adaptive adjacent signal difference lasso (AdaASDL) is proposed for bearing fault diagnosis. AdaASDL model is composed of l1-norm sparse regularization term and adjacent signal difference sparse regularization term, which are used to enhance the sparsity of the signal amplitude and adjacent signal difference, respectively. Moreover, a weighted method is proposed for setting the regularization parameter adaptively. The majorization–minimization (MM)-based optimization algorithm is derived to solve the objection function optimization of AdaASDL. In comparison with other state-of-the-art methods including the conventional sparse representation and spectral kurtosis (SK), AdaASDL has the better denoising performance on simulation signals and bearing vibration signals and the comparison results demonstrate the superiority of AdaASDL in bearing fault diagnosis.