Norm Regularization Research Articles

A Transfer Learning Model for Rolling Bearing Fault Diagnosis Based on Texture Loss Strategy and Nuclear Norm Regularization

Abstract In recent years, transfer learning (TL) approaches have seen extensive application in diagnosing bearing faults due to their exceptional performance. However, mechanical noise, equipment aging, and wear lead to notable disparities and differences in the multi-level feature distributions across the source and target domain signals. The issue is addressed by proposing a transfer learning model based on a texture loss strategy and nuclear norm regularization method. First, a Feature-Enhanced Network (MSRnet) is designed, which significantly improves the ability to capture local details and long-range dependencies by combining a multi-scale feature extraction module with a dilated residual module. Next, a texture loss strategy is proposed to align multi-scale features across domains by minimizing the Gram matrix of signal features. Finally, a nuclear norm regularization method is proposed to perform low-rank approximation on the signal matrix, facilitating the extraction of more robust feature data and mitigating the risk of overfitting. The experimental results demonstrate that the proposed method achieved an average accuracy of 98.58% on the University of Ottawa bearing fault dataset and 98.11% on the Jiangnan University (JNU) bearing dataset, surpassing eight other algorithms in bearing fault diagnosis.

Nuclear norm regularized loop optimization for tensor network

We propose a loop optimization algorithm based on nuclear norm regularization for the tensor network. The key ingredient of this scheme is to introduce a rank penalty term proposed in the context of data processing. Compared to the standard variational periodic matrix product states method, this algorithm can circumvent the local minima related to short-ranged correlation in a simpler fashion. We demonstrate its performance when used as a part of the tensor network renormalization algorithms [S. Yang, Z.-C. Gu, and X.-G. Wen, ] for the critical two-dimensional Ising model. The scale invariance of the renormalized tensors is attained with higher accuracy while the higher parts of the scaling dimension spectrum are obtained in a more stable fashion. Published by the American Physical Society 2024

Improving long‐tail classification via decoupling and regularisation

AbstractReal‐world data always exhibit an imbalanced and long‐tailed distribution, which leads to poor performance for neural network‐based classification. Existing methods mainly tackle this problem by reweighting the loss function or rebalancing the classifier. However, one crucial aspect overlooked by previous research studies is the imbalanced feature space problem caused by the imbalanced angle distribution. In this paper, the authors shed light on the significance of the angle distribution in achieving a balanced feature space, which is essential for improving model performance under long‐tailed distributions. Nevertheless, it is challenging to effectively balance both the classifier norms and angle distribution due to problems such as the low feature norm. To tackle these challenges, the authors first thoroughly analyse the classifier and feature space by decoupling the classification logits into three key components: classifier norm (i.e. the magnitude of the classifier vector), feature norm (i.e. the magnitude of the feature vector), and cosine similarity between the classifier vector and feature vector. In this way, the authors analyse the change of each component in the training process and reveal three critical problems that should be solved, that is, the imbalanced angle distribution, the lack of feature discrimination, and the low feature norm. Drawing from this analysis, the authors propose a novel loss function that incorporates hyperspherical uniformity, additive angular margin, and feature norm regularisation. Each component of the loss function addresses a specific problem and synergistically contributes to achieving a balanced classifier and feature space. The authors conduct extensive experiments on three popular benchmark datasets including CIFAR‐10/100‐LT, ImageNet‐LT, and iNaturalist 2018. The experimental results demonstrate that the authors’ loss function outperforms several previous state‐of‐the‐art methods in addressing the challenges posed by imbalanced and long‐tailed datasets, that is, by improving upon the best‐performing baselines on CIFAR‐100‐LT by 1.34, 1.41, 1.41 and 1.33, respectively.

A tensor completion method based on tensor QR decomposition with truncated nuclear norm and sparse regularization

A tensor completion method based on tensor QR decomposition with truncated nuclear norm and sparse regularization

NeRSI: Neural Implicit Representations for 5D Seismic Data Interpolation

Due to challenging field operations and resource constraints, seismic data acquisition often requires coping with missing traces. Interpolation algorithms are crucial for reconstructing these missing traces to enable improved subsurface analysis and interpretation. While deep learning has made exciting advances in seismic reconstruction, its focus has predominantly been on 2D and 3D datasets with relatively low rates of missing data. Five-dimensional (5D) seismic data reconstruction entails considering simultaneous sources and receivers deployed in areal arrays to solve the reconstruction problem. The latter offers greater data redundancy, which can be leveraged to enhance interpolation quality. Traditional 5D deep learning interpolation methods rely heavily on synthetic training pairs, posing challenges when applied to real-world data. This necessitates transfer learning techniques, which can be cumbersome. Addressing this, we introduce a self-supervised, coordinate-based deep interpolation algorithm that obviates the need for labeled data. Employing a multi-layer perceptron (MLP) network can effectively encode the continuous seismic wavefield inherent in 5D data. Once trained, the MLP can infer missing trace amplitudes from their coordinates. We contribute to boosting the MLP, enabling it to generate seismic profiles rather than single-point predictions. This enhancement significantly strengthens the model’s performance and efficiency. Moreover, we apply nuclear norm regularization to the output profiles, improving the reconstruction quality. The effectiveness of our algorithm is supported by both synthetic and field data experiments, demonstrating its superior reconstruction capabilities.

Enhancing SONAR performance through statistically-informed sub-array processing

Ocean fluctuations affect acoustic propagation and reduce the performance of SONAR arrays. In particular, internal waves can lead to a loss of coherence in incident signals, reducing the efficiency of classical processing. Precise knowledge of the environment is necessary to predict the degradation it causes. However, it is impossible to know the exact state of the ocean, it is therefore necessary to use statistical methods to study the ocean and acoustic propagation. In this work, statistical method, canonical correlation analysis (CCA), is used to find linear relationships between acoustic variables of interest and oceanographic measurements. In particular, the coherence radius is inferred from empirical modes of temperature using the model learned by CCA. A sub-array processing scheme can then be informed by defining the length of the sub-arrays from the coherence radius. To compensate for the loss of angular resolution inherent in sub-arrays, we introduce the use of a Bayesian algorithm exploiting l0 norm regularization. We show on experimental data from the ALMA 2017 campaign that the proposed processing improves detection when signal coherence is low.

Approximate Observation Weighted ℓ2/3 SAR Imaging under Compressed Sensing.

Compressed Sensing SAR Imaging is based on an accurate observation matrix. As the observed scene enlarges, the resource consumption of the method increases exponentially. In this paper, we propose a weighted ℓ2/3-norm regularization SAR imaging method based on approximate observation. Initially, to address the issues brought by the precise observation model, we employ an approximate observation operator based on the Chirp Scaling Algorithm as a substitute. Existing approximate observation models typically utilize ℓq(q = 1, 1/2)-norm regularization for sparse constraints in imaging. However, these models are not sufficiently effective in terms of sparsity and imaging detail. Finally, to overcome the aforementioned issues, we apply ℓ2/3 regularization, which aligns with the natural image gradient distribution, and further constrain it using a weighted matrix. This method enhances the sparsity of the algorithm and balances the detail insufficiency caused by the penalty term. Experimental results demonstrate the excellent performance of the proposed method.

Double-Structured Sparsity Guided Flexible Embedding Learning for Unsupervised Feature Selection.

In this article, we propose a novel unsupervised feature selection model combined with clustering, named double-structured sparsity guided flexible embedding learning (DSFEL) for unsupervised feature selection. DSFEL includes a module for learning a block-diagonal structural sparse graph that represents the clustering structure and another module for learning a completely row-sparse projection matrix using the l2,0 -norm constraint to select distinctive features. Compared with the commonly used l2,1 -norm regularization term, the l2,0 -norm constraint can avoid the drawbacks of sparsity limitation and parameter tuning. The optimization of the l2,0 -norm constraint problem, which is a nonconvex and nonsmooth problem, is a formidable challenge, and previous optimization algorithms have only been able to provide approximate solutions. In order to address this issue, this article proposes an efficient optimization strategy that yields a closed-form solution. Eventually, through comprehensive experimentation on nine real-world datasets, it is demonstrated that the proposed method outperforms existing state-of-the-art unsupervised feature selection methods.

Balance Sparse Decomposition Method with Nonconvex Regularization for Gearbox Fault Diagnosis

As an important part of rotating machinery, gearboxes often fail due to their complex working conditions and harsh working environment. Therefore, it is very necessary to effectively extract the fault features of the gearboxes. Gearbox fault signals usually contain multiple characteristic components and are accompanied by strong noise interference. Traditional sparse modeling methods are based on synthesis models, and there are few studies on analysis and balance models. In this paper, a balance nonconvex regularized sparse decomposition method is proposed, which based on a balance model and an arctangent nonconvex penalty function. The sparse dictionary is constructed by using Tunable Q-Factor Wavelet Transform (TQWT) that satisfies the tight frame condition, which can achieve efficient and fast solution. It is optimized and solved by alternating direction method of multipliers (ADMM) algorithm, and the non-convex regularized sparse decomposition algorithm of synthetic and analytical models are given. Through simulation experiments, the determination methods of regularization parameters and balance parameters are given, and compared with the L1 norm regularization sparse decomposition method under the three models. Simulation analysis and engineering experimental signal analysis verify the effectiveness and superiority of the proposed method.

Matrix Completion When Missing Is Not at Random and Its Applications in Causal Panel Data Models

This article develops an inferential framework for matrix completion when missing is not at random and without the requirement of strong signals. Our development is based on the observation that if the number of missing entries is small enough compared to the panel size, then they can be estimated well even when missing is not at random. Taking advantage of this fact, we divide the missing entries into smaller groups and estimate each group via nuclear norm regularization. In addition, we show that with appropriate debiasing, our proposed estimate is asymptotically normal even for fairly weak signals. Our work is motivated by recent research on the Tick Size Pilot Program, an experiment conducted by the Security and Exchange Commission (SEC) to evaluate the impact of widening the tick size on the market quality of stocks from 2016 to 2018. While previous studies were based on traditional regression or difference-in-difference methods by assuming that the treatment effect is invariant with respect to time and unit, our analyses suggest significant heterogeneity across units and intriguing dynamics over time during the pilot program. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

A Novel Truncated Norm Regularization Method for Multi-Channel Color Image Denoising

A Novel Truncated Norm Regularization Method for Multi-Channel Color Image Denoising

TAKAGI–SUGENO–KANG FUZZY SYSTEM MODELING BASED ON LOW-RANK SPARSE SUBSPACE LEARNING FOR MOTOR IMAGERY ELECTROENCEPHALOGRAM SIGNAL CLASSIFICATION

The classification of electroencephalogram (EEG) signals derived from motor imagery (MI) has always been a hot topic in the field of brain–computer interfaces. Due to its ability to handle the nonstationary and uncertain information contained in EEG signals, the Takagi–Sugeno–Kang fuzzy system (TSK-FS) has become an advantageous classification algorithm. To train a fuzzy system with strong discrimination capabilities from EEG data interspersed with redundant information, this paper proposes a TSK-FS modeling method based on low-rank sparse subspace learning (TSK-LSSL). This method focuses on consequent parameter learning, which transforms the traditional consequent parameter learning strategy into low-rank subspace and sparse subspace learning processes. Low-rank subspace learning is used to mine the global structural information of data and effectively reduce the number of fuzzy rules. During sparse subspace learning, [Formula: see text]-norm regularization is used to constrain the consequent parameters and causes the number of redundant consequent parameters to be zero, thereby simplifying the fuzzy rules. In addition, a local boundary term based on graph matrices is embedded into the objective function to mine the local structural information of the given data. TSK-LSSL simplifies the number of rules and the consequent part of the fuzzy rules. It exhibits good classification performance on two BCI Competition databases.

Kernel-free Reduced Quadratic Surface Support Vector Machine with 0-1 Loss Function and L$$_p$$-norm Regularization

Kernel-free Reduced Quadratic Surface Support Vector Machine with 0-1 Loss Function and L$$_p$$-norm Regularization

Drive-By Identification of Joint Bridge Damage and Surface Roughness Based on Sensitivity Analysis of Residual Contact Point Deflections

Drive-by inspection of bridge damage using a passing vehicle’s dynamic response has great potential in bridge damage identification. Among the current drive-by methodologies, the methods based on bridge modeling have been proposed for the quantitative identification of bridge damage and bridge surface roughness. However, the coupling of bridge damage and unknown surface roughness increases the difficulty of the identification problem and most previous studies conduct the identification task of bridge damage or bridge surface roughness separately. In this paper, a novel method is proposed for the identification of joint bridge damage and bridge surface roughness based on the residual bridge deflections from the front and rear wheels contact points of an instrumented passing vehicle. The vehicle–bridge interaction is analyzed using a half-car model of a four degrees of freedom (4-DOF) with front and rear vehicle wheels. First, the vehicle–bridge unknown forces and displacement of the vehicle axles are identified by the generalized Kalman filter under unknown input (GKF-UI) proposed by the authors. The sensors can be installed conveniently on the vehicle body due to GKF-UI. Then, the residual bridge deflections from the front and rear vehicle wheels contact points are utilized to eliminate the effect of road surface roughness. Afterward, bridge structural damage is identified based on the sensitivity analysis of residual bridge deflections from the front and rear contact points with [Formula: see text]1-norm regularization. Finally, bridge road surface roughness is estimated based on the identified unknown forces and updated bridge model with damage. The results of numerical identification examples demonstrate the effectiveness of the proposed method for the identification of joint bridge damage and surface roughness.

Robust unrolled network for lensless imaging with enhanced resistance to model mismatch and noise

Lensless imaging has gained popularity in various applications due to its user-friendly nature, cost-effectiveness, and compact design. However, achieving high-quality image reconstruction within this framework remains a significant challenge. Lensless imaging measurements are associated with distinct point spread functions (PSFs), resulting in many PSFs introducing artifacts into the underlying physical model. This discrepancy between the actual and prior models poses challenges for standard reconstruction methods to effectively address high-quality image reconstruction by solving a regularization-based inverse problem. To overcome these issues, we propose MN-FISTA-Net, an unrolled neural network that unfolds the fast iterative shrinkage/thresholding algorithm for solving mixed norm regularization with a deep denoiser prior. Our method enhances mask-based lensless imaging performance by efficiently addressing noise and model mismatch, as evidenced by significant improvements in image quality compared to existing approaches.

Impulsive Noise Suppression Based on ℓq -norm and Graph Laplacian Regularization for NOMA System

Impulsive Noise Suppression Based on <i>ℓ_q </i>-norm and Graph Laplacian Regularization for NOMA System

Learning latent disentangled embeddings and graphs for multi-view clustering

Graph based methods have recently attracted much attention for multi-view clustering. Most existing methods seek the latent shared embeddings to learn a unified similarity graph or fuse multiple view-specific graphs to a consensus one for clustering, which may not sufficiently explore the common and complementary information among views. Besides, the high-order inter-view correlations are not fully investigated. To address these issues, this paper proposes a latent Disentangled Embeddings and GRaphs based multi-viEw clustEring (DEGREE) method, which considers the common and view-specific information in a latent subspace by explicit embedding disentanglement and multiple graphs learning. We assume that each view can be generated from a shared latent embedding and a corresponding view-specific embedding, which model the common information and exclusive complementary information among views, respectively. The intra-view and inter-view exclusivities among embeddings are encouraged by an orthogonality regularizer. To fully use the underlying information, we excavate the pairwise instance relations in both shared embedding and diverse view-specific embeddings by learning multiple graphs. Besides, a tensor singular value decomposition (t-SVD) based tensor nuclear norm regularizer is imposed on view-specific graphs, which helps to explore the high-order inter-view correlations. An alternative optimization algorithm is designed to solve the proposed model. Experimental evaluations on several popular datasets demonstrate that our DEGREE method outperforms the state-of-the-art methods.

Minimization of peak stresses with the shape derivative.

This article is concerned with the minimization of peak stresses occurring in linear elasticity. We propose to minimize the maximal von Mises stress of the elastic body. This leads to a non-smooth shape functional. We derive the shape derivative and associate it with the Clarke sub-differential. Using a steepest descent algorithm, we present numerical simulations. We compare our results to the usual [Formula: see text]-norm regularization and show that our algorithm performs better in the presented tests.This article is part of the theme issue 'Non-smooth variational problems with applications in mechanics'.

SA-UCBSS: Sparsity-Based Adaptive Underdetermined Convolutive Blind Source Separation

Blind source separation is the process of separating unknown source signals based only on the received mixing signals when the source signal cannot be accurately known. Traditional methods are mainly based on linear approximate transformation models that often have large systematic errors in high reverberation environments, resulting in poor separation performance. To avoid these limitations, a sparsity-based adaptive underdetermined convolutive blind source separation (SA-UCBSS) algorithm is proposed by unifying a convolutive approximate transformation model and an Lp(0<p≤1)-norm regularisation penalty. A crosstalk canceller is designed for room impulse response pretreatment with p-norm optimisation to eliminate audible echoes, and a sparse optimisation scheme is designed for the parameter initialisation to accelerate the convergence of the iterative recovery process. Subsequently, a convolutive approximate transformation model is built to achieve a more accurate estimation of the mixing system. The SA-UCBSS algorithm is proposed to achieve better source separation by adaptively tuning the value of parameter p. Experimental results including BSS of the synthesised and live-recorded speech convolutive mixtures showed that the SA-UCBSS algorithm had superior separating properties and better robustness than several popular algorithms.

Total Third-Degree Variation for Noise Reduction in Atomic-Resolution STEM Images.

Scanning Transmission Electron Microscopy (STEM) enables direct determination of atomic arrangements in materials and devices. However, materials such as battery components are weak for electron beam irradiation and low electron doses are required to prevent beam-induced damages. Noise removal is thus essential for precise structural analysis of electron beam sensitive materials at atomic resolution. Total square variation (TSV) regularization is an algorithm that exhibits high noise removal performance. However, the use of the TSV regularization term leads to significant image blurring and intensity reduction. To address these problems, we here propose a new approach adopting L2 norm regularization based on higher-order total variation. An atomic-resolution STEM image can be approximated as a set of smooth curves represented by quadratic functions. Since the third-degree derivative of any quadratic function is 0, total third-degree variation (TTDV) is suitable for a regularization term. The application of TTDV for denoising the atomic-resolution STEM image of CaF2 observed along the [001] zone axis is shown, where we can clearly see the Ca and F atomic columns without compromising image quality.