L1 Optimization Research Articles

A hybrid approach for reconstruction of transonic buffet aerodynamic noise: Integrating random forest and compressive sensing algorithm

Experimental measurements and numerical simulation methods for obtaining aerodynamic noise face issues such as high costs and long periods. A single machine learning method for predicting aerodynamic noise also requires a sufficient amount of data. According to this, this paper proposes a hybrid method integrating Random Forest and Compressive Sensing (RF_CS) to accurately reconstruct transonic buffet aerodynamic noise from sparse data. First, the RF algorithm, known for its strong nonlinear feature extraction capabilities, is used to obtain the basis function. Then, the basis coefficients are calculated using the L1 optimization algorithm based on limited sensor data and basis functions. Finally, a linear combination of basis functions and basis coefficients is used to reconstruct aerodynamic noise, including power spectral density, sound pressure level, and flow modes. Compared to the Compressive Sensing based on Proper Orthogonal Decomposition (POD_CS), the proposed algorithm can effectively reduce error by approximately 2–20 times and decrease the absolute error of modes by about 2–3 orders of magnitude. Specifically, the RF_CS method ensures that the reconstruction errors for power spectral density across various flow conditions are all below 3E-3, achieving generalization from one flow condition to the entire sample space. Additionally, this approach can utilize approximately 10 sensors to reconstruct accurate sound pressure level and modes, with errors within 5E-3 and 5E-5, respectively. This allows for generalization across the entire Mach number space based on a single Mach number condition.

Channel Estimation for Underwater Acoustic Communications in Impulsive Noise Environments: A Sparse, Robust, and Efficient Alternating Direction Method of Multipliers-Based Approach

Channel estimation in Underwater Acoustic Communication (UAC) faces significant challenges due to the non-Gaussian, impulsive noise in ocean environments and the inherent high dimensionality of the estimation task. This paper introduces a robust channel estimation algorithm by solving an l1−l1 optimization problem via the Alternating Direction Method of Multipliers (ADMM), effectively exploiting channel sparsity and addressing impulsive noise outliers. A non-monotone backtracking line search strategy is also developed to improve the convergence behavior. The proposed algorithm is low in complexity and has robust performance. Simulation results show that it exhibits a small performance deterioration of less than 1 dB for Channel Impulse Response (CIR) estimation in impulsive noise environments, nearly matching its performance under Additive White Gaussian Noise (AWGN) conditions. For Delay-Doppler (DD) doubly spread channel estimation, it maintains Bit Error Rate (BER) performance comparable to using ground truth channel information in both AWGN and impulsive noise environments. At-sea experimental validations for channel estimation in Orthogonal Frequency Division Multiplexing (OFDM) systems further underscore the fast convergence speed and high estimation accuracy of the proposed method.

Aberration pre-correction for a simple optical system of HMDs.

Head-mounted displays (HMDs) are becoming increasingly popular as a crucial component of virtual reality (VR). However, contemporary HMDs enforce a simple optical structure due to their constrained form factor, which impedes the use of multiple lens elements that can reduce aberrations in general. As a result, they introduce severe aberrations and imperfections in optical imagery, causing visual fatigue and degrading the immersive experience of being present in VR. To address this issue without modifying the hardware system, we present a novel, to the best of our knowledge, software-driven approach that compensates for the aberrations in HMDs in real time. Our approach involves pre-correction that deconvolves an input image to minimize the difference between its after-lens image and the ideal image. We characterize the specific wavefront aberration and point spread function (PSF) of the optical system using Zernike polynomials. To achieve higher computational efficiency, we improve the conventional deconvolution based on hyper-Laplacian prior by adopting a regularization constraint term based on L2 optimization and the input-image gradient. Furthermore, we implement our solution entirely on a graphics processing unit (GPU) to ensure constant and scalable real-time performance for interactive VR. Our experiments evaluating our algorithm demonstrate that our solution can reliably reduce the aberration of the after-lens images in real time.

Function approximations via l1 - l2 optimization

Function approximations via l1 - l2 optimization

A machine learning approach for computation of cardiovascular intrinsic frequencies.

Analysis of cardiovascular waveforms provides valuable clinical information about the state of health and disease. The intrinsic frequency (IF) method is a recently introduced framework that uses a single arterial pressure waveform to extract physiologically relevant information about the cardiovascular system. The clinical usefulness and physiological accuracy of the IF method have been well-established via several preclinical and clinical studies. However, the computational complexity of the current L2 optimization solver for IF calculations remains a bottleneck for practical deployment of the IF method in real-time settings. In this paper, we propose a machine learning (ML)-based methodology for determination of IF parameters from a single carotid waveform. We use a sequentially-reduced Feedforward Neural Network (FNN) model for mapping carotid waveforms to the output parameters of the IF method, thereby avoiding the non-convex L2 minimization problem arising from the conventional IF approach. Our methodology also includes procedures for data pre-processing, model training, and model evaluation. In our model development, we used both clinical and synthetic waveforms. Our clinical database is composed of carotid waveforms from two different sources: the Huntington Medical Research Institutes (HMRI) iPhone Heart Study and the Framingham Heart Study (FHS). In the HMRI and FHS clinical studies, various device platforms such as piezoelectric tonometry, optical tonometry (Vivio), and an iPhone camera were used to measure arterial waveforms. Our blind clinical test shows very strong correlations between IF parameters computed from the FNN-based method and those computed from the standard L2 optimization-based method (i.e., R≥0.93 and P-value ≤0.005 for each IF parameter). Our results also demonstrate that the performance of the FNN-based IF model introduced in this work is independent of measurement apparatus and of device sampling rate.

A novel method for noise reduction of blade tip timing signals based on sparse representation and dictionary learning

Blade Tip Timing Technology (BTT) is a new type of blade vibration measurement method with non-contact measurement capabilities, which is associated with high efficiency and convenience. However, due to under-sampling feature of the BTT signal, the method can only measure part of the vibration information. Therefore, various BTT spectral reconstruction algorithms have been developed based on the under-sampling feature and sparsity of the blade vibration spectrum. Because of the harsh working environment of the blades, BTT signal typically contains high-level Gaussian noise in actual measurement, which significantly reduces frequency-domain sparsity and even aliases the vibration modals, resulting in a decrease in the performance of sparse reconstruction algorithms. In this paper, a multi-sequences BTT signal noise reduction method is proposed, which is based on sparse representation to suppress Gaussian noise. Through appropriately modifying the constraint in the ℓ0 optimization problem, the method filters out the noise elements in the sparse vector of the BTT signal. Then, denoising is completed by sparse inverse transformation. Finally, a global average-based singular value decomposition dictionary learning algorithm (GA-K-SVD) is proposed to generate an over-complete sparse dictionary which is adaptable to the original signal, to increase the effectiveness of GA-K-SVD. At last, simulation and experience are carried out to verify the performance optimization effect for sparse reconstruction algorithm and the effectiveness of proposed noise reduction method.

An inexact quasi-Newton algorithm for large-scale ℓ1 optimization with box constraints

In this paper, we develop an inexact quasi-Newton algorithm for ℓ1-regularization optimization problems subject to box constraints. The algorithm uses the identification technique of the proximal gradient algorithm to estimate the active set and free variables. To accelerate the convergence, we utilize the inexact quasi-Newton algorithm to update free variables. Under certain conditions, we show that the sequence generated by the algorithm converges R-linearly to a first-order optimality point of the problem. Moreover, the corresponding sequence of objective function values is also linearly convergent. Experiment results demonstrate the competitiveness of the proposed algorithm.

Efficient Distributed Inference of Deep Neural Networks via Restructuring and Pruning

In this paper, we consider the parallel implementation of an already-trained deep model on multiple processing nodes (a.k.a. workers). Specifically, we investigate as to how a deep model should be divided into several parallel sub-models, each of which is executed efficiently by a worker. Since latency due to synchronization and data transfer among workers negatively impacts the performance of the parallel implementation, it is desirable to have minimum interdependency among parallel sub-models. To achieve this goal, we propose to rearrange the neurons in the neural network, partition them (without changing the general topology of the neural network), and modify the weights such that the interdependency among sub-models is minimized under the computations and communications constraints of the workers while minimizing its impact on the performance of the model. We propose RePurpose, a layer-wise model restructuring and pruning technique that guarantees the performance of the overall parallelized model. To efficiently apply RePurpose, we propose an approach based on L0 optimization and the Munkres assignment algorithm. We show that, compared to the existing methods, RePurpose significantly improves the efficiency of the distributed inference via parallel implementation, both in terms of communication and computational complexity.

Sparse subsampling of flow measurements for finite-time Lyapunov exponent in domains with obstacles

We propose an efficient approach to estimate the finite-time Lyapunov exponent (FTLE). Instead of incorporating all available velocity measurements, we develop a sparse subsampling approach to detect relevant flow measurements for velocity reconstruction. This work has two main contributions. We first extend our previous algorithm in Ng and Leung (2019) to reconstruct a flow field with an impermeable condition. To solve the corresponding under-determined system for reconstructing the global velocity field, we propose a L1 optimization framework that can lead to a sparse reconstruction algorithm. Therefore, the overall algorithm can automatically identify and extract a sparse subset of velocity measurements for the FTLE computations. We will provide synthetic and real-life numerical examples to demonstrate the effectiveness of the proposed algorithm.

Estimation and fault diagnosis for non‐linear system with time‐varying faults and measurement noises: Application on two CSTRs in series

AbstractA continuously stirred tank reactor (CSTR) is largely used in water treatment and in chemical and biological processes. It is characterized by a complex non‐linear behaviour. Operating large reactors in industry can be expensive, so a common trick used to reduce costs is to operate multiple CSTRs in series. Consequently, the CSTR is usually exposed to faults and noises. This paper addresses the design of a robust observer for estimation and fault diagnosis strategy on two CSTRs in series. The considered system is affected simultaneously by time‐varying actuator and sensor faults with measurement noises. The Takagi‐Sugeno multimodel approach is proposed to transform the non‐linear model into an interpolation of several linear sub‐models with non‐measurable premise variables. The purpose of this brief is to provide the state and the fault estimation for the considered system using a proportional multiple integral (PMI) unknown input observer. The exponential stability conditions are studied with the Lyapunov theory and L2 optimization and formulated in terms of linear matrix inequalities. In addition, a comparative study with a PMI observer is conducted. Finally, the proposed observer is used for time‐varying fault detection and isolation.

Compressed sensing based fingerprint imaging system using a chaotic model-based deterministic sensing matrix

A secured compressed sensing (CS) systems design approach uses a novel deterministic sensing matrix to sense and transmit fingerprint images. The performance of the CS system was studied in detail by varying CS and security parameters. The sampling and sparse coefficient are the parameters considered from compressed sensing, whereas the encryption key is from the security scheme. The simultaneous compression and encryption has been achieved by multiplying the sparse modeled data with the proposed deterministic partial bounded orthogonal sensing matrix. A chaotic model-based permutation is applied to scramble the DCT matrix rows to build the sensing matrix. Recovering and decryption of the compressed image are accomplished with the help of the L1 optimization method. The experimental test shows that a sparse vector of 121 widths has been recovered by taking about 25 samples. This indicates that up to 1 : 5 compression ratio is supported without damaging the fingerprint minutiae. If only compression is required without encryption, up to a 1 : 16 ratio can be achieved. The peak signal-to-noise ratio (PSNR) is 27.65 dB for both compression ratios under fulfilments of all necessary security requirements. The 7.20 value of the entropy, histogram analysis, and the correlation analysis show the proposed scheme possesses adequate randomness. Furthermore, the ability of the system resistance against attacks is proved by 100% NPCR (Net Pixel Change Rate) and 0.92% UACI (Unified Average Changing Intensity) values.

Robust three-phase state estimation for PV-Integrated unbalanced distribution systems

As the increasing penetration of renewable energy brings uncertainty and fluctuation into the operating conditions of distribution systems, distribution system state estimation (DSSE) needs to track not only the state of the network, but also the state of the renewable energy sources robustly and under unbalanced conditions. Aiming at delivering comprehensive situational awareness of distribution systems hosting solar photovoltaic (PV) power plants, this paper proposed a joint state estimation model for unbalanced distribution systems integrated with single-phase and three-phase PV power plants, as well as an extended weighted least absolute value (EWLAV) estimation algorithm for handling simultaneous measurement errors and control input errors. The proposed state estimation model considers various loss components of single-phase and three-phase power electronic converters, and is applicable to general unbalanced conditions including unbalanced phasing and parameters of distribution lines, PV power plants, and loads. The proposed EWLAV algorithm leverages sparse l1 optimization principles and can identify and suppress the gross errors in both measurements and converter control inputs. Simulation results show that the developed model can reliably reflect the characteristics of three-phase unbalanced distribution networks and PV plants, and the EWLAV algorithm produces unbiased results under simultaneous measurement errors and control input errors, which conventional estimators such as the weighted least squares (WLS) and weighted least absolute values (WLAV) cannot achieve.

On-the-fly compressive single-pixel foveation using the STOne transform

Compressive imaging allows one to sample an image below the Nyquist rate yet still accurately recover it from the measurements by solving an L1 optimization problem. The L1 solvers, however, are iterative and can require significant time to reconstruct the original signal. Intuitively, the reconstruction time can be reduced by reconstructing fewer total pixels. The human eye reduces the total amount of data it processes by having a spatially varying resolution, a method called foveation. In this work, we use foveation to achieve a 4x improvement in L1 compressive sensing reconstruction speed for hyperspectral images and video. Unlike previous works, the presented technique allows the high-resolution region to be placed anywhere in the scene after the subsampled measurements have been acquired, has no moving parts, and is entirely non-adaptive.

Feedback Stabilization Robust Against Communication Constraints for Disturbed Linear Plants via Sparse Packetized Predictive Control

This paper investigates closed-loop stability of linear discrete-time plants subject to bounded disturbances when controlled according to packetized predictive control (PPC) policies. In the considered feedback loop, the controller is connected to the actuator via a digital communication channel imposing bounded dropouts. Two PPC strategies are taken into account. In both cases, the control packets are generated by solving sparsity-promoting optimization problems. One is based upon an l2-constrained £° optimization problem. Such problem is relaxed by an l1- l2 optimization problem in the other sparse PPC setting. We show that the l2-constrained £° sparse PPC and unconstrained l1- l2 sparse PPC render system states bounded if the design parameters satisfy certain conditions. The bounds we derive on states are increasing with respect to the disturbance magnitude. We illustrate the results via simulation.

Fast interval estimation for discrete-time linear systems: An [formula omitted] optimization method

This paper studies interval estimation for discrete-time linear systems with unknown but bounded disturbances. Inspired by the parity space approach, we propose a point estimator with fixed-time convergence property. The estimator is combined with the zonotope-based interval analysis to achieve fast interval estimation. The parameter matrix in the estimator is optimized by minimizing the length of the edges of the outer box of the error zonotope. It is formulated as L1 optimization problem and can be efficiently solved by linear programming. Comparison studies illustrate the superiority of the proposed method over existing techniques.

An Adaptive Observer Design for Nonlinear Systems Affected by Unknown Disturbance with Simultaneous Actuator and Sensor Faults. Application to a CSTR

Continuous Stirred Tank Reactor (CSTR) is an important system in the chemical and biological industries. It's characterized by a complex nonlinear behavior and is usually affected by faults and disturbances. Therefore, the states and faults estimation of a CSTR is always a challenging task for automated process researchers and engineers. This paper proposes an adaptive observer. This paper proposes an adaptive observer in order to estimate states and actuator and sensor faults simultaneously under unknown disturbance. Firstly, the approach of the Takagi-Sugeno multi-model is proposed to transform the complex nonlinear model into several simple linear sub-models. However, the states of the considered isotherm CSTR are not completely measurable, so the multi-model is represented with non-measurable premise variables. Then, in order to transform the considered system into a system with an unknown input, a mathematical transformation is introduced to describe the sensor faults as actuator faults. The proposed observer is designed, and the exponential stability conditions are studied with the Lyapunov theory and L2 optimization and formulated in terms of linear matrix inequalities. Finally, to improve the effectiveness of the proposed observer, a numerical simulation is carried out on a CSTR.

An L0-regularized global anisotropic gradient prior for single-image de-raining

We propose a method based on a global anisotropic gradient prior (GAGP) for addressing the problem of rain streak removal. We observe that both images and rain streaks have anisotropy at edges, and that the rain distribution in images is sparse. We therefore propose an L0-regularized sparse term with an L0-regularized GAGP term to efficiently detect the rain streaks. We applied the L0-regularized GAGP as the image regularization term to protect the image structure and details. We also developed an alternating half-quadratic algorithm to solve the proposed L0 optimization model by introducing the variable splitting method and analyzing the convergence of the algorithm. Experimental results show that the proposed method outperforms state-of-the-art methods in rain streak removal and preserving image structure on both synthesized and real-world images with different levels and types of rain.

Recent advances in knowledge‐based model structure optimization and extrapolation techniques for microwave applications

AbstractArtificial neural network modeling techniques have been recognized as important vehicles in the microwave computer‐aided design (CAD) area in addressing the growing challenges of designing next generation microwave device, circuits, and systems. This article provides an overview of recent advances in knowledge‐based neural network model generation and extrapolation techniques for microwave applications. We first introduce the unified knowledge‐based neural network structure optimization technique. Using the distinctive property for feature selection of l1 optimization, this unified modeling technique efficiently determines the type and topology of the mapping structure in a knowledge‐based model. This knowledge‐based model structure optimization technique is more flexible and systematic, and can further speed up the knowledge‐based neural model development. As a further advancement, we also discuss the advanced multi‐dimensional extrapolation technique for neural‐based microwave modeling. The purpose is to make the neural network model can be reliably used not only inside the training range but also outside the training range. Multi‐dimensional cubic polynomial extrapolation formulation and optimization over grids outside the training range are utilized to make neural models more robust and reliable when they are used outside the training range. The validity of these techniques is demonstrated by microwave modeling examples.

Sea state estimation based on vessel motion responses: Improved smoothness and robustness using Bézier surface and L1 optimization

Floating structures oscillate in waves, where these wave-induced motions may be critical for various marine operations. An important consideration is thereby given to the sea states at the planning and operating stages for an offshore project. The most important information extracted from a sea state is the directional wave spectrum, indicating wave direction, significant wave height, and wave spectrum peak period. Among several available methods of measuring and estimating the directional wave spectrum, the wave buoy analogy technique based on vessel motion responses is an in situ and almost real-time solution without extra costs of devices. If the forms of the wave spectra are not predefined in the estimation, the method is called a nonparametric approach. Its most remarkable advantage is the flexible form, but the smoothness should be regulated. After the discrete Fourier transform has been applied to the measured vessel motions, smoothing is necessary. However, this process results in disturbed vessel cross-spectra and a lowpass characteristic of the windowing function. This paper presents a nonparametric approach for directional wave spectrum estimation based on vessel motion responses. It introduces novel smoothness constraints using Bézier surface and includes a more robust estimate using L1 optimization. Both techniques are applied to the wave buoy analogy for the first time. Numerical simulations are conducted to verify the proposed algorithm.

Direct Least Absolute Deviation Fitting of Ellipses

Scattered data from edge detection usually involve undesired noise which seriously affects the accuracy of ellipse fitting. In order to alleviate this kind of degradation, a method of direct least absolute deviation ellipse fitting by minimizing the ℓ1 algebraic distance is presented. Unlike the conventional ℓ2 estimators which tend to produce a satisfied performance on ideal and Gaussian noise data, while do a poor job for non-Gaussian outliers, the proposed method shows very competitive results for non-Gaussian noise. In addition, an efficient numerical algorithm based on the split Bregman iteration is developed to solve the resulting ℓ1 optimization problem, according to which the computational burden is significantly reduced. Furthermore, two classes of ℓ2 solutions are introduced as the initial guess, and the selection of algorithm parameters is studied in detail; thus, it does not suffer from the convergence issues due to poor initialization which is a common drawback existing in iterative-based approaches. Numerical experiments reveal that the proposed method is superior to its ℓ2 counterpart and outperforms some of the state-of-the-art algorithms for both Gaussian and non-Gaussian artifacts.