L1-norm Minimization Research Articles

Automatic and robust segmentation of endoscopic OCT images and optical staining.

We report a generic method for automatic segmentation of endoscopic optical coherence tomography (OCT) images. In this method, OCT images are first processed with L1 -L0 norm minimization based de-noising and smoothing algorithms to increase the signal-to-noise ratio (SNR) and enhance the contrast between adjacent layers. The smoothed images are then formulated into cost graphs based on their vertical gradients. After that, tissue-layer segmentation is performed with the shortest path search algorithm. The efficacy and capability of this method are demonstrated by automatically and robustly identifying all five interested layers of guinea pig esophagus from in vivo endoscopic OCT images. Furthermore, thanks to the ultrahigh resolution, high SNR of endoscopic OCT images and the high segmentation accuracy, this method permits in vivo optical staining histology and facilitates quantitative analysis of tissue geometric properties, which can be very useful for studying tissue pathologies and potentially aiding clinical diagnosis in real time.

Improved Ultrasonic Computerized Tomography Method for STS (Steel Tube Slab) Structure Based on Compressive Sampling Algorithm

This paper developed a new ultrasonic computerized tomography (CT) method for damage inspections of a steel tube slab (STS) structure based on compressive sampling (CS). CS is a mathematic theory providing an approximate recovery for a sparse signal with minimal reconstruction error from under-sampled measurements. Considering the natural sparsity of the damage, CS algorithm is employed to image the defect in the concrete-filled steel tube of Shenyang Metro line 9 for reducing the work time. Thus, in the measurement stage, far fewer ultrasonic measurement paths were selected from the dense net of conventional ultrasonic CT techniques to capture the underlying damage information. Then, in the imaging stage, ℓ1-norm minimization algorithm of CS theory is selected to recover the internal damage via fusing measurement data and solving optimization problem. The functionality of the proposed method is validated by three numerical concrete tube models with various conditions. Additionally, both the conventional ultrasonic CT technique and the proposed one are employed for ultrasonic inspection of the STS structure in Shenyang Metro line 9. Both the numerical and experimental results indicate that the proposed ultrasonic CT improved by CS has a great potential for damage detection, which provides an alternative accurate and effective way for non-destructive testing/evaluation (NDT/E).

An antinoise sparse representation method for robust face recognition via joint l1 and l2 regularization

Sparse representation methods based on l1 and/or l2 regularization have shown promising performance in different applications. Previous studies show that the l1 regularization based representation has more sparse property, while the l2 regularization based representation is much simpler and faster. However, when dealing with noisy data, both naive l1 and l2 regularization suffer from the issue of unsatisfactory robustness. In this paper, we explore the method to implement an antinoise sparse representation method for robust face recognition based on a joint version of l1 and l2 regularization. The contributions of this paper are mainly shown in the following aspects. First, a novel objective function combining both l1 and l2 regularization is proposed to implement an antinoise sparse representation. An iterative fitting operation via l1 regularization is integrated with l2 norm minimization, to obtain an antinoise classification. Second, the rationale how the proposed method produces promising discriminative and antinoise performance for face recognition is analyzed. The l2 regularization enhances robustness and runs fast, and l1 regularization helps cope with the noisy data. Third, the classification robustness of the proposed method is demonstrated by extensive experiments on several benchmark facial datasets. The method can be considered as an option for the expert systems for biometrics and other recognition problems facing unstable and noisy data.

Demonstration of a compressive-sensing Fourier-transform on-chip spectrometer.

We demonstrate compressive-sensing (CS) spectroscopy in a planar-waveguide Fourier-transform spectrometer (FTS) device. The spectrometer is implemented as an array of Mach-Zehnder interferometers (MZIs) integrated on a photonic chip. The signal from a set of MZIs is composed of an undersampled discrete Fourier interferogram, which we invert using l1-norm minimization to retrieve a sparse input spectrum. To implement this technique, we use a subwavelength-engineered spatial heterodyne FTS on a chip composed of 32 independent MZIs. We demonstrate the retrieval of three sparse input signals by collecting data from restricted sets (8 and 14) of MZIs and applying common CS reconstruction techniques to this data. We show that this retrieval maintains the full resolution and bandwidth of the original device, despite a sampling factor as low as one-fourth of a conventional (non-compressive) design.

Incomplete data based parameter identification of nonlinear and time-variant oscillators with fractional derivative elements

Various system identification techniques exist in the literature that can handle non-stationary measured time-histories, or cases of incomplete data, or address systems following a fractional calculus modeling. However, there are not many (if any) techniques that can address all three aforementioned challenges simultaneously in a consistent manner. In this paper, a novel multiple-input/single-output (MISO) system identification technique is developed for parameter identification of nonlinear and time-variant oscillators with fractional derivative terms subject to incomplete non-stationary data. The technique utilizes a representation of the nonlinear restoring forces as a set of parallel linear sub-systems. In this regard, the oscillator is transformed into an equivalent MISO system in the wavelet domain. Next, a recently developed L1-norm minimization procedure based on compressive sensing theory is applied for determining the wavelet coefficients of the available incomplete non-stationary input-output (excitation-response) data. Finally, these wavelet coefficients are utilized to determine appropriately defined time- and frequency-dependent wavelet based frequency response functions and related oscillator parameters. Several linear and nonlinear time-variant systems with fractional derivative elements are used as numerical examples to demonstrate the reliability of the technique even in cases of noise corrupted and incomplete data.

Assessing the Impact of Building Volume on Land Subsidence in the Central Business District of Beijing with SAR Tomography

ABSTRACTHigh-rise buildings in the modern urban setting may have a considerable impact on land subsidence. The process of urbanization in Beijing has been found to be in direct relationship with the land subsidence. The permanent scatterers interferometric aperture radar (PSInSAR) technology is becoming a conventional tool for land subsidence monitoring. Synthetic aperture radar (SAR) tomography, as an extension of PSInSAR technology, allows a full 3D imaging, and is useful to reconstruct 3D urban building maps. In this study, a SAR tomography method “scale-down by L1 norm minimization, model selection, and estimation reconstruction” (SL1MMER) was applied to 33 TerraSAR-X images to retrieve building height. The object-oriented classification method based on the support vector machine was applied on the World View 2 image to extract geometry of building footprints and to aggregate SL1MMER derived building heights for each building. The building footprints and building heights were used to calculate the volume of buildings. The Stanford method for persistent scatterers, a PSInSAR methodology, was used to obtain land subsidence rates. The correlation coefficient between the building volumes and land subsidence showed no positive relationship for buildings that were below a certain volume. However, at local scale, with the increase of the building volumes, the correlation increased. The larger the volume of buildings, the stronger the relationship became. Once the volume of buildings became larger than 3.00 × 105 m3, the impact of building volume on land subsidence remained stable.

Iterative sparsity score for feature selection and its extension for multimodal data

As a key dimensionality reduction technique in pattern recognition, feature selection has been widely used in information retrieval, text classification and genetic data analysis. In recent years, structural information contained in samples for guiding feature selection has become a new hot spot in machine learning field. Although tremendous feature selection methods have been developed, less important features are still used to construct the structure in those conventional structure based feature selection approaches. In this paper, we propose a new filter-type feature selection method called iterative sparsity score, which is independent of any learning algorithm. The proposed method can preserve the structural information by sparse representation, which can be efficiently solved by a ℓ1-norm minimization problem. To exclude data noise, at one time we discard last m features and iteratively optimize the ℓ1-norm minimization problem. We perform clustering and classification experiments on numerous bench mark datasets. Furthermore, its extension for multimodal data is also developed. We adopt the multi-modality alzheimer’s disease data for classification to evaluate the extended method. The experimental results show the effectiveness of our proposed methods compared with several popular feature selection approaches.

Linearized dynamic warping with ℓ1-norm constraint for multi-component registration

Registration of the travel time of PP waves and PS waves on the same temporal coordinate is critical for joint interpretation of multi-component seismic data. We propose an ℓ1-norm minimization approach to register the corresponding amplitude of reflection events in the PP waves and PS waves data. The dynamic warping function is linearized in the ℓ2-norm fitting term, and a sparsity constrained is used to estimate smooth warping functions. The latter is possible by defining the warping function in terms of a cosine basis. Sparsity is used to minimize the number of basis functions that are used to model the warping function. We have also applied a fast iterative soft thresholding algorithm for solving the non-linear registration optimization problem. We apply the linearized dynamic warping approach with ℓ1-norm regularization to a synthetic example and to a field data set from the Shengli Oilfield, China.

Low-Dimensional Synergistic Representation of Bilateral Reaching Movements.

Kinematic and neuromuscular synergies have been found in numerous aspects of human motion. This study aims to determine how effectively kinematic synergies in bilateral upper arm movements can be used to replicate complex activities of daily living (ADL) tasks using a sparse optimization algorithm. Ten right-handed subjects executed 18 rapid and 11 natural-paced ADL tasks requiring bimanual coordination while sitting at a table. A position tracking system was used to track the subjects’ arms in space, and angular velocities over time for shoulder abduction, shoulder flexion, shoulder internal rotation, and elbow flexion for each arm were computed. Principal component analysis (PCA) was used to generate kinematic synergies from the rapid-paced task set for each subject. The first three synergies accounted for 80.3 ± 3.8% of variance, while the first eight accounted for 94.8 ± 0.85%. The first and second synergies appeared to encode symmetric reaching motions which were highly correlated across subjects. The first three synergies were correlated between left and right arms within subjects, whereas synergies four through eight were not, indicating asymmetries between left and right arms in only the higher order synergies. The synergies were then used to reconstruct each natural-paced task using the l1-norm minimization algorithm. Temporal dilations of the synergies were introduced in order to model the temporal scaling of movement patterns achieved by the cerebellum and basal ganglia as reported previously in the literature. Reconstruction error was reduced by introducing synergy dilations, and cumulative recruitment of several synergies was significantly reduced in the first 10% of training task time by introducing temporal dilations. The outcomes of this work could open new scenarios for the applications of postural synergies to the control of robotic systems, with potential applications in rehabilitation. These synergies not only help in providing near-natural control but also provide simplified strategies for design and control of artificial limbs. Potential applications of these bilateral synergies were discussed and future directions were proposed.

Randomness in neural networks: an overview

Neural networks, as powerful tools for data mining and knowledge engineering, can learn from data to build feature‐based classifiers and nonlinear predictive models. Training neural networks involves the optimization of nonconvex objective functions, and usually, the learning process is costly and infeasible for applications associated with data streams. A possible, albeit counterintuitive, alternative is to randomly assign a subset of the networks’ weights so that the resulting optimization task can be formulated as a linear least‐squares problem. This methodology can be applied to both feedforward and recurrent networks, and similar techniques can be used to approximate kernel functions. Many experimental results indicate that such randomized models can reach sound performance compared to fully adaptable ones, with a number of favorable benefits, including (1) simplicity of implementation, (2) faster learning with less intervention from human beings, and (3) possibility of leveraging overall linear regression and classification algorithms (e.g.,ℓ1norm minimization for obtaining sparse formulations). This class of neural networks attractive and valuable to the data mining community, particularly for handling large scale data mining in real‐time. However, the literature in the field is extremely vast and fragmented, with many results being reintroduced multiple times under different names. This overview aims to provide a self‐contained, uniform introduction to the different ways in which randomization can be applied to the design of neural networks and kernel functions. A clear exposition of the basic framework underlying all these approaches helps to clarify innovative lines of research, open problems, and most importantly, foster the exchanges of well‐known results throughout different communities.WIREs Data Mining Knowl Discov2017, 7:e1200. doi: 10.1002/widm.1200This article is categorized under:Technologies > Machine Learning

Block compressive sensing: Individual and joint reconstruction of correlated images

Compressive sensing provides simultaneous sensing and compression of data. Block compressive sensing (BCS) of images has gained a prominence in recent years due to low encoding complexity. In this paper, we propose the reconstruction algorithm for BCS framework based on iterative re-weighted l1 norm minimization. In the proposed algorithm, the desired signal sparsity is achieved using l1 norm minimization while Wiener filtering is incorporated as the smoothing operator. We also propose block based joint reconstruction algorithm for correlated images and video frames. The correlation is captured through the joint sparsity model by minimizing the objective function which promotes the common sparsity structure. The performance of proposed algorithms is tested on different stereo images and video data. Our analysis shows that the proposed individual reconstruction algorithm gives good compression performance as compared to existing schemes. Extensive analysis on correlated images demonstrates that the proposed joint reconstruction algorithm outperforms individual reconstruction algorithms.

Reweighted smoothed l0-norm based DOA estimation for MIMO radar

In this paper, a reweighted smoothed l0-norm algorithm is proposed for direction-of-arrival (DOA) estimation in monostatic multiple-input multiple-output (MIMO) radar. The proposed method firstly performs the vectorization operation on the covariance matrix, which is calculated from the latest received data matrix obtained by a reduced dimensional transformation. Then a weighted matrix is introduced to transform the covariance estimation errors into a Gaussian white vector, and the proposed method further constructs the other reweighted vector to enhance sparse solution. Finally, a reweighted smoothed l0-norm minimization framework with a reweighted continuous function is designed, based on which the sparse solution is obtained by using a decreasing parameter sequence and the steepest ascent algorithm. Consequently, DOA estimation is accomplished by searching the spectrum of the solution. Compared with the conventional l1-norm minimization based methods, the proposed reweighted smoothed l0-norm algorithm significantly reduces the computation time of DOA estimation. The proposed method is about two orders of magnitude faster than the l1-SVD, reweighted l1-SVD and RV l1-SRACV algorithms. Meanwhile, it provides higher spatial angular resolution and better angle estimation performance. Simulation results are used to verify the effectiveness and advantages of the proposed method.

Multiple-image super-resolution on mobile devices: an image warping approach

This paper discusses a super-resolution (SR) system implemented on a mobile device. We utilized an Android device’s camera to take successive shots and applied a classical multiple-image super-resolution (SR) technique that utilized a set of low-resolution (LR) images. Images taken from the mobile device are subjected to our proposed filtering scheme wherein images that have noticeable presence of blur are discarded to avoid outliers from affecting the produced high-resolution (HR) image. The remaining subset of images are subjected to non-local means denoising, then feature-matched against the first reference LR image. Successive images are then aligned with respect to the first image via affine and perspective warping transformations. The LR images are then upsampled using bicubic interpolation. An L2-norm minimization approach, which is essentially taking the pixel-wise mean of the aligned images, is performed to produce the final HR image.Our study shows that our proposed method performs better than the bicubic interpolation, which makes its implementation in a mobile device quite feasible. We have also proven in our experiments that there are substantial differences from images captured using burst mode that can be utilized by an SR algorithm to create an HR image.

Distribution system state estimation using compressive sensing

This paper leverages the big data provided by micro-phasor measurement units (μPMUs) placed along the smart distribution networks for distribution system state estimation (DSSE). We propose a novel and straightforward DSSE algorithm by solving a set of linear equations without any iterative process. The μPMUs are placed at few buses to measure the voltage phasors. The measured voltage vector is expressed as the product of current injections vector and impedance matrix of the system. Since number of μPMUs is less than number of buses, the constructed set of linear equations are underdetermined. Furthermore, the injection currents vector is sparse because any single load/generator current is negligible when compared with the total current injected from the external grid to the distribution network. Subsequently, we use Compressive Sensing and ℓ1-norm minimization to recover the sparse current vector from the limited number of μPMUs. The voltages at all buses are obtained by multiplying the reconstructed current vector by the impedance matrix. Performance of our method is demonstrated on the IEEE 123-bus test system and a 13.8-kV, 134-bus real network with different distributed generations (DGs) penetration level and under a weakly meshed operation mode. Also, the performance of the proposed technique is compared with that of the conventional weighted least-square (WLS) method.

Backtracking-Based Iterative Regularization Method for Image Compressive Sensing Recovery

This paper presents a variant of the iterative shrinkage-thresholding (IST) algorithm, called backtracking-based adaptive IST (BAIST), for image compressive sensing (CS) reconstruction. For increasing iterations, IST usually yields a smoothing of the solution and runs into prematurity. To add back more details, the BAIST method backtracks to the previous noisy image using L2 norm minimization, i.e., minimizing the Euclidean distance between the current solution and the previous ones. Through this modification, the BAIST method achieves superior performance while maintaining the low complexity of IST-type methods. Also, BAIST takes a nonlocal regularization with an adaptive regularizor to automatically detect the sparsity level of an image. Experimental results show that our algorithm outperforms the original IST method and several excellent CS techniques.

Sound field reconstruction using compressed modal equivalent point source method.

The accuracy, resolution, and economic cost of near-field acoustic holography (NAH) are highly dependent on the number of spatial sampling points. Generally, higher accuracy and resolution require more spatial sampling points, which may increase the workload of measurement or the hardware cost. Compressive sensing (CS) is able to solve the underdetermined problems by utilizing the sparsity of signals, and thus it can be applied to NAH to reduce the number of spatial sampling points but at the same time provide a high-resolution reconstruction image. Based on the CS theory, this paper proposes a compressed modal equivalent point source method (CMESM). In the method, a sparse basis that is obtained from the eigen-decomposition of the power resistance matrix is introduced to compress the equivalent point source strengths, and the ℓ1-norm minimization is used to promote sparse solutions. Both numerical simulation and experimental results demonstrate the validity of the proposed CMESM and show its advantage over the existing methods when the number of spatial sampling points is reduced.

Exploiting Spatial Sparsity in Vibration-based Damage Detection

One of the main limitations traditionally encountered in vibration-based structural health monitoring (SHM) is detecting, localizing and quantifying localized damage using global response measurements. This paper presents an impulse response sensitivity approach enhanced with a LASSO regularization in order to detect spatially sparse (localized) damage. The analytical expression for impulse response sensitivity was derived using Vetter calculus. The proposed algorithm exploits the fact that when damage is sparse, an l1-norm regularization is more suitable than the common least squares (l2-norm) minimization. The proposed methodology is successfully applied in the context of a simulated 21 degree of freedom non-uniform shear beam with noise-contaminated measurements, limited modal parameters, and single input-single output and single input-two output systems.

Compressing Sensing Based Source Localization for Controlled Acoustic Signals Using Distributed Microphone Arrays

In order to enhance the accuracy of sound source localization in noisy and reverberant environments, this paper proposes an adaptive sound source localization method based on distributed microphone arrays. Since sound sources lie at a few points in the discrete spatial domain, our method can exploit this inherent sparsity to convert the localization problem into a sparse recovery problem based on the compressive sensing (CS) theory. In this method, a two-step discrete cosine transform- (DCT-) based feature extraction approach is utilized to cover both short-time and long-time properties of acoustic signals and reduce the dimensions of the sparse model. In addition, an online dictionary learning (DL) method is used to adjust the dictionary for matching the changes of audio signals, and then the sparse solution could better represent location estimations. Moreover, we propose an improved block-sparse reconstruction algorithm using approximate l0 norm minimization to enhance reconstruction performance for sparse signals in low signal-noise ratio (SNR) conditions. The effectiveness of the proposed scheme is demonstrated by simulation results and experimental results where substantial improvement for localization performance can be obtained in the noisy and reverberant conditions.

Sparse inversion method of T2 spectrum based on the L1 norm for low-field nuclear magnetic resonance

The technology of low-field nuclear magnetic resonance (LF-NMR) is commonly used in food, agriculture, energy and chemical sectors due to its non-destructive, non-invasive, in situ, green and other advantages. Recently, this technology played an increasingly large role in the field of food-safety supervision especially. In oil product quality testing, conventional T2 spectrum inversion methods such as the non-negative singular value decomposition (SVD) algorithm can only reflect T2 spectrum in a smooth model. However, for a sparse model, the inversion result of non-negative SVD algorithm is expected to be very glossy, leading to low resolution of T2 spectrum and inaccurate analysis of sample property. To solve this problem, we propose a sparse T2 spectrum inversion algorithm based on the L1 norm minimization constraint. In this paper, we establish the sparse model expression of NMR echo curve, and obtain the T2 sparse spectrum inversion results based on the inner truncated Newton-point method. Furthermore, the effectiveness of L1 sparse inversion algorithm is examined by the synthetic data of the smooth model and the spare model which have different peak numbers and signaltonoise ratios (SNRs). Synthetic results show that compared with the non-negative SVD algorithm, the L1 sparse algorithm is appropriate for both the smooth model and the sparse model with higher inversion accuracy. When the number of T2 peaks in a sparse model changes from a single peak to a quad peak, the L1 sparse algorithm can still obtain accurate inversion results, while the SVD algorithm results in a gradual deterioration, and cannot even determine the peak number. Under the sparse model, when the SNR of the measured NMR curve is gradually changed from 5 dB to 50 dB, the L1 sparse algorithm at 20 dB or more can obtain accurate inversion results which have less than 10% peak error and less than 5% peak position error and amplitude average error. However, the non-negative SVD algorithm cannot obtain accurate results at each SNR. Finally, multiple sets of frying oil samples are utilized to prove the accuracy and robustness of L1 sparse inversion algorithm. Inversion results of seven sets of frying oil samples show that the L1 sparse algorithm prefers the non-negative SVD algorithm. The obtained T2 spectrum by the L1 sparse algorithm shows three peaks obviously, and the T21 peak area ratio S21 and the single component relaxation time T2w are higher linear with respect to frying time than the results by non-negative SVD algorithm, which is useful for detecting the frying oil quality change. The inversion results of the T2 spectrum at different SNRs are consistent with the synthetic results, i.e., when the SNR is reduced, the T2 spectrum inversion results from the L1 sparse algorithm are better than those from the non-negative SVD algorithm when SNR is greater than 20 dB.

Vehicle Trajectory Analysis System via Mutual Information and Sparse Reconstruction

In past years, the task of automatic vehicle trajectory analysis in video surveillance systems has gained increasing attention in the research community. Vehicle trajectory analysis can identify normal and abnormal vehicle motion patterns and is useful for traffic management. Although some analysis methods of vehicle trajectory have been developed, the application of these methods is still limited in practice. In this study, a novel adaptive vehicle trajectory classification method via sparse reconstruction and mutual information analysis based on video surveillance systems was proposed. The l0-norm minimization of sparse reconstruction in the method was relaxed to the lp-norm minimization (0 < p < 1). In addition, to consider the nonlinear correlation between the test trajectory and the dictionary, mutual information between the test trajectory and the reconstructed one was taken into account. A hybrid orthogonal matching pursuit–Newton method (HON) was developed to effectively find the sparse solutions for trajectory classification. Two real-world data sets (including the stop sign data set and straight data set) were used in the experiments to validate the performance and effectiveness of the proposed method. Experimental results show that the trajectory classification accuracy is significantly improved by the proposed method compared with most well-known classifiers, namely, NB, k–nearest neighbor, support vector machine, and typical extant sparse reconstruction methods.