Nonnegative Least-squares Method Research Articles

Physics-Informed Machine Learning for Surrogate Modeling of Heat Transfer Phenomena

Abstract In this paper, we propose a sparse modeling method for automatically creating a surrogate model for nonlinear time-variant systems from a very small number of time series data with nonconstant time steps. We developed three machine learning methods, namely, (1) a data preprocessing method for considering the correlation between errors, (2) a sequential thresholded non-negative least-squares method based on term size criteria, and (3) a solution space search method involving similarity model classification—to apply sparse identification of nonlinear dynamical systems, as first proposed in 2016, to temperature prediction simulations. The proposed method has the potential for wide application to fields where the concept of equivalent circuits can be applied. The effectiveness of the proposed method was verified using time series data obtained by thermofluid analysis of a power module. Two types of cooling systems were verified: forced air cooling and natural air cooling. The model created from the thermofluid analysis results with fewer than the number of input parameters, predicted multiple test data, including extrapolation, with a mean error of less than 1 K. Because the proposed method can be applied using a very small number of data, has a high extrapolation accuracy, and is easy to interpret, it is expected not only that design parameter can be fine-tuned and actual loads can be taken into account, but also that condition-based maintenance can be realized through real-time simulation.

Applications of the non-negative least-squares deconvolution method to analyze energy-dispersive x-ray fluorescence spectra.

We used the Monte Carlo simulation method to establish a detector response matrix and the non-negative least-squares method to deconvolute x-ray spectra. The simulation and experimental data verified the effectiveness of this method, and the influence of full-width at the half of the maximum calibration accuracy on the deconvolution results was investigated. The non-negative least-squares method had high accuracy and efficiency compared with others. The results showed that, except for Zn, the relative errors between the inversion and the standard values were less than 0.1% for the simulated spectra. For the experimental data, the relative errors were within 0.2%. The peaks with similar characteristic energies can be better distinguished in the deconvolution spectra, reducing the errors caused by overlapping peaks in subsequent analysis.

Low-frequency noise characterization of gate oxide trap depth distribution of MOSFETs

To accurately obtain the depth distribution of the gate oxide traps that cause low-frequency noises, this study developed a discrete form of the low-frequency noise model in MOSFETs based on McWhorter's theory. The non-negative least squares (NNLS) method is employed to numerically solve the designed model. The low-frequency noises of planar Ge nMOSFETs with a gate stack match the prerequisite of the proposed form, which is dominated by carrier number fluctuations and is consistent with the McWhorter model. These transistors are used as the object of analysis in this study. By introducing a La2O3 cap layer to the gate stack, the gate oxide trap profiles calculated by the discrete model and the NNLS method showed obvious suppression of the traps in the HfO2 body and concentration of the traps in and near the SiO2/HfO2 interface. The research on both energy-dispersive x-ray spectroscopy and electron energy-loss spectroscopy has shown during annealing the diffusion of La into HfO2 and the appearance of La-rich layer at the SiO2/HfO2 interface in the Ge nMOSFET with a La2O3 cap. According to the existing first-principles calculations, the substitution of Hf in HfO2 by La increases the formation energy of oxygen vacancies, and the formation energy of oxygen vacancies in the HfnLamSixOy layer formed near the SiO2/HfO2 interface is lower than that of HfO2. The experimental and theoretical results support the physical connotation of the calculations and validate the solution proposed by this study.

Improvement in magnetic nanoparticle tomography estimation accuracy by combining sLORETA and non-negative least squares methods

Imaging methods that can detect biofunctionalized magnetic nanoparticles (MNPs) accumulated at cancerous tumor sites are expected to be part of the in vivo cancer diagnostic techniques. An imaging technique called magnetic nanoparticle tomography (MNT), which uses a magnetic sensor array, has been proposed. High sensitivity and spatial resolution were achieved using the non-negative least-squares (NNLS) inverse solution in MNT. However, owing to the presence of measurement noise, the concentration and position of certain MNPs were estimated inaccurately, i.e., artifacts were generated. To overcome this issue, this study first applied standardized low-resolution brain electromagnetic tomography (sLORETA), a spatial filter method with no location bias, to approximate the position of MNPs. The region of analysis was restricted to where the estimated value exceeded the threshold. Subsequently, the NNLS method was applied to estimate the concentration and position of MNPs in the restricted region. In the experiment, two Resovist MNP samples (300 or 500 μgFe) were arranged at a distance of 25–502 mm, and the concentration and position estimation were performed. The estimation results demonstrated that the proposed method successfully suppresses artifacts and adequately estimates the concentration and position of MNPs within a position error of 10 mm and a concentration error of 20 %.

Revisiting the T2 spectrum imaging inverse problem: Bayesian regularized non-negative least squares

Multi-echo T2 magnetic resonance images contain information about the distribution of T2 relaxation times of compartmentalized water, from which we can estimate relevant brain tissue properties such as the myelin water fraction (MWF). Regularized non-negative least squares (NNLS) is the tool of choice for estimating non-parametric T2 spectra. However, the estimation is ill-conditioned, sensitive to noise, and highly affected by the employed regularization weight. The purpose of this study is threefold: first, we want to underline that the apparently innocuous use of two alternative parameterizations for solving the inverse problem, which we called the standard and alternative regularization forms, leads to different solutions; second, to assess the performance of both parameterizations; and third, to propose a new Bayesian regularized NNLS method (BayesReg). The performance of BayesReg was compared with that of two conventional approaches (L-curve and Chi-square (X2) fitting) using both regularization forms. We generated a large dataset of synthetic data, acquired in vivo human brain data in healthy participants for conducting a scan-rescan analysis, and correlated the myelin content derived from histology with the MWF estimated from ex vivo data. Results from synthetic data indicate that BayesReg provides accurate MWF estimates, comparable to those from L-curve and X2, and with better overall stability across a wider signal-to-noise range. Notably, we obtained superior results by using the alternative regularization form. The correlations reported in this study are higher than those reported in previous studies employing the same ex vivo and histological data. In human brain data, the estimated maps from L-curve and BayesReg were more reproducible. However, the T2 spectra produced by BayesReg were less affected by over-smoothing than those from L-curve. These findings suggest that BayesReg is a good alternative for estimating T2 distributions and MWF maps.

Inverse Problem Analysis in Magnetic Nanoparticle Tomography Using Minimum Variance Spatial Filter

In magnetic nanoparticle tomography (MNT), the reduction of artifacts and the calculation time can be used to estimate the position of magnetic nanoparticles (MNPs). A non-negative least-squares (NNLS) inverse problem analysis has been used in MNT systems for this task. However, due to the presence of measurement noise and the high sensitivity of the NNLS method, it often estimates certain MNPs inaccurately, i.e., it generates artifacts. In addition, its calculation time is very high. In this study, we applied the minimum variance spatial filter (MV-SF) inverse problem analysis to MNT and estimated the position of an MNP sample containing <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$100~\mu \text{g}$ </tex-math></inline-formula> of Fe. Using the MV-SF method, MNP samples placed at a depth of 25–40 mm were observed to have no artifacts. Moreover, the MV-SF method was also observed to be faster than the NNLS method by a factor of approximately 20. These results verify the feasibility of the MV-SF method for estimating the MNP positions in an MNT system.

Exact Complexity Certification of a Nonnegative Least-Squares Method for Quadratic Programming

In this letter we propose a method to exactly certify the complexity of an active-set method which is based on reformulating strictly convex quadratic programs to nonnegative least-squares problems. The exact complexity of the method is determined by proving the correspondence between the method and a standard primal active-set method for quadratic programming applied to the dual of the quadratic program to be solved. Once this correspondence has been established, a complexity certification method which has already been established for the primal active-set method is used to also certify the complexity of the nonnegative least-squares method. The usefulness of the proposed method is illustrated on a multi-parametric quadratic program originating from model predictive control of an inverted pendulum.

Developing Non-Negative Spatial Autoregressive Models for Better Exploring Relation Between Nighttime Light Images and Land Use Types

Exploring the relationship between nighttime light and land use is of great significance to understanding human nighttime activities and studying socioeconomic phenomena. Models have been studied to explain the relationships, but the existing studies seldom consider the spatial autocorrelation of night light data, which leads to large regression residuals and an inaccurate regression correlation between night light and land use. In this paper, two non-negative spatial autoregressive models are proposed for the spatial lag model and spatial error model, respectively, which use a spatial adjacency matrix to calculate the spatial autocorrelation effect of light in adjacent pixels on the central pixel. The application scenarios of the two models were analyzed, and the contribution of various land use types to nighttime light in different study areas are further discussed. Experiments in Berlin, Massachusetts and Shenzhen showed that the proposed methods have better correlations with the reference data compared with the non-negative least-squares method, better reflecting the luminous situation of different land use types at night. Furthermore, the proposed model and the obtained relationship between nighttime light and land use types can be utilized for other applications of nighttime light images in the population, GDP and carbon emissions for better exploring the relationship between nighttime remote sensing brightness and socioeconomic activities.

Molecular Dynamics Ensemble Refinement of Intrinsically Disordered Peptides According to Deconvoluted Spectra from Circular Dichroism

We have developed a computational method of atomistically refining the structural ensemble of intrinsically disordered peptides (IDPs) facilitated by experimental measurements using circular dichroism spectroscopy (CD). A major challenge surrounding this approach stems from the deconvolution of experimental CD spectra into secondary structure features of the IDP ensemble. Currently available algorithms for CD deconvolution were designed to analyze the spectra of proteins with stable secondary structures. Herein, our work aims to minimize any bias from the peptide deconvolution analysis by implementing a non-negative linear least-squares fitting algorithm in conjunction with a CD reference data set that contains soluble and denatured proteins (SDP48). The non-negative linear least-squares method yields the best results for deconvolution of proteins with higher disordered content than currently available methods, according to a validation analysis of a set of protein spectra with Protein Data Bank entries. We subsequently used this analysis to deconvolute our experimental CD data to refine our computational model of the peptide secondary structure ensemble produced by all-atom molecular dynamics simulations with implicit solvent. We applied this approach to determine the ensemble structures of a set of short IDPs, that mimic the calmodulin binding domain of calcium/calmodulin-dependent protein kinase II and its 1-amino-acid and 3-amino-acid mutants. Our study offers a, to our knowledge, novel way to solve the ensemble secondary structures of IDPs in solution, which is important to advance the understanding of their roles in regulating signaling pathways through the formation of complexes with multiple partners.

Assessment of phytoplankton class abundance using fluorescence excitation-emission matrix by parallel factor analysis and nonnegative least squares

The feasibility of using fluorescence excitation-emission matrix (EEM) along with parallel factor analysis (PARAFAC) and nonnegative least squares (NNLS) method for the differentiation of phytoplankton taxonomic groups was investigated. Forty-one phytoplankton species belonging to 28 genera of five divisions were studied. First, the PARAFAC model was applied to EEMs, and 15 fluorescence components were generated. Second, 15 fluorescence components were found to have a strong discriminating capability based on Bayesian discriminant analysis (BDA). Third, all spectra of the fluorescence component compositions for the 41 phytoplankton species were spectrographically sorted into 61 reference spectra using hierarchical cluster analysis (HCA), and then, the reference spectra were used to establish a database. Finally, the phytoplankton taxonomic groups was differentiated by the reference spectra database using the NNLS method. The five phytoplankton groups were differentiated with the correct discrimination ratios (CDRs) of 100% for single-species samples at the division level. The CDRs for the mixtures were above 91% for the dominant phytoplankton species and above 73% for the subdominant phytoplankton species. Sixteen of the 85 field samples collected from the Changjiang River estuary were analyzed by both HPLC-CHEMTAX and the fluorometric technique developed. The results of both methods reveal that Bacillariophyta was the dominant algal group in these 16 samples and that the subdominant algal groups comprised Dinophyta, Chlorophyta and Cryptophyta. The differentiation results by the fluorometric technique were in good agreement with those from HPLC-CHEMTAX. The results indicate that the fluorometric technique could differentiate algal taxonomic groups accurately at the division level.

Optimized dwell time algorithm in magnetorheological finishing

An optimized dwell time algorithm for magnetorheological finishing (MRF) is discussed. Based on the D-shape of the removal function of MRF, an optimized non-negative least-squares method is introduced to get dwell time from a linear matrix equation transferred from the de-convolution operation. Moreover, one kind of general surface error map extension is developed for any shape of optics to obtain a more precise optical surface in MRF. The simulation results show that the non-negative least-squares method of the constrained generalized minimal residual (GMRES) method with adaptive Tikhonov regulation is much faster to get highly stable dwell time distribution. In combination with the general surface error map extension, the peak to valley (PV) and root mean square (RMS) of the surface error of the diameter 400 mm converge from 184.41 and 21.26 nm to 7.56 and 0.632 nm with the consistency of the edge and the aperture inside. Finally, a fabricating experiment proves the validity of the optimized algorithm. Therefore, the algorithm developed and presented in this paper can facilitate the MRF process effectively.

LOR-interleaving image reconstruction for PET imaging with fractional-crystal collimation

Positron emission tomography (PET) has become an important modality in medical and molecular imaging. However, in most PET applications, the resolution is still mainly limited by the physical crystal sizes or the detector’s intrinsic spatial resolution. To achieve images with better spatial resolution in a central region of interest (ROI), we have previously proposed using collimation in PET scanners. The collimator is designed to partially mask detector crystals to detect lines of response (LORs) within fractional crystals. A sequence of collimator-encoded LORs is measured with different collimation configurations. This novel collimated scanner geometry makes the reconstruction problem challenging, as both detector and collimator effects need to be modeled to reconstruct high-resolution images from collimated LORs. In this paper, we present a LOR-interleaving (LORI) algorithm, which incorporates these effects and has the advantage of reusing existing reconstruction software, to reconstruct high-resolution images for PET with fractional-crystal collimation. We also develop a 3D ray-tracing model incorporating both the collimator and crystal penetration for simulations and reconstructions of the collimated PET. By registering the collimator-encoded LORs with the collimator configurations, high-resolution LORs are restored based on the modeled transfer matrices using the non-negative least-squares method and EM algorithm. The resolution-enhanced images are then reconstructed from the high-resolution LORs using the MLEM or OSEM algorithm. For validation, we applied the LORI method to a small-animal PET scanner, A-PET, with a specially designed collimator. We demonstrate through simulated reconstructions with a hot-rod phantom and MOBY phantom that the LORI reconstructions can substantially improve spatial resolution and quantification compared to the uncollimated reconstructions. The LORI algorithm is crucial to improve overall image quality of collimated PET, which can have significant implications in preclinical and clinical ROI imaging applications.

Facial Expression Recognition via Non-Negative Least-Squares Sparse Coding

Sparse coding is an active research subject in signal processing, computer vision, and pattern recognition. A novel method of facial expression recognition via non-negative least squares (NNLS) sparse coding is presented in this paper. The NNLS sparse coding is used to form a facial expression classifier. To testify the performance of the presented method, local binary patterns (LBP) and the raw pixels are extracted for facial feature representation. Facial expression recognition experiments are conducted on the Japanese Female Facial Expression (JAFFE) database. Compared with other widely used methods such as linear support vector machines (SVM), sparse representation-based classifier (SRC), nearest subspace classifier (NSC), K-nearest neighbor (KNN) and radial basis function neural networks (RBFNN), the experiment results indicate that the presented NNLS method performs better than other used methods on facial expression recognition tasks.

T(2)-relaxometry for myelin water fraction extraction using wald distribution and extended phase graph.

Quantitative assessment of myelin density in the white matter is an emerging tool for neurodegenerative disease related studies such as multiple sclerosis and Schizophrenia. For the last two decades, T2 relaxometry based on multi-exponential fitting to a single slice multi-echo sequence has been the most common MRI technique for myelin water fraction (MWF) mapping, where the short T2 is associated with myelin water. However, modeling the spectrum of the relaxations as the sum of large number of impulse functions with unknown amplitudes makes the accuracy and robustness of the estimated MWF's questionable. In this paper, we introduce a novel model with small number of parameters to simultaneously characterize transverse relaxation rate spectrum and B1 inhomogeneity at each voxel. We use mixture of three Wald distributions with unknown mixture weights, mean and shape parameters to represent the distribution of the relative amount of water in between myelin sheets, tissue water, and cerebrospinal fluid. The parameters of the model are estimated using the variable projection method and are used to extract the MWF at each voxel. In addition, we use Extended Phase Graph (EPG) method to compensate for the stimulated echoes caused by B1 inhomogeneity. To validate our model, synthetic and real brain experiments were conducted where we have compared our novel algorithm with the non-negative least squares (NNLS) as the state-of-the-art technique in the literature. Our results indicate that we can estimate MWF map with substantially higher accuracy as compared to the NNLS method.

Improvement of Odor Approximation Using Mass Spectrometry

Odor approximation is a technique of creating a scent similar to a target scent by blending multiple odor components. This technique expands the range of odors that can be presented even if the number of odor components is limited. This is a key technology for both odor reproduction using an odor recorder and an olfactory display. As a set of odor components that can cover a wide range of smells is not yet revealed, we study a selection of odor components using an essential-oil and food-flavor mass-spectrum database. Basis vectors are extracted by the nonnegative matrix factorization (NMF) method, and then the nonnegative least-squares method is used to determine the recipe. To increase the approximation accuracy, two methods are proposed. One method is to increase the contributions of the samples with less frequent occurrence. The other method is to set appropriate initial values of the basis vectors in the NMF method using clustering analysis. The accuracy of the odor approximation is increased using these methods.

Odor Approximation Using Mass Spectrometry

Since a set of odor components that can cover a wide range of smells have not been yet revealed, we studied a selection of odor components using an essential-oil mass spectrum database. Basis vectors were extracted using the nonnegative matrix factorization method, and then the nonnegative least-squares method was used to determine the recipe. The odor approximations of the three typical essential oils were evaluated by a sensory test. It was found that the mass spectrum data were correlated with the sensory test results. Moreover, this correlation was particularly high in the high m/z region.

Output noise analysis of optical fiber interferometric sensors using a 3 × 3 coupler

The output noise in a digital passive demodulation scheme for optical fiber interferometric sensors using a 3 × 3 coupler is investigated both theoretically and experimentally. In such a scheme, the measured signal is recovered by the linear combination of the three outputs and a digital arctangent approach. The impact of the additive noise (AN), multiplicative noise (MN) and square-root noise (SN) on the output noise is analyzed. Equations to calculate the output noise variance from the inputs AN, MN and SN are derived for both symmetric and asymmetric 3 × 3 couplers. According to our study, the initial phase and the correlation of different input noise also have great impact on the output noise level. Simulations show that the theoretical analysis is of high accuracy. The noise transfer function (NTF) defined as the ratio of output noise and mean input noise variances is proposed to describe the noise performance of a digital passive demodulation scheme using a 3 × 3 coupler. In our experiment, the distribution of the output noise variance as a function of the initial phase is observed. A method was proposed to predict the distribution of the output noise variance by estimating the variances and the correlation coefficients of the input noises using the non-negative least-squares method and polynomial fitting. The distribution of the output noise variance estimated by this method agrees well with the experimental results.

Time-varying spatial spectrum estimation with a maneuverable towed array

This paper addresses the problem of field directionality mapping (FDM) or spatial spectrum estimation in dynamic environments with a maneuverable towed acoustic array. Array processing algorithms for towed arrays are typically designed assuming the array is straight, and are thus degraded during tow-ship maneuvers. In this paper, maneuvering the array is treated as a feature allowing for left and right disambiguation as well as improved resolution toward endfire. The Cramér-Rao lower bound is used to motivate the improvement in source localization which can be theoretically achieved by exploiting array maneuverability. Two methods for estimating time-varying field directionality with a maneuvering array are presented: (1) Maximum likelihood (ML) estimation solved using the expectation maximization algorithm and (2) a non-negative least squares (NNLS) approach. The NNLS method is designed to compute the field directionality from beamformed power outputs, while the ML algorithm uses raw sensor data. A multi-source simulation is used to illustrate both the proposed algorithms' ability to suppress ambiguous towed array backlobes and resolve closely spaced interferers near endfire which pose challenges for conventional beamforming approaches especially during array maneuvers. Receiver operating characteristics are presented to evaluate the algorithms' detection performance versus signal-to-noise ratio. The results indicate that both FDM algorithms offer the potential to provide superior detection performance when compared to conventional beamforming with a maneuverable array.

Fault Source Modeling of the 2008 Wenchuan Earthquake Based on ALOS/PALSAR Data

Abstract We use crustal deformation data sets derived from satellite-based synthetic aperture radar (SAR) to develop a fault source model of the 2008 M w 7.9 Wenchuan earthquake, China, that occurred at the Longmen Shan fault zone. The data sets include interferometric SAR (InSAR), range offset, and azimuth offset data acquired at seven ascending paths. The range and azimuth offset data are particularly important, exhibiting a single major rupture to the northeast (NE) and multiple ruptures to the southwest (SW). Our preferred model consists of six segments; four follow the previously mapped traces of the Beichuan fault (BF) and its NE extension, one corresponds to the Pengguan fault (PF) to the SW, and the other is included to represent a conjugate fault to the SW. Fixing the location and geometry of those segments, we solve the variable slip distribution whose patch size increases toward the greater depth; we use a nonnegative least-squares method with a smoothing constraint on the distributed slip. The geodetically estimated moment is 1.05×10 21 N m ( M w =7.9), which is close to the seismological estimate and suggests that there are insignificant postseismic signals in the data. Maximum slip of ∼10 m, consisting of both thrust and right-lateral slip components, is identified at the shallowest patches to the NE along the BF. The multiple fault segments to the SW show that the thrust slip component initially dominates, and the strike slip becomes significant toward the NE. Examining the sensitivity of the predicted azimuth offset data to the assumed dip angle, we found that the dip angle changed significantly from 35°–45° at the SW fault segment for the PF to 80°–90° at the NE extension of the BF.

Use of Spatiotemporal Response Information from Sorption-Based Sensor Arrays to Identify and Quantify the Composition of Analyte Mixtures

Linear sensor arrays made from small molecule/carbon black composite chemiresistors placed in a low-headspace volume chamber, with vapor delivered at low flow rates, allowed for the extraction of new chemical information that significantly increased the ability of the sensor arrays to identify vapor mixture components and to quantify their concentrations. Each sensor sorbed vapors from the gas stream and, thereby, as in gas chromatography, separated species having high vapor pressures from species having low vapor pressures. Instead of producing only equilibrium-based sensor responses that were representative of the thermodynamic equilibrium partitioning of analyte between each sensor and the initial vapor phase, the sensor responses varied depending on the position of the sensor in the chamber and the time since the beginning of the analyte exposure. The concomitant spatiotemporal (ST) sensor array response therefore provided information that was a function of time, as well as of the position of the sensor in the chamber. The responses to pure analytes and to multicomponent analyte mixtures comprised of hexane, decane, ethyl acetate, chlorobenzene, ethanol, and/or butanol were recorded along each of the sensor arrays. Use of a non-negative least-squares (NNLS) method for analysis of the ST data enabled the correct identification and quantification of the composition of two-, three-, four-, and five-component mixtures from arrays using only four chemically different sorbent films. In contrast, when traditional time- and position-independent sensor response information was used, these same mixtures could not be identified or quantified robustly. The work has also demonstrated that, for ST data, NNLS yielded significantly better results than analyses using extended disjoint principal components modeling. The ability to correctly identify and quantify constituent components of vapor mixtures through the use of such ST information significantly expands the capabilities of such broadly cross-reactive arrays of sensors.