Sparse Version Research Articles

On Least Squares Support Vector Regression for Predicting Mechanical Properties of Steel Rebars

Aiming at ensuring the quality of the product and reducing the cost of steel manufacturing, an increasing number of studies have been developing nonlinear regression models for the prediction of the mechanical properties of steel rebars using machine learning techniques. Bearing this in mind, we revisit this problem by developing a design methodology that amalgamates two powerful concepts in parsimonious model building: (i) sparsity, in the sense that few support vectors are required for building the predictive model, and (ii) locality, in the sense that simpler models can be fitted to smaller data partitions. In this regard, two regression models based on the Least Squares Support Vector Regression (LSSVR) model are developed. The first one is an improved sparse version of the one introduced in a previous work. The second one is a novel local LSSVR-based regression model. The task of interest is the prediction of four output variables (the mechanical properties YS, UTS, UTS/YS, and PE) based on information about its chemical composition (12 variables) and the parameters of the heat treatment rolling (6 variables). The proposed LSSVR-based regression models are evaluated using real-world data collected from steel rebar manufacturing and compared with the global LSSVR model. The local sparse LSSVR approach was able to consistently outperform the standard single regression model approach in the task of interest, achieving improvements in the average R2 from previous studies: 5.04% for UTS, 5.19% for YS, 1.96% for UTS/YS, and 3.41% for PE. Furthermore, the sparsification of the dataset and the local modeling approach significantly reduce the number of SV operations on average, utilizing 34.0% of the total SVs available for UTS estimation, 44.0% for YS, 31.3% for UTS/YS, and 32.8% for PE.

Hausdorff metric based training of kernels to learn attractors with application to 133 chaotic dynamical systems

Kernel methods have been successfully used to learn surrogate models for dynamical systems by regressing the vector field of such systems. However, choosing an appropriate kernel for a specific problem is an essential challenge in practice. Kernel Flows (KFs) have emerged as an effective tool for learning data-adaptive kernels used to interpolate dynamical systems. In this paper, we introduce Hausdorff Metric based Kernel Flows (HMKFs) to discover a’good’ kernel from time series when a system under consideration has an attractor. First, under the premise that a kernel is good if there is no significant loss in accuracy when half of the data is used to reconstruct the attractor, we design an objective function based on the distance between true and forecasted attractors. Then, we optimize the proposal by adding an ℓ1 penalty term when the base kernel contains many terms, thereby generating sparse HMKFs. Furthermore, we apply HMKFs and its sparse version to a library of 133 chaotic systems from various fields such as biochemistry, fluid mechanics, and astrophysics. The results showcase the potential of HMKFs in accurately modeling complex dynamical systems using data-driven kernels.

Constrained Normalized Subband Adaptive Filter Algorithm and Its Performance Analysis

Constrained least mean square (CLMS) algorithm is the most popular constrained adaptive filtering algorithm due to its simple structure and easy implementation. However, its convergence slows down when the input signal is colored. To address this issue, this paper firstly introduces the normalized subband adaptive filter (NSAF) into the constrained filtering problem and derives a constrained NSAF (CNSAF) algorithm using the Lagrange multiplier method. Benefiting from the good decorrelation capability of the NSAF, the proposed CNSAF algorithm significantly improves the convergence performance of the CLMS algorithm under colored inputs. Then, the mean and mean-square stability of the CNSAF algorithm is analyzed, and the theoretical models to characterize the transient and steady-state mean square deviation (MSD) behaviors of the CNSAF algorithm are derived utilizing the Kronecker product property and vectorization method. Further to extend the CNSAF algorithm to the problem of sparse system identification, a sparse version of the CNSAF algorithm (S-CNSAF) is derived. Finally, the validity of the derived theoretical MSD prediction models and the superiority of the proposed algorithms are confirmed by extensive computer simulations on system identification with colored inputs.

On the conjugate symmetry and sparsity of the harmonic decomposition of parametric surfaces with the randomised Kaczmarz method

The downside of increasing the resolution of surface scanning devices is that the amount of acquired data makes the morphological analysis of the scanned surfaces computationally challenging. This limitation is circumvented by using scalable and non–memory–intensive harmonic expansions. In this paper, the projection of parametric surfaces onto disk and spherical harmonics bases is investigated, and three novel computationally efficient algorithms are proposed based on the randomised Kaczmarz (RK). To boost the computational performance and convergence of the root mean square error (RMSE) of the reconstructed surfaces we exploited the conjugate symmetry property of the harmonic basis functions. Further, the sparsity of the signals is used for estimating the projection coefficients from an undersampled surface. The first algorithm only takes into consideration the conjugate symmetry property for enhancing the convergence of the RMSE. The second algorithm endows the sparse version of the RK algorithm with the conjugate symmetry property. The third algorithm combines the previous two to further accelerate the convergence of the RMSE. The performance of the developed algorithms is tested on three surfaces where we demonstrate that they outperform conventional reconstruction techniques in terms of processing time with comparable precision.

Optogenetic action potentials and intrinsic pacemaker interplay in retrogradely identified midbrain dopamine neurons.

Dissecting the diversity of midbrain dopamine (DA) neurons by optotagging is a promising addition to better identify their functional properties and contribution to motivated behavior. Retrograde molecular targeting of DA neurons with specific axonal projection allows further refinement of this approach. Here, we focus on adult mouse DA neurons in the substantia nigra pars compacta (SNc) projecting to dorsal striatum (DS) by demonstrating the selectivity of a floxed AAV9-based retrograde channelrhodopsin-eYFP (ChR-eYFP) labeling approach in DAT-cre mice. Furthermore, we show the utility of a sparse labeling version for anatomical single-cell reconstruction and demonstrate that ChR-eYFR expressing DA neurons retain intrinsic functional properties indistinguishable from conventionally retrogradely red-beads-labeled neurons. We systematically explore the properties of optogenetically evoked action potentials (oAPs) and their interaction with intrinsic pacemaking in this defined subpopulation of DA neurons. We found that the shape of the oAP and its first derivative, as a proxy for extracellularly recorded APs, is highly distinct from spontaneous APs (sAPs) of the same neurons and systematically varies across the pacemaker duty cycle. The timing of the oAP also affects the backbone oscillator of the intrinsic pacemaker by introducing transient "compensatory pauses". Characterizing this systematic interplay between oAPs and sAPs in defined DA neurons will also facilitate a refinement of DA neuron optotagging in vivo.

DASS: Differentiable Architecture Search for Sparse Neural Networks

The deployment of Deep Neural Networks (DNNs) on edge devices is hindered by the substantial gap between performance requirements and available computational power. While recent research has made significant strides in developing pruning methods to build a sparse network for reducing the computing overhead of DNNs, there remains considerable accuracy loss, especially at high pruning ratios. We find that the architectures designed for dense networks by differentiable architecture search methods are ineffective when pruning mechanisms are applied to them. The main reason is that the current methods do not support sparse architectures in their search space and use a search objective that is made for dense networks and does not focus on sparsity. This paper proposes a new method to search for sparsity-friendly neural architectures. It is done by adding two new sparse operations to the search space and modifying the search objective. We propose two novel parametric SparseConv and SparseLinear operations in order to expand the search space to include sparse operations. In particular, these operations make a flexible search space due to using sparse parametric versions of linear and convolution operations. The proposed search objective lets us train the architecture based on the sparsity of the search space operations. Quantitative analyses demonstrate that architectures found through DASS outperform those used in the state-of-the-art sparse networks on the CIFAR-10 and ImageNet datasets. In terms of performance and hardware effectiveness, DASS increases the accuracy of the sparse version of MobileNet-v2 from 73.44% to 81.35% (+7.91% improvement) with a 3.87× faster inference time.

Excitation-emission fluorescence spectroscopy and sparse chemometric methods for grape seed oil classification and authentication

Grape seed oil (GSO) authentication and classification have recently become the most important aspects of edible oil quality control due to its high nutritional values. In the present contribution, excitation-emission fluorescence spectroscopy and sparse chemometric methods were used for the classification of GSO obtained from different Iranian grape genotypes and the detection of GSO adulteration with refined sunflower oil (SFO). The fluorescence spectra were collected in the wavelength ranges of λex = 200–500 nm and λem = 200–800 nm. To develop multivariate models, more than 200 samples from five different varieties of GSO were used. The sparse version of N-way partial least squares discriminant analysis (sNPLS-DA) was utilized to build an interpretable classification model. Classification accuracies for the sNPLS-DA model were 1.00 for predicting all grape genotypes. The collected fluorescence data revealed a distinct intensity difference between the Chafteh GSO and other GSOs in λex = 270–310 nm, and λem = 300–350 nm. To simulate the adulteration of GSO with SFO, 35 binary blends of Chafteh GSO samples (as a higher-quality oil) were prepared at adulterant levels from 10% to 50%. Sparse multivariate regression techniques, including the least absolute shrinkage and selection operator (Lasso), Ridge, and Elastic net were applied for quantitative analysis of GSO adulteration. The coefficient of multiple determination (R2) and the RMSE for the Lasso model were 0.914 and 0.013, respectively, for the external test sets. The results showed that the sparse chemometric methods coupled with excitation-emission fluorescence spectroscopy can be used as interpretable and reliable tools for quality control and adulteration detection of oil samples in the food industry and commerce.

Performance bounds of the intensity-based estimators for noisy phase retrieval

The aim of noisy phase retrieval is to estimate a signal x0∈Cd from m noisy intensity measurements bj=|〈aj,x0〉|2+ηj,j=1,…,m, where aj∈Cd are known measurement vectors and η=(η1,…,ηm)⊤∈Rm is a noise vector. A commonly used estimator for x0 is to minimize the intensity-based loss function, i.e., xˆ:=argminx∈Cd∑j=1m(|〈aj,x〉|2−bj)2. Although many algorithms for solving the intensity-based estimator have been developed, there are very few results about its estimation performance. In this paper, we focus on the performance of the intensity-based estimator and prove that the error bound satisfies minθ∈R⁡‖xˆ−eiθx0‖2≲min⁡{‖η‖2m1/4,‖η‖2‖x0‖2⋅m} under the assumption of m≳d and aj∈Cd,j=1,…,m, being complex Gaussian random vectors. We also show that the error bound is rate optimal when m≳dlog⁡m. In the case where x0 is an s-sparse signal, we present a similar result under the assumption of m≳slog⁡(ed/s). To the best of our knowledge, our results provide the first theoretical guarantees for both the intensity-based estimator and its sparse version.

Point Set Self-Embedding.

This work presents an innovative method for point set self-embedding, that encodes the structural information of a dense point set into its sparser version in a visual but imperceptible form. The self-embedded point set can function as the ordinary downsampled one and be visualized efficiently on mobile devices. Particularly, we can leverage the self-embedded information to fully restore the original point set for detailed analysis on remote servers. This new task is challenging, cause both the self-embedded point set and restored point set should resemble the original one. To achieve a learnable self-embedding scheme, we design a novel framework with two jointly-trained networks: one to encode the input point set into its self-embedded sparse point set and the other to leverage the embedded information for inverting the original point set back. Further, we develop a pair of up-shuffle and down-shuffle units in the two networks, and formulate loss terms to encourage the shape similarity and point distribution in the results. Extensive qualitative and quantitative results demonstrate the effectiveness of our method on both synthetic and real-scanned datasets. The source code and trained models will be publicly available at https://github.com/liruihui/Self-Embedding.

Adaptive L0 Regularization for Sparse Support Vector Regression

In this work, we proposed a sparse version of the Support Vector Regression (SVR) algorithm that uses regularization to achieve sparsity in function estimation. To achieve this, we used an adaptive L0 penalty that has a ridge structure and, therefore, does not introduce additional computational complexity to the algorithm. In addition to this, we used an alternative approach based on a similar proposal in the Support Vector Machine (SVM) literature. Through numerical studies, we demonstrated the effectiveness of our proposals. We believe that this is the first time someone discussed a sparse version of Support Vector Regression (in terms of variable selection and not in terms of support vector selection).

Naive Feature Selection: A Nearly Tight Convex Relaxation for Sparse Naive Bayes

Because of its linear complexity, naive Bayes classification remains an attractive supervised learning method, especially in very large-scale settings. We propose a sparse version of naive Bayes, which can be used for feature selection. This leads to a combinatorial maximum-likelihood problem, for which we provide an exact solution in the case of binary data, or a bound in the multinomial case. We prove that our convex relaxation bounds become tight as the marginal contribution of additional features decreases using a priori duality gap bounds derived from the Shapley–Folkman theorem. We show how to produce primal solutions satisfying these bounds. Both binary and multinomial sparse models are solvable in time almost linear in problem size, representing a very small extra relative cost compared with the classical naive Bayes. Numerical experiments on text data show that the naive Bayes feature selection method is as statistically effective as state-of-the-art feature selection methods such as recursive feature elimination, l1-penalized logistic regression, and LASSO while being orders of magnitude faster (a python implementation can be found at https://github.com/aspremon/NaiveFeatureSelection ). Funding: A. d’Aspremont acknowledges support from the Fonds AXA Pour la Recherche and Kamet Ventures [Machine Learning and Optimisation Joint Research Initiative] and a Google-focused award as well as funding from Agence Nationale de la Recherche [Grant ANR-19-P3IA-0001]. L. El Ghaoui acknowledges support from Berkeley Artificial Intelligence Research and the Tsinghua–Berkeley–Shenzhen Institute.

Depth-Based Dynamic Sampling of Neural Radiation Fields

Although the NeRF approach can achieve outstanding view synthesis, it is limited in practical use because it requires many views (hundreds) for training. With only a few input views, the Depth-DYN NeRF that we propose can accurately match the shape. First, we adopted the ip_basic depth-completion method, which can recover the complete depth map from sparse radar depth data. Then, we further designed the Depth-DYN MLP network architecture, which uses a dense depth prior to constraining the NeRF optimization and combines the depthloss to supervise the Depth-DYN MLP network. When compared to the color-only supervised-based NeRF, the Depth-DYN MLP network can better recover the geometric structure of the model and reduce the appearance of shadows. To further ensure that the depth depicted along the rays intersecting these 3D points is close to the measured depth, we dynamically modified the sample space based on the depth of each pixel point. Depth-DYN NeRF considerably outperforms depth NeRF and other sparse view versions when there are a few input views. Using only 10–20 photos to render high-quality images on the new view, our strategy was tested and confirmed on a variety of benchmark datasets. Compared with NeRF, we obtained better image quality (NeRF average at 22.47 dB vs. our 27.296 dB).

Binary-coupling sparse Sachdev-Ye-Kitaev model: An improved model of quantum chaos and holography

The sparse version of the Sachdev-Ye-Kitaev (SYK) model reproduces essential features of the original SYK model while reducing the number of disorder parameters. In this paper, we propose a further simplification of the model which we call the binary-coupling sparse SYK model. We set the nonzero couplings to be $\pm 1$, rather than being sampled from a continuous distribution such as Gaussian. Remarkably, this simplification turns out to be an improvement: the binary-coupling model exhibits strong correlations in the spectrum, which is the important feature of the original SYK model that leads to the quick onset of the random-matrix universality, more efficiently in terms of the number of nonzero terms. This model is better suited for analog or digital quantum simulations of quantum chaotic behavior and holographic metals due to its simplicity and scaling properties.

Dynamic Curriculum Learning for Great Ape Detection in the Wild

We propose a novel end-to-end curriculum learning approach for sparsely labelled animal datasets leveraging large volumes of unlabelled data to improve supervised species detectors. We exemplify the method in detail on the task of finding great apes in camera trap footage taken in challenging real-world jungle environments. In contrast to previous semi-supervised methods, our approach adjusts learning parameters dynamically over time and gradually improves detection quality by steering training towards virtuous self-reinforcement. To achieve this, we propose integrating pseudo-labelling with curriculum learning policies and show how learning collapse can be avoided. We discuss theoretical arguments, ablations, and significant performance improvements against various state-of-the-art systems when evaluating on the Extended PanAfrican Dataset holding approx. 1.8M frames. We also demonstrate our method can outperform supervised baselines with significant margins on sparse label versions of other animal datasets such as Bees and Snapshot Serengeti. We note that performance advantages are strongest for smaller labelled ratios common in ecological applications. Finally, we show that our approach achieves competitive benchmarks for generic object detection in MS-COCO and PASCAL-VOC indicating wider applicability of the dynamic learning concepts introduced. We publish all relevant source code, network weights, and data access details for full reproducibility.

Convergence of sparse grid Gaussian convolution approximation for multi-dimensional periodic functions

We consider the problem of approximating [ 0 , 1 ] d -periodic functions by convolution with a scaled Gaussian kernel. We start by establishing convergence rates to functions from periodic Sobolev spaces and we show that the saturation rate is O ( h 2 ) , where h is the scale of the Gaussian kernel. Taken from a discrete point of view, this result can be interpreted as the accuracy that can be achieved on the uniform grid with spacing h . In the discrete setting, the curse of dimensionality would place severe restrictions on the computation of the approximation. For instance, a spacing of 2 − n would provide an approximation converging at a rate of O ( 2 − 2 n ) but would require ( 2 n + 1 ) d grid points. To overcome this we introduce a sparse grid version of Gaussian convolution approximation, where substantially fewer grid points are required (from O ( 2 n d ) on the full grid to just O ( 2 n n d − 1 ) on the sparse grid) and show that the sparse grid version delivers a saturation rate of O ( n d − 1 2 − 2 n ) . This rate is in line with what one would expect in the sparse grid setting (where the full grid error only deteriorates by a factor of order n d − 1 ) however the analysis that leads to the result is novel in that it draws on results from the theory of special functions and key observations regarding the form of certain weighted geometric sums.

A canonical Ramsey theorem with list constraints in random graphs

The celebrated canonical Ramsey theorem of Erdős and Rado implies that for a given graph H, if n is sufficiently large then any colouring of the edges of Kn gives rise to copies of H that exhibit certain colour patterns, namely monochromatic, rainbow or lexicographic. We are interested in sparse random versions of this result and the threshold at which the random graph G(n,p) inherits the canonical Ramsey properties of Kn. Our main result here pins down this threshold when we focus on colourings that are constrained by some prefixed lists. This result is applied in an accompanying work of the authors on the threshold for the canonical Ramsey property (with no list constraints) in the case that H is an even cycle.

Growth and productivity of energy plantations of poplars in the Right-Bank Forest Steppe conditions

Purpose. To determine the features of the growth of an energy plantation of poplar variety ‘Robusta’ on leached chernozems in the conditions of the Central Forest Steppe of Ukraine. Methods. Field, laboratory, statistical. Results. One-year lignified cuttings 25 cm long and 0.8–1.0 cm in diameter were used as planting material. The designed layout of planting places: 2.0x0.8 m (more density version) and 2.0x0.9 m (more sparse version). After the first year of cultivation, the studied plots had the same height (1.6 m). During the second growing season, their growth in height was 2.0±0.05 m in the more sparse version, and 2.1±0.05 m in the thicker version. After the first year, the diameter at a height of 1.3 m was greater in the thicker version (0.5 cm versus 0.3 cm), but already in the next year, the trees of the more sparse stand became 2 mm thicker, and this difference persisted in the third year, when the average diameter of the trees of the denser version was 5.8 cm, and of the sparser — 6.2 cm. As a result of the higher diameter of trees of the more sparse version, they had a higher weight of the average tree, but due to the smaller number of trees, their productivity was lower. So, after three years of growing a poplar plantation created by cuttings, one hectare can yield 11.66–14.58 tons of dry biomass, or 207.5–259.5 GJ of energy. A sharp increase in morphometric indicators during the third year of cultivation indicates the feasibility of using a longer period of cultivation of such plantations to obtain the maximum amount of biomass. Conclusions. The study of a three-year energy plantation of the ‘Robusta’ variety, created by one-year lignified cuttings on the leached soils of the Right Bank Forest Steppe, with the placement of planting sites 2.0 x 0.8 m and 2.0 x 0.8 m, showed that after three years of growing a poplar plantation, one hectare can get 11.66–14.58 tons of dry biomass, or 207.5–259.5 GJ of energy. A significant increase in growth indicators and biomass productivity of poplar plantations during the third year of cultivation indicates the feasibility of using a 4–6-year period of cultivation of such plantations to obtain the maximum amount of biomass.

Sparse Pinball Twin Bounded Support Vector Clustering

Analyzing unlabeled data is of prime importance in machine learning. Creating groups and identifying an underlying clustering principle is essential to many fields, such as biomedical analysis and market research. Novel unsupervised machine learning algorithms, also called clustering algorithms, are developed and utilized for this task. Inspired by twin support vector machine (TWSVM) principles, a recently introduced plane-based clustering algorithm, the twin bounded support vector clustering (TBSVC), is used in widespread clustering problems. However, TBSVC is sensitive to noise and suffers from low resampling stability due to usage of hinge loss. Pinball loss is another type of loss function that is less sensitive toward noise in the datasets and is more stable for resampling of datasets. However, the use of pinball loss negatively affects the sparsity of the solution of the problem. In this article, we present a novel plane-based clustering method, the sparse TBSVC using pinball loss (pinSTBSVC). The proposed pinSTBSVC is the sparse version of our recently proposed TBSVC using pinball loss (pinTBSVC). Sparse solutions help create better-generalized solutions to clustering problems; hence, we attempt to use the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula> -insensitive pinball loss function to propose pinSTBSVC. The loss function used to propose pinSTBSVC provides sparsity to the solution of the problem and improves the aforementioned plane-based clustering algorithms. Experimental results performed on benchmark University of California, Irvine (UCI) datasets indicate that the proposed method outperforms other existing plane-based clustering algorithms. Additionally, we also give the application of our method in biomedical image clustering and marketing science. We show that the proposed method is more accurate on real-world datasets too. The code for the proposed algorithm is also provided on the author’s Github page: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/mtanveer1</uri> .

PRIN/SPRIN: On Extracting Point-Wise Rotation Invariant Features.

Point cloud analysis without pose priors is very challenging in real applications, as the orientations of point clouds are often unknown. In this paper, we propose a brand new point-set learning framework PRIN, namely, Point-wise Rotation Invariant Network, focusing on rotation invariant feature extraction in point clouds analysis. We construct spherical signals by Density Aware Adaptive Sampling to deal with distorted point distributions in spherical space. Spherical Voxel Convolution and Point Re-sampling are proposed to extract rotation invariant features for each point. In addition, we extend PRIN to a sparse version called SPRIN, which directly operates on sparse point clouds. Both PRIN and SPRIN can be applied to tasks ranging from object classification, part segmentation, to 3D feature matching and label alignment. Results show that, on the dataset with randomly rotated point clouds, SPRIN demonstrates better performance than state-of-the-art methods without any data augmentation. We also provide thorough theoretical proof and analysis for point-wise rotation invariance achieved by our methods. The code to reproduce our results will be made publicly available.

On the image in the torus of sparse points on dilating analytic curves

It is known that the image in ℝ2/ℤ2 of a circle of radius ρ in the plane becomes equidistributed as ρ → ∞. We consider the following sparse version of this phenomenon. Starting from a sequence of radii {ρn} ∞n=1 which diverges to ∞ and an angle ω ∈ ℝ/ℤ, we consider the projection to ℝ2/ℤ2 of the n’th roots of unity rotated by angle ω and dilated by a factor of ρn. We prove that if ρn is bounded polynomially in n, then the image of these sparse collections becomes equidistributed, and moreover, if ρngrows arbitrarily fast, then we show that equidistribution holds for almost all ω. Interestingly, we found that for any angle there is a sequence of radii growing to ∞ faster than any polynomial for which equidistribution fails dramatically. In greater generality, we prove this type of results for dilations of varying analytic curves in ℝd. A novel component of the proof is the use of the theory of o-minimal structures to control exponential sums.