Number Of Nonzero Elements Research Articles

Random sparse generators of Markovian evolution and their spectral properties.

The evolution of a complex multistate system is often interpreted as a continuous-time Markovian process. To model the relaxation dynamics of such systems, we introduce an ensemble of random sparse matrices which can be used as generators of Markovian evolution. The sparsity is controlled by a parameter φ, which is the number of nonzero elements per row and column in the generator matrix. Thus, a member of the ensemble is characterized by the Laplacian of a directed regular graph with D vertices (number of system states) and 2φD edges with randomly distributed weights. We study the effects of sparsity on the spectrum of the generator. Sparsity is shown to close the large spectral gap that is characteristic of nonsparse random generators. We show that the first moment of the eigenvalue distribution scales as ∼φ, while its variance is ∼sqrt[φ]. By using extreme value theory, we demonstrate how the shape of the spectral edges is determined by the tails of the corresponding weight distributions and clarify the behavior of the spectral gap as a function of D. Finally, we analyze complex spacing ratio statistics of ultrasparse generators, φ=const, and find that starting already at φ⩾2, spectra of the generators exhibit universal properties typical of Ginibre's orthogonal ensemble.

On preemptive scheduling on unrelated machines using linear programming

<abstract><p>We consider a basic preemptive scheduling problem where $ n $ non-simultaneously released jobs are to be processed by $ m $ unrelated parallel machines so as to minimize maximum job completion time. An optimal LP-solution has been used to construct an optimal preemptive schedule for simultaneously released jobs in time $ O(n^3) $. We propose fast $ O(m) $ time algorithm that finds an optimal schedule in case the above LP-solution possesses "small enough" number of non-zero elements. We propose another linear program for non-simultaneously released jobs and show how an optimal schedule can be constructed also in time $ O(m) $ from the optimal solution to that linear program. Based on another stronger linear program formulation, we extend the earlier known schedule construction procedure for non-simultaneously released jobs. The procedure is important, in particular, because there may exist no optimal schedule that agrees with an optimal LP-solution. An optimal LP-solution imposes a number of preemptions, and additional preemptions may occur during the schedule construction process, a job might be forced to be split on the same machine. We show that if no job split is allowed, even a restricted version of the problem on three unrelated machines is NP-hard. As a result, we obtain that, given an optimal LP-solution, it is NP-hard to find an optimal schedule that agrees with that LP-solution. As another side result, we obtain that it is NP-hard to find an optimal schedule with at most $ m-1 $ preemptions.</p></abstract>

Asymptotic Analysis of q-Recursive Sequences

For an integer qge 2, a q-recursive sequence is defined by recurrence relations on subsequences of indices modulo some powers of q. In this article, q-recursive sequences are studied and the asymptotic behavior of their summatory functions is analyzed. It is shown that every q-recursive sequence is q-regular in the sense of Allouche and Shallit and that a q-linear representation of the sequence can be computed easily by using the coefficients from the recurrence relations. Detailed asymptotic results for q-recursive sequences are then obtained based on a general result on the asymptotic analysis of q-regular sequences.Three particular sequences are studied in detail: We discuss the asymptotic behavior of the summatory functions ofStern’s diatomic sequence,the number of non-zero elements in some generalized Pascal’s triangle andthe number of unbordered factors in the Thue–Morse sequence. For the first two sequences, our analysis even leads to precise formulæ without error terms.

Co-sparse Non-negative Matrix Factorization.

Non-negative matrix factorization, which decomposes the input non-negative matrix into product of two non-negative matrices, has been widely used in the neuroimaging field due to its flexible interpretability with non-negativity property. Nowadays, especially in the neuroimaging field, it is common to have at least thousands of voxels while the sample size is only hundreds. The non-negative matrix factorization encounters both computational and theoretical challenge with such high-dimensional data, i.e., there is no guarantee for a sparse and part-based representation of data. To this end, we introduce a co-sparse non-negative matrix factorization method to high-dimensional data by simultaneously imposing sparsity in both two decomposed matrices. Instead of adding some sparsity induced penalty such as l1 norm, the proposed method directly controls the number of non-zero elements, which can avoid the bias issues and thus yield more accurate results. We developed an alternative primal-dual active set algorithm to derive the co-sparse estimator in a computationally efficient way. The simulation studies showed that our method achieved better performance than the state-of-art methods in detecting the basis matrix and recovering signals, especially under the high-dimensional scenario. In empirical experiments with two neuroimaging data, the proposed method successfully detected difference between Alzheimer's patients and normal person in several brain regions, which suggests that our method may be a valuable toolbox for neuroimaging studies.

Sample Complexity of Block-Sparse System Identification Problem

In this article, we study the system identification problem for sparse linear time-invariant systems. We propose a sparsity promoting block-regularized estimator to identify the dynamics of the system with only a limited number of input-state data samples. We characterize the properties of this estimator under high-dimensional scaling, where the growth rate of the system dimension is comparable to or even faster than that of the number of available sample trajectories. In particular, using contemporary results on high-dimensional statistics, we show that the proposed estimator results in a small elementwise error, provided that the number of sample trajectories is above a threshold. This threshold depends polynomially on the size of each block and the number of nonzero elements at different rows of input and state matrices, but only logarithmically on the system dimension. A by product of this result is that the number of sample trajectories required for sparse system identification is significantly smaller than the dimension of the system. Furthermore, we show that, unlike the recently celebrated least-squares estimators for system identification problems, the method developed in this work is capable of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">exact recovery</i> of the underlying sparsity structure of the system with the aforementioned number of data samples. Extensive case studies on switching networks and power systems are offered to demonstrate the effectiveness of the proposed method.

Utilizing the Heterogeneity of Clinical Data for Model Refinement and Rule Discovery Through the Application of Genetic Algorithms to Calibrate a High-Dimensional Agent-Based Model of Systemic Inflammation.

Introduction: Accounting for biological heterogeneity represents one of the greatest challenges in biomedical research. Dynamic computational and mathematical models can be used to enhance the study and understanding of biological systems, but traditional methods for calibration and validation commonly do not account for the heterogeneity of biological data, which may result in overfitting and brittleness of these models. Herein we propose a machine learning approach that utilizes genetic algorithms (GAs) to calibrate and refine an agent-based model (ABM) of acute systemic inflammation, with a focus on accounting for the heterogeneity seen in a clinical data set, thereby avoiding overfitting and increasing the robustness and potential generalizability of the underlying simulation model.Methods: Agent-based modeling is a frequently used modeling method for multi-scale mechanistic modeling. However, the same properties that make ABMs well suited to representing biological systems also present significant challenges with respect to their construction and calibration, particularly with respect to the selection of potential mechanistic rules and the large number of associated free parameters. We have proposed that machine learning approaches (such as GAs) can be used to more effectively and efficiently deal with rule selection and parameter space characterization; the current work applies GAs to the challenge of calibrating a complex ABM to a specific data set, while preserving biological heterogeneity reflected in the range and variance of the data. This project uses a GA to augment the rule-set for a previously validated ABM of acute systemic inflammation, the Innate Immune Response ABM (IIRABM) to clinical time series data of systemic cytokine levels from a population of burn patients. The genome for the GA is a vector generated from the IIRABM’s Model Rule Matrix (MRM), which is a matrix representation of not only the constants/parameters associated with the IIRABM’s cytokine interaction rules, but also the existence of rules themselves. Capturing heterogeneity is accomplished by a fitness function that incorporates the sample value range (“error bars”) of the clinical data.Results: The GA-enabled parameter space exploration resulted in a set of putative MRM rules and associated parameterizations which closely match the cytokine time course data used to design the fitness function. The number of non-zero elements in the MRM increases significantly as the model parameterizations evolve toward a fitness function minimum, transitioning from a sparse to a dense matrix. This results in a model structure that more closely resembles (at a superficial level) the structure of data generated by a standard differential gene expression experimental study.Conclusion: We present an HPC-enabled machine learning/evolutionary computing approach to calibrate a complex ABM to complex clinical data while preserving biological heterogeneity. The integration of machine learning, HPC, and multi-scale mechanistic modeling provides a pathway forward to more effectively representing the heterogeneity of clinical populations and their data.

Double Differencing by Demeaning: Applications to Hypocenter Location and Wavespeed Tomography

ABSTRACTDouble differencing of body-wave arrival times has proved to be a useful technique for increasing the resolution of earthquake locations and elastic wavespeed images, primarily because (1) differences in arrival times often can be determined with much greater precision than absolute onset times and (2) differencing reduces the effects of unknown, unmodeled, or otherwise unconstrained variables on the arrival times, at least to the extent that those effects are common to the observations in question. A disadvantage of double differencing is that the system of linearized equations that must be iteratively solved generally is much larger than the undifferenced set of equations, in terms of both the number of rows and the number of nonzero elements. In this article, a procedure based on demeaning subsets of the system of equations for hypocenters and wavespeeds that preserves the advantages of double differencing is described; it is significantly more efficient for both wavespeed-only tomography and joint hypocenter location-wavespeed tomography. Tests suggest that such demeaning is more efficient than double differencing for hypocenter location as well, despite double-differencing kernels having fewer nonzeros. When these subsets of the demeaned system are appropriately scaled and simplified estimates of observational uncertainty are used, the least-squares estimate of the perturbations to hypocenters and wavespeeds from demeaning are identical to those obtained by double differencing. This equivalence breaks down in the case of general, observation-specific weighting, but tests suggest that the resulting differences in least-squares estimates are likely to be inconsequential. Hence, demeaning offers clear advantages in efficiency and tractability over double differencing, particularly for wavespeed tomography.

Incomplete inverse matrices

AbstractThe solution of large sparse linear systems is required in many scientific fields such as computational fluid dynamics, computational electromagnetism, computational finance, etc. The computation of the solution of these systems is performed with preconditioned iterative methods, which rely on effective preconditioning schemes. A new class of approximate inverses is proposed, namely incomplete inverse matrices, which are computed using a recursive Schur complement‐based approach. This class of approximate inverses is based on a priori knowledge of a sparsity pattern. In order to have finer control over the density of the proposed approximate inverse, especially in the case of three‐dimensional problems, on‐the‐fly filtration is used, resulting in substantial reduction in the number of nonzero elements. Implementation details and analysis for computing the proposed scheme are given. Numerical results depicting the effectiveness and applicability of the proposed scheme are also provided.

ALBUS: A method for efficiently processing SpMV using SIMD and Load balancing

SpMV (Sparse matrix–vector multiplication) is widely used in many fields. Improving the performance of SpMV has been the pursuit of many researchers. Parallel SpMV using multi-core processors has been a standard parallel method used by researchers. In reality, the number of non-zero elements in many sparse matrices is not evenly distributed, so parallelism without preprocessing will cause a large amount of performance loss due to uneven load. In this paper, we propose ALBUS (Absolute Load Balancing Using SIMD (Single Instruction Multiple Data)), a method for efficiently processing SpMV using load balancing and SIMD vectorization. On the one hand, ALBUS can achieve multi-core balanced load processing; on the other hand, it gives full play to the ability of SIMD vectorization parallelism under the CPU. We selected 20 sets of regular matrices and 20 sets of irregular matrices to form the Benchmark suite. We performed SpMV performance comparison tests on ALBUS, CSR5 (Compressed Sparse Row 5), Merge(Merge-based SpMV), and MKL (Math Kernel Library) under the same conditions. On the E5-2670 v3 CPU platform, For 20 sets of regular matrices, ALBUS can achieve an average speedup of 1.59x, 1.32x, 1.48x (up to 2.53x, 2.22x, 2.31x) compared to CSR5, Merge, MKL, respectively. For 20 sets of irregular matrices, ALBUS can achieve an average speedup of 1.38x, 1.42x, 2.44x (up to 2.33x, 2.24x, 5.37x) compared to CSR5, Merge, MKL, respectively.

On the rank of random matrix over prime field consisting of independent rows with given numbers of nonzero elements

Недавно полученная явная оценка для распределения ранга матрицы, состоящей из независимых случайных строк, имеющих заданные веса, распространяется на более широкий класс двоичных матриц с независимыми строками, а также на матрицы над простым полем $GF(p)$ из строк, которые выбираются независимо из множеств векторов с заданным числом ненулевых элементов.

Dynamic reactive power optimization method based on the transformer dummy node model

This paper proposes a dynamic reactive power optimization method based on the transformer dummy node model. Compared to the method based on the traditional equivalent n circuit, the proposed method adds the voltage at the dummy node as a variable, together with the voltage at the other side of the transformer, to express the voltage change on the two sides of the transformer. So the turn ratio of transformer is removed from variables, and the power flow equations of transformer branches do not contain turn ratio and its square, both the dimension of power flow equations and the number of non-zero elements in the Jacobian and Hessian matrix reduce, which decreases the calculative amount and accelerates the solution process. The test results of IEEE4, 14, 30 bus systems show that the proposed method obtains the same optimal solution with the traditional method but converges more quickly. With higher computational efficiency, the proposed method is suitable for the practical application of power systems.

VLSI Architecture for Enhanced Approximate Message Passing Algorithm

Compressed sensing (CS) enables the reconstruction of sparse signals from a small number of linear measurements. However, sparse signal recovery algorithms require significant computational effort, even for problems of moderate size, and make their hardware realization a highly challenging task. In this paper, we present a novel architecture for the reconstruction of the sparse signal using the recently proposed Enhanced Approximate Message Passing (EAMP) algorithm. The EAMP algorithm shows better performance in terms of convergence rate and sparsity measurement trade-off as compared to AMP, IST and IHT techniques. The execution time of the proposed design has been reduced by maximizing parallelism with an appropriate level of unfolding. It receives CS measurements (y) and the number of non-zero elements (K) in the input vector from the encoder. Depending on the value of K, one sensing matrix (Φ j ) is selected from the code book Φ. The reconstruction algorithm (EAMP) recovers the input vector (s) by using y and Φ. Bitonic sorting has been used to identify the thresholding value in the EAMP algorithm. The proposed architecture for EAMP algorithm has been synthesized using Synopsys design compiler with 65 nm technology CMOS standard cell libraries. It occupies 719.15 KGE and consumes 315.5 mW power at a frequency of 400 MHz. It is suitable for any arbitrary sparsity-based signal restoration and CS problems.

Large-Scale Problems with Quasi-Block Matrices

Sparse large matrices with a block-staircase and block-treelike structure are studied. They are called quasi-block matrices and consist of independent blocks that are connected to each other pairwise or in a more general fashion. The interdependence of parameters of such matrices, such as the number of nonzero elements, the number of blocks, and the matrix size, is determined. Integer programming problems with large quasi-block matrices are described. For the efficient solution of these problems, a local elimination algorithm is used. This is an iterative algorithm in which certain variables are eliminated at each step. The issues concerning the optimal elimination order are studied. This problem turns out to be exponentially complex, which is proved using a graph interpretation of the concepts of block-treelike and block-staircase structures. The complexity of the local elimination algorithm is considered. This is important for deciding which methods are better in different situations. The numerical results are presented; in particular, efficient procedures for determining the optimal elimination order are described. Special attention is given to the parallelization of particular quasi-block Boolean programming problems on a computer GRID if these problems cannot be solved on a single processor due to their large size.

Compressive Sensing for Joint User Activity and Data Detection in Grant-Free NOMA

This letter studies the joint user activity and data detection problem in an uplink grant-free non-orthogonal multiple access system with binary phase-shift keying modulation. First, we propose a new compressive sensing (CS)-based algorithm, namely the information-enhanced adaptive matching pursuit (IE-AMP), which exploits both the frame-wise user sparsity and the ternary nature of transmit signals, to increase the detection performance with reduced computation complexity. Next, building upon this method, we propose a robust IE-AMP (RIE-AMP) algorithm for further performance improvement. The RIE-AMP first extracts the support information (i.e., the number of nonzero elements in every row) of the output signals of IE-AMP, and then employs a weighted $ {\ell }_{ {1, 2}}$ minimization to refine the result. Numerical results show that the IE-AMP outperforms several state-of-the-art CS-based algorithms, with reduced computational efforts required, while the RIE-AMP achieves a better and more robust detection performance than the IE-AMP at the cost of higher computation complexity.

An FPGA-Oriented Algorithm for Real-Time Filtering of Poisson Noise in Video Streams, with Application to X-Ray Fluoroscopy

In this paper we propose a new algorithm for real-time filtering of video sequences corrupted by Poisson noise. The algorithm provides effective denoising (in some cases overcoming the filtering performances of state-of-the-art techniques), is ideally suited for hardware implementation, and can be implemented on a small field-programmable gate array using limited hardware resources. The paper describes the proposed algorithm, using X-ray fluoroscopy as a case study. We use IIR filters for time filtering, which largely simplifies hardware cost with respect to previous FIR filter-based implementations. A conditional reset is implemented in the IIR filter, to minimize motion blur, with the help of an adaptive thresholding approach. Spatial filtering performs a conditional mean to further reduce noise and to remove isolated noisy pixels. IIR filter hardware implementation is optimized by using a novel technique, based on Steiglitz–McBride iterative method, to calculate fixed-point filter coefficients with minimal number of nonzero elements. Implementation results using the smallest StratixIV FPGA show that the system uses only, at most, the 22% of the resources of the device, while performing real-time filtering of 1024 × 1024@49fps video stream. For comparison, a previous FIR filter-based implementation, on the same FPGA, in the same conditions and constraints (1024 × 1024@49fps), requires the 80% of the logic resources of the FPGA.

Linear Hypothesis Testing in Dense High-Dimensional Linear Models

ABSTRACTWe propose a methodology for testing linear hypothesis in high-dimensional linear models. The proposed test does not impose any restriction on the size of the model, that is, model sparsity or the loading vector representing the hypothesis. Providing asymptotically valid methods for testing general linear functions of the regression parameters in high-dimensions is extremely challenging—especially without making restrictive or unverifiable assumptions on the number of nonzero elements. We propose to test the moment conditions related to the newly designed restructured regression, where the inputs are transformed and augmented features. These new features incorporate the structure of the null hypothesis directly. The test statistics are constructed in such a way that lack of sparsity in the original model parameter does not present a problem for the theoretical justification of our procedures. We establish asymptotically exact control on Type I error without imposing any sparsity assumptions on model parameter or the vector representing the linear hypothesis. Our method is also shown to achieve certain optimality in detecting deviations from the null hypothesis. We demonstrate the favorable finite-sample performance of the proposed methods, via a number of numerical and a real data example. Supplementary materials for this article are available online.

Manuilov Algebra, C*-Hilbert Modules, and Kuiper Type Theorems

We generalize (in two natural ways) the C*-algebra generated by matrices of bounded operators in a separable Hilbert space H with a bounded number of nonzero elements in each row and each column, introduced recently by V. Manuilov. We consider the standard C*-Hilbert module HA instead of H = Hℂ. Also we consider the algebras with finiteness conditions only on rows or only on columns. For related general linear groups, we prove the contractibility (Kuiper type theorems) and some other properties.

Binary Codification Design for Compressive Imaging by Uniform Sensing.

Recently, an important set of high dimensional signals (HDS) applications has successfully implemented compressive sensing (CS) sensors in which their efficiency depends on physical elements that perform a binary codification over the HDS. The structure of the binary codification is crucial as it determines the HDS sensing matrices. For a correct reconstruction, this class of matrices drastically differs from the dense or i.i.d. assumptions usually made in CS. Therefore, current CS matrix design algorithms are impractical. This paper proposes a novel strategy to design structured, sparse, and binary HDS measurement matrices based on promoting linear independence between rows by minimizing the number of its zero singular values. The design constraints lead to keep uniform both, the number of non-zero elements per row and also the number of non-zero elements per column. An algorithm based on an optimal selection of non-zero entries positions is developed to implement this strategy. Simulations show that the proposed optimization improves the quality of the reconstructed HDS in up to 8 dB of PSNR compared with non-optimized matrices.

Anomaly detection in high-dimensional data with the Mahalanobis–Taguchi system

The Mahalanobis–Taguchi (MT) system is a typical Taguchi method and plays an important role in several fields. This study aims at improving the statistical procedure employed for anomaly detection in high-dimensional data with the MT system. The proposed study focuses on estimating the eigenvalues and eigenvectors of the covariance matrix and introduces an estimation procedure based on sparse principal component analysis (SPCA) in the MT system. By incorporating SPCA, eigenvalues and eigenvectors can be accurately estimated for high-dimensional data. In addition, the interpretation of the principal components can become simplified with decreasing number of nonzero elements in the estimated eigenvectors. Numerical experiments have confirmed that the proposed procedure is beneficial for both anomaly detection performance and investigating the cause of anomalies in high-dimensional data. Furthermore, a limitation of the proposed study is its emphasis on improving anomaly detection procedures founded on the first principal component and its residual component. However, the scope of such an anomaly detection procedure can be easily expanded for further improvement.

A discontinuous-grid finite-difference scheme for frequency-domain 2D scalar wave modeling

The discontinuous-grid method can greatly reduce the storage requirement and computational cost in finite-difference modeling, especially for models with large velocity contrasts. However, this technique is mostly applied to time-domain methods. We have developed a discontinuous-grid finite-difference scheme for frequency-domain 2D scalar wave modeling. Special frequency-domain finite-difference stencils are designed in the fine-coarse grid transition zone. The coarse-to-fine-grid spacing ratio is restricted to [Formula: see text], where [Formula: see text] is a positive integer. Optimization equations are formulated to obtain expansion coefficients for irregular stencils in the transition zone. The proposed method works well when teamed with commonly used 9- and 25-point schemes. Compared with the conventional frequency-domain finite-difference method, the proposed discontinuous-grid method can largely reduce the size of the impedance matrix and number of nonzero elements. Numerical experiments demonstrated that the proposed discontinuous-grid scheme can significantly reduce memory and computational costs, while still yielding almost identical results compared with those from conventional uniform-grid simulations. When tested for a very long elapsed time, the frequency-domain discontinuous-grid method does not show instability problems as its time-domain counterpart usually does.