Input Matrices Research Articles

Cluster consensus for leader-following high-order linear multiagent systems

<p indent="0mm">Cluster consensus for leader-following high-order linear multiagent systems (MASs) is investigated. Based on the information of an agent and its neighbors, state feedback and output feedback control protocols are designed for each follower, respectively. Sufficient conditions for leader-following cluster consensus are given using tools from the algebraic Riccati equation and matrix analysis. It is shown that if the communication topology of an MAS composed of the leader and followers is interactively balanced and contains a spanning tree, the MAS can achieve leader-following cluster consensus under the proposed control protocols by properly selecting control gains. Particularly, it is found that the leader-following cluster consensus problem based on the output feedback protocol can be transformed into a static output feedback problem using an error system. If the input and output matrices of a follower satisfy a certain rank condition, the MAS can achieve leader-following cluster consensus under the designed output feedback control protocol.

An extended GCRD algorithm for parametric univariate polynomial matrices and application to parametric Smith form

The first extended greatest common right divisor (GCRD) algorithm for parametric univariate polynomial matrices is presented. The starting point of this GCRD algorithm is the free property of submodules over univariate polynomial rings. We convert the computation of GCRDs to that of free basis for modules and prove that a free basis of the submodule generated by row vectors of input matrices forms just a GCRD of these matrices. The GCRD algorithm is obtained by computing a minimal Gröbner basis for the corresponding submodule since a minimal Gröbner basis of submodules is a free basis for univariate cases. While the key idea of extended algorithm is to construct a special module by adding the unit vectors which can record the representation coefficients. This method based on modules can be naturally generalized to the parametric case because of the comprehensive Gröbner systems for modules. As a consequence, we obtain an extended GCRD algorithm for parametric univariate polynomial matrices. More importantly, we apply the proposed extended GCD algorithm for univariate polynomials (as a special case of matrices) to the computation of Smith normal form, and give the first algorithm for reducing a univariate polynomial matrix with parameters to its Smith normal form.

CONNA: Configurable Matrix Multiplication Engine for Neural Network Acceleration

Convolutional neural networks (CNNs) have demonstrated promising results in various applications such as computer vision, speech recognition, and natural language processing. One of the key computations in many CNN applications is matrix multiplication, which accounts for a significant portion of computation. Therefore, hardware accelerators to effectively speed up the computation of matrix multiplication have been proposed, and several studies have attempted to design hardware accelerators to perform better matrix multiplications in terms of both speed and power consumption. Typically, accelerators with either a two-dimensional (2D) systolic array structure or a single instruction multiple data (SIMD) architecture are effective only when the input matrix has shapes that are close to or similar to a square. However, several CNN applications require multiplications of non-squared matrices with various shapes and dimensions, and such irregular shapes lead to poor utilization efficiency of the processing elements (PEs). This study proposes a configurable engine for neural network acceleration, called CONNA, whose computation engine can conduct matrix multiplications with highly utilized computing units, regardless of the access patterns, shapes, and dimensions of the input matrices by changing the shape of matrix multiplication conducted in the physical array. To verify the functionality of the CONNA accelerator, we implemented CONNA as an SoC platform that integrates a RISC-V MCU with CONNA on Xilinx VC707 FPGA. SqueezeNet on CONNA achieved an inference performance of 100 frames per second (FPS) with 2.36 mm2 and 83.55 mW in a 65 nm process, improving efficiency by up to 34.1 times better than existing accelerators in terms of FPS, silicon area, and power consumption.

Observer-based sliding mode control for discrete-time nonlinear system with time-varying delay

This work deals with the design observer-based sliding mode control for discrete-time Takagi–Sugeno fuzzy models with time-varying delay and subject to measurement noises. The sliding surface is developed by introducing the state and input vectors to overcome the restrictive assumption in most existing sliding mode control schemes which forces the input matrices of all linear subsystems to be equal. Sufficient stability conditions of the error system and sliding mode dynamics with disturbance attenuation level are developed using the augmented Lyapunov–Krasovskii functional and free-weighting matrix. Finally, a simulation example is presented to demonstrate the efficiency of the proposed results.

Towards Efficient Sparse Matrix Vector Multiplication on Real Processing-In-Memory Architectures

Several manufacturers have already started to commercialize near-bank Processing-In-Memory (PIM) architectures, after decades of research efforts. Near-bank PIM architectures place simple cores close to DRAM banks. Recent research demonstrates that they can yield significant performance and energy improvements in parallel applications by alleviating data access costs. Real PIM systems can provide high levels of parallelism, large aggregate memory bandwidth and low memory access latency, thereby being a good fit to accelerate the Sparse Matrix Vector Multiplication (SpMV) kernel. SpMV has been characterized as one of the most significant and thoroughly studied scientific computation kernels. It is primarily a memory-bound kernel with intensive memory accesses due its algorithmic nature, the compressed matrix format used, and the sparsity patterns of the input matrices given. This paper provides the first comprehensive analysis of SpMV on a real-world PIM architecture, and presents SparseP, the first SpMV library for real PIM architectures. We make two key contributions. First, we design efficient SpMV algorithms to accelerate the SpMV kernel in current and future PIM systems, while covering a wide variety of sparse matrices with diverse sparsity patterns. Second, we provide the first comprehensive analysis of SpMV on a real PIM architecture. Specifically, we conduct our rigorous experimental analysis of SpMV kernels in the UPMEM PIM system, the first publicly-available real-world PIM architecture. Our extensive evaluation provides new insights and recommendations for software designers and hardware architects to efficiently accelerate the SpMV kernel on real PIM systems. For more information about our thorough characterization on the SpMV PIM execution, results, insights and the open-source SparseP software package [21], we refer the reader to the full version of the paper [3, 4]. The SparseP software package is publicly and freely available at https://github.com/CMU-SAFARI/SparseP.

Robust attitude tracking on the Special Orthogonal Group [formula omitted] using PD-type state feedback and linear matrix inequalities

The problem of rigid-body attitude tracking in the presence of exogenous disturbances is addressed. Attitude is parameterized using the rotation matrix, an element of the Special Orthogonal Group SO(3), as it provides a singularity-free and unambiguous attitude description. The closed-loop stability and robustness properties of a PD-type state-feedback control law, proposed in literature for attitude tracking using rotation matrices, are investigated using the nonlinear H∞ control framework. Starting from a dissipation inequality, sufficient conditions are derived which ensure that the closed-loop energy gain from bounded, finite-energy exogenous disturbances to a specified error signal respects a given upper bound. Then, the sufficient conditions are reformulated using the state and input matrices for the translational double integrator, and recast as linear matrix inequalities (LMIs). Lastly, the reformulated LMIs are used to synthesize controller gains for the proportional and derivative state-feedback terms in the original SO(3) control law. The controller synthesis problem for a microsatellite is considered as a case study. The controller gains are obtained using the proposed LMI-based procedure, and the tracking and disturbance rejection capabilities of the SO(3) controller are illustrated.

Zeroing Neural Network Approaches Based on Direct and Indirect Methods for Solving the Yang–Baxter-like Matrix Equation

This research introduces three novel zeroing neural network (ZNN) models for addressing the time-varying Yang–Baxter-like matrix equation (TV-YBLME) with arbitrary (regular or singular) real time-varying (TV) input matrices in continuous time. One ZNN dynamic utilizes error matrices directly arising from the equation involved in the TV-YBLME. Moreover, two ZNN models are proposed using basic properties of the YBLME, such as the splitting of the YBLME and sufficient conditions for a matrix to solve the YBLME. The Tikhonov regularization principle enables addressing the TV-YBLME with an arbitrary input real TV matrix. Numerical experiments, including nonsingular and singular TV input matrices, show that the suggested models deal effectively with the TV-YBLME.

Recovering single precision accuracy from Tensor Cores while surpassing the FP32 theoretical peak performance

Tensor Core is a mixed-precision matrix–matrix multiplication unit on NVIDIA GPUs with a theoretical peak performance of more than 300 TFlop/s on Ampere architectures. Tensor Cores were developed in response to the high demand of dense matrix multiplication from machine learning. However, many applications in scientific computing such as preconditioners for iterative solvers and low-precision Fourier transforms can exploit these Tensor Cores. To compute a matrix multiplication on Tensor Cores, we need to convert input matrices to half-precision, which results in loss of accuracy. To avoid this, we can keep the mantissa loss in the conversion using additional half-precision variables and use them for correcting the accuracy of matrix–matrix multiplication. Even with this correction, the use of Tensor Cores yields higher throughput compared to FP32 SIMT Cores. Nevertheless, the correcting capability of this method alone is limited, and the resulting accuracy cannot match that of a matrix multiplication on FP32 SIMT Cores. We address this problem and develop a high accuracy, high performance, and low power consumption matrix–matrix multiplication implementation using Tensor Cores, which exactly matches the accuracy of FP32 SIMT Cores while achieving superior throughput. The implementation is based on NVIDIA’s CUTLASS. We found that the key to achieving this accuracy is how to deal with the rounding inside Tensor Cores and underflow probability during the correction computation. Our implementation achieves 51 TFlop/s for a limited exponent range using FP16 Tensor Cores and 33 TFlop/s for full exponent range of FP32 using TF32 Tensor Cores on NVIDIA A100 GPUs, which outperforms the theoretical FP32 SIMT Core peak performance of 19.5 TFlop/s.

On the Connectivity Properties of Feasible Regions of Optimal Decentralized Control Problems

The optimal decentralized control (ODC) is an NP-hard problem with many applications in real-world systems. There is a recent trend of using local search algorithms for solving optimal control problems. However, the effectiveness of these methods depends on the connectivity property of the feasible region of the underlying optimization problem. In this article, for ODC problems with static controllers, we develop a novel criterion for certifying the connectivity of the feasible region in the case where the input and output matrices of the system dynamics are identity matrices. This criterion can be checked via an efficient algorithm, and it is used to prove that the number of communication networks leading to connected feasible regions is greater than a square root of the exponential number of possible communication networks (named patterns). For ODC problems with dynamic controllers, we prove that under certain mild conditions, the closure of the feasible region is always connected after some parameterization, for general communication networks and system dynamics.

Robust Inverse Optimal Cooperative Control for Uncertain Linear Multiagent Systems

This article studies robust inverse optimal control design for cooperative control of uncertain linear multiagent systems (MASs) on directed graph. While designing optimal cooperative control solutions are desired and have been widely considered, ignoring the system uncertainties during the design phase can jeopardize their performance and even cause instability during the operation phase. To alleviate this issue, distributed controllers are designed that are robust to uncertainty and are optimal with respect to some meaningful performance index (referred to as being inverse optimal). It is shown that using the presented distributed robust inverse optimal controller the network of agents converge to a common value of interest or follow the trajectory of a leader node despite uncertainties. A comprehensive study is performed with the time varying bounded parameters uncertainties on the drift and the input matrices of the state-space representation of the MASs. Sufficient conditions for stability analysis are provided using the Lyapunov theory to show that the distributed controller can stabilize the MAS globally. The proposed ideas are confirmed via numerical examples.

A Unified Treatment of Partial Stragglers and Sparse Matrices in Coded Matrix Computation

The overall execution time of distributed matrix computations is often dominated by slow worker nodes (stragglers) within the computation clusters. Recently, coding-theoretic techniques have been utilized to mitigate the effect of stragglers where worker nodes are assigned the job of processing encoded submatrices of the original matrices. In many machine learning or optimization problems the relevant matrices are often sparse. Several prior coded computation methods operate with dense linear combinations of the original submatrices; this can significantly increase the worker node computation times and consequently the overall job execution time. Moreover, several existing techniques treat the stragglers as failures (erasures) and discard their computations. In this work, we present a coding approach which operates with limited encoding of the original submatrices and utilizes the partial computations done by the slower workers. While our scheme can continue to have the optimal threshold of prior work, it also allows us to trade off the straggler resilience with the worker computation speed for sparse input matrices. Extensive numerical experiments done over cloud platforms confirm that the proposed approach enhances the speed of the worker computations (and thus the whole process) significantly.

Robust Stochastic Sensitivity Analysis of the Threonine Synthesis Pathway

The theory of robust sensitivity of feedback controlled biochemical metabolic pathways is extended to the stochastic case where the deterministic parameter uncertainties are replaced by stochastic state-multiplicative ones. As a case study, we consider the well studied Threonine Synthesis pathway in E. coli, leading from Aspartate to the end product Threonine, via a complex five step mechanism. Following the linearization of each step in the pathway around pre-chosen, biochemical-sound, set-point, the original system is assembled as a stochastic linear system where the uncertainties in the enzymes concentrations appear as state-multiplicative stochastic terms in both the dynamical and the input matrices. The effect of the three negative feedback-loops that regulate the system dynamics is first investigated. Given a disturbance signal, we then study the sensitivity of the system to variations in certain kinetic and thermodynamical variables in order to asses the possible optimality of the system in the and the senses.

Secure tracking control against sensor and actuator attacks: A robust model-reference adaptive control method

This paper is concerned with secure tracking control problems for cyber-physical systems subjected to a class of sensor and actuator attacks via the model-reference control techniques. The considered attack signals are the unknown bounded signals generated from any linear or nonlinear finite-L2-gain exogenous dynamical system. The auxiliary outputs produced from a high-gain approximate differentiator are first used in co-design for the inputs of the systems and introduced filter, so that the matching condition on the system input and output matrices in the existing results is relaxed. Furthermore, with the aid of the auxiliary outputs, attack’s property and small-gain control methods, a novel model-reference controller with a robust adaptive attack compensator is proposed, such that (a) the tracking error converges to a small set around zero where the set bound can be adjusted by the design parameters, and (b) the desired disturbance attenuation performance is obtained. Different from the existing results, the developed one not only compensates the effect of the attacks automatically by online adjusting the adaptive parameter, but also improves the robustness against the attacks efficiently. Finally, the developed method is validated by a simulation example.

Sliding mode learning control for T‐S fuzzy system and an application to hypersonic flight vehicle

AbstractSliding mode‐based learning control is presented for T‐S fuzzy system. A T‐S fuzzy model with both uncertainties and unmodeled dynamics is proposed firstly, in which the information of uncertainties and unmodeled dynamics are assumed to be unknown. Then, according to a given reference model, state‐tracking error system is built. Respecting facts, the input matrices of the built T‐S fuzzy model are different from each other. An extended state observer is built for estimating the unknown uncertainties and unmodeled dynamics, and a corresponding sliding surface is proposed. A learning controller is then presented for the closed loop system. Moreover, a numerical simulation result on hypersonic flight vehicles is considered to testify the controller's availability.

Zeroing Neural Network for Pseudoinversion of an Arbitrary Time-Varying Matrix Based on Singular Value Decomposition

Many researchers have investigated the time-varying (TV) matrix pseudoinverse problem in recent years, for its importance in addressing TV problems in science and engineering. In this paper, the problem of calculating the inverse or pseudoinverse of an arbitrary TV real matrix is considered and addressed using the singular value decomposition (SVD) and the zeroing neural network (ZNN) approaches. Since SVD is frequently used to compute the inverse or pseudoinverse of a matrix, this research proposes a new ZNN model based on the SVD method as well as the technique of Tikhonov regularization, for solving the problem in continuous time. Numerical experiments, involving the pseudoinversion of square, rectangular, singular, and nonsingular input matrices, indicate that the proposed models are effective for solving the problem of the inversion or pseudoinversion of time varying matrices.

Statistical Analysis on Random Matrices of Echo State Network in PEMFC System’s Lifetime Prediction

The data-driven method of echo state network (ESN) has been successfully used in the proton exchange membrane fuel cell (PEMFC) system’s lifetime prediction area. Nevertheless, the uncertainties of the randomly generated input and internal weight matrices in ESN have not been reported yet. In view of this, an ensemble ESN structure is proposed in this paper to analyze the effects of random matrices from a statistical point of view. For each ESN, the particle swarm optimization (PSO) method is utilized to optimize the hyperparameters of the leaking rate, spectral radius, and regularization coefficient. The statistical results of each ensemble ESN are analyzed from 100 repeated tests whose weight matrices are generated randomly. The mean value of the ensemble ESN and a confidence interval (CI) of 95% are given during the long-term lifetime prediction. The effects of two different distribution shapes, i.e., uniform distribution and Gaussian distribution, are fully compared. Finally, the effects of the ensemble structure and two different distribution shapes are tested by three experimental datasets under steady-state, quasi-dynamic, and full dynamic operating conditions.

Incorporating non-empty initial states into MRP Theory

MRP theory is a theoretical body treating production-inventory systems, in which produced items are made up of sets of produced or purchased sub-items, required to be available a lead-time before each product is completed. The hierarchical dependence between items is captured using input matrices from input-output-analysis, the necessary advanced timing by employing Laplace transform methodology, and the economic consequences by applying the net present value. Little attention has hitherto been given to aspects of a non-empty initial state, e.g. an initial inventory position. Since such states are common in industry, there is a strong need for this theory to include such aspects, in order to gain further practical acceptance. In this paper, theoretical consequences from having a non-empty initial state are investigated. A method for finding the Lot-for-Lot solution is developed using the concept of a truncated monotonically non-decreasing time function, generalising the approach of the generalised Leontief inverse and instrumental for designing plans meeting the necessary inner-corner requirement for optimality. Also the definition of inventory-related costs needs a modification for this concept to be consistent with NPV. These findings are applied in an extensive numerical example. Immediate future research concerns investigating principles for the optimal removal of initial backlogs.

SparseP

Several manufacturers have already started to commercialize near-bank Processing-In-Memory (PIM) architectures, after decades of research efforts. Near-bank PIM architectures place simple cores close to DRAM banks. Recent research demonstrates that they can yield significant performance and energy improvements in parallel applications by alleviating data access costs. Real PIM systems can provide high levels of parallelism, large aggregate memory bandwidth and low memory access latency, thereby being a good fit to accelerate the Sparse Matrix Vector Multiplication (SpMV) kernel. SpMV has been characterized as one of the most significant and thoroughly studied scientific computation kernels. It is primarily a memory-bound kernel with intensive memory accesses due its algorithmic nature, the compressed matrix format used, and the sparsity patterns of the input matrices given. This paper provides the first comprehensive analysis of SpMV on a real-world PIM architecture, and presents SparseP, the first SpMV library for real PIM architectures. We make three key contributions. First, we implement a wide variety of software strategies on SpMV for a multithreaded PIM core, including (1) various compressed matrix formats, (2) load balancing schemes across parallel threads and (3) synchronization approaches, and characterize the computational limits of a single multithreaded PIM core. Second, we design various load balancing schemes across multiple PIM cores, and two types of data partitioning techniques to execute SpMV on thousands of PIM cores: (1) 1D-partitioned kernels to perform the complete SpMV computation only using PIM cores, and (2) 2D-partitioned kernels to strive a balance between computation and data transfer costs to PIM-enabled memory. Third, we compare SpMV execution on a real-world PIM system with 2528 PIM cores to an Intel Xeon CPU and an NVIDIA Tesla V100 GPU to study the performance and energy efficiency of various devices, i.e., both memory-centric PIM systems and conventional processor-centric CPU/GPU systems, for the SpMV kernel. SparseP software package provides 25 SpMV kernels for real PIM systems supporting the four most widely used compressed matrix formats, i.e., CSR, COO, BCSR and BCOO, and a wide range of data types. SparseP is publicly and freely available at https://github.com/CMU-SAFARI/SparseP. Our extensive evaluation using 26 matrices with various sparsity patterns provides new insights and recommendations for software designers and hardware architects to efficiently accelerate the SpMV kernel on real PIM systems.

Task-parallel tiled direct solver for dense symmetric indefinite systems

This paper proposes a direct solver for symmetric indefinite linear systems. The program is parallelized via the OpenMP task construct and outperforms existing programs. The proposed solver avoids pivoting, which requires a lot of data movement, during factorization with preconditioning using the symmetric random butterfly transformation. The matrix data layout is tiled after the preconditioning to more efficiently use cache memory during factorization. Given the low-rank property of the input matrices, an adaptive crossing approximation is used to make a low-rank approximation before the update step to reduce the computation load. Iterative refinement is then used to improve the accuracy of the final result. Finally, the performance of the proposed solver is compared to that of various symmetric indefinite linear system solvers to show its superiority.

UINMF performs mosaic integration of single-cell multi-omic datasets using nonnegative matrix factorization

Single-cell genomic technologies provide an unprecedented opportunity to define molecular cell types in a data-driven fashion, but present unique data integration challenges. Many analyses require “mosaic integration”, including both features shared across datasets and features exclusive to a single experiment. Previous computational integration approaches require that the input matrices share the same number of either genes or cells, and thus can use only shared features. To address this limitation, we derive a nonnegative matrix factorization algorithm for integrating single-cell datasets containing both shared and unshared features. The key advance is incorporating an additional metagene matrix that allows unshared features to inform the factorization. We demonstrate that incorporating unshared features significantly improves integration of single-cell RNA-seq, spatial transcriptomic, SNARE-seq, and cross-species datasets. We have incorporated the UINMF algorithm into the open-source LIGER R package (https://github.com/welch-lab/liger).