Matrix Vector Research Articles

The Development of Fast DST-II Algorithms for Short-Length Input Sequences

The subject of this work is the development of fast algorithms for the discrete sinusoidal transformation of the second type (DST-II) for sequences of input data of small length N = 2, 3, 4, 5, 6, 7, 8. The starting point for the development of algorithms is the well-known possibility of representing any discrete transformation in the form of a matrix–vector product. Due to the remarkable structural properties of the matrices of the DST-II transformation base, these matrices can be successfully factorized, which should lead to a reduction in the computational complexity of the procedure as a whole. You can factorize matrices in different ways. The art of designing fast algorithms is to find the factorization that produces the maximum effect. We justified the correctness of the obtained algorithmic solutions theoretically, using strict mathematical derivations of each of them. The developed algorithms were then further tested using MATLAB R2023b software to finally confirm their performance. Finally, we presented estimates of the computational complexity for each solution obtained and compared them with direct computational methods that rely on the direct calculation of matrix–vector products.

High dimensional discriminant rules with shrinkage estimators of the covariance matrix and mean vector

Linear discriminant analysis (LDA) is a typical method for classification problems with large dimensions and small samples. There are various types of LDA methods that are based on the different types of estimators for the covariance matrices and mean vectors. In this paper, we consider shrinkage methods based on a non-parametric approach. For the precision matrix, methods based on the sparsity structure or data splitting are examined. Regarding the estimation of mean vectors, Non-parametric Empirical Bayes (NPEB) methods and Non-parametric Maximum Likelihood Estimation (NPMLE) methods, also known as f-modeling and g-modeling, respectively, are adopted. The performance of linear discriminant rules based on combined estimation strategies of the covariance matrix and mean vectors are analyzed in this study. Particularly, the study presents a theoretical result on the performance of the NPEB method and compares it with previous studies. Simulation studies with various covariance matrices and mean vector structures are conducted to evaluate the methods discussed in this paper. Furthermore, real data examples such as gene expressions and EEG data are also presented.

On topological bound states and secular equations for quantum-graph eigenvalues

Quantum graphs without interaction which contain equilateral cycles possess “topological” bound states which do not correspond to zeros of one of the two variants of the secular equation for quantum graphs. Instead, their eigenvalues lie in the set of singularities of the vertex-scattering secular matrix. This observation turns out to be representative of a wider phenomenon. We introduce a notion of topological bound states and show that they are linear combinations of functions supported on generators of the fundamental group of the graph (hence the “topological” in the name), including for graphs that have interactions on the edges. Using an Ihara-style theorem, we elucidate the role of such topological bound states in the spectral analysis of quantum graph Hamiltonians using secular matrices. En route we determine the set of the fixed vectors of the bond-scattering matrix.

Efficiency analysis of discontinuous Galerkin approaches for the application onto quantum Liouville-type equations

The simulation of nanodevices is computationally inefficient with current algorithms. The discontinuous Galerkin approach has been demonstrated in the field of computational fluid dynamics to deliver high order accuracy and efficiency due to its reliance on matrix–vector multiplications. Previously, the discontinuous Galerkin approach was successfully used in conjunction with the finite volume technique to solve the Liouville–von Neumann equation in center-mass coordinates and thus simulate nanodevices. To exploit its full potential regarding high-performance computing, this work aims to substitute the aforementioned finite volume technique with the discontinuous Galerkin method. To arrive at the said formalism, a finite element method is implemented as an intermediate step.

A Common Source 3D FeFET with Disturb Inhibition Program and Erase Method

A common source p-type single-crystal channel three-dimensional ferroelectric field-effect transistor (3D FeFET) in a 2 × 2 × 3 array is proposed. Two programming and erasing conditions are introduced. A large memory window (>1.2 V), good retention (>10 years), and high speed (<100 ns) was presented under high voltage (±6 V) conditions. The endurance, >103, was observed under relatively low voltage (±3 V) conditions. Based on these two conditions, a novel asymmetric bias program and erase method is proposed to obtain good disturb inhibition. A more than 0.5 V threshold voltage shift in target cell was achieved while threshold voltage shift in unselected cell was limited, and analysis of long term disturb in novel method is proposed, showing good disturb inhibition. Additional investigation in word line disturbance shows causation and efficiency of disturb. Building upon the proposed structure of the 3D FeFET array, a vector matrix multiplication able to calculate 2-bit weights was designed and demonstrated. This work provides a potential solution for increasing integration density with 3D FeFET array.

Decoherence time control by interconnection for finite-level quantum memory systems

This paper is concerned with open quantum systems whose dynamic variables have an algebraic structure, similar to that of the Pauli matrices for finite-level systems. The Hamiltonian and the operators of coupling of the system to the external bosonic fields depend linearly on the system variables. The fields are represented by quantum Wiener processes which drive the system dynamics according to a quasilinear Hudson-Parthasarathy quantum stochastic differential equation whose drift vector and dispersion matrix are affine and linear functions of the system variables. This setting includes the zero-Hamiltonian isolated system dynamics as a particular case, where the system variables are constant in time, which makes them potentially applicable as a quantum memory. In a more realistic case of nonvanishing system-field coupling, we define a memory decoherence time when a mean-square deviation of the system variables from their initial values becomes relatively significant as specified by a weighting matrix and a fidelity parameter. We consider the decoherence time maximization over the energy parameters of the system and obtain a condition under which the zero Hamiltonian provides a suboptimal solution. This optimization problem is also discussed for a direct energy coupling interconnection of such systems.

A structural after measurement approach to structural equation modeling.

In structural equation modeling (SEM), the measurement and structural parts of the model are usually estimated simultaneously. In this article, we revisit the long-standing idea that we should first estimate the measurement part, and then estimate the structural part. We call this the "structural-after-measurement" (SAM) approach to SEM. We describe a formal framework for the SAM approach under settings where the latent variables and their indicators are continuous. We review earlier SAM methods and establish how they are specific instances of the SAM framework. Decoupled estimation for the measurement and structural parts using SAM possesses three key advantages over simultaneous estimation in standard SEM. First, estimates are more robust against local model misspecifications. Second, estimation routines are less vulnerable to convergence issues in small samples. Third, estimates exhibit smaller finite sample biases under correctly specified models. We propose two variants of the SAM approach. "Local" SAM expresses the mean vector and variance-covariance matrix of the latent variables as a function of the observed summary statistics and the parameters of the measurement model. "Global" SAM holds the parameters of the measurement part fixed while estimating the parameters of the structural part. Our framework includes two-step corrected standard errors, and permits computing both local and global fit measures. Nonetheless, the SAM approach is an estimation strategy, and should not be regarded as a model-building tool. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

CRPIM: An efficient compute-reuse scheme for ReRAM-based Processing-in-Memory DNN accelerators

Resistive random access memory (ReRAM) is a promising technology for AI Processing-in-Memory (PIM) hardware because of its compatibility with CMOS, small footprint, and ability to complete matrix–vector multiplication workloads inside the memory device itself. However, redundant computations are brought on by duplicate weights and inputs when an MVM has to be split into smaller-granularity sequential sub-works in the real world. Recent studies have proposed repetition-pruning to address this issue, but the buffer allocation strategy for enhancing buffer device utilization remains understudied. In preliminary experiments observing input patterns of neural layers with different datasets, the similarity of repetition allows us to transfer the buffer allocation strategy obtained from a small dataset to the computation with a large dataset. Hence, this paper proposes a practical compute-reuse mechanism for ReRAM-based PIM, called CRPIM, which replaces repetitive computations with buffering and reading. Moreover, the subsequent buffer allocation problem is resolved at both inter-layer and intra-layer levels. Our experimental results demonstrate that CRPIM significantly reduces ReRAM cells and execution time while maintaining adequate buffer and energy overhead.

Binarized Neural Network Comprising Quasi‐Nonvolatile Memory Devices for Neuromorphic Computing

AbstractThis study presents a binarized neural network (BNN) comprising quasi‐nonvolatile memory (QNVM) devices that operate in a positive feedback loop mechanism and exhibit an extremely low subthreshold swing (≤ 5 mV dec−1) and a high on/off ratio (≥ 107). A pair of QNVM devices are used for a single synaptic cell in a cell array, in which its memory state represents the synaptic weight, and the voltages applied to the pair act as input in a complementary fashion. The array of synaptic cells performs matrix multiply‐accumulate (MAC) operations between the weight matrix and input vector using XNOR and current summation. All the results of the MAC operations and vector‐matrix multiplications are equivalent. Moreover, the BNN features a high accuracy of 93.32% in the MNIST image recognition simulation owing to high device uniformity (1.35%), which demonstrates the feasibility of compact and high‐performance neuromorphic computing.

RSDA: A Reliable and Secure Data Aggregation Scheme for Smart Grid

Smart grids are modern power distribution systems that utilize advanced technologies to optimize energy generation, distribution, and consumption. The huge amount of data generated by various sources, such as smart meters, sensors, and other devices, can be aggregated for efficient energy management, supporting demand response etc. However, this data can contain sensitive information about individual’s energy usage patterns, behaviour, and preferences, which raises several privacy concerns. Data reliability is another major issue, as the data send from the smart meters to the aggregators may get corrupted due to errors introduced by the communication channel. In this paper, a private-key additive homomorphic encryption scheme based on polar codes is proposed for privacy preserving data aggregation. The proposed scheme not only ensures data confidentiality but also provide data reliability. The security of this scheme relies on the difficulty in decoding the codeword and is achieved by embedding security in the generator matrix and error vector. Security analysis shows that the proposed scheme is resistant against all known attacks against private-key code-based cryptosystems. The performance analysis is evaluated in terms of error performance, communication overhead and computational complexity. The performance comparison with related additive homomorphic encryption scheme shows the efficiency of the proposed scheme.

Variable transformations in combination with wavelets and ANOVA for high-dimensional approximation

We use hyperbolic wavelet regression for the fast reconstruction of high-dimensional functions having only low-dimensional variable interactions. Compactly supported periodic Chui-Wang wavelets are used for the tensorized hyperbolic wavelet basis on the torus. With a variable transformation, we are able to transform the approximation rates and fast algorithms from the torus to other domains. We perform and analyze scattered data approximation for smooth but arbitrary density functions by using a least squares method. The corresponding system matrix is sparse due to the compact support of the wavelets, which leads to a significant acceleration of the matrix vector multiplication. For non-periodic functions, we propose a new extension method. A proper choice of the extension parameter together with the piecewise polynomial Chui-Wang wavelets extends the functions appropriately. In every case, we are able to bound the approximation error with high probability. Additionally, if the function has a low effective dimension (i.e., only interactions of a few variables), we qualitatively determine the variable interactions and omit ANOVA terms with low variance in a second step in order to decrease the approximation error. This allows us to suggest an adapted model for the approximation. Numerical results show the efficiency of the proposed method.

Accuracy of linear transformations performed on a nonideal Mach–Zehnder interferometer

We consider a mathematical model of a planar Mach–Zehnder interferometer with nonideal beam splitters. For this model, we obtain two fidelity estimates of the matrix–vector multiplication in the form of dependences of the multiplication error on the beam splitting error in directional couplers. The first estimate is obtained as a measure of the difference between the transfer matrices implemented by interferometers and is presented in the form of the norm of the difference between two unitary matrices corresponding to ideal and non-ideal interferometers. The second estimate is obtained as a measure of the difference in output intensities. It is shown that in the latter case the fidelity depends both on the beam splitter error and on the parameters of the input signals. We verified the second estimate using computer simulations of MZI in COMSOL Multiphysics.

Efficient Task-driven Video Data Privacy Protection for Smart Camera Surveillance System

As one of the most commonly used AIoT sensors, smart cameras and their supporting services, namely cloud video surveillance (CVS) systems, have brought great convenience to people’s lives. Recent CVS providers use different machine learning techniques to improve their services (regarded as tasks) based on the uploaded video. However, uploading data to the CVS providers may cause severe privacy issues. Existing works that remove privacy information could not achieve a high tradeoff between data usability and privacy, because the importance of information varies with the task. In addition, it is challenging to design a real-time privacy protection mechanism, especially in resource-constrained smart cameras. In this work, we design a task-driven and efficient video privacy protection mechanism for a better tradeoff between privacy and data usability. We use Class Activation Mapping to protect privacy while preserving data usability. To improve the efficiency, we utilize the motion vector and residual matrix produced during video codec. Our work outperforms the region of interest–based methods in data protection while preserving data usability. The attack accuracy drops 70%, while the task accuracy is comparable to those without protection (within ± 4%). The average protection frame rate of the High Definition video can exceed 16 fps+ even on a CPU.

Vectorization of the density matrix and quantum simulation of the von Neumann equation of time-dependent Hamiltonians

Based oh the properties of Lie algebras, in this work we develop a general framework to linearize the von Neumann equation rendering it in a suitable form for quantum simulations. Departing from the conventional method of expanding the density matrix in the Liouville space formed by matrices unit we express the von Neumann equation in terms of Pauli strings. This provides several advantages related to the quantum tomography of the density matrix and the formulation of the unitary gates that generate the time evolution. The use of Pauli strings facilitates the quantum tomography of the density matrix whose elements are purely real. As for any other basis of Hermitian matrices, this eliminates the need to calculate the phase of the complex entries of the density matrix. This approach also enables to express the evolution operator as a sequence of commuting Hamiltonian gates of Pauli strings that can readily be synthetized using Clifford gates. Additionally, the fact that these gates commute with each other along with the unique properties of the algebra formed by Pauli strings allows to avoid the use of Trotterization hence considerably reducing the circuit depth. The algorithm is demonstrated for three Hamiltonians using the IBM noisy quantum circuit simulator.

Knowledge-dominated and data-driven rigid-flexible coupling dynamics for rotating flexible structure

Solving a rigid–flexible coupling nonlinear dynamic equation necessitates updating dynamic quantities in each time step, which is the main factor that hampers computational efficiency. Therefore, this study develops an innovative framework that combines knowledge-based and data-driven solutions, for which a data-driven numerical integration (DDNI) method is established to update the mass matrix and load vector. The DDNI model is obtained from offline training based on a deep neural network model and is embedded in the floating frame of reference formulation for online simulation of rigid–flexible coupling problems. As benchmark cases, planar and curved shell structures are analyzed, and the results show that the DDNI method exhibits excellent performance in terms of accuracy and adaptability. In particular, compared to a fully numeric integration (FNI) scheme, the DDNI method shows significantly improved efficiency, surpassing that of commercial software. These noteworthy characteristics aid the investigation of intricate structures in engineering problems. We utilize the developed DDNI method in a rigid–flexible coupling analysis of a manipulator, which shows that it is highly time-efficient, approximately 20 times faster than the FNI scheme, while producing results that are consistent with those of experiments.

Adaptive dynamic programming and distributionally robust optimal control of linear stochastic system using the Wasserstein metric

SummaryIn this paper, we consider the optimal control of unknown stochastic dynamical system for both the finite‐horizon and infinite‐horizon cases. The objective of this paper is to find an optimal controller to minimize the expected value of a function which depends on the random disturbance. Throughout this paper, it is assumed that the mean vector and covariance matrix of the disturbance distribution is unknown. An uncertainty set in the space of mean vector and the covariance matrix is introduced. For the finite‐horizon case, we derive a closed‐form expression of the unique optimal policy and the opponents policy that generates the worst‐case distribution. For the infinite‐horizon case, we simplify the Riccati equation obtained in the finite‐hozion setting to an algebraic Riccati equation, which can guarantee the existence of the solution of the Riccati equation. It is shown that the resulting optimal policies obtained in these two cases can stabilize the expected value of the system state under the worst‐case distribution. Furthermore, the unknown system matrices can also be explicitly computed using the adaptive dynamic programming technique, which can help compute the optimal control policy by solving the algebraic Riccati equation. Finally, a simulation example is presented to demonstrate the effectiveness of our theoretical results.

Multivariate Approach for Texture Segmentation using Probabilistic Statistical Model

The analysis of the regions of the image is of the prerogatives in the fields of medical and global systems meant for location identification. This analysis is strongly associated with partitions of regions of interest such as segmentation. For an effective strategy of analyzing the regions of interest, texture of the image plays a major concern. The texture is generally characterized using signal processing methods namely Discrete Cosine Transformation coefficients and their specific insights leading to feature vector selection. Further, to identify regions, a statistical model needs to be identified for the feature matrix vector and thus make use of Gaussian mixture model with extensions. The Expectation Maximization approach is used, and performance is assessed by experimenting with random images from the Brodatz data store domain. Performance measurements for texture segmentation that can be attributed are Global Cons. Error (GC), Prob. Rand. Index (PR) and Variation of data (VA). These are determined alongside the confusion matrix. To assess the improvement, a comparison was made with other existing models and showed better. The algorithms will be exceptionally helpful for clinical analysis and in radio navigation map systems.

Stochastic dynamics of mechanical systems with impacts via the Step Matrix multiplication based Path Integration method

AbstractIn this work we propose the Step Matrix Multiplication based Path Integration method (SMM-PI) for nonlinear vibro-impact oscillator systems. This method allows the efficient and accurate deterministic computation of the time-dependent response probability density function by transforming the corresponding Chapman–Kolmogorov equation to a matrix–vector multiplication using high-order numerical time-stepping and interpolation methods. Additionally, the SMM-PI approach yields the computation of the joint probability distribution for response and impact velocity, as well as the time between impacts and other important characteristics. The method is applied to a nonlinear oscillator with a pair of impact barriers, and to a linear oscillator with a single barrier, providing relevant densities and analysing energy accumulation and absorption properties. We validate the results with the help of stochastic Monte-Carlo simulations and show the superior ability of the introduced formulation to compute accurate response statistics.

Computing a compact local Smith–McMillan form

We define a compact local Smith–McMillan form of a rational matrix R ( λ ) as the diagonal matrix whose diagonal elements are the nonzero entries of a local Smith-McMillan form of R ( λ ) . We show that a recursive rank search procedure, applied to a block-Toeplitz matrix built on the Laurent expansion of R ( λ ) around an arbitrary complex point λ 0 , allows us to compute a compact local Smith-McMillan form of that rational matrix R ( λ ) at the point λ 0 , provided we keep track of the transformation matrices used in the rank search. It also allows us to recover the root polynomials of a polynomial matrix and root vectors of a rational matrix, at an expansion point λ 0 . Numerical tests illustrate the promising performance of the resulting algorithm.

Feed Forward Neural Network With Column-Wise Matrix–Vector Multiplication on FPGAs

This article presents a reconfigurable accelerator for recurrent neural networks with fine grained column wise matrix vector multiplication (RENOWN).We propose a novel latency hiding architecture for recurrent neural network accelerator using column wise matrix vector multiplication instead of the state of the art row wise operation. This hardware (HW) architecture can eliminate data dependencies to improve the throughput of RNN inference systems. Besides, we introduce a configurable checkerboard tiling strategy which allows large weight matrices, while incorporating various configurations of element-based parallelism (EP) and vector-based parallelism (VP). These optimizations improve the exploitation of parallelism to increase HW utilization and enhance system throughput. Evaluation results show that our design can achieve over 29.6 tera operations per second (TOPS) which would be among the highest for field-programmable gate array (FPGA)-based RNN designs. Compared to state-of-the-art accelerators on FPGAs, our design achieves 3.7–14.8 times better performance and has the highest HW.