Multiplicative Update Research Articles

Multiplicative updates for symmetric-cone factorizations

Multiplicative updates for symmetric-cone factorizations

Global Interconnect Optimization

We propose a new comprehensive solution to global interconnect optimization. Traditional buffering algorithms mostly insert repeaters on a net-by-net basis based on slacks and possibly guided by global wires. We show how to integrate routing congestion, placement congestion, global timing constraints, power consumption, and additional constraints into a single resource sharing formulation. The core of our algorithm is a new buffered routing subroutine. Given a net and Lagrangean resource prices for routing, timing, placement, and power, it computes a buffered Steiner tree. The resource sharing framework provides a special multiplicative price update for fast convergence. Our algorithm is fast enough for practical instances. We demonstrate experimentally on 7nm microprocessor units that it significantly improves timing while reducing netlength and power consumption in an industrial design flow. Our implementation scales well under parallelization with up to 128 threads.

A bicriteria approximation algorithm for the minimum hitting set problem in measurable range spaces

We consider the problem of finding a minimum-size hitting set in a range space F=(Q,R) defined by a measure on a family of subsets of an infinite set R. We observe that, under reasonably general assumptions, the infinite-dimensional convex relaxation can be solved (approximately) efficiently by multiplicative weight updates. As a consequence, we get an algorithm that finds, for any δ>0, a set of size O(αFzF⁎) that hits a (1−δ)-fraction of the ranges in R (with respect to the given measure) in time proportional to log⁡1δ, where zF⁎ is the value of the fractional optimal solution and αF is the integrality gap of the standard LP relaxation of the hitting set problem for the restriction of F on a set of points of size O(zF⁎log⁡1δ). Some applications of this result in Computational Geometry are given.

HMT: A Hardware-centric Hybrid Bonsai Merkle Tree Algorithm for High-performance Authentication

The Bonsai Merkle tree (BMT) is a widely used tree structure for authentication of metadata such as encryption counters in a secure computing system. Common BMT algorithms were designed for traditional Von Neumann architectures with a software-centric implementation in mind and as such, they are predominantly recursive and sequential in nature. However, the modern heterogeneous computing platforms employing Field-Programmable Gate Array (FPGA) devices require concurrency-focused algorithms to fully utilize the versatility and parallel nature of such systems. The recursive nature of traditional BMT algorithms makes them challenging to implement in such hardware-based setups. Our goal for this work is to introduce HMT, a hardware-friendly BMT algorithm that enables the verification and update processes to function independently and provides the benefits of relaxed update while being comparable to the eager update in terms of update complexity. The methodology of HMT contributes both novel algorithmic revisions and innovative hardware techniques to implementing BMT. We mathematically demonstrate the challenges of potentially unbounded recursions in relaxed BMT updates. To solve this problem, we use a partitioned BMT caching scheme that allocates a separate write-back cache for each BMT level—thus allowing for low and fixed upper bounds for dirty evictions compared to the traditional BMT caches. Then we introduce the aforementioned hybrid BMT algorithm that is hardware-targeted, parallel, and relaxes the update depending on BMT cache hit but makes the update conditions more flexible compared to lazy update to save additional write-backs. Deploying this new algorithm, we have designed a new BMT controller with a dataflow architecture including speculative buffers and parallel write-back engines to facilitate performance-enhancing mechanisms (like multiple concurrent authentication and independent updates) that were not possible with the conventional lazy algorithm. Our empirical performance measurements on a Xilinx U200 accelerator FPGA have demonstrated that HMT can achieve up to 7× improvement in bandwidth and 4.5× reduction in latency over lazy-update BMT baseline and up to 14% faster execution in standard benchmarks compared to a state-of-the-art, eager-update BMT solution.

Analog Ion‐Slicing LiNbO3 Memristor Based on Hopping Transport for Neuromorphic Computing

Inspired by human brain, the emerging analog‐type memristor employed in neuromorphic computing systems has attracted tremendous interest. However, existing analog memristors are still far from accurate tuning of multiple conductance states, which are crucial from the device‐level view. Herein, a reliable analog memristor based on ion‐slicing single‐crystalline LiNbO3 (LNO) thin film is demonstrated. The highly ordered LNO crystal structure provides a stable pathway of oxygen vacancy migration, which is contributed to a stable Mott variable‐range hopping process in trap sites. Excellent analog switching characteristics with high reliability and repeatability, including long retention/great endurance with small fluctuation (fluctuated within 0.22%), a large dynamic range of two orders of magnitude, hundreds of distinguishable conductance states with tunable linearity, and ultralow cyclic variances for multiple weight updating (down to 0.75%), are realized with the proposed memristor. As a result, a multilayer perceptron with a high recognition accuracy of 95.6% for Modified National Institute of Standards and Technology dataset is realized. The proposed analog memristive devices based on ion‐slicing single‐crystalline thin films offer a novel strategy for fabricating high‐performance memristors that combined linear tunability and long‐term repeatability, opening a novel avenue for neuromorphic computing application.

Integrated Energy Station Optimal Dispatching Using a Novel Many-Objective Optimization Algorithm Based on Multiple Update Strategies

Regarding the need to decrease carbon emissions, the electric vehicle (EV) industry is growing rapidly in China; the charging needs of EVs require the number of EV charging stations to grow significantly. Therefore, many refueling stations have been modified to integrated energy stations, which contain photovoltaic systems. The key issue in current times is to figure out how to operate these integrated energy stations in an efficient way. Therefore, an effective scheduling model is needed to operate an integrated energy station. Photovoltaic (PV) and energy storage systems are integrated into EV charging stations to transform them into integrated energy stations (PE-IES). Considering the demand for EV charging during different time periods, the PV output, the loss rate of energy storage systems, the load status of regional grids, and the dynamic electricity prices, a multi-objective optimization scheduling model was established for operating integrated energy stations that are connected to a regional grid. The model aims to simultaneously maximize the daily profits of the PE-IES, minimize the daily loss rate of the energy storage system, and minimize the peak-to-valley difference of the load in the regional grid. To validate the effectiveness of the model, simulation experiments under three different scenarios for the PE-IES were conducted in this research. Each object weight was determined using the entropy weight method, and the optimal solution was selected from the Pareto solution set using an order-preference technique according to the similarity to an ideal solution (TOPSIS). The results demonstrate that, compared to traditional charging stations, the daily revenue of the PE-IES stations increases by 26.61%, and the peak-to-valley difference of the power load in the regional grid decreases by 30.54%, respectively. The effectiveness of PE-IES is therefore demonstrated. Furthermore, to solve the complex optimization problem for PE-IES, a novel multi-objective optimization algorithm based on multiple update strategies (MOMUS) was proposed in this paper. To evaluate the performance of the MOMUS, a detailed comparison with seven other algorithms was demonstrated. These results indicate that our algorithm exhibits an outstanding performance in solving this optimization problem, and that it is capable of generating high-quality optimal solutions.

Deep Multiplicative Update Algorithm for Nonnegative Matrix Factorization and Its Application to Audio Signals

Deep Multiplicative Update Algorithm for Nonnegative Matrix Factorization and Its Application to Audio Signals

Guided wave signal‐based sensing and classification for small geological structure

AbstractSensing, Computing and Communication Integration (SC2) is widely believed as a new enabling technology. A non‐negative tensor sparse factorisation (NTSF) algorithm based on tensor analysis is proposed for sensing and classification of Small Geological Structure in coal mines. Utilising this method, advanced detection of geological anomalies hidden in coal seams was achieved. The morphological properties of geological anomalies in coal seams and the propagation characteristics of guided waves were first thoroughly studied. A three‐dimensional (3D) medium geometry model was developed for a complicated coal seam with Goaf, collapse column, scouring zone, and tiny fault based on COMSOL Multiphysics. On this model, the third‐order tensors data was constructed. Then, the TUCKER‐based NTSF algorithm was employed for feature extraction and classification. To achieve multi‐dimensional feature, the two‐dimensional data in the form of a matrix is collected, and a multiplicative update method is introduced to update the algorithm iteratively. Finally, the Support Vector Machine (SVM) multi‐classifier with Gaussian radial basis kernel function is selected for classification of Small Geological Structure. The experimental results show that the classification accuracy based on the NTSF and SVM is as high as 97.33%, which demonstrates that the proposed algorithm is suitable for Sensing and Classification of Small Geological Structure in coal mines.

ACE: Cooperative Multi-Agent Q-learning with Bidirectional Action-Dependency

Multi-agent reinforcement learning (MARL) suffers from the non-stationarity problem, which is the ever-changing targets at every iteration when multiple agents update their policies at the same time. Starting from first principle, in this paper, we manage to solve the non-stationarity problem by proposing bidirectional action-dependent Q-learning (ACE). Central to the development of ACE is the sequential decision making process wherein only one agent is allowed to take action at one time. Within this process, each agent maximizes its value function given the actions taken by the preceding agents at the inference stage. In the learning phase, each agent minimizes the TD error that is dependent on how the subsequent agents have reacted to their chosen action. Given the design of bidirectional dependency, ACE effectively turns a multi-agent MDP into a single-agent MDP. We implement the ACE framework by identifying the proper network representation to formulate the action dependency, so that the sequential decision process is computed implicitly in one forward pass. To validate ACE, we compare it with strong baselines on two MARL benchmarks. Empirical experiments demonstrate that ACE outperforms the state-of-the-art algorithms on Google Research Football and StarCraft Multi-Agent Challenge by a large margin. In particular, on SMAC tasks, ACE achieves 100% success rate on almost all the hard and super hard maps. We further study extensive research problems regarding ACE, including extension, generalization and practicability.

Hypergraph-Regularized Lp Smooth Nonnegative Matrix Factorization for Data Representation

Nonnegative matrix factorization (NMF) has been shown to be a strong data representation technique, with applications in text mining, pattern recognition, image processing, clustering and other fields. In this paper, we propose a hypergraph-regularized Lp smooth nonnegative matrix factorization (HGSNMF) by incorporating the hypergraph regularization term and the Lp smoothing constraint term into the standard NMF model. The hypergraph regularization term can capture the intrinsic geometry structure of high dimension space data more comprehensively than simple graphs, and the Lp smoothing constraint term may yield a smooth and more accurate solution to the optimization problem. The updating rules are given using multiplicative update techniques, and the convergence of the proposed method is theoretically investigated. The experimental results on five different data sets show that the proposed method has a better clustering effect than the related state-of-the-art methods in the vast majority of cases.

Multi-strategy and self-adaptive differential sine–cosine algorithm for multi-robot path planning

Researchers have conducted studies using various metaheuristic algorithms for multi-robot path planning recently. One of these algorithms, sine–cosine algorithm (SCA) cannot produce satisfactory results in path planning problems, due to a single update strategy. It is necessary to adopt multiple update strategies and improve its performance for a wider set of problems. We have proposed a new multi-strategy self-adaptive differential sine–cosine algorithm (sdSCA) that uses a pool of strategies and allows for more frequent selection of strategies that lead to better solutions. Thus, dependency of SCA on a single strategy has been removed and it has become a more stable for a wider set of problems and convergence of SCA is improved. Firstly, effectiveness of sdSCA was tested in CEC2015 benchmark functions and CEC2020 real-world optimization problems. Performance of sdSCA at these tests is satisfactory. Secondly, sdSCA was applied to online multi-robot path planning in complex environments with static and dynamic obstacles. In this path planning simulation, the proposed algorithm achieved an average improvement of 42% compared to SCA. It also appears to produce results superior to state-of-the-art metaheuristic algorithms.

Priority Criteria (PC) Based Particle Swarm Optimization of Reinforced Concrete Frames (PCPSO)

Simplicity and optimality are commonly associated with the particle swarm optimization (PSO) algorithm. As a result, numerous variants and hybrids of PSO have been developed and implemented to address structural optimization problems. The undeniable importance of the initialization technique in determining the overall performance of a given optimization algorithm cannot be overstated. Optimization algorithms, such as PSO, typically rely on a random, uniformly distributed initialization. Through multiple iterations and updates, these algorithms aim to achieve optimal results. The underlying assumption behind such an initialization approach is that a fair or reasonable arrangement of particles is best accomplished through randomization, and thus the entire optimization process is iterated based on this assumption. However, this initialization technique raises concerns regarding the attainment of optimality and convergence, leaving room for further examination. In this paper, we challenge this assumption by introducing a priority concept. The key idea is that particles should not be initialized randomly since randomness alone does not guarantee a reasonable allocation of design variable values in iterative optimization. This can lead to misguided velocity updates and ultimately, a time-consuming pursuit of optimality. To address this issue, we formulate priority criteria (PC) and propose an enhanced PSO variant called Priority Criteria PSO (PCPSO). The PC can be incorporated into any PSO variant or hybrid without impacting the parameter settings, constraints, and penalty approaches of the respective algorithms. A case study involving 2D reinforced concrete frames was conducted to compare the performance of the ordinary PSO algorithm with the PCPSO. The results clearly demonstrate that the introduction of the PC leads to a significant cost reduction when compared to PSO with an inertia damping factor. Additionally, the PCPSO algorithm exhibits accelerated convergence. Furthermore, to alleviate the computational burden associated with structural analysis at each iteration, a reanalysis approach called Combined Approximations (CA) is mathematically formulated and implemented.

HTA63 The Data Lifecycle: Towards a Standardized Relational Data Model to Support Living Systematic Literature Reviews? – Use Case Demonstration with LiveSLR®

Systematic literature reviews (SLR) offer an unbiased evidence summary for informing policy. Keeping SLRs current is paramount for healthcare decision making. However, traditional, static SLRs cannot accommodate constant data inflow especially resulting from multiple updates of studies. LiveSLR® is an up-to-date Cochrane/NICE-compliant curated SLR library following PRISMA standards, maintained by experienced reviewers. We seek understanding if the LiveSLR® data model could bridge this gap, leading to a “living" workflow.

TopoBERT: Exploring the topology of fine-tuned word representations

Transformer-based language models such as BERT and its variants have found widespread use in natural language processing (NLP). A common way of using these models is to fine-tune them to improve their performance on a specific task. However, it is currently unclear how the fine-tuning process affects the underlying structure of the word embeddings from these models. We present TopoBERT, a visual analytics system for interactively exploring the fine-tuning process of various transformer-based models – across multiple fine-tuning batch updates, subsequent layers of the model, and different NLP tasks – from a topological perspective. The system uses the mapper algorithm from topological data analysis (TDA) to generate a graph that approximates the shape of a model’s embedding space for an input dataset. TopoBERT enables its users (e.g. experts in NLP and linguistics) to (1) interactively explore the fine-tuning process across different model-task pairs, (2) visualize the shape of embedding spaces at multiple scales and layers, and (3) connect linguistic and contextual information about the input dataset with the topology of the embedding space. Using TopoBERT, we provide various use cases to exemplify its applications in exploring fine-tuned word embeddings. We further demonstrate the utility of TopoBERT, which enables users to generate insights about the fine-tuning process and provides support for empirical validation of these insights.

Non-negative matrix factorization for overlapping community detection in directed weighted networks with sparse constraints.

Detecting overlapping communities is essential for analyzing the structure and function of complex networks. However, most existing approaches only consider network topology and overlook the benefits of attribute information. In this paper, we propose a novel attribute-information non-negative matrix factorization approach that integrates sparse constraints and optimizes an objective function for detecting communities in directed weighted networks. Our algorithm updates the basic non-negative matrix adaptively, incorporating both network topology and attribute information. We also add a sparsity constraint term of graph regularization to maintain the intrinsic geometric structure between nodes. Importantly, we provide strict proof of convergence for the multiplication update rule used in our algorithm. We apply our proposed algorithm to various artificial and real-world networks and show that it is more effective for detecting overlapping communities. Furthermore, our study uncovers the intricate iterative process of system evolution toward convergence and investigates the impact of various variables on network detection. These findings provide insights into building more robust and operable complex systems.

Joint majorization-Minimization for nonnegative matrix factorization with the [formula omitted]-divergence

This article proposes new multiplicative updates for nonnegative matrix factorization (NMF) with the β-divergence objective function. Our new updates are derived from a joint majorization-minimization (MM) scheme, in which an auxiliary function (a tight upper bound of the objective function) is built for the two factors jointly and minimized at each iteration. This is in contrast with the classic approach in which a majorizer is derived for each factor separately. Like that classic approach, our joint MM algorithm also results in multiplicative updates that are simple to implement. They however yield a significant drop of computation time (for equally good solutions), in particular for some β-divergences of important applicative interest, such as the quadratic loss and the Kullback-Leibler or Itakura-Saito divergences. We report experimental results using diverse datasets: face images, an audio spectrogram, hyperspectral data and song play counts. Depending on the value of β and on the dataset, our joint MM approach can yield CPU time reductions from about 13% to 86% in comparison to the classic alternating scheme.

Scalable NMF via linearly optimized data compression.

Orthonormal projective non-negative matrix factorization (opNMF) has been widely used in neuroimaging and clinical neuroscience applications to derive representations of the brain in health and disease. The non-negativity and orthonormality constraints of opNMF result in intuitive and well-localized factors. However, the advantages of opNMF come at a steep computational cost that prohibits its use in large-scale data. In this work, we propose novel and scalable optimization schemes for orthonormal projective non-negative matrix factorization that enable the use of the method in large-scale neuroimaging settings. We replace the high-dimensional data matrix with its corresponding singular value decomposition (SVD) and QR decompositions and combine the decompositions with opNMF multiplicative update algorithm. Empirical validation of the proposed methods demonstrated significant speed-up in computation time while keeping memory consumption low without compromising the accuracy of the solution.

Fast and Accurate Non-Negative Latent Factor Analysis of High-Dimensional and Sparse Matrices in Recommender Systems

A fast non-negative latent factor (FNLF) model for a high-dimensional and sparse (HiDS) matrix adopts a Single Latent Factor-dependent, Non-negative, Multiplicative and Momentum-incorporated Update (SLF-NM <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> U) algorithm, which enables its fast convergence. It is crucial to achieve a rigorously theoretical proof regarding its fast convergence, which has not been provided in prior research. Aiming at addressing this critical issue, this work theoretically proves that with an appropriately chosen momentum coefficient, SLF-NM <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> U enables the fast convergence of an FNLF model in both continuous and discrete time cases. Empirical analysis of HiDS matrices generated by representative industrial applications provides empirical evidences for the theoretical proof. Hence, this study represents an important milestone in the field of HiDS matrix analysis.

A cascading l0 regularization reconstruction method in nonsubsampled contourlet domain for limited-angle CT

In some computed tomography (CT) practical applications, the projection data is incomplete, which makes some limited-angle artifacts presented in the reconstructed image. Total variation (TV) regularization method is used to suppress the artifacts, but the edge of reconstructed image is distorted or some tiny details are missed. To preserve the edge of reconstructed image, a cascading ℓ0 regularization-based reconstruction model in nonsubsampled contourlet transform (NSCT) domain is proposed, which considers both the directional peculiarity of artifacts and the smooth of low frequency of the reconstructed image. In the proposed reconstruction model, a ℓ0 regularization term in the high frequency is used to suppress the artifacts and noise, and a ℓ0 regularization of gradient image in low frequency is used to suppress the limited-angle artifacts, to preserve the edge of object and to smooth the reconstructed image. To solve the proposed model, a proximal alternating linearized Peaceman–Rachford splitting method (PAL-PRSM) is proposed, which uses the Peaceman–Rachford splitting method that updates the Lagrange multiplier twice, one after each minimization of the subproblem, to deal with the optimization problem firstly, and then the proximal linearization is used to avoid computing the inverse of a huge system matrix. In addition, to compensate the proximal linearization, we add an additional term into Lagrange multiplier updating. Real data experiments are investigated to demonstrate the effectiveness of PAL-PRSM method, and the results show that the proposed method outperforms two other CT reconstruction methods on preserving the edge of reconstructed image. On the whole, the proposed method is a compromise between denoising and preserving the edge of reconstructed image.

Alpha-divergence two-dimensional nonnegative matrix factorization for biomedical blind source separation

An alpha-divergence two-dimensional nonnegative matrix factorization (NMF2D) for biomedical signal separation is presented. NMF2D is a popular approach for retrieving low-rank approximations of nonnegative data such as image pixel, audio signal, data mining, pattern recognition and so on. In this paper, we concentrate on biomedical signal separation by using NMF2D with alpha-divergence family which decomposes a mixture into two-dimensional convolution factor matrices that represent temporal code and the spectral basis. The proposed iterative estimation algorithm (alpha-divergence algorithm) is initialized with random values, and it updated using multiplicative update rules until the values converge. Simulation experiments were carried out by comparing the original and estimated signal in term of signal-to-distortion ratio (SDR). The performances have been evaluated by including and excluding the sparseness constraint which sparseness is favored by penalizing nonzero gains. As a result, the proposed algorithm improved the iteration speed and sparseness constraints produce slight improvement of SDR.