Linear Complexity Algorithm Research Articles

Fast Multiview Clustering by Optimal Graph Mining.

Multiview clustering (MVC) aims to exploit heterogeneous information from different sources and was extensively investigated in the past decade. However, far less attention has been paid to handling large-scale multiview data. In this brief, we fill this gap and propose a fast multiview clustering by an optimal graph mining model to handle large-scale data. We mine a consistent clustering structure from landmark-based graphs of different views, from which the optimal graph based on the one-hot encoding of cluster labels is recovered. Our model is parameter-free, so intractable hyperparameter tuning is avoided. An efficient algorithm of linear complexity to the number of samples is developed to solve the optimization problems. Extensive experiments on real-world datasets of various scales demonstrate the superiority of our proposal.

Features gradient-based signals selection algorithm of linear complexity for convolutional neural networks

<abstract><p>Recently, convolutional neural networks (CNNs) for classification by time domain data of multi-signals have been developed. Although some signals are important for correct classification, others are not. The calculation, memory, and data collection costs increase when data that include unimportant signals for classification are taken as the CNN input layer. Therefore, identifying and eliminating non-important signals from the input layer are important. In this study, we proposed a features gradient-based signals selection algorithm (FG-SSA), which can be used for finding and removing non-important signals for classification by utilizing features gradient obtained by the process of gradient-weighted class activation mapping (grad-CAM). When we defined $ n_ \mathrm{s} $ as the number of signals, the computational complexity of FG-SSA is the linear time $ \mathcal{O}(n_ \mathrm{s}) $ (i.e., it has a low calculation cost). We verified the effectiveness of the algorithm using the OPPORTUNITY dataset, which is an open dataset comprising of acceleration signals of human activities. In addition, we checked the average of 6.55 signals from a total of 15 signals (five triaxial sensors) that were removed by FG-SSA while maintaining high generalization scores of classification. Therefore, FG-SSA can find and remove signals that are not important for CNN-based classification. In the process of FG-SSA, the degree of influence of each signal on each class estimation is quantified. Therefore, it is possible to visually determine which signal is effective and which is not for class estimation. FG-SSA is a white-box signal selection algorithm because it can understand why the signal was selected. The existing method, Bayesian optimization, was also able to find superior signal sets, but the computational cost was approximately three times greater than that of FG-SSA. We consider FG-SSA to be a low-computational-cost algorithm.</p></abstract>

Optimizing vehicle dynamics co-simulation performance by introducing mesoscopic traffic simulation

Microscopic traffic simulation is often used in conjunction with vehicle dynamics simulation to test cooperative or perception-based driver assistance functions. On the other hand, visualization and the interaction of a large number of swarm vehicles is computationally burdensome, limiting the size of scenarios. To remedy this problem, this paper introduces a mesoscopic traffic model, namely the extension of the shockwave profile model to road networks, that handles traffic as continuous flows of vehicles. Adopting the idea of level of detail from computer graphics, the modeling of swarm vehicles is carried out in less detail farther away from the EGO vehicle. These levels are defined using the classical (macroscopic, mesoscopic, and microscopic) categorization of traffic modeling. The macroscopic traffic model is responsible for the traffic demand and traffic assignment. The proposed mesoscopic model is capable of capturing the fluctuating nature of traffic on lane level. Closer to the EGO vehicle microscopic traffic simulation is employed while the EGO vehicle is modeled in full-detail including vehicle dynamics too. The 3D rendering of the simulation is performed by the vehicle dynamics simulator. The challenge in the proposed methodology is transitioning between the mesoscopic and the microscopic models, i.e., selecting the boundary and spawning/destroying vehicle agents. The paper addresses this challenge with a linear (w.r.t. the vehicle number) time complexity algorithm. Practically, a dynamic downscaling of the microscopic simulation to mesoscopic level is realized outside the vicinity of the EGO vehicle. The proposed methodology is generic, and can be adapted to most existing vehicle dynamics and microscopic traffic simulator software. The solution is tested with SUMO as microsimulation and Carla as vehicle dynamics simulation through a simple path following test case in two scenarios: a congested residential area and a complex scenario with both a highway section and an urban area. Simulation results suggest that the simulation performance could be improved by 200–500% while retaining modeling accuracy, compared to the case when only microscopic simulation is used.

Multiple Kernel Clustering With Compressed Subspace Alignment.

Multiple kernel clustering (MKC) has recently achieved remarkable progress in fusing multisource information to boost the clustering performance. However, the O(n2) memory consumption and O(n3) computational complexity prohibit these methods from being applied into median- or large-scale applications, where n denotes the number of samples. To address these issues, we carefully redesign the formulation of subspace segmentation-based MKC, which reduces the memory and computational complexity to O(n) and O(n2) , respectively. The proposed algorithm adopts a novel sampling strategy to enhance the performance and accelerate the speed of MKC. Specifically, we first mathematically model the sampling process and then learn it simultaneously during the procedure of information fusion. By this way, the generated anchor point set can better serve data reconstruction across different views, leading to improved discriminative capability of the reconstruction matrix and boosted clustering performance. Although the integrated sampling process makes the proposed algorithm less efficient than the linear complexity algorithms, the elaborate formulation makes our algorithm straightforward for parallelization. Through the acceleration of GPU and multicore techniques, our algorithm achieves superior performance against the compared state-of-the-art methods on six datasets with comparable time cost to the linear complexity algorithms.

Efficient computations with counting functions on free groups and free monoids

We present efficient algorithms to decide whether two given counting functions on nonabelian free groups or monoids are at bounded distance from each other and to decide whether two given counting quasimorphisms on nonabelian free groups are cohomologous. We work in the multi-tape Turing machine model with nonconstant-time arithmetic operations. In the case of integer coefficients we construct an algorithm of linear time complexity (assuming that the rank is at least $3$ in the monoid case). In the case of rational coefficients we prove that the time complexity is $O(N\log N)$, where $N$ denotes the size of the input, that is, it is the same as in addition of rational numbers (implemented using the Harvey-van der Hoeven algorithm for integer multiplication). These algorithms are based on our previous work which characterizes bounded counting functions. Bibliography: 20 titles.

Point Cloud Sampling via Graph Balancing and Gershgorin Disc Alignment.

Point cloud (PC)-a collection of discrete geometric samples of a 3D object's surface-is typically large, which entails expensive subsequent operations. Thus, PC sub-sampling is of practical importance. Previous model-based sub-sampling schemes are ad-hoc in design and do not preserve the overall shape sufficiently well, while previous data-driven schemes are trained for specific pre-determined input PC sizes and sub-sampling rates and thus do not generalize well. Leveraging advances in graph sampling, we propose a fast PC sub-sampling algorithm of linear time complexity that chooses a 3D point subset while minimizing a global reconstruction error. Specifically, to articulate a sampling objective, we first assume a super-resolution (SR) method based on feature graph Laplacian regularization (FGLR) that reconstructs the original high-res PC, given points chosen by a sampling matrix H. We prove that minimizing a worst-case SR reconstruction error is equivalent to maximizing the smallest eigenvalue λmin of matrix HT H+ μL, where L is a symmetric, positive semi-definite matrix derived from a neighborhood graph connecting the 3D points. To arrive at a fast algorithm, instead of maximizing λmin, we maximize a lower bound λ-min(HT H+ μL) via selection of H-this translates to a graph sampling problem for a signed graph G with self-loops specified by graph Laplacian L. We tackle this general graph sampling problem in three steps. First, we approximate G with a balanced graph GB specified by Laplacian LB. Second, leveraging a recent linear algebraic theorem called Gershgorin disc perfect alignment (GDPA), we perform a similarity transform Lp = SLB S-1, so that all Gershgorin disc left-ends of Lp are aligned exactly at λmin(LB). Finally, we choose samples on GB using a previous graph sampling algorithm to maximize λ-min(HT H+ μLp) in linear time. Experimental results show that 3D points chosen by our algorithm outperformed competing schemes both numerically and visually in reconstruction quality.

Structure-preserving numerical method for Maxwell-Ampère Nernst-Planck model

Charge dynamics play essential role in many practical applications such as semiconductors, electrochemical devices and transmembrane ion channels. A Maxwell-Ampère Nernst-Planck (MANP) model that describes charge dynamics via concentrations and the electric displacement is able to take effects beyond mean-field approximations into account. To obtain physically faithful numerical solutions, we develop a structure-preserving numerical method for the MANP model whose solution has several physical properties of importance. By the Slotboom transform with entropic-mean approximations, a positivity preserving scheme with Scharfetter-Gummel fluxes is derived for the generalized Nernst-Planck equations. To deal with the curl-free constraint, the dielectric displacement from the Maxwell-Ampère equation is further updated with a local relaxation algorithm of linear computational complexity. We prove that the proposed numerical method unconditionally preserves the mass conservation and the solution positivity at the discrete level, and satisfies the discrete energy dissipation law with a time-step restriction. Numerical experiments verify that our numerical method has expected accuracy and structure-preserving properties. Applications to ion transport with large convection, arising from boundary-layer electric field and Born solvation interactions, further demonstrate that the MANP formulation with the proposed numerical scheme has attractive performance and can effectively describe charge dynamics with large convection of high numerical cell Péclet numbers.

Fast Multi Step-Index Mode Solver for Analysis and Optimization of Optical Fiber Performance

We propose a competitive approach for analysis and optimization of optical fiber performance. Taking into account the standard fiber fabrication cycle, we develop a model of highly multistepped optical fiber. We introduce a fast and accurate algorithm which allows one to obtain propagation constants and transverse fields for systems with thousands of steps within seconds. The approach relies on both the exact analytical formulation of the corresponding eigenvalue problem and an original solving algorithm of linear complexity with respect to the number of steps. The model accurately represents arbitrary smooth profiles with azimuthal symmetry, which, together with low computation time, opens up ample opportunities for various optimization schemes. Obtained results are compared with those of other numerical and analytical methods.

Structure-Preserving Numerical Method for Ampere-Nernst-Planck Model

Charge dynamics play essential role in many practical applications such as semiconductors, electrochemical devices and transmembrane ion channels. An Amp\'{e}re-Nernst-Planck (ANP) model that describes charge dynamics via concentrations and the electric displacement is able to take effects beyond mean-field approximations into account. To obtain physically faithful numerical solutions, we develop a structure-preserving numerical method for the ANP model whose solution has several physical properties of importance. By the Slotboom transform with entropy-mean approximations, a positivity preserving scheme with Scharfetter-Gummel fluxes is derived for the generalized Nernst-Planck equations. To deal with the curl-free constraint, the dielectric displacement from the Amp\'{e}re equation is further updated with a local relaxation algorithm of linear computational complexity. We prove that the proposed numerical method unconditionally preserves the mass conservation and the solution positivity at the discrete level, and satisfies the discrete energy dissipation law with a mild time-step restriction. Numerical experiments verify that our numerical method has expected accuracy and structure-preserving properties. Applications to ion transport with large convection, arising from boundary-layer electric field and Born solvation interactions, further demonstrate that the ANP formulation with the proposed numerical scheme has attractive performance and can effectively describe charge dynamics with large convection of high numerical cell P\'{e}clet numbers.

3D Urban UAV Relay Placement: Linear Complexity Algorithm and Analysis

Optimal unmanned aerial vehicle (UAV) placement in a 3-dimensional (3D) space to build a connection between a base station (BS) and a ground user is studied herein. A key challenge is to avoid signal propagation blockage due to obstacles. Much prior work uses probabilistic terrain models with model parameters learned from the statistics over a large area, and therefore, the optimization for a specific user in a small local area is poor. In contrast, this paper seeks the optimal UAV position over actual and fine-grained terrain, and develops efficient UAV positioning strategy adaptive to the degree of location-dependent line-of-sight (LOS) condition measured on the fly. It is proven that the globally optimal UAV position in 3D can be determined from the proposed search trajectory which has merely linear length in the diameter of the target area. Therefore, the proposed strategy can be practically implemented. Numerical experiments are performed over a real-world urban topology and demonstrate superior performance gain over existing strategies based on probabilistic models.

Measurement of material surface defect intensity by distributed cumulative histogram and clustering

The object of research is a distributed cumulative histogram of a digital image and its advantages for auto-mated determination of the location and intensity of defects of different nature on the surfaces of materials: metal, paper, etc. The technique considered in the study is aimed at minimization of human interference in the process of material surface control from the moment of its photographing to the moment of making a decision about the surface quality. Three-dimensional distributed cumulative histogram (DCH) is presented as a two-dimensional image in which the pixel intensity corresponds to the third dimension – the number of pixels of a certain intensity in the original surface image. Informative distributed cumulative histogram (IDCH) is used to recognize black, dark and light defects, and to measure their intensity and location by the clustering algorithm. The average value of the pixel intensity in the columns and rows of the pixel matrix of the cumulative histogram image is calculated to estimate the intensity of the defects. Measurement of the intensity of defects is carried out in two ways: directly on the image of the surface sample and by comparing the image of the sample and the reference image of the sample without defects. To solve the problem, an algorithm of hierarchical clustering of data to rectangular segments of the surface image is used. In the image, each cluster is marked with a corresponding color of gray. The image for analysis is transformed using segmentation and inversion algorithms. This allows to get more accurate estimates of the intensity of light and dark defects. The clustering algorithm groups the image segments of the surface samples, as well as the images of the distributed cumulative histogram to group the level of surface damage. Distributed cumulative histogram was used to detect defects on the surface of materials as a method of linking the number and intensity of pixels to image coordinates. Cluster analysis helps to find their coordinates and intensity. In comparison with known approaches, the method has a linear algorithmic complexity to the number of pixels in the input image, which allows to do a significant number of experiments to identify the types of surfaces of materials for use and the features of algorithms.

Authenticated secret key generation in delay-constrained wireless systems

With the emergence of 5G low-latency applications, such as haptics and V2X, low-complexity and low-latency security mechanisms are needed. Promising lightweight mechanisms include physical unclonable functions (PUF) and secret key generation (SKG) at the physical layer, as considered in this paper. In this framework, we propose (i) a zero round trip time (0-RTT) resumption authentication protocol combining PUF and SKG processes, (ii) a novel authenticated encryption (AE) using SKG, and (iii) pipelining of the AE SKG and the encrypted data transfer in order to reduce latency. Implementing the pipelining at PHY, we investigate a parallel SKG approach for multi-carrier systems, where a subset of the subcarriers are used for SKG and the rest for data transmission. The optimal solution to this PHY resource allocation problem is identified under security, power, and delay constraints, by formulating the subcarrier scheduling as a subset-sum 0−1 knapsack optimization. A heuristic algorithm of linear complexity is proposed and shown to incur negligible loss with respect to the optimal dynamic programming solution. All of the proposed mechanisms have the potential to pave the way for a new breed of latency aware security protocols.

Identifying highly influential nodes in multilayer networks based on global propagation

Based on percolation theory and the independent cascade model, this paper considers the selection of the optimal propagation source when the propagation probability is greater than the percolation threshold. First, based on the percolation characteristics of real networks, this paper presents an iterative algorithm of linear complexity to solve the probability of the propagation source transmitting information to the network's giant component, that is, the global propagation probability. Compared with the previous multiple local simulation algorithm, this algorithm eliminates random errors and significantly reduces the operation time. A sufficient and necessary condition is provided, and it is proved that the final propagation range of the propagation source obeys the bimodal distribution. Based on this sufficient and necessary condition, we extend the efficient iterative algorithm proposed in this article to multi-layer networks and find that for two-layer networks, the final propagation range of the propagation source follows a four-peak distribution. Through iterations and calculations, the probability of each peak and the number of nodes included can be directly obtained, and the propagation expectations of the nodes in the multi-layer network can then be calculated, which can result in a better ranking of the propagation influence of the nodes. In addition, to maximize the influence of multi-propagation sources, this paper also presents a de-overlapping method, which has evident advantages over traditional methods.

Finding strongly connected components of simple digraphs based on granulation strategy

Strongly connected components (SCCs) are an important kind of subgraphs in digraphs. It can be viewed as a kind of knowledge in the viewpoint of knowledge discovery. In our previous work, a knowledge discovery algorithm called RSCC was proposed for finding SCCs of simple digraphs based on two operators, k-step R-related set and k-step upper approximation, of rough set theory (RST). RSCC algorithm can find SCCs more efficiently than Tarjan algorithm of linear complexity. However, on the one hand, as the theoretical basis of RST applied to SCCs discovery of digraphs, the theoretical relationships between RST and graph theory investigated in previous work only include four equivalences between fundamental RST and graph concepts relating to SCCs. The reasonability of using the two RST operators to find SCCs still need to be investigated. On the other hand, it is found that there are three SCCs correlations between vertices after we use three RST concepts, R-related set, lower and upper approximation sets, to analyze SCCs. RSCC algorithm ignores these SCCs correlations so that the efficiency of RSCC is affected negatively. For the above two issues, firstly, we explore the equivalence between the two RST operators and Breadth-First Search (BFS) which is one of the most basic graph search algorithms and the most direct way to find SCCs. These equivalences explain the reasonability of using the two RST operators to find SCCs, and enrich the content of the theoretical relationships between RST and graph theory. Secondly, we design a granulation strategy according to these three SCCs correlations. Then an algorithm called GRSCC for finding SCCs of simple digraphs based on granulation strategy is proposed. Experimental results show that GRSCC provides better performance to RSCC.

Accelerating Flexible Manifold Embedding for Scalable Semi-Supervised Learning

In this paper, we address the problem of large-scale graph-based semi-supervised learning for multi-class classification. Most existing scalable graph-based semi-supervised learning methods are based on the hard linear constraint or cannot cope with the unseen samples, which limits their applications and learning performance. To this end, we build upon our previous work flexible manifold embedding (FME) [1] and propose two novel linear-complexity algorithms called fast flexible manifold embedding (f-FME) and reduced flexible manifold embedding (r-FME). Both of the proposed methods accelerate FME and inherit its advantages. Specifically, our methods address the hard linear constraint problem by combining a regression residue term and a manifold smoothness term jointly, which naturally provides the prediction model for handling unseen samples. To reduce computational costs, we exploit the underlying relationship between a small number of anchor points and all data points to construct the graph adjacency matrix, which leads to simplified closed-form solutions. The resultant f-FME and r-FME algorithms not only scale linearly in both time and space with respect to the number of training samples but also can effectively utilize information from both labeled and unlabeled data. Experimental results show the effectiveness and scalability of the proposed methods.

SVRG-MKL: A Fast and Scalable Multiple Kernel Learning Solution for Features Combination in Multi-Class Classification Problems.

In this paper, we present a novel strategy to combine a set of compact descriptors to leverage an associated recognition task. We formulate the problem from a multiple kernel learning (MKL) perspective and solve it following a stochastic variance reduced gradient (SVRG) approach to address its scalability, currently an open issue. MKL models are ideal candidates to jointly learn the optimal combination of features along with its associated predictor. However, they are unable to scale beyond a dozen thousand of samples due to high computational and memory requirements, which severely limits their applicability. We propose SVRG-MKL, an MKL solution with inherent scalability properties that can optimally combine multiple descriptors involving millions of samples. Our solution takes place directly in the primal to avoid Gram matrices computation and memory allocation, whereas the optimization is performed with a proposed algorithm of linear complexity and hence computationally efficient. Our proposition builds upon recent progress in SVRG with the distinction that each kernel is treated differently during optimization, which results in a faster convergence than applying off-the-shelf SVRG into MKL. Extensive experimental validation conducted on several benchmarking data sets confirms a higher accuracy and a significant speedup of our solution. Our technique can be extended to other MKL problems, including visual search and transfer learning, as well as other formulations, such as group-sensitive (GMKL) and localized MKL (LMKL) in convex settings.

A versatile, linear complexity algorithm for flow routing in topographies with depressions

Abstract. We present a new algorithm for solving the common problem of flow trapped in closed depressions within digital elevation models, as encountered in many applications relying on flow routing. Unlike other approaches (e.g., the Priority-Flood depression filling algorithm), this solution is based on the explicit computation of the flow paths both within and across the depressions through the construction of a graph connecting together all adjacent drainage basins. Although this represents many operations, a linear time complexity can be reached for the whole computation, making it very efficient. Compared to the most optimized solutions proposed so far, we show that this algorithm of flow path enforcement yields the best performance when used in landscape evolution models. In addition to its efficiency, our proposed method also has the advantage of letting the user choose among different strategies of flow path enforcement within the depressions (i.e., filling vs. carving). Furthermore, the computed graph of basins is a generic structure that has the potential to be reused for solving other problems as well, such as the simulation of erosion. This sequential algorithm may be helpful for those who need to, e.g., process digital elevation models of moderate size on single computers or run batches of simulations as part of an inference study.

A new statistic for efficient detection of repetitive sequences.

Detecting sequences containing repetitive regions is a basic bioinformatics task with many applications. Several methods have been developed for various types of repeat detection tasks. An efficient generic method for detecting most types of repetitive sequences is still desirable. Inspired by the excellent properties and successful applications of the D2 family of statistics in comparative analyses of genomic sequences, we developed a new statistic D2R that can efficiently discriminate sequences with or without repetitive regions. Using the statistic, we developed an algorithm of linear time and space complexity for detecting most types of repetitive sequences in multiple scenarios, including finding candidate clustered regularly interspaced short palindromic repeats regions from bacterial genomic or metagenomics sequences. Simulation and real data experiments show that the method works well on both assembled sequences and unassembled short reads. The codes are available at https://github.com/XuegongLab/D2R_codes under GPL 3.0 license. Supplementary data are available at Bioinformatics online.

On Optimal Power Allocation over Time-Varying Rayleigh Fading Channels for Maximum-Ratio Combining in Diversity Systems with Partial CSI

This paper studies the problem of optimal power allocation (OPA) over independent but not necessarily identically distributed time-varying Rayleigh fading channels with maximum-ratio combining to exploit transmit and/or receive diversity. Specifically, the optimal average rate-maximizing and M-ary phase shift keying symbol-error-rate (SER)-minimizing power allocation problems subject to a total transmit power constraint are formulated as stochastic optimization problems. After that, the stochastic problems are reformulated into deterministic ones in terms of the average rate and SER, respectively, which require only partial channel state information (CSI) in the form of channel gains. Novel closed-form expressions are derived for the deterministic OPA for average rate-maximization and SER-minimization, and a linear-complexity water-filling algorithm is then proposed to efficiently determine the OPA based on derived closed-form solutions. Moreover, dynamic OPA (with full CSI knowledge) is analyzed and closed-form solutions for the optimal average rate and SER are obtained. Simulation results are presented to validate the theoretical analysis, and demonstrate that the deterministic OPA algorithm outperforms equal power allocation and yields comparable average rate and SER performances to the dynamic OPA.

Superpixel Segmentation Using Gaussian Mixture Model.

Superpixel segmentation partitions an image into perceptually coherent segments of similar size, namely, superpixels. It is becoming a fundamental preprocessing step for various computer vision tasks because superpixels significantly reduce the number of inputs and provide a meaningful representation for feature extraction. We present a pixel-related Gaussian mixture model (GMM) to segment images into superpixels. GMM is a weighted sum of Gaussian functions, each one corresponding to a superpixel, to describe the density of each pixel represented by a random variable. Different from previously proposed GMMs, our weights are constant, and Gaussian functions in the sums are subsets of all the Gaussian functions, resulting in segments of similar size and an algorithm of linear complexity with respect to the number of pixels. In addition to the linear complexity, our algorithm is inherently parallel and allows fast execution on multicore systems. During the expectation-maximization iterations of estimating the unknown parameters in the Gaussian functions, we impose two lower bounds to truncate the eigenvalues of the covariance matrices, which enables the proposed algorithm to control the regularity of superpixels. Experiments on a wellknown segmentation dataset show that our method can efficiently produce superpixels that adhere to object boundaries better than the current state-of-the-art methods.