Sparse Block Research Articles

Application of Optimal Control to Inversion of Self-Potential Data: Theory and Synthetic Examples

Last decades, there has been an increased interest in the use of the self-potential (SP) method in hydrogeophysics. In response to this strong interest, we develop a novel approach to the inversion of SP data. Mathematically, the SP inverse problem is the source identification problem for the Poisson equation. Our approach substantially differs from the standard regularization approach, which explicitly includes the forward-problem operator into the cost functional. We formulated the inverse problem is as an optimal control problem and then translate it into a variational system. The system is approximated in suitable finite-element spaces giving rise to an algebraic problem with the saddle-point structure. In contrast to the standard approach, which leads to a dense linear system, our method results in a system with a sparse block matrix. It can be efficiently solved by either direct sparse solvers or preconditioned iterative solvers. In this paper, we present the formulation of the problem and its finite-element approximation. We discuss the iterative solution and preconditioning strategies. Our software implementation is based on an industrial finite-element package. We also present a numerical experiment with node-based linear finite elements on tetrahedral grids. Our results suggest that the proposed approach may serve as a rapid and reliable tool for large-scale SP inverse problems. Moreover, the same technique can easily be extended to a wide range of geophysical linear inverse problems, such as inversions of magnetic and gravity data.

Learning interlaced sparse Sinkhorn matching network for video super-resolution

How to effectively fuse inter- and intra-frame spatio-temporal information plays a key role in video super-resolution (VSR). Most existing works rely heavily on the accuracy of motion estimation and compensation for spatio-temporal feature alignment. However, they cannot perform well when suffering from large-scale and complex motions. To this end, this paper introduces an efficient and effective Interlaced Sparse Sinkhorn Matching (ISSM) network for VSR, which aligns supporting frames with the reference one in the feature space by learning optimal matching between image regions across frames. Specifically, the ISSM divides the input dense affinity matrix into two sparse block matrixes: one can match long-distance regions while the other can match short-distance regions, and then we leverage an efficient Sinkhorn method on each block to learn optimal matching. Moreover, we insert a residual atrous spatial pyramid pooling module before the ISSM, which can flexibly generate multi-scale features frame by frame to capture the multi-scale context information in images. The aligned features of each adjacent frame are then fed to a bidirectional temporal fusion module to capture the rich temporal information. Finally, the fused features are sent into a frame-wise dynamic reconstruction network to produce an HR frame. Extensive evaluations on three benchmark datasets demonstrate the superiority of our method over the state-of-the-art methods in terms of PSNR and SSIM.

Sparse attention block: Aggregating contextual information for object detection

It is well recognized that the contextual information of surrounding objects is beneficial for object detection. Such contextual information can often be obtained from long-range dependencies. This paper proposes a sparse attention block to capture long-range dependencies in an efficient way. Unlike the conventional non-local block, which generates a dense attention map to characterize the dependency between any two positions of the input feature map, our sparse attention block samples the most representative positions for contextual information aggregation. After searching for local peaks in a heat map of the given input feature map, it adaptively selects a sparse set of positions to represent the relationship between query and key elements. With the obtained sparse positions, our sparse attention block can well model long-range dependencies, and greatly improve the object detection performance at the additional cost of <2% GPU memory and computation of the conventional non-local block. This sparse attention block can be easily plugged into various object detection frameworks, such as Faster R-CNN, RetinaNet and Mask R-CNN. Experiments on COCO benchmark confirm that our sparse attention block can boost the detection accuracy with significant gains ranging from 1.4% to 1.9% and negligible overhead of computation and memory usage.

An Improved Image Filtering Algorithm for Mixed Noise

In recent years, image filtering has been a hot research direction in the field of image processing. Experts and scholars have proposed many methods for noise removal in images, and these methods have achieved quite good denoising results. However, most methods are performed on single noise, such as Gaussian noise, salt and pepper noise, multiplicative noise, and so on. For mixed noise removal, such as salt and pepper noise + Gaussian noise, although some methods are currently available, the denoising effect is not ideal, and there are still many places worthy of improvement and promotion. To solve this problem, this paper proposes a filtering algorithm for mixed noise with salt and pepper + Gaussian noise that combines an improved median filtering algorithm, an improved wavelet threshold denoising algorithm and an improved Non-local Means (NLM) algorithm. The algorithm makes full use of the advantages of the median filter in removing salt and pepper noise and demonstrates the good performance of the wavelet threshold denoising algorithm and NLM algorithm in filtering Gaussian noise. At first, we made improvements to the three algorithms individually, and then combined them according to a certain process to obtain a new method for removing mixed noise. Specifically, we adjusted the size of window of the median filtering algorithm and improved the method of detecting noise points. We improved the threshold function of the wavelet threshold algorithm, analyzed its relevant mathematical characteristics, and finally gave an adaptive threshold. For the NLM algorithm, we improved its Euclidean distance function and the corresponding distance weight function. In order to test the denoising effect of this method, salt and pepper + Gaussian noise with different noise levels were added to the test images, and several state-of-the-art denoising algorithms were selected to compare with our algorithm, including K-Singular Value Decomposition (KSVD), Non-locally Centralized Sparse Representation (NCSR), Structured Overcomplete Sparsifying Transform Model with Block Cosparsity (OCTOBOS), Trilateral Weighted Sparse Coding (TWSC), Block Matching and 3D Filtering (BM3D), and Weighted Nuclear Norm Minimization (WNNM). Experimental results show that our proposed algorithm is about 2–7 dB higher than the above algorithms in Peak Signal-Noise Ratio (PSNR), and also has better performance in Root Mean Square Error (RMSE), Structural Similarity (SSIM), and Feature Similarity (FSIM). In general, our algorithm has better denoising performance, better restoration of image details and edge information, and stronger robustness than the above-mentioned algorithms.

A Coupling between Integral Equations and On-Surface Radiation Conditions for Diffraction Problems by Non Convex Scatterers

The aim of this paper is to introduce an orignal coupling procedure between surface integral equation formulations and on-surface radiation condition (OSRC) methods for solving two-dimensional scattering problems for non convex structures. The key point is that the use of the OSRC introduces a sparse block in the surface operator representation of the wave field while the integral part leads to an improved accuracy of the OSRC method in the non convex part of the scattering structure. The procedure is given for both the Dirichlet and Neumann scattering problems. Some numerical simulations show the improvement induced by the coupling method.

A supervised multi-view feature selection method based on locally sparse regularization and block computing

With the increasing scale of obtained multi-view data, how to deal with large-scale multi-view data quickly and efficiently is a significant problem. In this paper, a novel supervised multi-view feature selection method based on locally sparse regularization and block computing is proposed to solve the problem. Specifically, the multi-view dataset is firstly divided into sub-blocks according to classes and views. Then with the aid of the Alternating Direction Method of Multipliers (ADMM), a sharing sub-model is proposed to perform feature selection on each class by integrating each view’s locally sparse regularizers and shared loss that makes all views share a common penalty and regresses samples to their labels. Finally, all the sharing sub-models are fused to form the final general additive feature selection model, in which each sub-block adjusts its corresponding variables to perform block-based feature selection. In the optimization process, the proposed model can be decomposed into multiple separate subproblems, and an efficient optimization algorithm is proposed to solve them quickly. The comparison experiments with several state-of-the-art feature selection methods show that the proposed method is superior in classification accuracy and training speed.

Fast numerical optimization for genome sequencing data in population biobanks.

Large-scale and high-dimensional genome sequencing data poses computational challenges. General-purpose optimization tools are usually not optimal in terms of computational and memory performance for genetic data. We develop two efficient solvers for optimization problems arising from large-scale regularized regressions on millions of genetic variants sequenced from hundreds of thousands of individuals. These genetic variants are encoded by the values in the set {0,1,2,NA}. We take advantage of this fact and use two bits to represent each entry in a genetic matrix, which reduces memory requirement by a factor of 32 compared to a double precision floating point representation. Using this representation, we implemented an iteratively reweighted least square algorithm to solve Lasso regressions on genetic matrices, which we name snpnet-2.0. When the dataset contains many rare variants, the predictors can be encoded in a sparse matrix. We utilize the sparsity in the predictor matrix to further reduce memory requirement and computational speed. Our sparse genetic matrix implementation uses both the compact two-bit representation and a simplified version of compressed sparse block format so that matrix-vector multiplications can be effectively parallelized on multiple CPU cores. To demonstrate the effectiveness of this representation, we implement an accelerated proximal gradient method to solve group Lasso on these sparse genetic matrices. This solver is named sparse-snpnet, and will also be included as part of snpnet R package. Our implementation is able to solve Lasso and group Lasso, linear, logistic and Cox regression problems on sparse genetic matrices that contain 1 000 000 variants and almost 100 000 individuals within 10 min and using less than 32GB of memory. https://github.com/rivas-lab/snpnet/tree/compact.

Practical implementation of genetic groups in single-step genomic evaluations with Woodbury matrix identity–based genomic relationship inverse

The growing amount of genomic information in dairy cattle has increased computational and modeling challenges in the single-step evaluations. The computational challenges are due to the dense inverses of genomic (G) and pedigree (A22) relationship matrices of genotyped animals in the single-step mixed model equations. An equivalent mixed model equation is given by single-step genomic BLUP that are based on the T matrix (ssGTBLUP), where these inverses are avoided by expressing G-1 through a product of 2 rectangular matrices, and (A22)-1 through sparse matrix blocks of the inverse of full relationship matrix A-1. A proper way to account genetic groups through unknown parent groups (UPG) after the Quaas-Pollak transformation (QP) is one key factor in a single-step model. When the UPG effects are incompletely accounted, the iterative solving method may have convergence problems. In this study, we investigated computational and predictive performance of ssGTBLUP with residual polygenic (RPG) effect and UPG. The QP transformation used A-1 and, in the complete form, T and (A22)-1 matrices as well. The models were tested with official Nordic Holstein milk production test-day data and model. The results show that UPG can be easily implemented in ssGTBLUP having RPG. The complete QP transformation was computationally feasible when preconditioned conjugate gradient iteration and iteration on data without explicitly setting up G or A22 matrices were used. Furthermore, for good convergence of the preconditioned conjugate gradient method, a complete QP transformation was necessary.

TSSN-Net: Two-step Sparse Switchable Normalization for learning correspondences with heavy outliers

In this paper, we solve the problem of feature matching by designing an end-to-end network (called TSSN-Net). Given putative correspondences of feature points in two views, existing deep learning based methods formulate the feature matching problem as a binary classification problem. In these methods, a normalizer plays an important role in the networks. However, they adopt the same normalizer in all normalization layers of the entire networks, which will result in suboptimal performance. To address this problem, we propose a Two-step Sparse Switchable Normalization Block, which involves the advantage of adaptive normalization for different convolution layers from Sparse Switchable Normalization and robust global context information from Context Normalization. Moreover, to capture local information of correspondences, we propose a Multi-Scale Correspondence Grouping algorithm, by defining a multi-scale neighborhood representation, to search for consistent neighbors of each correspondence. Finally, with a series of convolution layers, the end-to-end TSSN-Net is proposed to learn correspondences with heavy outliers for feature matching. Our experimental results have shown that our network achieves the state-of-the-art performance on benchmark datasets.

Fast and Accurate Classification of Meta-Genomics Long Reads With deSAMBA.

There is still a lack of fast and accurate classification tools to identify the taxonomies of noisy long reads, which is a bottleneck to the use of the promising long-read metagenomic sequencing technologies. Herein, we propose de Bruijn graph-based Sparse Approximate Match Block Analyzer (deSAMBA), a tailored long-read classification approach that uses a novel pseudo alignment algorithm based on sparse approximate match block (SAMB). Benchmarks on real sequencing datasets demonstrate that deSAMBA enables to achieve high yields and fast speed simultaneously, which outperforms state-of-the-art tools and has many potentials to cutting-edge metagenomics studies.

Hybrid evolutionary network architecture search (HyENAS) for convolution class of deep neural networks with applications

AbstractConvolutional Neural Networks (CNNs) and its variants are increasingly used across wide domain of applications achieving high performance measures. For high performance, application specific CNN architecture is required, hence the need for network architecture search (NAS) becomes essential. This paper proposes a hybrid evolutionary approach for network architecture search (HyENAS), and targets convolution class of neural networks. One of the significant contribution of this technique is to completely evolve the high performance network by simultaneously finding network structures and their corresponding parameters. An elegant string representation has been proposed which efficiently represents the network. The concept of sparse block evolving requisite layer wise features for dense network is deployed. This permits the network to dynamically evolve for a specific application. In comparison to the other state‐of‐art methods, the high performance of the proposed HyENAS approach is demonstrated across various benchmark data sets belonging to the domain of malariology, oncology, neurology, ophthalmology, and genomics. Further, to deploy the proposed model on lower hardware specification devices, another salient feature of the HyENAS technique is to seamlessly sift out the simpler network architecture with comparable accuracy.

Community detection in the sparse hypergraph stochastic block model

AbstractWe consider the community detection problem in sparse random hypergraphs. Angelini et al. in [6] conjectured the existence of a sharp threshold on model parameters for community detection in sparse hypergraphs generated by a hypergraph stochastic block model. We solve the positive part of the conjecture for the case of two blocks: above the threshold, there is a spectral algorithm which asymptotically almost surely constructs a partition of the hypergraph correlated with the true partition. Our method is a generalization to random hypergraphs of the method developed by Massoulié (2014) for sparse random graphs.

Joint direct and transposed sparse matrix‐vector multiplication for multithreaded CPUs

AbstractRepeatedly performing sparse matrix‐vector multiplication (SpMV) followed by transposed sparse matrix‐vector multiplication (SpMTV) with the same matrix is a part of several algorithms, for example, the Lanczos biorthogonalization algorithm and the biconjugate gradient method. Such algorithms can benefit from combining parallel SpMV and SpMTV into a single operation we call joint direct and transposed sparse matrix‐vector multiplication (SpMMTV). In this article, we present a parallel SpMMTV algorithm for shared‐memory CPUs. The algorithm uses a sparse matrix format that divides the stored matrix into sparse matrix blocks and compresses the row and column indices of the matrix. This sparse matrix format can be also used for SpMV, SpMTV, and similar sparse matrix‐vector operations. We expand upon existing research by suggesting new variants of the parallel SpMMTV algorithm and by extending the algorithm to efficiently support symmetric matrices. We compare the performance of the presented parallel SpMMTV algorithm with alternative approaches, which use state‐of‐the‐art sparse matrix formats and libraries, using sparse matrices from real‐world applications. The performance results indicate that the median performance of our proposed parallel SpMMTV algorithm is up to 45% higher than of the alternative approaches.

A circulant-matrix-based new accelerated GSOR preconditioned method for block two-by-two linear systems from image restoration problems

In this paper, we construct a circulant-matrix-based new accelerated GSOR (CNAGSOR) iteration method for a class of large and sparse block two-by-two linear systems of generalized saddle-point structure. Theoretical results about the convergence properties and eigenvalues distribution of the preconditioning matrix are studied in detail. Implementations in the image restoration problem and in the PDE-constraint optimization problem are made to verify the feasibility and the efficiency of the new methods.

DETERIORATED HSS-LIKE METHODS FOR THE WEIGHTED TOEPLITZ LEAST SQUARES PROBLEM FROM IMAGE RESTORATION

<abstract> In this paper, we construct a deteriorated HSS-like (DHSS-like) iteration method for a class of large and sparse block two-by-two linear systems from image restoration. The detailed spectral properties and the quasi-optimal iteration parameters are investigated in detail. Because the DHSS-like iteration method naturally leads to a DHSS-like preconditioner, then we can use the circulant matrix to replace the Toeplitz matrix in the DHSS-like preconditioner approximately to obtain a circulant matrix-based DHSS-like (CDHSS-like) preconditioner. It is pointed out that the workload of the new preconditioned method is about $ O(n\log n) $. Theoretically analysis shows that the eigenvalues of the CDHSS-like preconditioned matrix are clustered around 1. Implementations in linear systems from the image restoration problems are made to verify the correctness of the theoretical results and the efficiency of both the CDHSS-like iteration method and the CDHSS-like preconditioned method. </abstract>

Structured sparsity modeling for improved multivariate statistical analysis based fault isolation

In order to improve the fault diagnosis capability of multivariate statistical methods, this article introduces a fault isolation framework based on structured sparsity modeling. The developed method relies on the reconstruction based contribution analysis and the process structure information can be incorporated into the reconstruction objective function in the form of structured sparsity regularization terms. The structured sparsity terms allow selection of fault variables over structures like blocks or networks of process variables, hence more accurate fault isolation can be achieved. Four structured sparsity terms corresponding to different kinds of process information are considered, namely, partially known sparse support, block sparsity, clustered sparsity and tree-structured sparsity. The optimization problems involving the structured sparsity terms can be solved using the Alternating Direction Method of Multipliers (ADMM) algorithm, which is fast and efficient. Through a simulation example and an application study to a coal-fired power plant, it is verified that the proposed method can better isolate faulty variables by incorporating process structure information.

Optimal couplings between sparse block models

We study the problem of coupling a stochastic block model with a planted bisection to a uniform random graph having the same average degree. Focusing on the regime where the average degree is a constant relative to the number of vertices n n , we show that the distance to which the models can be coupled undergoes a phase transition from O ( n ) O(\sqrt {n}) to Ω ( n ) \Omega (n) as the planted bisection in the block model varies. This settles half of a conjecture of Bollobás and Riordan and has some implications for sparse graph limit theory. In particular, for certain ranges of parameters, a block model and the corresponding uniform model produce samples which must converge to the same limit point. This implies that any notion of convergence for sequences of graphs with Θ ( n ) \Theta (n) edges which allows for samples from a limit object to converge back to the limit itself must identify these models.

Research on real-time data transmission and multi-scale video image decomposition of embedded optical sensor array based on machine learning

Aiming at the research of real-time data transmission and multi-scale image decomposition of embedded optical sensor array, the principle, method and fusion strategy of multi-sensor image fusion are studied comprehensively, thoroughly and systematically by combining the imaging characteristics of source image with multi-scale geometric analysis tools using machine learning algorithm. A new quality scalable video image coding framework is also proposed in this paper, which is implemented by a multi-scale online dictionary learning algorithm based on structured sparse video signals. For the purpose of different types of images and image fusion, a new high quality scalable video image coding framework based on machine learning algorithm is proposed on the basis of comprehensive analysis of prior information such as imaging mechanism of image sensor and imaging characteristics of source image. A multi-scale online dictionary learning algorithm based on machine learning for sparse video signal structure is proposed. Through the hierarchical structure of wavelet decomposition, the searching domain of online learning is optimized to a hierarchical sparse block, and its sparse representation coefficients are obtained by using machine learning sparse coding idea. The real-time data transmission of embedded optical sensor array based on machine learning and multi-scale image decomposition algorithm proposed in this paper have good fusion performance, which is of great significance for further research and engineering application of image fusion technology.

Accelerating sparse matrix–matrix multiplication with GPU Tensor Cores

Sparse general matrix–matrix multiplication (spGEMM) is an essential component in many scientific and data analytics applications. However, the sparsity pattern of the input matrices and the interaction of their patterns make spGEMM challenging. Modern GPUs include Tensor Core Units (TCUs), which specialize in dense matrix multiplication. Our aim is to re-purpose TCUs for sparse matrices. The key idea of our spGEMM algorithm, tSparse, is to multiply sparse rectangular blocks using the mixed precision mode of TCUs. tSparse partitions the input matrices into tiles and operates only on tiles which contain one or more elements. It creates a task list of the tiles, and performs matrix multiplication of these tiles using TCUs. To the best of our knowledge, this is the first time that TCUs are used in the context of spGEMM. We show that spGEMM, with our tiling approach, benefits from TCUs. Our approach significantly improves the performance of spGEMM in comparison to cuSPARSE, CUSP, RMerge2, Nsparse, AC-SpGEMM and spECK.

An Image-Based Real-Time Georeferencing Scheme for a UAV Based on a New Angular Parametrization

Simultaneous localization and mapping (SLAM) of a monocular projective camera installed on an unmanned aerial vehicle (UAV) is a challenging task in photogrammetry, computer vision, and robotics. This paper presents a novel real-time monocular SLAM solution for UAV applications. It is based on two steps: consecutive construction of the UAV path, and adjacent strip connection. Consecutive construction rapidly estimates the UAV path by sequentially connecting incoming images to a network of connected images. A multilevel pyramid matching is proposed for this step that contains a sub-window matching using high-resolution images. The sub-window matching increases the frequency of tie points by propagating locations of matched sub-windows that leads to a list of high-frequency tie points while keeping the execution time relatively low. A sparse bundle block adjustment (BBA) is employed to optimize the initial path by considering nuisance parameters. System calibration parameters with respect to global navigation satellite system (GNSS) and inertial navigation system (INS) are optionally considered in the BBA model for direct georeferencing. Ground control points and checkpoints are optionally included in the model for georeferencing and quality control. Adjacent strip connection is enabled by an overlap analysis to further improve connectivity of local networks. A novel angular parametrization based on spherical rotation coordinate system is presented to address the gimbal lock singularity of BBA. Our results suggest that the proposed scheme is a precise real-time monocular SLAM solution for a UAV.