Sparsity Rate Research Articles

A novel RCACycleGAN model is proposed for the high-precision reconstruction of sparse TFM images

The sparse total focusing method (TFM) has been shown to enhance the computational efficacy of ultrasound imaging but the image quality of ultrasound regrettably deteriorates with an increase in the sparsity rate of array elements. Deep learning has made remarkable advancements in image processing and cycle-consistent generative adversarial networks (CycleGANs) have been extensively employed to reconstruct diverse image categories. However, due to the incomplete extraction of image feature information by the generator and discriminator in a CycleGAN, high-quality sparse TFM images cannot be directly reconstructed using CycleGANs. There is also a risk of losing crucial feature information related to minor defects. As a result, this paper modifies the generator and discriminator in the CycleGAN to construct a new relativistic discriminator and coordinate attention CycleGAN (RCACycleGAN) model, which enables high-precision reconstruction of sparse TFM images. The addition of the coordinate attention module to the CycleGAN enhances the defective feature representation by fully considering the channel and spatial correlation between regions and using the fusion of spatially perceived feature maps in different directions. It solves the problem of easy loss of defective key feature information. The relativistic discriminator replaces the PatchGAN discriminator in the CycleGAN and evaluates the quality of both real and sparse TFM reconstructed images to ensure a relative image quality evaluation. This process solves the problem of unstable image quality of the sparse TFM reconstructed image. Experimental results demonstrate that RCACycleGAN can stably reconstruct sparse TFM images even in small sample dataset scenarios. The proposed network model reconstructs images with better accuracy, including in terms of structural similarity, defect roundness and area, and has a shorter training time than several existing network models.

Fast and Controllable Post-training Sparsity: Learning Optimal Sparsity Allocation with Global Constraint in Minutes

Neural network sparsity has attracted many research interests due to its similarity to biological schemes and high energy efficiency. However, existing methods depend on long-time training or fine-tuning, which prevents large-scale applications. Recently, some works focusing on post-training sparsity (PTS) have emerged. They get rid of the high training cost but usually suffer from distinct accuracy degradation due to neglect of the reasonable sparsity rate at each layer. Previous methods for finding sparsity rates mainly focus on the training-aware scenario, which usually fails to converge stably under the PTS setting with limited data and much less training cost. In this paper, we propose a fast and controllable post-training sparsity (FCPTS) framework. By incorporating a differentiable bridge function and a controllable optimization objective, our method allows for rapid and accurate sparsity allocation learning in minutes, with the added assurance of convergence to a predetermined global sparsity rate. Equipped with these techniques, we can surpass the state-of-the-art methods by a large margin, e.g., over 30\% improvement for ResNet-50 on ImageNet under the sparsity rate of 80\%. Our plug-and-play code and supplementary materials are open-sourced at https://github.com/ModelTC/FCPTS.

Efficient and Secure Federated Learning for Financial Applications

Conventional machine learning (ML) and deep learning approaches require sharing customers’ sensitive information with an external credit bureau to generate a prediction model, thereby increasing the risk of privacy leakage. This poses a significant challenge for financial companies. To address this challenge, federated learning has emerged as a promising approach to protect data privacy. However, the high communication costs associated with federated systems, particularly for large neural networks, can be a bottleneck. To mitigate this issue, it is necessary to limit the number and size of communications for practical training of large neural structures. Gradient sparsification is a technique that has gained increasing attention as a method to reduce communication costs, as it updates only significant gradients and accumulates insignificant gradients locally. However, the secure aggregation framework cannot directly employ gradient sparsification. To overcome this limitation, this article proposes two sparsification methods for reducing the communication costs of federated learning. The first method is a time-varying hierarchical sparsification method for model parameter updates, which addresses the challenge of maintaining model accuracy after a high sparsity ratio. This method can significantly reduce the cost of a single communication. The second method is to apply sparsification to the secure aggregation framework. Specifically, the encryption mask matrix is sparsified to reduce communication costs while protecting privacy. Experiments demonstrate that our method can reduce the upload communication costs to approximately 2.9% to 18.9% of the conventional federated learning algorithm under different non-IID experiment settings when the sparsity rate is 0.01.

A Sparse Design for Aperture-Level Simultaneous Transmit and Receive Arrays

The aperture-level simultaneous transmit and receive (ALSTAR) system uses full digital architecture with an observation channel to achieve remarkably effective isotropic isolation (EII). However, the number of observation channels must be the same as the number of transmit channels, which increases the system’s complexity. To balance the system cost and performance of the ALSTAR, this paper proposes a joint design of sparse arrays and beamforming, which are achieved by a genetic algorithm and an alternating optimization algorithm, respectively. In the sparse design, we introduce beamforming technology to guarantee the EII while decreasing the corresponding elements of observation channel that contribute slightly to the EII. The simulation results are presented for a 32-element array that achieves 185.87 dB of the EII with 1000 W of transmit power. In the cases of sparsity rates at 0.875 and 0.75 (≥0.6), i.e., the number of observation channels decreases by 12.5% (2/16) and 25% (4/16), the reductions in EII do not exceed 1 dB and 3 dB, respectively. However, the EII decreases rapidly with a sparsity rate less than 0.25. Results demonstrate that our proposed joint design of sparse arrays and beamforming can reduce the system cost with little performance loss of EII.

Sparse discrete least squares meshless method on multicore computers

In this paper, a well-parallelized and scalable sparse assembly algorithm that plays a key role in large-scale simulations is developed. Being sometimes the bottleneck of making the most of the computational resources, assembly operation is of great significance to be implemented in a highly efficient manner. The proposed algorithm employs just nodes to construct the compressed global stiffness matrix leading to the optimized assembly procedure in problems containing several unknowns per node. Moreover, Discrete Least Squares Meshless Method is presented in which producing time-consuming background cells for performing Gaussian integration is avoided. However, it takes significant CPU time to construct the coefficient matrix (like other meshless methods) instead of the computationally demanding process of generating FEM meshes. Unlike the finite element method, meshless methods with their optimal radius of support domain (2.0–3.0 times of nodal spacing) contain numerous collisions and low sparsity rate in the assembly operation, hence, the assembly of meshless methods is targeted to demonstrate the superiority of the proposed algorithm. This study compares sparse and dense assembly between meshless and finite element method probably for the first time. The current paper also introduces and evaluates an open-source Parallel Nodal Sparse Assembly package in shared memory architecture which can be easily extended to other parallel schemes. The proposed algorithm not only is compatible with problems including high sparsity rate but also can be highly effective in cases with a large number of collisions and low sparsity rate.

Modelling short-term variations of differential code bias aiding in extraction of ionospheric observables with sparse kernel learning

As short-term variations existing in differential code bias of receivers (RDCB) will decrease the accuracies of extracting ionospheric observables by carrier-to-code leveling (CCL) method, a modified carrier-to code leveling (MCCL) method had been proposed to estimate RDCB offsets epoch by epoch in retrieval of ionospheric information. However, MCCL method is hampered by efficiency and precision at higher sampling intervals. To address RDCB offsets in higher sampling rate domain, a kernel model in the form of radial basis function (RBF) was chosen to model RDCB offsets produced by MCCL using lower sampling rate data in this paper. Owing to ill-posed problem induced by setting to many kernels, L1 norm regulation was employed to promote the sparsity of the problem and Fast iterative shrinkage thresholding algorithm (FISTA) was adopted to find the sparse solution. Data sets of 350 IGS stations evenly distributed in high-solar and low-solar activity years were chosen to illustrate the general existences of RDCB offsets and their dependence on solar activity. To validate the reliability and efficiency of the proposed method, two stations of them were selected out to construct the spares kernel model. As shown in experimental results, kernel model is of good performance in training data with over 90% sparsity rate and all standard deviations of the errors between original and fitting data are around 0.4 ns. It is a striking finding in terms of efficiency that the proposed method and CCL are much more efficient than MCCL in extracting ionospheric observables by means of fixed equipment. For further validations, generalized triangular series function (GTSF) was constructed using ionospheric observables derived from precise point positioning (PPP), CCL, MCCL and the proposed method. With Center for Orbit Determination in Europe (CODE) Global Ionospheric Map (GIM) regarded as reference, compared with VTEC estimates derived from CCL, those derived from MCCL and the proposed method have improved accuracies ranging from 10% to 20%. Overall, these results indicate that the proposed method has a promising application in retrieval of ionospheric observables in terms of accuracy and efficiency.

3-D Acoustic Image Denoising for a Sonar System With Sparse Planar Arrays

The development of three-dimensional (3-D) underwater imaging systems is restricted by the high hardware costs associated with the use of a large number of transducers, as well as poor image quality due to degrading noise. This report presents the design of a low-complexity 3-D underwater imaging system that can generate high-quality images. In particular, a complex-weight sparse synthesis method for a large planar array is presented to reduce the number of active elements. Then, a denoising network referred to as an asymmetric-pyramid globe residual network is proposed to enhance the acoustic images generated by the sparse planar arrays. This method can perceive the entirety of the 3-D acoustic images for denoising and effectively reduce the degradation effects due to speckle noise and sidelobes. The simulation results demonstrate that the resulting image quality is higher than that achieved using previously developed networks in terms of the peak signal-to-noise ratio and structural similarity index. To validate the proposed system further, an actual system based on the proposed methods was devised and tested via lake-based trials. The experimental results demonstrate the notable improvements obtained considering the sparsity rate and image quality relative to those presented in previous literature.

Sparse sampling in frequency domain and laser imaging

The monochromaticity of the laser and the characteristics of the natural image’s spectrum, including sparsity and concentrating in the low frequency range, make it possible to sample the image spectrum sparsely. Based on small-scale laser detectors and the introduced laser reference signals, a method of laser imaging with sparse sampling in frequency domain is proposed in this paper. The principle of frequency sparse sampling laser imaging and the imaging system structure are introduced. The simulation results of spectrum and complex images reconstructed are given. Both the effects of the signals’ parameters, such as the ratio of the reference laser signal amplitude to the laser echo spectrum amplitude and the initial phase of the laser reference signal, on reconstruction results are investigated. The reconstruction results are evaluated by correlation coefficient, mean square error (MSE), and structural similarity index (SSIM). For the strong correlation between phase and amplitude of the laser echo complex image, the amplitude image and the phase image are both set to be 256 × 256 diagram. The sparse laser detector plane array consists of 5 64 × 64 frequency domain laser detector arrays, which form a cross and make a sparsity rate of 31.25%(5/16). The simulation results show that the correlation coefficient, MSE and SSIM of the spectrum reconstructed are 0.96, 22.14, 1.00 and those of the complex image reconstructed are 0.96, 1857.25 and 0.67 respectively. The simulation results indicate that the method proposed is effective. However, the method requires the laser reference signal amplitude to be about 30 times the mean value of the laser echo spectrum amplitude, which reduces the dynamic range of the detectors. The initial phase of the laser reference signal has no obvious effect on the reconstruction results.

Pruned Distributed and Parallel Subarray Beamforming for 3-D Underwater Imaging With Fine-Grid Sparse Arrays

The development of real-time 3-D underwater imaging systems with large planar arrays involves high hardware costs and great computational burden. This study proposes the concept of fine-grid sparse arrays and the pruned distributed and parallel subarray (P-DPS) beamforming algorithm to achieve 3-D narrowband underwater imaging systems with low complexity. Fine-grid sparse arrays refine traditional half-wavelength grids to improve the freedom of the sparse optimization process and reduce the number of active elements. However, grid refinement increases the computational load of conventional fast beamforming algorithms. To solve this problem, a new pruned fast Fourier transform is proposed to eliminate all the redundant operations in the P-DPS beamforming, which is suitable for fine-grid sparse arrays. The computational load of the P-DPS beamforming seldom increases with grid refinement. The validity of the proposed method is verified with a fine-grid sparse planar array designed in the study. The computational load is then analyzed and compared with those of other methods. Results show the notable improvements in array sparsity rate and computational efficiency relative to those in the literature.

Compressed Sensing for THz FMCW Radar 3D Imaging

A terahertz (THz) frequency-modulated continuous wave (FMCW) imaging radar system is developed for high-resolution 3D imaging recently. Aiming at the problems of long data acquisition periods and large sample sizes for the developed imaging system, an algorithm based on compressed sensing is proposed for THz FMCW radar 3D imaging in this paper. Firstly, the FMCW radar signal model is built, and the conventional range migration algorithm is introduced for THz FMCW radar imaging. Then, compressed sensing is extended for THz FMCW radar 3D imaging, and the Newton smooth L0-norm (NSL0) algorithm is presented for sparse measurement data reconstruction. Both simulation and measurement experiments demonstrate the feasibility of reconstructing THz images from measurements even at the sparsity rate of 20%.

A Consistent Estimator of Structural Distribution

We consider sparse count data models with the sparsity rate ? = N/n = O(1) where N = N (n) is the number of observations and n ? ? is the number of cells. In this case the plug-in estimator of the structural distribution of expected frequencies is inconsistent. If ? = O(n ?? ) for some ? > 0, the nonparametric maximum likelihood estimator, in general, is also inconsistent. Assuming that some auxiliary information on the expected frequencies is available, we construct a consistent estimator of the structural distribution.

A Sharp Sufficient Condition for Sparsity Pattern Recovery

Sufficient number of linear and noisy measurements for exact and approximate sparsity pattern/support set recovery in the high dimensional setting is derived. Although this problem as been addressed in the recent literature, there is still considerable gaps between those results and the exact limits of the perfect support set recovery. To reduce this gap, in this paper, the sufficient condition is enhanced. A specific form of a Joint Typicality decoder is used for the support recovery task. Two performance metrics are considered for the recovery validation; one, which considers exact support recovery, and the other which seeks partial support recovery. First, an upper bound is obtained on the error probability of the sparsity pattern recovery. Next, using the mentioned upper bound, sufficient number of measurements for reliable support recovery is derived. It is shown that the sufficient condition for reliable support recovery depends on three key parameters of the problem; the noise variance, the minimum nonzero entry of the unknown sparse vector and the sparsity level. Simulations are performed for different sparsity rate, different noise variances, and different distortion levels. The results show that for all the mentioned cases the proposed methodology increases convergence rate of upper bound of the error probability of support recovery significantly which leads to a lower error probability bound compared with previously proposed bounds.

SparseMaps: Convolutional networks with sparse feature maps for tiny image classification

Deep convolutional models have been able to get extraordinary results in visual, speech and textual processing domains. Nevertheless, the complexity of manifold of images in data space necessitates the deep networks to have a large number of parameters and subsequently these deep models prone to redundancy and overfitting. One of the most effective methods to tackle this problem is devising special regularization methods in the context of convolutional models. In this paper, after presenting and discussing the most important models in the field, sparse feature maps are proposed and employed as a penalty term in the cost function. This conducts the learning process to construct kernels so that the feature maps at every point are sparse along the depth. A sparse representation is able to adapt to the varying level of information of natural images and can help to extract more independent and informative representations. Also, the DropMaps concept is proposed and employed in the last convolutional layer of the model. This technique applies dropout on the feature maps that causes coincidence of feature maps to be avoided. As is shown in this paper, sparse feature maps and DropMaps can handle the problem of overfitting in large models for tiny images. We have studied the effect of the sparsity rate on the accuracy of the model, and it is observed the accuracy of the test dataset reaches its maximum at a sparsity rate of 0.05. Moreover, by designing appropriate learning rate curves, we were able to obtain ensemble machines with much less cost for training. It is noticeable that the test accuracy is higher than the validation accuracy of the ensemble, indicating that the model has not overfitted. In the input of the model, a random online preprocessing layer is employed for the training phase that helps regularization of the model. Comparing input space and feature space of the model we found that the proposed network is able to successfully separate images of different classes. Finally, testing the proposed model with MNIST dataset has shown that the test set can be classified with accuracy 99.75. The same test with CIFAR 10 dataset attained an accuracy of 94.05.

Reduction of data acquisition time in Raman spectroscopy imaging using structure based compressive sampling algorithm

The Bayesian approach that utilizes the sparsity constraint and a priori statistical information to obtain near optimal estimates is presented. In addition, the wealthy structure of the sensing matrix including modularity, orthogonality and order recursive calculations is used to develop a fast sparse recovery algorithm. The performance of this algorithm is quite close to Convex Relaxation and Fast Bayesian Matching Pursuit algorithms at low sparsity rate while it outperforms Orthogonal Matching Pursuit algorithm by approximately 3 dB for the studied range of sparsity. The results show that the Structure based Compressive Sampling is a promising tool for obtaining Raman image reconstructions of quality in a reduced time of acquisition.

희소 투영행렬 획득을 위한 RSR 개선 방법론

본 논문은 패턴인식에서 자주 사용되는 투영행렬을 희소화하는 문제를 다룬다. 최근 임베디드 시스템이 널리 사용됨에 따라 탑재되는 프로그램의 용량이 제한받는 경우가 빈번히 발생한다. 개발된 프로그램은 상수 데이터를 포함하는 경우가 많다. 예를 들어, 얼굴인식과 같은 패턴인식 프로그램의 경우 고차원 벡터를 저차원 벡터로 차원을 축소하는 투영행렬을 사용하는 경우가 많다. 인식성능 향상을 위해 영상으로부터 매우 높은 차원의 고차원 특징벡터를 추출하는 경우 투영행렬의 사이즈는 매우 크다. 최근 라소 회귀분석 방법을 이용한 RSR(rotated sparse regression) 방법론[1]이 제안되었다. 이 방법론은 여러 실험을 통해 희소행렬을 구하는 가장 우수한 알고리즘 중 하나로 평가받고 있다. 우리는 본 논문에서 RSR을 개선할 수 있는 세 가지 방법론을 제안한다. 즉, 학습데이터에서 이상치를 제거하여 일반화 성능을 높이는 방법, 학습데이터를 랜덤 샘플링하여 희소율을 높이는 방법, RSR의 목적함수에 엘라스틱 넷 회귀분석의 패널티 항을 사용한 E-RSR(elastic net-RSR) 방법을 제안한다. 우리는 실험을 통해 제안한 방법론이 인식률을 희생하지 않으며 희소율을 크게 증가시킴으로써 기존 RSR 방법론을 개선할 수 있음을 보였다. This paper addresses the problem to make sparse the projection matrix in pattern recognition method. Recently, the size of computer program is often restricted in embedded systems. It is very often that developed programs include some constant data. For example, many pattern recognition programs use the projection matrix for dimension reduction. To improve the recognition performance, very high dimensional feature vectors are often extracted. In this case, the projection matrix can be very big. Recently, RSR(roated sparse regression) method[1] was proposed. This method has been proved one of the best algorithm that obtains the sparse matrix. We propose three methods to improve the RSR; outlier removal, sampling and elastic net RSR(E-RSR) in which the penalty term in RSR optimization function is replaced by that of the elastic net regression. The experimental results show that the proposed methods are very effective and improve the sparsity rate dramatically without sacrificing the recognition rate compared to the original RSR method.

Maximum Hands-Off Control: A Paradigm of Control Effort Minimization

In this paper, we propose a new paradigm of control, called a maximum hands-off control. A hands-off control is defined as a control that has a short support per unit time. The maximum hands-off control is the minimum support (or sparsest) per unit time among all controls that achieve control objectives. For finite horizon control, we show the equivalence between the maximum hands-off control and L1-optimal control under a uniqueness assumption called normality. This result rationalizes the use of L1 optimality in computing a maximum hands-off control. We also propose an L1/L2-optimal control to obtain a smooth hands-off control. Furthermore, we give a self-triggered feedback control algorithm for linear time-invariant systems, which achieves a given sparsity rate and practical stability in the case of plant disturbances. An example is included to illustrate the effectiveness of the proposed control.

Quickest Detection of Multi-channel Based on STFT and Compressed Sensing

This paper proposes a multi-channel quickest detection method based on compressed sensing and short-time Fourier transform. Quickest detection performs a statistical test to obtain the minimal detection delay subject to given false alarm constrains. Short-time Fourier transform, which reflects the time---frequency information, implements the multi-channel quickest detection. Compressed sensing reduces the sampling rate at first. Compared with single-channel spectrum sensing, this method substantially improves the spectrum access opportunity in time and frequency domain. The relationship between the detection delay and other parameters, such as the probability of false alarm, SNR, sparsity, and sampling rate, verifies the validity of the method. While simulation results show that this method can perform spectrum sensing in high detection probability and low probability of false alarm.

Receiver-Based Recovery of Clipped OFDM Signals for PAPR Reduction: A Bayesian Approach

Clipping is one of the simplest peak-to-average power ratio reduction schemes for orthogonal frequency division multiplexing (OFDM). Deliberately clipping the transmission signal degrades system performance, and clipping mitigation is required at the receiver for information restoration. In this paper, we acknowledge the sparse nature of the clipping signal and propose a low-complexity Bayesian clipping estimation scheme. The proposed scheme utilizes a priori information about the sparsity rate and noise variance for enhanced recovery. At the same time, the proposed scheme is robust against inaccurate estimates of the clipping signal statistics. The undistorted phase property of the clipped signal, as well as the clipping likelihood, is utilized for enhanced reconstruction. Furthermore, motivated by the nature of modern OFDM-based communication systems, we extend our clipping reconstruction approach to multiple antenna receivers and multi-user OFDM.We also address the problem of channel estimation from pilots contaminated by the clipping distortion. Numerical findings are presented that depict favorable results for the proposed scheme compared to the established sparse reconstruction schemes.

Sparse Reconstruction Using Distribution Agnostic Bayesian Matching Pursuit

A fast matching pursuit method using a Bayesian approach is introduced for sparse signal recovery. This method performs Bayesian estimates of sparse signals even when the signal prior is non-Gaussian or unknown. It is agnostic on signal statistics and utilizes a priori statistics of additive noise and the sparsity rate of the signal, which are shown to be easily estimated from data if not available. The method utilizes a greedy approach and order-recursive updates of its metrics to find the most dominant sparse supports to determine the approximate minimum mean-square error (MMSE) estimate of the sparse signal. Simulation results demonstrate the power and robustness of our proposed estimator.

Structure-Based Bayesian Sparse Reconstruction

Sparse signal reconstruction algorithms have attracted research attention due to their wide applications in various fields. In this paper, we present a simple Bayesian approach that utilizes the sparsity constraint and a priori statistical information (Gaussian or otherwise) to obtain near optimal estimates. In addition, we make use of the rich structure of the sensing matrix encountered in many signal processing applications to develop a fast sparse recovery algorithm. The computational complexity of the proposed algorithm is relatively low compared with the widely used convex relaxation methods as well as greedy matching pursuit techniques, especially at a low sparsity rate.