Alternating Direction Optimization Research Articles

PiSL: Physics-informed Spline Learning for data-driven identification of nonlinear dynamical systems

Nonlinear dynamics is ubiquitous in nature and commonly seen in many science and engineering disciplines. Distilling analytical expressions that govern the behavior of nonlinear dynamical systems from limited data is vital but remains challenging. To tackle this fundamental issue, we propose a novel Physics-informed Spline Learning (PiSL) framework to discover parsimonious governing equations for nonlinear dynamics, based on sparsely sampled noisy data. Specifically, splines are employed to locally interpolate the dynamics and perform analytical differentiation, feeding the discovery of underlying equations in form of either a linear interpolation of candidate terms or a symbolic-activated neural network model. The physics residual in turn informs the spline learning. The synergy between splines and discovered governing equations produces great robustness against high-level data sparsity and noise. Subsequently, a hybrid sparsity-promoting alternating direction optimization strategy is developed for fine-tuning the coefficients with a sparsity enforcement approach to obtain a parsimonious structure of discovered governing equations. The effectiveness and supremacy of the proposed PiSL architectures have been demonstrated by several numerical and experimental examples, in comparison with two state-of-the-art methods serving as baselines.

Fast and high-quality single-pixel imaging.

The imaging quality of the conventional single-pixel-imaging (SPI) technique seriously degrades at a low sampling rate. To tackle this problem, we propose an efficient sampling method and a high-quality real-time image reconstruction strategy: first, different from the conventional simple circular path sampling strategy or variable density random sampling technique, the proposed method samples the Fourier spectrum using the spectrum distribution of the image, that is, sampling the significant spectrum coefficients first, which will help to improve the image quality at a relevantly low sampling rate; second, to handle the long image reconstruction time caused by the iterative algorithm, the sparsity of the image and the alternating direction optimization strategy are combined to ameliorate the reconstruction process in the image gradient space. Compared with the state-of-the-art techniques, the proposed method significantly improves the imaging quality and achieves real-time reconstruction on the time scale of milliseconds.

Direct interaction network inference for compositional data via codaloss.

16S rRNA gene sequencing and whole microbiome sequencing make it possible and stable to quantitatively analyze the composition of microbial communities and the relationship among microbial communities, microbes, and hosts. One essential step in the analysis of microbiome compositional data is inferring the direct interaction network among microbial species, bringing to light the potential underlying mechanism that regulates interaction in their communities. However, standard statistical analysis may obtain spurious results due to compositional nature of microbiome data; therefore, network recovery of microbial communities remains challenging. Here, we propose a novel loss function called codaloss for direct microbes interaction network estimation under the sparsity assumptions. We develop an alternating direction optimization algorithm to obtain sparse solution of codaloss as estimator. Compared to other state-of-the-art methods, our model makes less assumptions about the microbial networks. The simulation and real microbiome data results show that our method outperforms other methods in network inference. An implementation of codaloss is available from https://github.com/xuebaliang/Codaloss.

Low rank spectral regression via matrix factorization for efficient subspace learning

Subspace learning aims to obtain the corresponding low-dimensional representation of high dimensional data in order to facilitate the subsequent data storage and processing. Graph-based subspace learning is a kind of effective subspace learning methods by modeling the data manifold with a graph, which can be included in the general spectral regression (SR) framework. By using the least square regression form as objective function, spectral regression mathematically avoids performing eign-decomposition on dense matrices and has excellent flexibility. Recently, spectral regression has obtained promising performance in diverse applications; however, it did not take the underlying classes/tasks correlation patterns of data into consideration. In this paper, we propose to improve the performance of spectral regression by exploring the correlation among classes with low-rank modeling. The newly formulated low-rank spectral regression (LRSR) model is achieved by decomposing the projection matrix in SR by two factor matrices which were respectively regularized. The LRSR objective function can be handled by the alternating direction optimization framework. Besides some analysis on the differences between LRSR and existing related models, we conduct extensive experiments by comparing LRSR with its full rank counterpart on benchmark data sets and the results demonstrate its superiority.

Multi-nonlinear multi-view locality-preserving projection with similarity learning for random cross-view gait recognition

View variation is one of the greatest challenges in the field of gait recognition. Subspace learning approaches are designed to solve this issue by projecting cross-view features into a common subspace before recognition. However, similarity measures are data-dependent, which results in low accuracy when cross-view gait samples are randomly arranged. Inspired by the recent developments of data-driven similarity learning and multi-nonlinear projection, we propose a new unsupervised projection approach, called multi-nonlinear multi-view locality-preserving projections with similarity learning (M2LPP-SL). The similarity information among cross-view samples can be learned adaptively in our M2LPP-SL. Besides, the complex nonlinear structure of original data can be well preserved through multiple explicit nonlinear projection functions. Nevertheless, its performance is largely affected by the choice of nonlinear projection functions. Considering the excellent ability of kernel trick for capturing nonlinear structure information, we further extend M2LPP-SL into kernel space, and propose its multiple kernel version MKMLPP-SL. As a result, our approaches can capture linear and nonlinear structure more precisely, and also learn similarity information hidden in the multi-view gait dataset. The proposed models can be solved efficiently by alternating direction optimization method. Extensive experimental results over various view combinations on the multi-view gait database CASIA-B have demonstrated the superiority of the proposed algorithms.

Energy-Efficient Resource Allocation for Cognitive Industrial Internet of Things With Wireless Energy Harvesting

Cognitive industrial Internet of Things (CIIoT) can extend available spectrum resources by accessing the spectrum licensed to primary user (PU) on the premise of not disturbing the PU's communications. However, additional spectrum sensing and long-time working may consume much energy of CIIoT. In this article, a CIIoT with wireless energy harvesting (WEH) is proposed to harvest the radio frequency energy of PU's signal, and energy-efficient resource allocations in different spectrum access modes are presented to maximize the average transmission rate of CIIoT while guaranteeing its energy saving requirements. The underlay and overlay spectrum access modes for CIIoT with WEH are described, respectively, in which the energy-efficient resource allocations are formulated as joint optimization problems that can be solved using the alternating direction optimization and water-filling algorithm. By combining underlay and overlay modes, a hybrid spectrum access mode is proposed to enable the CIIoT to access both idle and busy spectrum without limiting its transmission power at the absence of PU. Simulation results show that the CIIoT with WEH can consume less power to achieve larger transmission rate, and the hybrid mode outperforms the underlay and overlay modes in the aspects of transmission rate and energy saving.

Joint non-negative and fuzzy coding with graph regularization for efficient data clustering

Non-negative matrix factorization (NMF) is an effective model in converting data into non-negative coefficient representation whose discriminative ability is usually enhanced to be used for diverse pattern recognition tasks. In NMF-based clustering, we often need to perform K-means on the learned coefficient as postprocessing step to get the final cluster assignments. This breaks the connection between the feature learning and recognition stages. In this paper, we propose to learn the non-negative coefficient matrix based on which we jointly perform fuzzy clustering, by viewing that each column of the dictionary matrix as a concept of each cluster. As a result, we formulate a new fuzzy clustering model, termed Joint Non-negative and Fuzzy Coding with Graph regularization (G-JNFC), and design an effective optimization method to solve it under the alternating direction optimization framework. Besides the convergence and computational complexity analysis on G-JNFC, we conduct extensive experiments on both synthetic and representative benchmark data sets. The results show that the proposed G-JNFC model is effective in data clustering.

SAR Image Despeckling Based on Combination of Fractional-Order Total Variation and Nonlocal Low Rank Regularization

Regularization method is an effective tool for synthetic aperture radar (SAR) image despeckling. Design of the effective regularization terms describing the image priors plays a vital role in this kind of method. In this article, a new combinational regularization model for speckle reduction (CRM-SR) is proposed, in which a regularization term is elaborately designed to contain both a fractional-order total variation (FrTV) regularization and a nonlocal low rank (NLR) regularization. The new regularization model inherits both the advantages of FrTV and NLR and improves the performance of SAR despeckling and, therefore, better preserves the edges and geometrical features of the images during the despeckling process. An efficient algorithm based on alternating direction optimization is derived to solve the proposed combinational regularization model. Experimental results show that the proposed model can effectively remove SAR image speckle and preserve the geometrical features of images according to both subjective visual assessment of image quality and objective evaluation.

A scalable and robust trust-based nonnegative matrix factorization recommender using the alternating direction method

Matrix Factorization (MF) has been proven to be an effective approach for the generation of a successful recommender system. However, most current MF-based recommenders cannot obtain high prediction accuracy due to the sparseness of the user–item matrix in collaborative filtering models. Moreover, they suffer from scalability issues when applied to large-scale real-world tasks. To tackle these issues, a social regularization method, called TrustANLF, is proposed, which incorporates users’ social trust information in a nonnegative matrix factorization framework. The proposed method integrates trust statements as an additional information source along with rating values into the recommendation model to deal with the data sparsity and cold-start issues. Moreover, the alternating direction optimization method is used for solving the trust-based nonnegative MF model in order to improve convergence speed as well as reduce computational and memory costs. To evaluate the effectiveness of the proposed method, several experiments are performed on three real-world datasets. The obtained results demonstrate the significant improvements of the proposed method over several state-of-the-art methods.

Bilinear Matrix Factorization Methods for Time-Varying Narrowband Channel Estimation: Exploiting Sparsity and Rank

In this paper, the estimation of a narrowband time-varying channel under the practical assumptions of finite block length and finite transmission bandwidth is investigated. It is shown that the signal, after passing through a time-varying narrowband channel reveals a useful parametric low-rank structure that can be represented as a bilinear form. To estimate the channel, two strategies are developed. The first method exploits the low-rank bilinear structure of the channel via a non-convex strategy based on an alternating direction optimization between the delay and Doppler directions. While prior Wirtinger flow methods have exhibited good performance with proper initialization, this is not true in the current scenario. Due to the non-convex nature of this approach, this first approach is sensitive to local minima. Furthermore, the convergence rate of the Wirtinger flow method is shown to be provably modest. Thus, a novel convex approach based on the minimization of the atomic norm using measurements of the signal at the time domain is proposed based on a second bilinear parametrization of the channel. For the convex approach, optimality and uniqueness conditions, and a theoretical guarantee for noiseless channel estimation with small number of measurements are characterized. Numerical results show that the performance of the proposed algorithm is independent of the leakage effect and the new methods can achieve a 5–12 dB improvement, on average, compared to a classical $l_1$ -based sparse approximation method that does not consider the leakage effect, and 2 dB improvement over a basis expansion method that considers the leakage effect.

Large-Scale Price Interval Prediction at OTA Sites

With the rapid growing proliferation of online travel agent (OTA) services, personalized recommendations are highly valuable as they can improve customer experiences by preventing the information overload problem. The accurate prediction of users’ expectations for price plays an important role in the personalized recommendation of hotels in OTA platforms. Considering that customers’ preferences for hotel prices are actually acceptable ranges and traditional point estimations may neglect some informative aspects of the prediction, interval estimation is more suitable for the problem investigated in this paper. However, existing related methods are not applicable due to some specific issues. To provide a better personalized recommendation of hotels in OTA platforms, this paper proposes a novel interval forecasting solution to improve the accuracy of predicting users’ price preferences. The novel interval forecasting solution first puts forward a customized objective function which could directly measure the quality of constructed intervals, while also allowing for adjustable tradeoffs between interval tightness and prediction reliability. Then, it combines alternating direction optimization and the gradient boosting framework to efficiently aggregate weak individual predictors to optimize the introduced learning objective. Empirical comparisons conducted on several benchmark standard datasets and a large-scale dataset shared with us by a major Chinese OTA site demonstrate the effectiveness of the proposed approach.

Cooperative spectrum sensing and data transmission optimization for multichannel cognitive sonar communication network

The natural acoustic system used by marine mammals and the artificial sonar system used by humans coexist in the underwater cognitive sonar communication networks (CSCN). They share the spectrum when they are in the same waters. The CSCN detects the natural acoustic signal depending on cooperative spectrum sensing of sonar nodes. In order to improve spectrum sensing performance of CSCN, the optimization of cooperative spectrum sensing and data transmission is investigated. We seek to obtain spectrum efficiency maximization (SEM) and energy efficiency maximization (EEM) of CSCN through jointly optimizing sensing time, subchannel allocation, and transmission power. We have formulated a class of optimization problems and obtained the optimal solutions by alternating direction optimization and Dinkelbach’s optimization. The simulation results have indicated that SEM can achieve higher spectrum efficiency while EEM may get higher energy efficiency.

Restoration of hyperspectral image contaminated by Poisson noise using spectral unmixing

Abstract Restoring Poissonian images is particularly considered due to astronomical and medical applications in recent years. In this paper, we propose a novel noise removal method for restoration of hyperspectral images corrupted by Poisson noise, based on spectral unmixing technique. We formulate Poissonian hyperspectral image problem as an optimization problem where the cost function consists of three terms. The likelihood of the observation with Poisson distribution is used as the data-fidelity term. The total variation of the abundance for piecewise smooth information and the l1-norm of the abundance for sparse information are introduced as two regularization terms. Finally, the optimization problem is effectively solved by alternating direction optimization algorithm. Therefore, the hyperspectral image can be well restored after Poisson noise is successfully removed. The experiment results show that the proposed method has a better performance than current Poissonion hyperspectral image denoising methods, in terms of both image quality and computation time.

Hyperspectral Image Superresolution Based on Double Regularization Unmixing

This letter proposes a novel double regularization unmixing-based method for hyperspectral image (HSI) superresolution. The proposed cost function contains two data-fidelity terms, the endmember regularization term and the abundance regularization term. Since the double regularization unmixing terms are able to exploit the spatial structure information of endmember and abundance, respectively, the nonnegative factorization (spectral unmixing) error is minimized. As a result, the performance of the proposed HSI superresolution method can be enhanced in terms of noise suppression and the special structure information preservation of reconstruction images. Finally, the associated optimization problem is effectively solved by an alternating direction optimization algorithm. Simulation results illustrate that the proposed method has a better performance than the state-of-the-art methods in terms of both visual effectiveness and quality indices.

Joint Resource Allocation of Spectrum Sensing and Energy Harvesting in an Energy-Harvesting-Based Cognitive Sensor Network.

The cognitive sensor (CS) can transmit data to the control center in the same spectrum that is licensed to the primary user (PU) when the absence of the PU is detected by spectrum sensing. However, the battery energy of the CS is limited due to its small size, deployment in atrocious environments and long-term working. In this paper, an energy-harvesting-based CS is described, which senses the PU together with collecting the radio frequency energy to supply data transmission. In order to improve the transmission performance of the CS, we have proposed the joint resource allocation of spectrum sensing and energy harvesting in the cases of a single energy-harvesting-based CS and an energy-harvesting-based cognitive sensor network (CSN), respectively. Based on the proposed frame structure, we have formulated the resource allocation as a class of joint optimization problems, which seek to maximize the transmission rate of the CS by jointly optimizing sensing time, harvesting time and the numbers of sensing nodes and harvesting nodes. Using the half searching method and the alternating direction optimization, we have achieved the sub-optimal solution by converting the joint optimization problem into several convex sub-optimization problems. The simulation results have indicated the predominance of the proposed energy-harvesting-based CS and CSN models.

Effective Alternating Direction Optimization Methods for Sparsity-Constrained Blind Image Deblurring.

Single-image blind deblurring for imaging sensors in the Internet of Things (IoT) is a challenging ill-conditioned inverse problem, which requires regularization techniques to stabilize the image restoration process. The purpose is to recover the underlying blur kernel and latent sharp image from only one blurred image. Under many degraded imaging conditions, the blur kernel could be considered not only spatially sparse, but also piecewise smooth with the support of a continuous curve. By taking advantage of the hybrid sparse properties of the blur kernel, a hybrid regularization method is proposed in this paper to robustly and accurately estimate the blur kernel. The effectiveness of the proposed blur kernel estimation method is enhanced by incorporating both the -norm of kernel intensity and the squared -norm of the intensity derivative. Once the accurate estimation of the blur kernel is obtained, the original blind deblurring can be simplified to the direct deconvolution of blurred images. To guarantee robust non-blind deconvolution, a variational image restoration model is presented based on the -norm data-fidelity term and the total generalized variation (TGV) regularizer of second-order. All non-smooth optimization problems related to blur kernel estimation and non-blind deconvolution are effectively handled by using the alternating direction method of multipliers (ADMM)-based numerical methods. Comprehensive experiments on both synthetic and realistic datasets have been implemented to compare the proposed method with several state-of-the-art methods. The experimental comparisons have illustrated the satisfactory imaging performance of the proposed method in terms of quantitative and qualitative evaluations.

Optimal Resource Allocation in Simultaneous Cooperative Spectrum Sensing and Energy Harvesting for Multichannel Cognitive Radio

In this paper, a simultaneous cooperative spectrum sensing and energy harvesting model is proposed to improve the transmission performance of the multichannel cognitive radio. The frame structure is divided into sensing slot and transmission slot. In the sensing slot, the secondary user (SU) splits the subchannels into two subchannel sets, one for sensing the primary user (PU) by multichannel cooperative spectrum sensing and the other one for collecting the radio frequency energy of the PU signal and noise by multichannel energy harvesting. In the transmission slot, the harvested energy is supplied to compensate the sensing energy loss in order to guarantee the throughput. We have formulated the resource allocation of the proposed model as a class of optimization problems, which maximize aggregate throughput, harvested energy, and energy efficiency of the SU over all the subchannels through jointly optimizing subchannel set, sensing time, and transmission power, respectively. To achieve the sub-optimal solutions to the optimization problems, we have proposed the subchannel allocation algorithm and the joint optimization algorithm of sensing time and transmission power based on the Greedy algorithm and the alternating direction optimization. The stopping criteria of SU is described, when the PU is not present but the harvested energy is not enough. The simulation results are presented to demonstrate the validity and predominance of our proposed algorithms.

Robust Online Spectrum Prediction With Incomplete and Corrupted Historical Observations

A range of emerging applications, from adaptive spectrum sensing to proactive spectrum mobility, depend on the ability to foresee spectrum state evolution. Despite a number of studies appearing about spectrum prediction, fundamental issues still remain unresolved: 1) The existing studies do not explicitly account for anomalies, which may incur serious performance degradation; 2) they focus on the design of batch spectrum prediction algorithms, which limit the scalability to analyze massive spectrum data in real time; 3) they assume the historical data are complete, which may not hold in reality. To address these issues, we develop a Robust Online Spectrum Prediction (ROSP) framework, with incomplete and corrupted observations, in this paper. We first present data analytics of real-world spectrum measurements to reveal the correlation structures of spectrum evolution and to analyze the impact of anomalies on the rank distribution of spectrum matrices. Then, from a spectral–temporal 2-D perspective, we formulate the ROSP as a joint optimization problem of matrix completion and recovery by effectively integrating the time series forecasting techniques and develop an alternating direction optimization method to efficiently solve it. We apply ROSP to a wide range of real-world spectrum matrices of popular wireless services. Experiment results show that ROSP outperforms state-of-the-art spectrum prediction schemes.

A novel spectrum handoff-based sensing-throughput tradeoff scheme in cognitive radio

In cognitive radio (CR), there is a tradeoff between spectrum sensing time and throughput of secondary user (SU). In the conventional sensing-throughput tradeoff scheme, when the presence of the primary user (PU) is detected, the SU will stop forwarding data and wait to redetect the PU in the following frame, yielding great throughput loss. In order to improve the SU's throughput, a novel sensing-throughput tradeoff scheme is proposed, which allows the SU to search for a new idle channel through spectrum searching and transfer to this channel to continue communication, when the presence of the PU is detected. An optimization problem is proposed to maximize the SU's throughput in the proposed scheme through jointly optimizing the sensing time, the searching time and the number of available channels, providing that the detection probability to the PU is guaranteed. By fixing the number of channels, based on alternating direction optimization, the joint optimization algorithm of sensing time and searching time is proposed, and then the optimal number of channels is obtained through enumerative searching. The simulation results how that there exist the optimal solutions to the proposed scheme, and the proposed scheme outperforms the conventional scheme notably, with different detection probabilities and sampling frequencies.

Poissonian Hyperspectral Image Superresolution Using Alternating Direction Optimization

The reconstruction of Poissonian image is an active research area in recent years. This paper proposes a novel method for Poissonian hyperspectral image superresolution by fusing a low-spatial-resolution hyperspectral image and a low-spectral-resolution multispectral image. The fusion scheme is designed as an optimization problem, whose cost function consists of the two data-fidelity terms about Poisson distribution, the sparse representation term, and the nonlocal regularization term. The two data-fidelity terms can capture statistical information of Poisson noise. The sparse representation term is used for enhancing the quality of sparsity-based signal reconstruction, and the nonlocal regularization term exploits the spatial similarity of hyperspectral image. As a result, the hyperspectral image and multispectral image are well fused. Finally, the designed optimization problem is effectively solved by an alternating direction optimization algorithm. Simulation results illustrate that the proposed method has a better performance than several well-known methods both in terms of quality indexes and reconstruction visual effect.