Block Coordinate Descent Approach Research Articles

A Graph-Informed Modeling Framework Empowering Gene Pathway Discovery.

This study introduces a novel graph-informed modeling framework for improving the statistical analysis of gene expression data, particularly in the context of identifying differentially expressed gene pathways and gene expression-assisted disease classification in a high-dimensional data setting. By integrating gene regulatory network information into hypothesis testing for the difference between mean vectors and linear discriminant analysis, we aim to effectively capture and utilize previously validated external gene interaction information. Our method leverages a block-coordinate descent approach which enables us to incorporate mixed graph information into linear structural equation modeling, accommodating directed/undirected edges and potential cycles in gene regulatory networks. Extensive simulations under various data scenarios have demonstrated the effectiveness of our approach with improved power for gene pathway tests and disease classification over existing methods. An application to a lung cancer dataset from the Cancer Genome Atlas Program (TCGA) further exemplifies the potential of our graph-informed approach in empowering the detection of differentially expressed gene pathways and gene expression-based classification of different lung cancer stages. Our findings underscore the potential utility of incorporating gene regulatory network information in gene pathway analysis, setting the stage for future advancements in gene pathway discovery, disease diagnosis, and treatment strategies.

Joint Transmit and Receive Beamforming Design for DPC-Based MIMO DFRC Systems

This paper proposes an optimal beamforming strategy for a downlink multi-user multi-input–multi-output (MIMO) dual-function radar communication (DFRC) system with dirty paper coding (DPC) adopted at the transmitter. We aim to achieve the maximum weighted sum rate of communicating users while adhering to a predetermined transmit covariance constraint for radar performance assurance. To make the intended problem trackable, we leverage the equivalence of the weighted sum rate and the weighted minimum mean squared error (MMSE) to reframe the issue and devise a block coordinate descent (BCD) approach to iteratively calculate transmit and receive beamforming solutions. Through this methodology, we demonstrate that the optimal receive beamforming aligns with the traditional MMSE approach, whereas the optimal transmit beamforming design can be cast into a quadratic optimization problem defined on a complex Stiefel manifold. Based on the majorization–minimization (MM) method, an iterative algorithm is then developed to compute the optimal transmit beamforming design by solving a series of orthogonal Procrustes problems (OPPs) that admit closed-form optimal solutions. Numerical findings serve to validate the efficacy of our scheme. It is demonstrated that our approach can achieve at least 73% higher spectral efficiency than the existing methods in a high signal-to-noise ratio (SNR) regime.

Secure two-way transmission via untrusted UAV Relay: Joint path design and slot-pairing strategy

This paper proposes a novel two-way secure communication system that allows numerous pairs of ground users (GUs) communicate with one another using an untrusted UAV relay (UUR), where the UUR can eavesdrop on information while helping GUs perform two-way information transmission. We propose a safe scheme using slot pairing and joint path design to ensure the security of communication. Specifically, according to the physical-layer network coding (PNC) protocol, the collected information from GUs is cached in the UAV’s shock absorber before being transmitted as the UAV approaches optimal channel conditions. By matching the channel status of two-way transmission with the slot-pairing approach, our goal is to optimize the system’s secrecy sum rate (SSR). Due to the hybrid integer non-convex optimization problem, the formulated problem is difficult to address. As a result, we decompose it into three subproblems and propose an effective iterative algorithm that uses the block coordinate descent (BCD) approach and the sequential convex approximation (SCA) technique. The convergence and efficacy of our proposed scheme are demonstrated by numerical results.

Secure Beamforming in Multi-User Multi-IRS Millimeter Wave Systems

We study the secrecy rate maximization problem in a millimeter wave (mmWave) network, consisting of a base station (BS), multiple intelligent reflecting surfaces (IRSs) (or reconfigurable intelligent surfaces (RISs)), multiple users, and a single eavesdropper. To ensure a fair secrecy rate among all the users, we adopt a max-min fairness criterion which results in a mixed integer problem. We first relax discrete IRSs phase shifts to the continuous ones. To cope with the non-convexity of the relaxed optimization problem, we leverage the penalty method and block coordinate descent approach to divide it into two sub-problems, which are solved by successive convex approximation (SCA). Then, we propose a low-complexity mapping algorithm where feasible IRSs phase shifts are obtained. Mathematical evaluation shows the convergence of sub-problems to a Karush-Kuhn-Tucker (KKT) point of the original ones. Furthermore, the convergence guarantee of the overall proposed algorithm and computational complexity are investigated. Finally, simulation results show our proposed algorithm outweighs the conventional solutions based on the semi-definite programming (SDP) in terms of convergence and secrecy rate, especially in a larger number of IRSs and phase shifts where SDP suffers from rank-one approximation. Maximum ratio transmission (MRT) and IRS-free systems are also considered as other benchmarks.

RBDL: Robust block-Structured dictionary learning for block sparse representation

Dictionary learning methods have been extensively used in different types of image and signal processing tasks. In a number of applications, the collected data/signal may have a multi-subspace structure and be perturbed with outliers. These motivate the use of robust and block-sparse signal representations. In this paper, a new algorithm for learning a block-structured dictionary in the presence of outliers is proposed. It is based on α−divergence and has the advantage of tolerating the presence of outliers. A block coordinate descent approach is adopted to obtain simple closed-form solutions for both the sparse coding and dictionary update stages. Finally, experimental results illustrating the superiority of the proposed method over some state-of-the-art dictionary learning methods, are provided.

Robust Resource Allocation Design for Secure IRS-Aided WPCN

This paper studies the robust resource allocation design for secure intelligent reflecting surface (IRS)-aided wireless powered communication networks (WPCN). In particular, deploying an IRS can establish favorable end-to-end radio propagation environment for achieving the desired performance gain in secure wireless-powered systems. We aim to minimize the total hybrid base station (HBS) transmit power by jointly designing the active transmitting and receiving beamforming at the HBS, the passive beamforming at the IRS, and the transmit power of each wireless-powered device (WD) and jammer node (JN). We formulate a non-convex optimization problem for the robust resource allocation design taking into account the secrecy rate requirement of the WDs and the power budgets for both the WDs and the JNs. To handle this intractable problem, we propose a computationally efficient iterative suboptimal algorithm exploiting the block coordinate descent approach, the successive convex approximation, and the penalty method, which attains a Karush-Kuhn-Tucker (KKT) solution of the transformed problem. Also, we reveal that the optimal energy beamforming matrices are rank-one sharing the same spatial direction. Simulation results unveil that the proposed scheme is able to dramatically reduce the HBS transmit power over various baseline schemes adopting existing solutions. Besides, our results show the superiority of introducing IRS for secure communication in wireless-powered systems.

Accelerating probabilistic tensor canonical polyadic decomposition with nonnegative factors: An inexact BCD Approach

Recently, Bayesian modeling and variational inference (VI) were leveraged to enable the nonnegative factor matrix learning with automatic rank determination in tensor canonical polyadic decomposition (CPD), which has found various applications in big data analytics. However, since VI inherently performs block coordinate descent (BCD) steps over the functional space, it generally does not allow integration with modern large-scale optimization methods, making the scalability a critical issue. In this paper, it is revealed that the expectations of the variables updated by the VI algorithm is equivalent to the block minimization steps of a deterministic optimization problem. This equivalence further enables the adoption of inexact BCD method for devising a fast nonnegative factor matrix learning algorithm with automatic tensor rank determination. Numerical results using synthetic data and real-world applications show that the performance of the proposed algorithm is comparable with that of the VI-based algorithm, but with computation times reduced significantly.

Parsimonious Identification of Continuous-Time Systems: A Block-Coordinate Descent Approach⋆

The identification of electrical, mechanical, and biological systems using data can benefit greatly from prior knowledge extracted from physical modeling. Parametric continuous-time identification methods can naturally incorporate this knowledge, which leads to inter-pretable and parsimonious models. However, some applications lead to model structures that lack parsimonious descriptions using unfactored transfer functions, which are commonly used in standard direct approaches for continuous-time system identification. In this paper we characterize this parsimony problem, and develop a block-coordinate descent algorithm that delivers parsimonious models by sequentially estimating an additive decomposition of the transfer function of interest. Numerical simulations show the efficacy of the proposed approach.

Numerical methods for CT reconstruction with unknown geometry parameters

Computed tomography (CT) techniques are well known for their ability to produce high-quality images needed for medical diagnostic purposes. Unfortunately, standard CT machines are extremely large, heavy, require careful and regular calibration, and are expensive, which can limit their availability in point-of-care situations. An alternative approach is to use portable machines, but parameters related to the geometry of these devices (e.g., distance between source and detector, orientation of source to detector) cannot always be precisely calibrated, and these parameters may change slightly when the machine is adjusted during the image acquisition process. In this work, we describe the non-linear inverse problem that models this situation, and discuss algorithms that can jointly estimate the geometry parameters and compute a reconstructed image. In particular, we propose a hybrid machine learning and block coordinate descent (ML-BCD) approach that uses an ML model to calibrate geometry parameters, and uses BCD to refine the predicted parameters and reconstruct the imaged object simultaneously. We show using numerical experiments that our new method can efficiently improve the accuracy of both the image and geometry parameters.

Resource Reservation in Backhaul and Radio Access Network With Uncertain User Demands

Resource reservation is an essential step to enable wireless data networks to support a wide range of user demands. In this paper, we consider the problem of joint resource reservation in the backhaul and Radio Access Network (RAN) based on the statistics of user demands and channel states, and also network availability. The goal is to maximize the sum of expected traffic flow rates, subject to link and access point budget constraints, while minimizing the expected outage of downlinks. The formulated problem turns out to be non-convex and difficult to solve to global optimality. We propose an efficient Block Coordinate Descent (BCD) algorithm to approximately solve the problem. The proposed BCD algorithm optimizes the link capacity reservation in the backhaul using a novel multi-path routing algorithm that decomposes the problem down to link-level and parallelizes the computation across backhaul links, while the reservation of transmission resources in RAN is carried out via a novel scalable and distributed algorithm based on Block Successive Upper-bound Minimization (BSUM). We prove that the proposed BCD algorithm converges to a Karush-Kuhn-Tucker (KKT) solution. Simulation results verify the efficiency and the efficacy of our BCD approach against two heuristic algorithms.

Robust Resource Allocation for Lightweight Secure Transmission in Multicarrier NOMA-Assisted Full Duplex IoT Networks

In this article, with the aim to enhance the secure transmission and improve the utilization of spectrum resources in Internet of Things (IoT), a multicarrier nonorthogonal multiple access (MC-NOMA)-assisted full duplex (FD) network is investigated, in which nonorthogonal multiple access (NOMA) is implemented in both uplink and downlink transmissions. The lightweight and low-power physical layer security (PLS) technology is employed to protect the information from eavesdropping. Taking the imperfect channel state information (CSI) into account, we formulate a problem to optimize the beamforming vector, artificial noise (AN), transmit power, and subcarrier assignment policy aiming to maximize the worst case sum secrecy rate under the Quality of Service (QoS) and power consumption constraints. Since the formulated problem is nonconvex and difficult to be solved, we decompose it into two joint optimization subproblems. The first is resource allocation with given subcarrier assignment, which is solved by using the block coordinate descent (BCD) approach. The second is subcarrier assignment solved by the matching theory. Our simulation shows that the proposed scheme is robust against the CSI imperfectness of the eavesdropping and self-interference channels, while providing significant sum secrecy rate improvement compared with the orthogonal multiple access (OMA), half duplex (HD) systems, and other benchmark schemes.

Secrecy Energy Efficiency Maximization in UAV-Enabled Wireless Sensor Networks Without Eavesdropper’s CSI

Unmanned aerial vehicles (UAVs) are anticipated to be a potential data collection solution for wireless sensor networks (WSNs). The main challenges of integrating UAVs in WSNs are security threats and UAV’s onboard energy limitation. To cope with these two challenges, this article examines the secrecy energy efficiency (SEE) maximization problem in UAV-enabled WSN. Specifically, a full-duplex (FD) UAV gathers confidential information from ground sensor nodes (SNs) in the uplink while sending jamming signals to confound a ground eavesdropper (Eve) in the downlink. Considering a passive eavesdropping scenario lacking Eve’s instantaneous channel state information (CSI), the resulting problem is subject to the constraints of connection outage probability (COP), secrecy outage probability (SOP), securely collected bits, and flight trajectory. To tackle the intractable nonconvex problem, we first derive the optimal codeword rate and redundancy rate in closed-form expressions and then develop a low-complexity algorithm using the block coordinate descent (BCD) approach to alternatively optimize the SN scheduling, SN transmit power, UAV transmit power, and UAV trajectory. Simulation results verify the performance gains of the proposed scheme compared with the benchmark schemes. In particular, the proposed scheme achieves nearly the same secrecy rate gains at a lower UAV’s energy consumption cost than the sum secrecy rate maximization (SSRM) baseline. Moreover, it is revealed that trajectory optimization of the proposed scheme plays a crucial role in improving SEE performance compared with the circle trajectory (CT) baseline.

TU85. POLYGENIC RISK SCORE FOR CANNABIS USE DISORDER PREDICTS ACUTE INTOXICATING EFFECTS AND COGNITIVE DEFICITS INDUCED BY DELTA-9-TETRAHYDROCANNABINOL: A PILOT STUDY

Algorithms for learning overcomplete dictionaries for sparse signal representation are mostly iterative minimization methods that alternate between a sparse coding stage and a dictionary update stage. For most however, the notion of consistency of the learned quantities has not been addressed. Based on the observation that the observed signals can be approximated as a sum of rank one matrices, a new adaptive dictionary learning algorithm is proposed in this paper. It is derived via sequential adaptive penalized rank one matrix approximation where the ℓ1-norm is introduced as a penalty promoting sparsity. The proposed algorithm uses a block coordinate descent approach to consistently estimate the unknowns and has the advantage of having simple closed form solutions for both the sparse coding and dictionary update stages. The consistency properties of both the estimated sparse code and dictionary atom are provided. The performance of the proposed algorithm compared to some state of the art algorithms is illustrated on both simulated data and a real functional magnetic resonance imaging (fMRI) data set from a finger tapping experiment.

A Block Coordinate Descent-Based Projected Gradient Algorithm for Orthogonal Non-Negative Matrix Factorization

This article uses the projected gradient method (PG) for a non-negative matrix factorization problem (NMF), where one or both matrix factors must have orthonormal columns or rows. We penalize the orthonormality constraints and apply the PG method via a block coordinate descent approach. This means that at a certain time one matrix factor is fixed and the other is updated by moving along the steepest descent direction computed from the penalized objective function and projecting onto the space of non-negative matrices. Our method is tested on two sets of synthetic data for various values of penalty parameters. The performance is compared to the well-known multiplicative update (MU) method from Ding (2006), and with a modified global convergent variant of the MU algorithm recently proposed by Mirzal (2014). We provide extensive numerical results coupled with appropriate visualizations, which demonstrate that our method is very competitive and usually outperforms the other two methods.

Block layer decomposition schemes for training deep neural networks

Deep Feedforward Neural Networks' (DFNNs) weights estimation relies on the solution of a very large nonconvex optimization problem that may have many local (no global) minimizers, saddle points and large plateaus. As a consequence, optimization algorithms can be attracted toward local minimizers which can lead to bad solutions or can slow down the optimization process. Furthermore, the time needed to find good solutions to the training problem depends on both the number of samples and the number of variables. In this work, we show how Block Coordinate Descent (BCD) methods can be applied to improve performance of state-of-the-art algorithms by avoiding bad stationary points and flat regions. We first describe a batch BCD method ables to effectively tackle the network's depth and then we further extend the algorithm proposing a \textit{minibatch} BCD framework able to scale with respect to both the number of variables and the number of samples by embedding a BCD approach into a minibatch framework. By extensive numerical results on standard datasets for several architecture networks, we show how the application of BCD methods to the training phase of DFNNs permits to outperform standard batch and minibatch algorithms leading to an improvement on both the training phase and the generalization performance of the networks.

Robust convex clustering

Objective-based clustering is a class of important clustering analysis techniques; however, these methods are easily beset by local minima due to the non-convexity of their objective functions involved, as a result, impacting final clustering performance. Recently, a convex clustering method (CC) has been on the spot light and enjoys the global optimality and independence on the initialization. However, one of its downsides is non-robustness to data contaminated with outliers, leading to a deviation of the clustering results. In order to improve its robustness, in this paper, an outlier-aware robust convex clustering algorithm, called as RCC, is proposed. Specifically, RCC extends the CC by modeling the contaminated data as the sum of the clean data and the sparse outliers and then adding a Lasso-type regularization term to the objective of the CC to reflect the sparsity of outliers. In this way, RCC can both resist the outliers to great extent and still maintain the advantages of CC, including the convexity of the objective. Further we develop a block coordinate descent approach with the convergence guarantee and find that RCC can usually converge just in a few iterations. Finally, the effectiveness and robustness of RCC are empirically corroborated by numerical experiments on both synthetic and real datasets.

Tonic and Phasic Decomposition of Skin Conductance Data: A Generalized-Cross-Validation-Based Block Coordinate Descent Approach.

Salty sweat secretions in the epidermis change the skin's electrical activity resulting in the measured skin conductance signal. While the relatively fast variation of skin conductance (i.e. phasic component) reflects sympathetic nervous system activity, the slow variation (i. e. tonic component) is related to thermoregulation and general arousal. To better understand the neural information encoded in a skin conductance signal, it is necessary to decompose it into its constituent components. We model the fast variations using a second order differential equation incorporating a sparse impulsive input to the model. Furthermore, we model the tonic component with several cubic basis spline functions. Finally, we develop a block coordinate descent approach for skin conductance signal decomposition by employing generalized-cross-validation for balancing between smoothness of the tonic component, the sparsity of the neural stimuli, and residual error. We analyze experimental and simulated data to validate the performance of the proposed approach. We successfully illustrate its ability to recover the neural stimuli, the underlying physiological system parameters, and both tonic and phasic components. In summary, we develop a novel approach for decomposition of phasic and tonic components of skin conductance signal using a generalized-cross-validation-based block coordinate descent approach. Recovering the underlying neural stimuli and the tonic component accurately could potentially improve cognitive-stress-related arousal states estimation for better stress regulation in mental health disorders.

An α -Divergence-Based Approach for Robust Dictionary Learning.

In this paper, a robust sequential dictionary learning (DL) algorithm is presented. The proposed algorithm is motivated from the maximum likelihood perspective on dictionary learning and its link to the minimization of the Kullback-Leibler divergence. It is obtained by using a robust loss function in the data fidelity term of the DL objective instead of the usual quadratic loss. The proposed robust loss function is derived from the α -divergence as an alternative to the Kullback-Leibler divergence, which leads to a quadratic loss. Compared to other robust approaches, the proposed loss has the advantage of belonging to class of redescending M-estimators, guaranteeing inference stability from large deviations from the Gaussian nominal noise model. The algorithm is obtained by solving a sequence of penalized rank-1 matrix approximation problems, where the l1 -norm is introduced as a penalty promoting sparsity and then using a block coordinate descent approach to estimate the unknowns. Performance comparison with similar robust DL algorithms on digit recognition, background removal, and gray-scale image denoising is performed highlighting efficacy of the proposed algorithm.

A Sequential Block-Structured Dictionary Learning Algorithm for Block Sparse Representations

Dictionary learning algorithms have been successfully applied to a number of signal and image processing problems. In some applications, however, the observed signals may further have a multisubspace structure that enables block-sparse signal representations. For this case, a new algorithm for learning block-structured dictionaries for block-sparse signal representations is proposed in this paper. It is obtained by solving a sequence of penalized low rank matrix approximation problems where the $\ell _{1,2}$ -norm is introduced as a penalty promoting block sparsity and then using a block coordinate descent approach to estimate the unknowns. The proposed algorithm has the advantage of involving simple closed form solutions for both, the sparse coding and dictionary update stages. In particular, the sparse coding stage involves a simple shrinkage operation related to soft thresholding and it is related to the uniformly most powerful invariant testing procedure. Experimental results showing the improved efficacy and significant gain in computational time, offered by the proposed algorithm over the usual K singular value decomposition block extension are presented.

Total variation regularization-based compressed sensing synthetic aperture radar imaging

A total variation (TV) regularization-based compressed sensing (CS) synthetic aperture radar (SAR) imaging algorithm is proposed. Using chirp scaling algorithm as the approximation observation operator and taking TV minimization as the magnitude sparse constraint, an optimization model is designed by separating the magnitude from complex-valued scene. The reconstruction of the SAR image is formulated as a double-parameter joint optimization problem, which is solved by the second-order convex approximation and block coordinate descent approach. Compared with matching filtering method, TV-based image denoising method, and approximation observation-based CS method, the proposed algorithm can reconstruct both sparse and nonsparse scenes with higher accuracy using downsampling raw data. The experimental results via real data validate the effectiveness of the proposed method.