Efficient Iterative Optimization Algorithm Research Articles

Dual-learning Multi-hop Nonnegative Matrix Factorization for community detection

As an important branch of network science, community detection has garnered significant attention. Among various community detection methods, nonnegative matrix factorization (NMF)-based community detection approaches have become a popular research topic. However, most NMF-based methods overlook the network’s multi-hop information, let alone the community detection results specific to each hop of the network. In this paper, we propose Dual-learning Multi-hop NMF (DL-MHNMF), a method that considers not only the multi-hop connectivity between two nodes but also factors in the shared results across multiple hops and the impact of differences in the specific results at each hop on the shared outcomes. An efficient iterative optimization algorithm with guaranteed theoretical convergence is proposed for solving DL-MHNMF. Methodologically, by iteratively removing the specific results during the optimization process of DL-MHNMF, we achieve enhanced detection accuracy, which is also verified by subsequent experiments. Specifically, we compare fourteen algorithms on eleven publicly available datasets, and experimental results show that our algorithm outperforms most state-of-the-art methods. The source code is availiable at https://github.com/bx20000827/DL-MHNMF.git.

Robust and Sparse Principal Component Analysis With Adaptive Loss Minimization for Feature Selection.

Principal component analysis (PCA) is one of the most successful unsupervised subspace learning methods and has been used in many practical applications. To deal with the outliers in real-world data, robust principal analysis models based on various measure are proposed. However, conventional PCA models can only transform features to unknown subspace for dimensionality reduction and cannot perform features' selection task. In this article, we propose a novel robust PCA (RPCA) model to mitigate the impact of outliers and conduct feature selection, simultaneously. First, we adopt σ -norm as reconstruction error (RE), which plays an important role in robust reconstruction. Second, to conduct feature selection task, we apply l2,0 -norm constraint to subspace projection. Furthermore, an efficient iterative optimization algorithm is proposed to solve the objective function with nonconvex and nonsmooth constraint. Extensive experiments conducted on several real-world datasets demonstrate the effectiveness and superiority of the proposed feature selection model.

T[formula omitted]LR-Net: An unrolling network learning transformed tensor low-rank prior for dynamic MR image reconstruction

The tensor low-rank prior has attracted considerable attention in dynamic MR reconstruction. Tensor low-rank methods preserve the inherent high-dimensional structure of data, allowing for improved extraction and utilization of intrinsic low-rank characteristics. However, most current methods are still confined to utilizing low-rank structures either in the image domain or predefined transformed domains. Designing an optimal transformation adaptable to dynamic MRI reconstruction through manual efforts is inherently challenging. In this paper, we propose a deep unrolling network that utilizes the convolutional neural network (CNN) to adaptively learn the transformed domain for leveraging tensor low-rank priors. Under the supervised mechanism, the learning of the tensor low-rank domain is directly guided by the reconstruction accuracy. Specifically, we generalize the traditional t-SVD to a transformed version based on arbitrary high-dimensional unitary transformations and introduce a novel unitary transformed tensor nuclear norm (UTNN). Subsequently, we present a dynamic MRI reconstruction model based on UTNN and devise an efficient iterative optimization algorithm using ADMM, which is finally unfolded into the proposed T2LR-Net. Experiments on two dynamic cardiac MRI datasets demonstrate that T2LR-Net outperforms the state-of-the-art optimization-based and unrolling network-based methods.

Shared Manifold Regularized Joint Feature Selection for Joint Classification and Regression in Alzheimer's Disease Diagnosis.

In Alzheimer's disease (AD) diagnosis, joint feature selection for predicting disease labels (classification) and estimating cognitive scores (regression) with neuroimaging data has received increasing attention. In this paper, we propose a model named Shared Manifold regularized Joint Feature Selection (SMJFS) that performs classification and regression in a unified framework for AD diagnosis. For classification, unlike the existing works that build least squares regression models which are insufficient in the ability of extracting discriminative information for classification, we design an objective function that integrates linear discriminant analysis and subspace sparsity regularization for acquiring an informative feature subset. Furthermore, the local data relationships are learned according to the samples' transformed distances to exploit the local data structure adaptively. For regression, in contrast to previous works that overlook the correlations among cognitive scores, we learn a latent score space to capture the correlations and employ the latent space to design a regression model with l2,1 -norm regularization, facilitating the feature selection in regression task. Moreover, the missing cognitive scores can be recovered in the latent space for increasing the number of available training samples. Meanwhile, to capture the correlations between the two tasks and describe the local relationships between samples, we construct an adaptive shared graph to guide the subspace learning in classification and the latent cognitive score learning in regression simultaneously. An efficient iterative optimization algorithm is proposed to solve the optimization problem. Extensive experiments on three datasets validate the discriminability of the features selected by SMJFS.

Energy Consumption Minimization in Unmanned Aerial Vehicle-Enabled Secure Wireless Sensor Networks

In wireless sensor networks (WSNs), unmanned aerial vehicles (UAVs) are considered an effective data collection tool. In this paper, we investigate the energy-efficient data collection problem in a UAV-enabled secure WSN without knowing the instantaneous channel state information of the eavesdropper (Eve). Specifically, the UAV collected the information from all the wireless sensors at the scheduled time and forward it to the fusion center while Eve tries to eavesdrop on this confidential information from the UAV. To surmount this intractable and convoluted mixed-integer non-convex problem, we propose an efficient iterative optimization algorithm using the block coordinate descent (BCD) method to minimize the maximum energy consumption of the ground sensor nodes (GSNs) under the constraints of secrecy outage probability (SOP), connection outage probability (COP), minimum secure data, information causality, and UAV trajectory. Numerical results demonstrate the superiority of the algorithm we proposed in energy consumption and secrecy rate compared with other schemes.

Learning correlation information for multi-label feature selection

In many real-world multi-label applications, the content of multi-label data is usually characterized by high dimensional features, which contains complex correlation information, i.e., label correlations and redundant features. To alleviate the problem, we present a novel scheme, called learning correlation information for multi-label feature selection (LCIFS) method, by jointly digging up label correlations and controlling feature redundancy. To be specific, the regression model via manifold framework is presented to fit the relationship between feature space and label distribution, during which adaptive spectral graph is leveraged to learn more precise structural correlations of labels simultaneously. Besides, we utilize the relevance of features to constrain the redundancy of the generated feature subset, and a general ℓ2,p-norm regularized model is employed to fulfill more robust feature selection. The proposed method is transformed into an explicit optimization function, which is conquered by an efficient iterative optimization algorithm. Finally, we conduct comprehensive experiments on twelve realistic multi-label datasets, including text domain, image domain, and audio domain. The statistic results demonstrate the effectiveness and superiority of the proposed method among nine competition methods.

Graph structure learning layer and its graph convolution clustering application

To learn the embedding representation of graph structure data corrupted by noise and outliers, existing graph structure learning networks usually follow the two-step paradigm, i.e., constructing a “good” graph structure and achieving the message passing for signals supported on the learned graph. However, the data corrupted by noise may make the learned graph structure unreliable. In this paper, we propose an adaptive graph convolutional clustering network that alternatively adjusts the graph structure and node representation layer-by-layer with back-propagation. Specifically, we design a Graph Structure Learning layer before each Graph Convolutional layer to learn the sparse graph structure from the node representations, where the graph structure is implicitly determined by the solution to the optimal self-expression problem. This is one of the first works that uses an optimization process as a Graph Network layer, which is obviously different from the function operation in traditional deep learning layers. An efficient iterative optimization algorithm is given to solve the optimal self-expression problem in the Graph Structure Learning layer. Experimental results show that the proposed method can effectively defend the negative effects of inaccurate graph structures. The code is available at https://github.com/HeXiax/SSGNN.

Unsupervised Feature Selection via Graph Regularized Nonnegative CP Decomposition.

Unsupervised feature selection has attracted remarkable attention recently. With the development of data acquisition technology, multi-dimensional tensor data has been appeared in enormous real-world applications. However, most existing unsupervised feature selection methods are non-tensor-based which results the vectorization of tensor data as a preprocessing step. This seemingly ordinary operation has led to an unnecessary loss of the multi-dimensional structural information and eventually restricted the quality of the selected features. To overcome the limitation, in this paper, we propose a novel unsupervised feature selection model: Nonnegative tensor CP (CANDECOMP/PARAFAC) decomposition based unsupervised feature selection, CPUFS for short. In specific, we devise new tensor-oriented linear classifier and feature selection matrix for CPUFS. In addition, CPUFS simultaneously conducts graph regularized nonnegative CP decomposition and newly-designed tensor-oriented pseudo label regression and feature selection to fully preserve the multi-dimensional data structure. To solve the CPUFS model, we propose an efficient iterative optimization algorithm with theoretically guaranteed convergence, whose computational complexity scales linearly in the number of features. A variation of the CPUFS model by incorporating nonnegativity into the linear classifier, namely CPUFSnn, is also proposed and studied. Experimental results on ten real-world benchmark datasets demonstrate the effectiveness of both CPUFS and CPUFSnn over the state-of-the-arts.

An exact algorithm for the static pricing problem under discrete mixed logit demand

Price differentiation is a common strategy in many markets. In this paper, we study a static multiproduct price optimization problem with demand given by a discrete mixed multinomial logit model. By considering a mixed logit model that includes customer specific variables and parameters in the utility specification, our pricing problem reflects well the discrete choice models used in practice. To solve this pricing problem, we design an efficient iterative optimization algorithm that asymptotically converges to the optimal solution. To this end, a linear optimization (LO) problem is formulated, based on the trust-region approach, to find a “good” feasible solution and approximate the problem from below. A convex optimization problem is designed using a convexification technique to approximate the optimization problem from above. Then, using a branching method, we tighten the optimality gap. The effectiveness of our algorithm is illustrated on several cases, and compared against solvers and existing state-of-the-art methods in the literature.

Max–Min Robust Principal Component Analysis

Principal Component Analysis (PCA) is a powerful unsupervised dimensionality reduction algorithm, which uses squared ℓ2-norm to cleverly connect reconstruction error and projection variance, and those improved PCA methods only consider one of them, which limits their performance. To alleviate this problem, we propose a novel Max–Min Robust Principal Component Analysis via binary weight, which ingeniously combines reconstruction error and projection variance to learn projection matrix more accurately, and uses ℓ2-norm as evaluation criterion to make the model rotation invariant. In addition, we design binary weight to remove outliers to improve the robustness of model and obtain the ability of anomaly detection. Subsequently, we exploit an efficient iterative optimization algorithm to solve this problem. Extensive experimental results show that our model outperforms related state-of-the-art PCA methods.

Self-paced non-convex regularized analysis–synthesis dictionary learning for unsupervised feature selection

Due to the ability to prevent over-fitting, reduce computational complexity and storage cost, and enhance interpretability, unsupervised feature selection (UFS) has received widespread attention in a large number of application scenarios. However, a majority of the existing UFS methods cannot make full use of the diversity between data points, which usually leads to a suboptimal solution. In this paper, a novel UFS approach, referred to as self-paced analysis–synthesis dictionary learning (SPASDL) is proposed by integrating dictionary learning and self-paced learning into a joint framework. Specifically, we reconstruct and code data with the synthesis dictionary and analysis dictionary, respectively. This strategy can avoid using pseudo labels of data, which may mislead the feature selection process. Further, we introduce self-paced learning into dictionary learning to realize the reconstruction of data points from easy to hard in the training process. To identify more discriminative features, we impose the non-convex sparse constraint on the synthesis dictionary. For the purpose of optimizing the resulting model, we exploit an efficient iterative optimization algorithm based on alternative search strategy (ASS). Besides, the theoretical results on the optimization algorithm, including the convergence analysis and computational complexity, are investigated. Finally, experimental results on public available real-world datasets demonstrate the effectiveness and the superiority of the proposed method.

Towards Robust Discriminative Projections Learning via Non-Greedy l2,1-Norm MinMax.

Linear Discriminant Analysis (LDA) is one of the most successful supervised dimensionality reduction methods and has been widely used in many real-world applications. However, l2-norm is employed as the distance metric in the objective of LDA, which is sensitive to outliers. Many previous works improve the robustness of LDA by using l1-norm distance. However, the robustness against outliers is limited and the solver of l1-norm is mostly based on the greedy search strategy, which is time-consuming and easy to get stuck in a local optimum. In this paper, we propose a novel robust LDA measured by l2,1-norm to learn robust discriminative projections. The proposed model is challenging to solve since it needs to minimize and maximize (minmax) l2,1-norm terms simultaneously. As a result, we first systematically derive an efficient iterative optimization algorithm to solve a general ratio minimization problem, and then rigorously prove its convergence. More importantly, an alternately non-greedy iterative re-weighted optimization algorithm is developed based on the preceding approach for solving proposed l2,1-norm minmax problem. Besides, an optimal weighted mean mechanism is driven according to the designed objective and solver, which can be applied to other approaches for robustness improvement. Experimental results on several real-world datasets show the effectiveness of proposed method.

L2,1-norm regularized multivariate regression model with applications to genomic prediction.

MotivationGenomic selection (GS) is currently deemed the most effective approach to speed up breeding of agricultural varieties. It has been recognized that consideration of multiple traits in GS can improve accuracy of prediction for traits of low heritability. However, since GS forgoes statistical testing with the idea of improving predictions, it does not facilitate mechanistic understanding of the contribution of particular single nucleotide polymorphisms (SNP).ResultsHere, we propose a -norm regularized multivariate regression model and devise a fast and efficient iterative optimization algorithm, called -joint, applicable in multi-trait GS. The usage of the -norm facilitates variable selection in a penalized multivariate regression that considers the relation between individuals, when the number of SNPs is much larger than the number of individuals. The capacity for variable selection allows us to define master regulators that can be used in a multi-trait GS setting to dissect the genetic architecture of the analyzed traits. Our comparative analyses demonstrate that the proposed model is a favorable candidate compared to existing state-of-the-art approaches. Prediction and variable selection with datasets from Brassica napus, wheat and Arabidopsis thaliana diversity panels are conducted to further showcase the performance of the proposed model.Availability and implementation: The model is implemented using R programming language and the code is freely available from https://github.com/alainmbebi/L21-norm-GS.Supplementary informationSupplementary data are available at Bioinformatics online.

Internet financing credit risk evaluation using multiple structural interacting elastic net feature selection

Internet financing is an important alternative to banks where individuals or SMEs borrow money using online trading platforms. A central problem for internet financing is how to identify the most influential factors that are closely related to the credit risks. This problem is inherently challenging because the raw data of internet financing is often associated with complex structural correlations and usually contains many irrelevant and redundant features. To effectively identify the most salient features for credit risk evaluation in internet financing, we develop a new multiple structural interacting elastic net model for feature selection (MSIEN). Our idea is based on converting the original vectorial features into structure-based feature graph representations to encapsulate structural relationship between pairwise samples, and defining two new information theoretic criteria. One criterion maximizes joint relevance of different pairwise feature combinations in relation to the target feature graph and the other minimizes the redundancy between pairwise features. Then two structural interaction matrices are obtained with the elements representing the proposed information theoretic measures. To identify the most informative features, we formulate a new optimization model which combines the interaction matrices and an elastic net regularization model for the feature subset selection problem. We exploit an efficient iterative optimization algorithm to solve the proposed problem and also provide the theoretical analyses on its convergence property and computational complexity. Finally, experimental results on datasets of internet financing demonstrate the effectiveness of the proposed MSIEN method.

Subspace Sparse Discriminative Feature Selection.

In this article, we propose a novel feature selection approach via explicitly addressing the long-standing subspace sparsity issue. Leveraging l2,1 -norm regularization for feature selection is the major strategy in existing methods, which, however, confronts sparsity limitation and parameter-tuning trouble. To circumvent this problem, employing the l2,0 -norm constraint to improve the sparsity of the model has gained more attention recently whereas, optimizing the subspace sparsity constraint is still an unsolved problem, which only can acquire an approximate solution and without convergence proof. To address the above challenges, we innovatively propose a novel subspace sparsity discriminative feature selection (S2DFS) method which leverages a subspace sparsity constraint to avoid tuning parameters. In addition, the trace ratio formulated objective function extremely ensures the discriminability of selected features. Most important, an efficient iterative optimization algorithm is presented to explicitly solve the proposed problem with a closed-form solution and strict convergence proof. To the best of our knowledge, such an optimization algorithm of solving the subspace sparsity issue is first proposed in this article, and a general formulation of the optimization algorithm is provided for improving the extensibility and portability of our method. Extensive experiments conducted on several high-dimensional text and image datasets demonstrate that the proposed method outperforms related state-of-the-art methods in pattern classification and image retrieval tasks.

Local structured feature learning with dynamic maximum entropy graph

In recent years, Linear Discriminant Analysis (LDA) has seen huge adoption in data mining applications. Due to its globality, it is incompetent to handle multimodal data. Besides, most of LDA’s variants learn the projection matrix based on the pre-defined similarity matrix, which is easily affected by noisy and irrelevant features. To address above two issues, a novel local structured feature learning with Dynamic Maximum Entropy Graph (DMEG) method is developed which firstly develops a more discriminative LDA with whitening constraint that can minimize the within-class scatter while keeping the total samples scatter unchanged simultaneously. Second, for exploring the local structure of data, the ℓ0-norm constraint is imposed on similarity matrix to ensure the k connectivity on graph. More importantly, proposed model learns the similarity and projection matrix simultaneously to ensure that the neighborships can be found in the optimal subspace where the noise have been removed already. Moreover, a maximum entropy regularization is employed to reinforce the discriminability of graph and avoid the trivial solution. Last but not least, an efficient iterative optimization algorithm is provided to optimize proposed model with a NP-hard constraint. Extensive experiments conducted on synthetic and several real-world data sets demonstrate the efficiency in classification task and robustness to noise of proposed method.

Simultaneous Wireless Information and Power Transfer for OFDM-based Cooperative Communication

In cooperative communication, relay reduces the stored energy for its own transmission due to the energy consumption for forwarding the information from source node. In this paper, simultaneous wireless information and power transfer (SWIPT) for OFDM-based cooperative communication is proposed to save the relay’s energy by harvesting the radio frequency (RF) energy from source node. Subcarrier allocation based SWIPT (SA-SWIPT) is designed to use different subcarrier sets for information decoding and energy harvesting. In the first phase, the relay uses the SA-SWIPT to receive information and energy from source node. And in the second phase, the relay forwards the information both from source node and itself to the destinations using the energy collected from source node. The transmission rates of the relay in DF and AF modes are maximized by jointly optimizing subcarrier allocations and subcarrier powers, respectively, while ensuring the required transmission rate of source node and the energy supply of relay. An efficient iterative optimization algorithm is proposed to solve the proposed optimization problem through Lagrangian dual decomposition. Simulation results show the convergence of the optimization algorithm after very few iterations and explains the potential benefits for using SA-SWIPT in the OFDM-based cooperative communication.

Joint local structure preservation and redundancy minimization for unsupervised feature selection

Unsupervised feature selection is an indispensable pre-processing step in many data mining and pattern recognition tasks where the unlabeled high dimensional data are ubiquitous. Most of existing methods fail to explore the local geometric structure consistency (preservation) of the input data and minimize redundancy of selected features simultaneously. In this paper we propose a novel unsupervised feature selection method which jointly integrates the local geometric structure consistency and redundancy minimization (JLSPRM) into an unified framework. JLSPRM utilizes nonnegative spectral analysis to learn the cluster labels of the input data, then the local geometric structure consistency is developed to make the learned cluster labels more accurate, during which the feature selection operation is performed. To minimize the redundancy rate among selected features, the maximal information coefficient (MIC) is utilized to evaluate the correlation of the pairwise features. Besides, the l2,1-norm is exerted on feature selection matrix which makes the framework decent for selecting features. An efficient iterative optimization algorithm is designed to obtain the solution of the unsupervised feature selection model. The superiority and effectiveness of our proposed approach over the state-of-the-art feature selection methods have also been validated through the extensive experiments on nine benchmark datasets.

Robust cognitive transmit waveform and receive filter design for airborne MIMO radar in signal-dependent clutter environment

This paper investigates the robust cognitive joint design of transmit waveform and receive filter to improve the system performance of airborne multiple-input multiple-output (MIMO) radar. Considering the target Doppler frequency and spatial cone angle uncertainties are present, we formulate the averaged signal-to-clutter-plus-noise ratio (SCNR) as the optimization goal. Specifically, the averaged SCNR is maximized under only constant modulus constraint and that under constant modulus and similarity constraints. Four iterative optimization algorithms are developed to deal with the joint design problem. The first kind of iterative optimization algorithm is based on semidefinite programming (SDP) relaxation and randomization or rank-one decomposition techniques. The second kind of computational efficient iterative optimization algorithm utilizes the fractional programming and power method-like iteration. The proposed algorithms can achieve a monotonic output SCNR enhancement and are robust against the inaccuracies of target parameters. Several simulations results are implemented to validate the superiority of the proposed algorithms considering the output SCNR, space-time beampattern and computational complexity.

Adaptive Local Linear Discriminant Analysis

Dimensionality reduction plays a significant role in high-dimensional data processing, and Linear Discriminant Analysis (LDA) is a widely used supervised dimensionality reduction approach. However, a major drawback of LDA is that it is incapable of extracting the local structure information, which is crucial for handling multimodal data. In this article, we propose a novel supervised dimensionality reduction method named Adaptive Local Linear Discriminant Analysis (ALLDA), which adaptively learns a k -nearest neighbors graph from data themselves to extract the local connectivity of data. Furthermore, the original high-dimensional data usually contains noisy and redundant features, which has a negative impact on the evaluation of neighborships and degrades the subsequent classification performance. To address this issue, our method learns the similarity matrix and updates the subspace simultaneously so that the neighborships can be evaluated in the optimal subspaces where the noises have been removed. Through the optimal graph embedding, the underlying sub-manifolds of data in intra-class can be extracted precisely. Meanwhile, an efficient iterative optimization algorithm is proposed to solve the minimization problem. Promising experimental results on synthetic and real-world datasets are provided to evaluate the effectiveness of proposed method.