Efficient Iterative Algorithm Research Articles

Robust Spherical Laplacian Embedding.

Laplacian embedding (LE) aims to project high-dimensional input data samples, which often contain nonlinear structures, into a low-dimensional space. However, existing distance functions used in the embedding space fail to provide discriminative representations for real-world datasets, especially those related to text analysis or image processing. Cosine similarity measurements are advantageous in dealing with sparse data but are fragile to the impact of outliers and noise from samples or data features. In this work, we propose robust spherical LE (RS-LE), a powerful method that builds the LE on a novel metric that unifies the Euclidean distance and cosine similarity in a spherical space using a robust lp,p -norm. We introduce an efficient iterative algorithm, the proximal alternating linearized minimization (ALM) method, to solve the difficult optimization problem that arises from the new metric, which is neither convex nor smooth. This algorithm allows the solution to achieve both global and objective convergence. Our findings are presented in rigorous theoretical analysis and validated in experiments.

On Mean-Optimal Robust Linear Discriminant Analysis

Linear discriminant analysis (LDA) is widely used for dimensionality reduction under supervised learning settings. Traditional LDA objective aims to minimize the ratio of the squared Euclidean distances that may not perform optimally on noisy datasets. Multiple robust LDA objectives have been proposed to address this problem, but their implementations have two major limitations. One is that their mean calculations use the squared \(\ell_{2}\) -norm distance to center the data, which is not valid when the objective depends on other distance functions. The second problem is that there is no generalized optimization algorithm to solve different robust LDA objectives. In addition, most existing algorithms can only guarantee the solution to be locally optimal, rather than globally optimal. In this paper, we review multiple robust loss functions and propose a new and generalized robust objective for LDA. Besides, to better remove the mean value within data, our objective uses an optimal way to center the data through learning. As one important algorithmic contribution, we derive an efficient iterative algorithm to optimize the resulting non-smooth and non-convex objective function. We theoretically prove that our solution algorithm guarantees that both the objective and the solution sequences converge to globally optimal solutions at a sub-linear convergence rate. The results of comprehensive experimental evaluations demonstrate the effectiveness of our new method, achieving significant improvements compared to the other competing methods.

A sum‐rate maximization for IRS‐aided broadcast SWIPT with time‐switching structure and nonlinear energy harvester

AbstractThis study evaluates the sum‐rate gain of the intelligent reflecting surface (IRS) in a broadcast simultaneous wireless information and power transfer network. Users are equipped with a time‐switching (TS) structure for an alternating information decoder and a nonlinear energy harvester. The sum‐rate maximization (SRM) problem is formulated subject to the constraints of the energy‐harvesting threshold required by users and limited transmission power at the base station (BS). To address the SRM problem, we jointly designed a broadcast beamforming precoding vector at the BS, reflective phase shifters at the IRS, and TS factors at the users. Subsequently, after certain manipulations, we propose efficient iterative algorithms that combine alternating optimization, successive convex approximation, semidefinite relaxation, and rank‐one penalty function methods to obtain a suboptimal solution. Finally, the numerical results show the promising sum‐rate gain of the proposed scheme compared with the conventional schemes.

Dual-learning Multi-hop Nonnegative Matrix Factorization for community detection

As an important branch of network science, community detection has garnered significant attention. Among various community detection methods, nonnegative matrix factorization (NMF)-based community detection approaches have become a popular research topic. However, most NMF-based methods overlook the network’s multi-hop information, let alone the community detection results specific to each hop of the network. In this paper, we propose Dual-learning Multi-hop NMF (DL-MHNMF), a method that considers not only the multi-hop connectivity between two nodes but also factors in the shared results across multiple hops and the impact of differences in the specific results at each hop on the shared outcomes. An efficient iterative optimization algorithm with guaranteed theoretical convergence is proposed for solving DL-MHNMF. Methodologically, by iteratively removing the specific results during the optimization process of DL-MHNMF, we achieve enhanced detection accuracy, which is also verified by subsequent experiments. Specifically, we compare fourteen algorithms on eleven publicly available datasets, and experimental results show that our algorithm outperforms most state-of-the-art methods. The source code is availiable at https://github.com/bx20000827/DL-MHNMF.git.

Comprehensive Multiview Representation Learning via Deep Autoencoder-Like Nonnegative Matrix Factorization.

Learning a comprehensive representation from multiview data is crucial in many real-world applications. Multiview representation learning (MRL) based on nonnegative matrix factorization (NMF) has been widely adopted by projecting high-dimensional space into a lower order dimensional space with great interpretability. However, most prior NMF-based MRL techniques are shallow models that ignore hierarchical information. Although deep matrix factorization (DMF)-based methods have been proposed recently, most of them only focus on the consistency of multiple views and have cumbersome clustering steps. To address the above issues, in this article, we propose a novel model termed deep autoencoder-like NMF for MRL (DANMF-MRL), which obtains the representation matrix through the deep encoding stage and decodes it back to the original data. In this way, through a DANMF-based framework, we can simultaneously consider the multiview consistency and complementarity, allowing for a more comprehensive representation. We further propose a one-step DANMF-MRL, which learns the latent representation and final clustering labels matrix in a unified framework. In this approach, the two steps can negotiate with each other to fully exploit the latent clustering structure, avoid previous tedious clustering steps, and achieve optimal clustering performance. Furthermore, two efficient iterative optimization algorithms are developed to solve the proposed models both with theoretical convergence analysis. Extensive experiments on five benchmark datasets demonstrate the superiority of our approaches against other state-of-the-art MRL methods.

Energy Efficient and Differentially Private Federated Learning via a Piggyback Approach

This paper aims to develop a differential private federated learning (FL) scheme with the least artificial noises added while minimizing the energy consumption of participating mobile devices. By observing that some communication efficient FL approaches and even the nature of wireless communications contribute to the differential privacy (DP) preservation of training data on mobile devices, in this paper, we propose to jointly leverage gradient compression techniques (i.e., gradient quantization and sparsification) and additive white Gaussian noises (AWGN) in wireless channels to develop a piggyback DP approach for FL over mobile devices. Even with the piggyback DP approach, information distortion caused by gradient compression and noise perturbation may slow down FL convergence, which in turn consumes more energy of mobile devices for local computing and model update communications. Thus, we theoretically analyze FL convergence and formulate an energy efficient FL optimization under piggyback DP, transmission power, and FL convergence constraints. Furthermore, we propose an efficient iterative algorithm where closed-form solutions for artificial DP noise and power control are derived. Extensive simulation and experimental results demonstrate the effectiveness of the proposed scheme in terms of energy efficiency and privacy preservation.

Penalized weighted smoothed quantile regression for high-dimensional longitudinal data.

Quantile regression, known as a robust alternative to linear regression, has been widely used in statistical modeling and inference. In this paper, we propose a penalized weighted convolution-type smoothed method for variable selection and robust parameter estimation of the quantile regression with high dimensional longitudinal data. The proposed method utilizes a twice-differentiable and smoothed loss function instead of the check function in quantile regression without penalty, and can select the important covariates consistently using the efficient gradient-based iterative algorithms when the dimension of covariates is larger than the sample size. Moreover, the proposed method can circumvent the influence of outliers in the response variable and/or the covariates. To incorporate the correlation within each subject and enhance the accuracy of the parameter estimation, a two-step weighted estimation method is also established. Furthermore, we prove the oracle properties of the proposed method under some regularity conditions. Finally, the performance of the proposed method is demonstrated by simulation studies and two real examples.

Robust and Sparse Principal Component Analysis With Adaptive Loss Minimization for Feature Selection.

Principal component analysis (PCA) is one of the most successful unsupervised subspace learning methods and has been used in many practical applications. To deal with the outliers in real-world data, robust principal analysis models based on various measure are proposed. However, conventional PCA models can only transform features to unknown subspace for dimensionality reduction and cannot perform features' selection task. In this article, we propose a novel robust PCA (RPCA) model to mitigate the impact of outliers and conduct feature selection, simultaneously. First, we adopt σ -norm as reconstruction error (RE), which plays an important role in robust reconstruction. Second, to conduct feature selection task, we apply l2,0 -norm constraint to subspace projection. Furthermore, an efficient iterative optimization algorithm is proposed to solve the objective function with nonconvex and nonsmooth constraint. Extensive experiments conducted on several real-world datasets demonstrate the effectiveness and superiority of the proposed feature selection model.

A fast closed-form approximate iterative fitting algorithm based on laser absorption spectrum

This paper presents a novel approach—an efficient closed-form approximation iterative fitting algorithm based on laser absorption spectra. Through this closed-form approximation iterative fitting, key parameters such as peak value, spectral line width, and normalized signal area serve as indicators for iteration completion, improving the speed without compromising accuracy. Furthermore, it employs the spectral signal of n cycles as a window for further processing, minimizing external interference. The results show that the proposed method averages 9.75 iterations, while the Levenberg–Marquardt fitting method averages 60.17 iterations. The average iteration time for the proposed method is 588.83 ms, a substantial 81.7% reduction compared to the 3210.5 ms required by the Levenberg–Marquardt fitting. These results decisively demonstrate the efficacy of the proposed method in reducing iteration time and enhancing measurement precision.

Asymmetric Supervised Fusion-Oriented Hashing for Cross-Modal Retrieval.

Hashing technologies have been widely applied for large-scale multimodal retrieval tasks owing to their excellent performance in search and storage tasks. Although some effective hashing methods have been proposed, it is still difficult to handle the intrinsic linkages that exist among different heterogeneous modalities. Moreover, optimizing the discrete constraint problem through a relaxation-based strategy results in a large quantization error and leads to a suboptimal solution. In this article, we present a novel asymmetric supervised fusion-oriented hashing method, named (ASFOH), which investigates three novel schemes to remedy the above issues. Specifically, we first explicitly formulate the problem as matrix decomposition into a common latent representation and a transformation matrix, combined with an adaptive weight scheme and nuclear norm minimization to ensure the information completeness of multimodal data. Then, we associate the common latent representation with the semantic label matrix, thereby increasing the discriminative capability of the model by constructing an asymmetric hash learning framework, thus, making the generated hash codes more compact. Finally, an efficient discrete optimization iterative algorithm based on nuclear norm minimization is proposed to decompose the nonconvex multivariate optimization problem into several subproblems with analytical solutions. Comprehensive experiments on the MIRFlirck, NUS-WIDE, and IARP-TC12 datasets testify that ASFOH outperforms the compared state-of-the-art approaches.

T[formula omitted]LR-Net: An unrolling network learning transformed tensor low-rank prior for dynamic MR image reconstruction

The tensor low-rank prior has attracted considerable attention in dynamic MR reconstruction. Tensor low-rank methods preserve the inherent high-dimensional structure of data, allowing for improved extraction and utilization of intrinsic low-rank characteristics. However, most current methods are still confined to utilizing low-rank structures either in the image domain or predefined transformed domains. Designing an optimal transformation adaptable to dynamic MRI reconstruction through manual efforts is inherently challenging. In this paper, we propose a deep unrolling network that utilizes the convolutional neural network (CNN) to adaptively learn the transformed domain for leveraging tensor low-rank priors. Under the supervised mechanism, the learning of the tensor low-rank domain is directly guided by the reconstruction accuracy. Specifically, we generalize the traditional t-SVD to a transformed version based on arbitrary high-dimensional unitary transformations and introduce a novel unitary transformed tensor nuclear norm (UTNN). Subsequently, we present a dynamic MRI reconstruction model based on UTNN and devise an efficient iterative optimization algorithm using ADMM, which is finally unfolded into the proposed T2LR-Net. Experiments on two dynamic cardiac MRI datasets demonstrate that T2LR-Net outperforms the state-of-the-art optimization-based and unrolling network-based methods.

Bosonic excitation spectra of superconducting Bi2Sr2CaCu2O8+δ and YBa2Cu3O6+x extracted from scanning tunneling spectra

A detailed interpretation of scanning tunneling spectra obtained on unconventional superconductors enables one to gain information on the pairing boson. Decisive for this approach are inelastic tunneling events. Due to the lack of momentum conservation in tunneling from or to the sharp tip, those are enhanced in the geometry of a scanning tunneling microscope compared to planar tunnel junctions. This work extends the method of obtaining the bosonic excitation spectrum by deconvolution from tunneling spectra to nodal d-wave superconductors. In particular, scanning tunneling spectra of slightly underdoped Bi2Sr2CaCu2O8+δ with a Tc of 82 K and optimally doped YBa2Cu3O6+x with a Tc of 92 K reveal a resonance mode in their bosonic excitation spectrum at Ωres≈63 meV and Ωres≈61 meV respectively. In both cases, the overall shape of the bosonic excitation spectrum is indicative of predominant spin scattering with a resonant mode at Ωres<2Δ and overdamped spin fluctuations for energies larger than 2Δ. To perform the deconvolution of the experimental data, we implemented an efficient iterative algorithm that significantly enhances the reliability of our analysis.

A Comparative Study of Optimal PV Allocation in a Distribution Network Using Evolutionary Algorithms

The growing distributed energy resource (DER) penetration into distribution networks, such as through residential and commercial photovoltaics (PV), has emerged through a transition from passive to active networks, which takes the complexity of planning and operations to the next level. Optimal PV allocation (sizing and location) is challenging because it involves mixed-integer non-linear programming with three-phase non-linear unbalanced power flow equations. Meta-heuristic algorithms have proven their effectiveness in many complex engineering problems. Thus, in this study, we propose to achieve optimal PV allocation by using several basic evolutionary algorithms (EAs), particle swarm optimization (PSO), artificial bee colony (ABC), differential evolution (DE), and their variants, all of which are applied for a study of their performance levels. Two modified unbalanced IEEE test feeders (13 and 37 bus) are developed to evaluate these performance levels, with two objectives: one is to maximize PV penetration, and the other is to minimize the voltage deviation from 1.0 p.u. To handle the computational burden of the sequential power flow and unbalanced network, we adopt an efficient iterative load flow algorithm instead of the commonly used and yet highly simplified forward–backward sweep method. A comparative study of these basic EAs shows their general success in finding a near-optimal solution, except in the case of the DE, which is known for solving continuous optimization problems efficiently. From experiments run 30 times, it is observed that PSO-related algorithms are more efficient and robust in the maximum PV penetration case, while ABC-related algorithms are more efficient and robust in the minimum voltage deviation case.

Kernel-based diffusion approximated Markov decision processes for autonomous navigation and control on unstructured terrains

We propose a diffusion approximation method to the continuous-state Markov decision processes that can be utilized to address autonomous navigation and control in unstructured off-road environments. In contrast to most decision-theoretic planning frameworks that assume fully known state transition models, we design a method that eliminates such a strong assumption that is often extremely difficult to engineer in reality. We first take the second-order Taylor expansion of the value function. The Bellman optimality equation is then approximated by a partial differential equation, which only relies on the first and second moments of the transition model. By combining the kernel representation of the value function, we design an efficient policy iteration algorithm whose policy evaluation step can be represented as a linear system of equations characterized by a finite set of supporting states. We first validate the proposed method through extensive simulations in 2 D obstacle avoidance and 2.5 D terrain navigation problems. The results show that the proposed approach leads to a much superior performance over several baselines. We then develop a system that integrates our decision-making framework with onboard perception and conduct real-world experiments in both cluttered indoor and unstructured outdoor environments. The results from the physical systems further demonstrate the applicability of our method in challenging real-world environments.

Shared Manifold Regularized Joint Feature Selection for Joint Classification and Regression in Alzheimer's Disease Diagnosis.

In Alzheimer's disease (AD) diagnosis, joint feature selection for predicting disease labels (classification) and estimating cognitive scores (regression) with neuroimaging data has received increasing attention. In this paper, we propose a model named Shared Manifold regularized Joint Feature Selection (SMJFS) that performs classification and regression in a unified framework for AD diagnosis. For classification, unlike the existing works that build least squares regression models which are insufficient in the ability of extracting discriminative information for classification, we design an objective function that integrates linear discriminant analysis and subspace sparsity regularization for acquiring an informative feature subset. Furthermore, the local data relationships are learned according to the samples' transformed distances to exploit the local data structure adaptively. For regression, in contrast to previous works that overlook the correlations among cognitive scores, we learn a latent score space to capture the correlations and employ the latent space to design a regression model with l2,1 -norm regularization, facilitating the feature selection in regression task. Moreover, the missing cognitive scores can be recovered in the latent space for increasing the number of available training samples. Meanwhile, to capture the correlations between the two tasks and describe the local relationships between samples, we construct an adaptive shared graph to guide the subspace learning in classification and the latent cognitive score learning in regression simultaneously. An efficient iterative optimization algorithm is proposed to solve the optimization problem. Extensive experiments on three datasets validate the discriminability of the features selected by SMJFS.

Joint Optimization of Mobility and Reliability-Guaranteed Air-to-Ground Communication for UAVs

Aerial unmanned vehicles (UAVs) play a significant role in improving the connectivity and coverage of terrestrial communication networks. However, UAV-assisted air-to-ground (A2G) data transmissions usually encounter several fundamental challenges, such as terminal mobility, random nature in channel fading and contention, resource constraints, and application-specific transmission requirements. To tackle these challenges, we formulate a bi-level optimization problem that jointly considers the control of the UAV mobility and transmission power and the scheduling of A2G data transmissions. The objective is to optimize energy consumption and maximize A2G transmission reliability. Particularly, we first theoretically characterize the A2G transmission reliability from a probabilistic perspective concerning the effects of channel fading, channel access contention, and application requirements. We then derive a closed-form expression for the optimal expected transmission reliability. Using the closed-form reliability, we transform the bi-level optimization into a mathematically-tractable optimal control problem and propose an efficient iterative algorithm to solve it. Simulation results show that our approach provides a comprehensive improvement in terms of both energy utilization and A2G transmission reliability, in particular, with a reduction of more than 12.1% in energy consumption and an increase of 7.53% in reliability on average, compared to several baselines.

Multiview Spectral Clustering Based on Consensus Neighbor Strategy.

Multiview spectral clustering, renowned for its spatial learning capability, has garnered significant attention in the data mining field. However, existing methods assume that the optimal consensus adjacency matrix is confined within the space spanned by each view's adjacency matrix. This constraint restricts the feasible domain of the algorithm and hinders the exploration of the optimal consensus adjacency matrix. To address this limitation, we propose a novel and convex strategy, termed the consensus neighbor strategy, for learning the optimal consensus adjacency matrix. This approach constructs the optimal consensus adjacency matrix by capturing the consensus local structure of each sample across all views, thereby expanding the search space and facilitating the discovery of the optimal consensus adjacency matrix. Furthermore, we introduce the concept of a correlation measuring matrix to prevent trivial solution. We develop an efficient iterative algorithm to solve the resulting optimization problem, benefitting from the convex nature of our model, which ensures convergence to a global optimum. Experimental results on 16 multiview datasets demonstrate that our proposed algorithm surpasses state-of-the-art methods in terms of its robust consensus representation learning capability. The code of this article is uploaded to https://github.com/PhdJiayiTang/Consensus-Neighbor-Strategy.git.

Group sparse representation and saturation-value total variation based color image denoising under multiplicative noise

&lt;abstract&gt;&lt;p&gt;In this article, we propose a novel group-based sparse representation (GSR) model for restoring color images in the presence of multiplicative noise. This model consists of a convex data-fidelity term, and two regularizations including GSR and saturation-value-based total variation (SVTV). The data-fidelity term is suitable for handling heavy multiplicative noise. GSR enables the retention of textures and details while sufficiently removing noise in smooth regions without producing the staircase artifacts engendered by total variation-based models. Furthermore, we introduce a multi-color channel-based GSR that involves coupling between three color channels. This avoids the generation of color artifacts caused by decoupled color channel-based methods. SVTV further improves the visual quality of restored images by diminishing certain artifacts induced by patch-based methods. To solve the proposed nonconvex model and its subproblem, we exploit the alternating direction method of multipliers, which contributes to an efficient iterative algorithm. Numerical results demonstrate the outstanding performance of the proposed model compared to other existing models regarding visual aspect and image quality evaluation values.&lt;/p&gt;&lt;/abstract&gt;

The algorithm for canonical forms of neural ideals

&lt;abstract&gt;&lt;p&gt;To elucidate the combinatorial architecture of neural codes, the neural ideal $ J_C $, an algebraic object, was introduced. Represented in its canonical form, $ J_C $ provides a succinct characterization of the inherent receptive field architecture within the code. The polynomials in $ J_C $ are also instrumental in determining the relationships among the neurons' receptive fields. Consequently, the computation of the collection of canonical forms is pivotal. In this paper, based on the study of relations between pseudo-monomials, the authors present a computationally efficient iterative algorithm for the canonical forms of the neural ideal. Additionally, we introduce a new relationship among the neurons' receptive fields, which can be characterized by if-and-only-if statements, relating both to $ J_C $ and to a larger ideal of a code $ I(C) $.&lt;/p&gt;&lt;/abstract&gt;

Efficient iterative algorithm for improving frequency sweep linearity of external cavity lasers

Efficient iterative algorithm for improving frequency sweep linearity of external cavity lasers