Alternating Optimization Procedure Research Articles

Blind Super-Resolution via Meta-Learning and Markov Chain Monte Carlo Simulation.

Learning based approaches have witnessed great successes in blind single image super-resolution (SISR) tasks, however, handcrafted kernel priors and learning based kernel priors are typically required. In this paper, we propose a meta-learning and Markov Chain Monte Carlo (MCMC) based SISR approach to learn kernel priors from organized randomness. In concrete, a lightweight network is adopted as kernel generator, and is optimized via learning from the MCMC simulation on random Gaussian distributions. This procedure provides an approximation for the rational blur kernel, and introduces a network-level Langevin dynamics into SISR optimization processes, which contributes to preventing bad local optimal solutions for kernel estimation. Meanwhile, a meta-learning based alternating optimization procedure is proposed to optimize the kernel generator and image restorer, respectively. In contrast to the conventional alternating minimization strategy, a meta-learning based framework is applied to learn an adaptive optimization strategy, which is less-greedy and results in better convergence performance. These two procedures are iteratively processed in a plug-and-play fashion, for the first time, realizing a learning-based but plug-and-play blind SISR solution in unsupervised inference. Extensive simulations demonstrate the superior performance and generalization ability of the proposed approach when compared with the Start-of-the-Art solutions on synthesis and real-world datasets.

Estimating Asset Pricing Models in the Presence of Cross-Sectionally Correlated Pricing Errors

In this study, we propose an adversarial learning approach to the asset pricing model estimation problem which aims to find estimates of factors and loadings that capture time-series covariations while minimizing the worst-case cross-sectional pricing errors. The proposed estimator is defined by a novel min-max optimization problem in which finding a solution is known to be difficult. This contrasts with other related estimators that admit a well-defined analytic solution but do not effectively account for correlations among the pricing errors. To this end, we propose an approximate algorithm based on the alternating optimization procedure and empirically demonstrate that our proposed adversarial estimation framework outperforms other existing factor models, especially when the explanatory power of the pricing model is limited.

Partial Multi-Label Learning via Exploiting Instance and Label Correlations

The goal of partial multi-label learning is to induce a multi-label classifier from partial multi-label data where each instance is annotated with a number of candidate labels but only a subset of them are valid. Many of the existing studies either fail to fully utilize instance and label correlations to eliminate noisy labels or build an oversimplified multi-label classifier, both of which are unfavorable for the improvement of generalization performance. In this article, we put forward a novel model named Pml-ilc to learn a multi-label classifier from partial multi-label data. Specifically, Pml-ilc first encodes instances and labels into a compact semantic space and takes full advantage of instance and label correlations to eliminate noisy labels. Then, it induces a linear mapping from the feature space to the label space while exploiting label-specific features and instance correlations to facilitate the multi-label classifier learning process. Finally, the above two steps are combined into a joint optimization problem and an efficient alternating optimization procedure is developed to find a satisfactory solution. Extensive experiments show that Pml-ilc achieves superior performance on both real-world and synthetic partial multi-label datasets in terms of different evaluation metrics.

Decentralized distributionally robust chance-constrained operation of integrated electricity and hydrogen transportation networks

Hydrogen energy offers a promising pathway to decarbonizing future energy systems. Renewable energy-based hydrogen production and road transportation-based hydrogen delivery have strengthened the coupling between the power distribution network (PDN) and the hydrogen transportation network (HTN). In this article, we propose a decentralized distributionally robust chance-constrained (DRCC) method for the integrated electricity and hydrogen transportation network (IEHTN) to foster synergies between PDN and HTN. First, an optimal scheduling framework is developed to coordinate hydrogen production and transportation in the IEHTN, where multiple operational states of water electrolyzers and the time-constrained tube trailer routing are modeled. Second, a data-driven DRCC approach is proposed to model the uncertain behaviors of renewable power generation and hydrogen demand. A conditional value-at-risk approximation is then employed to convert the DRCC problem to a tractable mixed-integer linear program that can be solved by the off-the-shelf solver. Third, an alternating direction method of multipliers (ADMM) based solving scheme is designed to solve the problem in a distributed manner with limited information exchange between PDN and HTN. To address the non-convexity brought by binary variables in the ADMM, a tractable alternating optimization procedure (AOP) strategy is utilized to guarantee the convergence of solution. Finally, case studies are conducted on an IEHTN composed of the modified IEEE 33-bus PDN and 52-node HTN to verify the validity of the proposed method.

Distributed Adaptive Robust Restoration Scheme of Cyber-Physical Active Distribution System With Voltage Control

The increasing penetration of distributed generators (DGs) and the advancement of information and communication technologies (ICTs) will facilitate the transformation of the traditional passive distribution network towards a cyber-physical active distribution system (CPADS). With the increasing risks of extreme events, such as natural disasters (e.g., flooding) and cyber-physical attacks, it is critical for CPADS to formulate a restoration scheme to improve its resilience. Therefore, in this paper, a distributed adaptive robust restoration scheme with voltage/var control is presented to cope with the high-impact but low-frequency events. First, a detailed cyber-physical system model is established, including the dynamic routing and the quality-of-services (QoS) in both optical fiber networks and 5G wireless networks. Then, the interactions between the cyber system and the physical system are analyzed. Based on the cyber-physical system model, a two-stage restoration scheme with voltage/var control is proposed by coordinately scheduling different network assets in day-ahead and in real-time. The formulated problem is solved by adaptive robust optimization (ARO). To further enhance the resilience of the CPADS, a distributed restoration framework is proposed. The distributed problem is solved by the alternating direction method of multipliers (ADMM) algorithm, and the convergence of the discrete problem is ensured by introducing the alternating optimization procedure (AOP). Considering the cyber faults, a boundary variable compensation and residual relaxation mechanism is proposed in ADMM. The proposed framework and methodology are verified in the case study. The convergence and the efficiency of the proposed algorithm are verified. Compared with the state-of-art works, the advantages in load restoration capability of the proposed method are shown.

A Customer-Centric Distributed Data-Driven Stochastic Coordination Method for Residential PV and BESS

An aggregation scheme is an effective transactive manner of Distributed Energy Resources (DER) spreading across distribution networks. Distributed approach locally achieves cost minimization of an aggregator and customers. The uncertainties of wholesale market price and rooftop PV output will impact on aggregator's scheduling decision and each customer's cost, while solar energy fluctuation can cause an overvoltage problem in distribution networks. However, the probability distributions of these uncertainties always have errors, even in emerging data-based methods. There is no stochastic method using real data with an out-of-sample guarantee suitable for this distributed approach so far to help an aggregator avoid price risk and manage customers' energy against solar energy fluctuation. To address these unsolved issues, we propose a data-driven Wasserstein distributionally robust formulation of the aggregator's agent and customer's agent respectively. The Wasserstein metric is employed to construct the Wasserstein ambiguity set. The mathematical models are then reformulated equivalently to convex programming respectively so that the operating model can be solved by the off-the-shelf solver. To improve the efficiency of the distributed solving framework, an alternating optimization procedure (AOP) process is proposed to overcome the issue caused by binary variables in the alternating direction method of multipliers (ADMM). The proposed operation framework is verified on the modified IEEE 33-bus distribution network and realistic single-feeder LV network.

Distributed data-driven distributionally robust Volt/Var control for distribution network via an accelerated alternating optimization procedure

This paper proposes a distributed data-driven distributionally robust volt/var control (DDDR-VVC) approach which schedules on-load-tap changer (OLTC), capacitor banks (CBs) and Photovoltaic (PV) inverters coordinately. With integration of distributed optimization and data-driven distributionally robust optimization, the DDDR-VVC model is constructed to reduce power losses and maintain voltage within allowable range under uncertainty. To fully address the uncertainty impacts, construction of uncertain set is improved through that the probability distribution sets and trapezoidal fuzzy functions are developed to model PV supply and load demand respectively. An accelerated alternating optimization procedure which integrates the alternating direction method of multipliers and the column-and-constraint generation algorithm is proposed. This method improves the dual information update via Nesterov’s accelerated gradient method, thus solving the DDDR-VVC model at a high convergence rate. The effectiveness of proposed method is verified through numerical simulations using IEEE 123 bus test system.

Robust Coordinated Optimization With Adaptive Uncertainty Set for a Multi-Energy Microgrid

With the increasing integration of the multi-energy microgrid (MEM) with the distribution network (DN), the distributed coordination between MEM and DN becomes critical. This paper proposes a distributed scheduling method for the coupled MEM-DN under operational uncertainties. First, a worst-expectation min-max-max-min robust optimization model is formulated for MEM considering the uncertainties of renewable distributed generation (wind and photovoltaic) as well as its class probability. Second, the column and constraint generation algorithm with an alternating iteration strategy (C&CG-AIS) is proposed to accelerate the solution by decoupling the subproblems. Third, the multi-interval convex hull uncertainty set (MCHUS) is proposed to reduce the conservatism of robust optimization by decreasing the low-probability scenarios. Furthermore, the Bregman alternating direction method with multipliers (BADMM) is combined with the alternating optimization procedure (AOP) to overcome the convergence difficulty in the nonconvex distributed MEM-DN model. Finally, the effectiveness of the proposed model and method is verified by simulation tests based on IEEE-33 node DN and a park-level MEM.

Multistage Scheduling of Regional Power Grids Against Sequential Outage and Power Uncertainties

Aiming at the scheduling problem for regional power grids under both outage and power uncertainties, this paper puts forward a multistage robust optimization (RO) method that ensures the reliable power supply in system operations. Firstly, a multistage RO model is proposed according to the practical scheduling request of the regional power grid. The startup and shutdown of slow-acting generators are optimized in the first stage. The optimal topology is then determined in the second stage in preparation to the outage uncertainty of grid-connected lines. In the third to 2+ <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${T}$ </tex-math></inline-formula> stages, the scheduling plans of generation units in each time period are further formulated regarding the sequential realizations of renewable-load power uncertainties, which ensures the nonanticipativity of the obtained robust schedules. Secondly, a tailored and efficient transformation-decomposition algorithm is exploited for the solution of multistage RO problem. Based on the affine control function, the nested max-min optimization of the third to 2+ <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${T}$ </tex-math></inline-formula> stages is explicitly transformed to a single-stage max-min model via the multi-to-one strategy. A column and constraint generation (C&CG) algorithm with looped alternative optimization procedure (AOP) handles the equivalent three-stage RO problem with binary recourses, thus avoiding numerous forward and backward iterations as well as speeding up the modeling solution. Numerical experiments of real-world regional power grids verify the effectiveness, superiorities and scalability of the proposed multistage RO scheduling method, which indicates great guidance for engineering applications.

Distributionally robust unit commitment of integrated electricity and heat system under bi-directional variable mass flow

This paper proposes a distributionally robust unit commitment of integrated electricity and heat system, in which the district heating systems are operated under variable flow and variable temperature (VFVT) with bi-directional mass flow considered rather than the commonly used constant flow and variable temperature (CFVT) control method to increase the flexibility capacity for renewable energy accommodation, and the distributionally robust optimization scheme adopted can additionally help increase the operation flexibility of system under high penetration of renewable energy. Piecewise linear method is adopted to convert the mixed integer nonlinear problem into its mixed integer linear counterpart, and tractable alternating optimization procedure is adopted to solve the problem in a decentralized scheme. Case studies conducted on a small scale system and a large scale system respectively demonstrates the effectiveness and the applicability of this method.

Distributed robust cooperative scheduling of multi-region integrated energy system considering dynamic characteristics of networks

Due to the advantages of reliability and economy, multiple adjacent region integrated energy systems (IESs) are interconnected to form a multi-region IES through multi-energy network. However, the limitation on private data exchange, multi-uncertainty and the complex dynamic characteristics bring challenges to the optimal scheduling of the multi-region IES. This paper proposes a distributed robust cooperative scheduling method for the multi-region electricity–natural gas–heat IES considering the dynamic characteristics and the uncertainties of wind turbine (WT) and electricity load. Firstly, a robust scheduling model of the multi-region IES considering dynamic characteristics and the uncertainties is built. In particular, the dynamic transmission models of gas and heating networks are constructed, and a regulation model of virtual heat storage is built to control the virtual heat storage in district heating network (DHN). Further, to preserve the information privacy and decision-making independence for each sub-region IES, a distributed cooperative scheduling framework based on the consensus-based alternating direction method of multipliers (ADMM) is developed for the multi-region IES, where the original robust scheduling model is decomposed into several subproblems that are solved independently. Moreover, the nonconvex dynamic natural gas flow function is addressed by convex relaxation technique, and the max–min robust model with binary variables is solved by alternative optimization procedure (AOP) method and duality theory. Finally, the simulation results show that the proposed method can converge reliably, realize the balance between the operation cost and the heat loss and provide a flexible scheduling scheme for the multi-region IES.

Semi-supervised partial multi-label classification via consistency learning

Partial multi-label learning refers to the problem that each instance is associated with a candidate label set involving both relevant and noisy labels. Existing solutions mainly focus on label disambiguation, while ignoring the negative effect of the inconsistency between feature information and label information. Specifically, the existence of completely unlabeled instances makes the estimation of label co-occurrence difficult. To tackle these problems, we propose a novel framework for partial multi-label learning in semi-supervised scenarios by solving the inconsistency between features and labels. In the first stage, the label-level correlation matrix on both labeled and unlabeled instances is derived via Hilbert-Schmidt Independence Criterion (HSIC). The correlation matrix can characterize the label correlation of labeled instances and can propagate the label correlation of unlabeled instances. In the second stage, the proposed framework achieves the training of feature mapping, the recovery of ground-truth labels, and the alleviation of noisy labels in a mutually beneficial manner, and develops an alternative optimization procedure to optimize them. In addition, a nonlinear version is extended by using kernel trick. Experimental studies demonstrate that the proposed methods can achieve competitive superiority against existing well-established methods.

Decentralized Game-Based Robustly Planning Scheme for Distribution Network and Microgrids Considering Bilateral Energy Trading

In the emerging electricity market, bilateral transactions can be conducted among the distribution network (DN) and microgrids (MGs). It is essential to consider the effects of bilateral energy trading among multiple stakeholders at the planning level. In this paper, a bi-level robustly game-based planning framework is proposed for DN and MGs. To address the multiple non-convexities of traditional robust planning problems, a linear robust counterpart is developed to model the uncertainty of renewable energy sources. Then, the non- continuous robust planning problem is further transformed into an iterative process by alternating optimization procedure (AOP) for the proof of Nash equilibrium (NE) and convergence of distributed algorithms. In each iteration, the upper level utilizes the Nash bargaining solution (NBS) to clear the bilateral transaction electricity and payments in the convex subset generated by the lower level, while the lower level determines the individual planning problem locally with fixed transaction contracts obtained from the upper level. It is proved that the bi-level planning framework benefits all investors and minimizes social costs. Meanwhile, distributed algorithms are adopted to ensure the synchronization and privacy of stakeholders. Simulations on the IEEE 33-bus system demonstrate the effectiveness of the proposed framework.

R-CTSVM+: Robust capped L1-norm twin support vector machine with privileged information

In the new paradigm, learning using privileged information (LUPI) creates a more informative strategy for tasks to achieve better prediction. SVM based methods including SVM+ and TSVM+, have achieved considerable success in LUPI on the clean data. However, these methods typically suffer from the noise and outliers contained in the data, which leads to larger fluctuations in performance. To handle this problem, in this paper, we propose a novel Robust Capped L1-norm Twin Support Vector Machine with Privileged Information (R-CTSVM+). The proposed pair of regularization functions (up- and down-bound) can definitely help to increase the learning model’s tolerance to disturbance, because the up-bound function aims to maximize the lower bound of the perturbation of both main feature and privilege feature, meanwhile, the down-bound function aims to minimize the upper bound of the perturbation of both main feature and privilege feature. Moreover, as the widely employed squared L2-norm distance typically exaggerates the impact of outliers, we adopt the capped L1 regularized distance to further guarantee the robustness of the model. We present that the resulted optimization problem is theoretically converged and can be solved using an effective alternating optimization procedure. Experimental results on benchmark datasets indicate that the proposed robust model can produce superior performance in the case where data samples contain much noise and outliers.

Reconfigurable Intelligent Surface Empowered Downlink Non-Orthogonal Multiple Access

Power-domain non-orthogonal multiple access (NOMA) has become a promising technology to exploit the new dimension of the power domain to enhance the spectral efficiency of wireless networks. However, most existing NOMA schemes rely on the strong assumption that users’ channel gains are quite different, which may be invalid in practice. To unleash the potential of power-domain NOMA, we propose a reconfigurable intelligent surface (RIS)-empowered NOMA scheme to introduce desirable channel gain differences among the users by adjusting the phase shifts at the RIS. Our goal is to minimize the total transmit power by jointly optimizing the beamforming vectors at the base station, the phase-shift matrix at the RIS, and user ordering. To address challenge due to the highly coupled optimization variables, we present an alternating optimization framework to decompose the non-convex bi-quadratically constrained quadratic problem under a specific user ordering into two rank-one constrained matrices optimization problems via matrix lifting. To accurately detect the feasibility of the non-convex rank-one constraints and improve performance by avoiding early stopping in the alternating optimization procedure, we equivalently represent the rank-one constraint as the difference between nuclear norm and spectral norm. A difference-of-convex (DC) algorithm is further developed to solve the resulting DC programs via successive convex relaxation, followed by establishing the convergence of the proposed DC-based alternating optimization method. We further propose an efficient user ordering scheme with closed-form expressions, considering both the channel conditions and users’ target data rates. Simulation results validate the ability of an RIS in enlarging the channel-gain difference when the users’ original channel conditions are similar and the superiority of the proposed DC-based alternating optimization method in reducing the total transmit power.

Enhanced knowledge transfer for collaborative filtering with multi-source heterogeneous feedbacks

Collaborative filtering (CF) is a widely used method in recommender systems due to its simplicity and efficiency. But most existing CF methods suffer from data scarcity, which arises from the situation with only a limited number of interactions between users and items. One solution to address is how to introduce Transfer Learning (TL)-based CF methods with heterogeneous feedbacks to deal with multi-source and heterogeneous data, such as rating vs. clicks or rating vs. purchase. However, in some applications, extremely sparse (i.e., sparsity level ≤ 0.1%) target data could cause under-transfer and negative transfer. To address the above issue, we propose an Enhanced Knowledge Transfer for Collaborative Filtering with Multi-Source Heterogeneous Feedbacks (EKT). Specifically, we first propose a weighted collective matrix tri-factorization framework. The proposed framework constrains the auxiliary data and the target data to share the same latent factors of users and items as well as partial cluster-level user-item rating pattern in order to enhance knowledge transfer and alleviate the under-transfer issue. Then, to alleviate the negative transfer issue, we integrate the graph co-regularization terms into a proposed framework, which contains the neighborhood structure information of users and items. At last, we simultaneously minimize the objective function of EKT, which consists of weighted collective matrix tri-factorization and the graph co-regularization of user and item graphs. Since the EKT framework is a non-convex optimization problem, we use an alternating optimization procedure to solve it and further prove its convergence. The experimental results on two benchmark datasets show that our proposed EKT method performs better than other baseline methods at almost all sparsity levels except for the denser case of 1% on ML10M and Netflix.

Distributed modeling considering uncertainties for robust operation of integrated energy system

To improve the energy system flexibility and robustness, a distributed synergistic model with the min-max-min robust optimization is proposed for a 3-block integrated energy system (IES). First, the uncertainties of distributed generation (DG) output, and electric vehicle (EV) aggregator’s state of charge (SOC) in IES are thoroughly considered, and the uncertainty set considering the spatial-temporal correlation and symmetry is constructed. Second, a three-level min-max-min robust optimization model for IES is built to deal with the multiple uncertainties. Furthermore, to accelerate the solution of the third-level with integer variables, this paper develops a column and constraint generation with an alternative optimization procedure (C&CG-AOP) algorithm in the robust optimization model. Finally, based on the prediction-correction-based alternating direction method with multipliers (PCB-ADMM) algorithm, a distributed model considering the multi-agent characteristics of IES is proposed to solve the 3-block IES distributed operation problem. The simulation results show that the constructed uncertainty set considering the spatial-temporal correlation and symmetry has lower operating cost than the traditional uncertainty set, which can eliminate some low probability scenarios and make the conservatism of robust optimization reduced in a certain degree. Meanwhile, the developed C&CG-AOP algorithm effectively reduces the solving time while keeping the results consistent, and has higher efficiency than the Nested-C&CG algorithm in solving complex multi-level models. Moreover, the 3-block IES distributed model based on the PCB-ADMM algorithm can converge reliably and has a faster convergence rate than the compared ADMM-based algorithm.

Compressed Sensing-Based Simultaneous Recovery of Magnitude and Phase MR Images via Dual Trigonometric Sparsity

Complex-valued Magnetic Resonance Imaging (MRI) is widely used in clinical diagnosis. The magnitude images are mainly used for structure visualization, and the phase images reveal tissue properties, such as magnetic susceptibility and fluid flow information. While MRI is slow, compressed sensing (CS) can be used to reconstruct the accelerated acquisitions. Current CS-based MRI reconstruction algorithms mostly focus on magnitude image recovery, with less attention to the recovery of phase images. In this paper, we propose a novel CS algorithm to simultaneously recover the magnitude and phase MR images based on the sparsity of the trigonometric function. The CS method requires a sparse representation of the original images, and it is observed the trigonometric functions of phase images in the Wavelet domain promote sparsity. Therefore, rather than transforming the phase images directly into the Wavelet domain, we calculate the sine and cosine of the phase images whose Wavelet transforms are set as the L1-norm regularization term. The combination of the dual trigonometric functions captures a unique, faithful four-quadrant phase information, which also improves the reconstructed magnitude images through an alternating optimization procedure. Reconstructions of simulated images and in vivo images are studied, and both show the superiority of the proposed method over compared phase recovery algorithms.

Power Allocation for Coexisting Multicarrier Radar and Communication Systems in Cluttered Environments

In this paper, power allocation is examined for the coexistence of a radar and a communication system that employ multicarrier waveforms. We propose two designs for the considered spectrum sharing problem by maximizing the output signal-to-interference-plus-noise ratio (SINR) at the radar receiver while maintaining certain communication throughput and power constraints. The first is a joint design where the subchannel powers of both the radar and communication systems are jointly optimized. Since the resulting problem is highly nonconvex, we introduce a reformulation by combining the power variables of both systems into a single stacked variable, which allows us to bypass a conventional computationally intensive alternating optimization procedure. The resulting problem is then solved via a quadratic transform method along with a sequential convex programming (SCP) technique. The second is a unilateral design which optimizes the radar transmission power with fixed communication power. The unilateral design is suitable for cases where the communication system pre-exists while the radar occasionally joins the channel as a secondary user. The problem is solved by a Taylor expansion based iterative SCP procedure. Numerical results are presented to demonstrate the effectiveness of the proposed joint and unilateral designs in comparison with a subcarrier allocation based method.

Unsupervised Optimized Bipartite Graph Embedding

Graph embedding is a widely used method for dimensionality reduction due to its computational effectiveness. The quality of the graph and the efficiency of graph construction will directly affect the performance and the efficiency of the graph embedding methods. However, in the unsupervised graph embedding methods, the graph is not considered as an optimized graph since there is no label that can be used to construct this graph. In addition, the running of traditional graph embedding methods become very time-consuming on large-scale datasets due to the high computational cost in the step of graph construction. Aiming to solve these problems, we propose an unsupervised dimensionality reduction method based on bipartite graph, called Unsupervised Optimized Bipartite Graph Embedding (UOBGE). Representative anchors are firstly identified in the data. Then, we construct the bipartite graph between the projected samples and the projected anchors and the intrinsic graph connecting all the projected sample pairs with equal weights, which keep the local and global geometric structures of the data, respectively. Finally, the bipartite graph and the projection matrix are optimized simultaneously by introducing an alternating optimization procedure. Extensive experiments on several datasets demonstrate that the effectiveness and efficiency of the proposed method.