Alternating Optimization Procedure Research Articles

A Historical-Correlation-Driven Robust Optimization Approach for Microgrid Dispatch

Robust optimization (RO) has been a proficient approach for uncertainty dispatch in microgrids. In view of the strong conservativeness of traditional RO models, this article proposes a historical-correlation-driven (HCD) RO method for the MG dispatch problem. Firstly, the distinct spatiotemporal correlations in renewable energy (RE) generation are confirmed based on the historical data series of RE power. To effectively extract the correlation characteristics, the new intervals that envelop all correlation data points of similar days are formulated via the line-fitting approach, and the interval boundaries are added as liner constraints into the polyhedral uncertainty set that actively integrates the spatiotemporal correlations and avoids unreasonable scenarios. Secondly, a gradient-approximating decomposition algorithm with equilibrium constraints is put forward to address the nonlinear RO problem caused by non-independent uncertainties. The alternative optimization procedure ensures the gradient ascent of the objective, thus achieving the fast solution convergence of the Max-Min model with binary recourse variables. Besides, the equilibrium constraints realize the precise linearization of bilinear terms, as well as avoid the introduction of numerous binaries. Case tests verify the effectiveness and superiority of the proposed HCD-RO method comparing with existing models and algorithms.

Improving Hosting Capacity of Unbalanced Distribution Networks via Robust Allocation of Battery Energy Storage Systems

Distribution system operators aim to improve hosting capacity (HC) of distribution networks (DNs) to accommodate more distributed rooftop photovoltaics (PVs). Although PV power generation delivers numerous benefits, power unbalance and voltage rise are two major obstacles that limit the network HC. To mitigate these issues, battery energy storage systems (BESSs) can be applied. Thus, this paper proposes a robustly optimal allocation method for BESSs, which aims to reduce the power unbalance and alleviate the voltage rise, and thus improve the HC of the unbalanced three-phase DNs. Considering that locations and capacities of distributed rooftop PVs are determined by customers, future PV installations are regarded as uncertainties. In addition, to deal with the uncertainties, the proposed BESS allocation problem is formulated as an adaptive robust optimization (ARO) model with integer recourse variables. Accordingly, a solution algorithm which integrates an alternating optimization procedure into a column-and-constraint generation algorithm is developed to efficiently solve the ARO model. With the proposed BESS allocation method, a new perspective on HC improvement is provided, which not only considers the worst power unbalance situation but also satisfies the allowed maximum PV capacity. The simulation results verify high efficiency and solution robustness of the proposed allocation method.

A Novel Graphics-Based Multi-Epoch Power Adjustment Strategy for WPCNs

In this paper, we focus on a wireless powered communication network where a hybrid access point can wirelessly activate the power of source nodes, and receive their information. Considering the time-switching protocol that implies energy harvesting, and energy usage are separated by time, a novel graphics-based multi-epoch power adjustment (GB-MPA) strategy is proposed to maximize the sum-throughput over a finite time horizon consisting of multiple transmission epochs. The proposed strategy relies on graphical constraint conditions, and allows the elimination of restrictions corresponding to the objective of throughput maximization. Moreover, the proposed GB-MPA strategy can be combined with the time-switching factor optimization by employing an alternating optimization procedure for performance enhancement. Simulation results show the accuracy of the proposed strategy, reduced complexity compared with the conventional convex optimization method, and the existing multi-epoch power adjustment strategy, and the compatibility with the alternating optimization procedure for developing the joint power, and time allocation scheme.

A Novel Spectrally-Efficient Uplink Hybrid-Domain NOMA System

This paper proposes a novel hybrid-domain (HD) non-orthogonal multiple access (NOMA) approach to support a larger number of uplink users than the recently proposed code-domain NOMA approach, i.e., sparse code multiple access (SCMA). HD-NOMA combines the code-domain and power-domain NOMA schemes by clustering the users in small path loss (strong) and large path loss (weak) groups. The two groups are decoded using successive interference cancellation while within the group users are decoded using the message passing algorithm. To further improve the performance of the system, a spectral-efficiency maximization problem is formulated under a user quality-of-service constraint, which dynamically assigns power and subcarrier to the users. The problem is non-convex and has sparsity constraints. The alternating optimization procedure is used to solve it iteratively. We apply successive convex approximation and reweighted $\ell_1$ minimization approaches to deal with the non-convexity and sparsity constraints, respectively. The performance of the proposed HD-NOMA is evaluated and compared with the conventional SCMA scheme through numerical simulation. The results show the potential of HD-NOMA in increasing the number of uplink users.

Recourse-Cost Constrained Robust Optimization for Microgrid Dispatch With Correlated Uncertainties

To accomplish more practical scheduling of microgrids under source-load uncertainties, this article first proposes a novel recourse-cost constrained adaptive robust optimization (RC-ARO) model with binary recourse variables. The dispatch plan in the nominal scenario is optimized in the first-stage to get the minimal operation cost, then the adjustment plan in the worst scenario is determined in the second-stage that minimizes the recourse-cost. This model has overcome the defect of conventional adaptive robust optimization (ARO), which can only get the scheduling plans in the worst scenario. Second, a spatiotemporal correlation model of wind power uncertainty is further developed based on the similarities of power time sequences, aimed at avoiding impossible scenarios in reality and reducing the conservativeness of independent uncertainty sets. Third, a new column-and-constraint generation (C&CG) algorithm with alternating optimization procedure (AOP) is developed to directly obtain the binary solution, which helps accelerating the solution of RC-ARO model using traditional nested-C&CG. Finally, case studies demonstrate the effectiveness and superiority of the proposed RC-ARO model, the developed uncertainty sets, and the novel solving algorithm. The solving time of C&CG-AOP reduces by half compared with nested-C&CG, and a larger scale of decision variables under uncertainties brings more significant speedup by the proposed algorithm.

Aperture-Level Simultaneous Transmit and Receive With Digital Phased Arrays

While purely digital phased arrays were once discarded as simultaneous transmit and receive (STAR) capable platforms, this notion has recently been reconsidered. Previous work demonstrated that adaptive digital beamforming and digital self-interference cancellation (SIC) can enable transmitting and receiving subapertures in an array to operate simultaneously in the same frequency band. This approach, referred to as Aperture-Level Simultaneous Transmit and Receive (ALSTAR), uses only adaptive digital beamforming and digital SIC techniques. The ALSTAR architecture does not require custom radiators or analog canceling circuits that can increase front end losses and add significant size, weight, and cost to the array. This paper extends the previously proposed effective isotropic isolation (EII) metric to account for fixed dynamic range transmit and receive channels. An alternating optimization procedure that exploits the interdependence of the transmit and receive beamformers is proposed based on the symmetry of the EII metric, achieving higher EII than in previous work. This optimization procedure balances the goal of null-placement for interference and noise rejection with the goal of maintaining high transmit and receive gain. Simulated results are presented for a $\mathbf {50}$ -element array that achieves $\mathbf {187.1}$ dB of EII in narrowband operation with $\mathbf {2500}$ W of transmit power. We explore the effectiveness of the architecture and proposed optimization methods by demonstrating the high EII achieved across the full scan space of the array at several transmit power levels. Results are also presented for a regularized version of the beamformer optimization problem that allows the designer to trade EII for array gain.

Learning Object Detectors With Semi-Annotated Weak Labels

For alleviating the human labor associated with annotating the training data for learning object detectors, recent research has focused on semi-supervised object detection (SSOD) and weakly supervised object detection (WSOD) approaches. In SSOD, instead of annotating all the instances in the whole training set, people only need to annotate the part of the training instances using bounding boxes. In WSOD, people need to annotate the image-level tags on all training images to indicate the object categories contained by the corresponding images since more detailed bounding box annotations are no longer needed. Along this line of research, this paper makes a further step to alleviate the human labor in annotating training data, leading to the problem of object detection with semi-annotated weak labels (ODSAWLs). Instead of labeling image-level tags on all training images, ODSAWL only needs the image-level tags for a small portion of the training images, and then, the object detectors can be learned from a small portion of the weakly-labeled training images and from the remaining unlabeled training images. To address such a challenging problem, this paper proposes a cross model co-training framework that collaborates an object localizer and a tag generator in an alternative optimization procedure. Specifically, during the learning procedure, these two (deep) models can transfer the needed knowledge (including labels and visual patterns) between each other. The whole learning procedure is accomplished in a few stages under the guidance of a progressive learning curriculum. To demonstrate the effectiveness of the proposed approach, we implement the comprehensive experiments on three benchmark datasets, where the obtained experimental results are quite encouraging. Notably, by using only about 15% weakly labeled training images, the proposed approach can effectively approach, or even outperform, the state-of-the-art WSOD methods.

Distributed Adaptive Robust Voltage/VAR Control With Network Partition in Active Distribution Networks

This paper proposes a distributed adaptive robust voltage/var control (DAR-VVC) method in active distribution networks to minimize power loss while keeping operating constraints under uncertainties. The DAR-VVC aims to coordinate on-load tap changers, capacitor banks and PV inverters in multiple operation stages through a distributed algorithm. To improve efficiency of the distributed algorithm, an affinity propagation clustering algorithm is employed to divide the distribution network by aggregating “the close nodes” together and setting “the far nodes” apart, leading to the network partition where the information exchange between adjacent sub-networks is reduced. Moreover, the virtual load which describes load characteristics of the sub-networks is applied to enhance the boundary conditions. To fully deal with the uncertainties, the proposed DAR-VVC is formulated in a robust optimization model which considers the worst case to guarantee solution robustness against uncertainty realization. Besides, this paper develops an alternating optimization procedure integrating a column-and-constraint generation algorithm and an alternating direction method of multipliers to solve the DAR-VVC problem. The proposed approach is tested on IEEE 33 and IEEE 123 bus distribution test system and numerical simulations verify high efficiency and full solution robustness of the DAR-VVC.

Discovering Multiple Co-Clusterings With Matrix Factorization.

Clustering is a fundamental data exploration task which aims at discovering the hidden grouping structure in the data. The traditional clustering methods typically compute a single partition. However, there often exist different and equally meaningful clusterings in complex data. To solve this issue, multiple clustering approaches have emerged with the goal of exploring alternative clusterings from different perspectives. Existing solutions to this problem mainly focus on one-way clustering, that is, they cluster either the samples or the features. However, for many practical tasks, it is meaningful and desirable to explore alternative two-way clusterings (or co-clusterings), which capture not only the sample cluster structure but also the feature cluster structure. To tackle this interesting and unresolved task, we introduce an approach, called multiple co-clusterings (MultiCCs), to generate multiple alternative co-clusterings at the same time. MultiCC takes advantage of matrix tri-factorization to seek the co-clustering indicator matrices for samples and features and defines the row and column redundancy quantification terms to enforce diversity among co-clusterings based on these indicator matrices. After that, it integrates matrix tri-factorization and two nonredundancy terms into a unified objective function and gives an alternative optimization procedure to optimize the objective function. Extensive experimental results demonstrate that MultiCC performs significantly better than the existing multiple clustering methods. In addition, MultiCC can find out interesting co-clusters, which cannot be made by those comparing methods.

Extreme semi-supervised learning for multiclass classification

Semi-Supervised Support Vector Machines (S3VMs) provide a powerful framework for Semi-Supervised Learning (SSL) tasks which leverage widely available unlabeled data to improve performance. However, there exist three issues in S3VMs: (i) S3VMs require concurrently training c one-against-all (OAA) classifiers (c is the number of classes) for multiclass classification, which is prohibitive for large c; (ii) S3VMs require huge computational time and large storage (because of the large kernel matrix) in large-scale training and testing; (iii) S3VMs require the balance constraint in the unlabeled data, which not only needs prior knowledge from the unlabeled data (the prior knowledge is unavailable in some applications), but also makes their nonconvex optimization problem more intractable. To address these issues, a novel method called Extreme Semi-Supervised Learning (ESSL) is proposed in this paper. First, the framework of Extreme Learning Machine (ELM) is adopted to handle both binary and multiclass classification problems in a unified model. Second, the hidden layer is encoded by an extremely small approximate empirical kernel map (AEKM) to greatly reduce the computational cost and the memory usage for training and testing. Third, the balance constraint (prior knowledge) in the unlabeled data is removed through the elaborative design of weighting function (which emphasizes the importance of labeled data and the minority pattern in the labeled data).By these three ways, ESSL can be solved effectively and efficiently based on alternating optimization (AO). More specifically, ESSL can be analytically and simply solved by generalized pseudoinverse and oneHotMap function (without any optimization solver and the OAA strategy) in the AO procedure, and consequently, better performance and much faster training speed are always achieved in ESSL. Our empirical study shows that ESSL significantly outperforms existing efficient SSL methods (e.g., meanS3VM and SS-ELM) in terms of accuracy, efficiency and memory, especially for large-scale multiclass problems. As an example, on the 20Newsgroups dataset, ESSL respectively runs 45 and 120 times faster than meanS3VM for training and testing with the improvement in accuracy of 3%, while the memory usage is reduced to 1/14. It is noteworthy that even though all the model parameters are with default values, ESSL already produces very excellent performance without fine-tuning parameters.

Synchronously Decentralized Adaptive Robust Planning Method for Multi-Stakeholder Integrated Energy Systems

In the future integrated energy market, the traditional centralized planning method cannot solve the planning problem for multi-stakeholder integrated energy systems (MSIESs). To cope with this problem, this paper proposes a synchronously decentralized adaptive robust programming (SDARP) model for MSIESs based on the analytical target cascading (ATC) algorithm. In this model, different stakeholders are planned in parallel under the source-load uncertainties, aiming at minimizing the annual investment and operation costs and fully taking into account the interactive heating/cooling power transmitted by the district heating/cooling network. In each stakeholder, the column-and-constraint generation algorithm is used to address the robustness problems in the three-layer adaptive robust programming model. In addition, a tractable alternating optimization procedure is used to solve the non-convex SDARP model with finite convergence. Case studies verify the superiority and effectiveness of the SDARP method in dealing with the planning problem of MSIESs.

Supervised Nonnegative Matrix Factorization to Predict ICU Mortality Risk.

ICU mortality risk prediction is a tough yet important task. On one hand, due to the complex temporal data collected, it is difficult to identify the effective features and interpret them easily; on the other hand, good prediction can help clinicians take timely actions to prevent the mortality. These correspond to the interpretability and accuracy problems. Most existing methods lack of the interpretability, but recently Subgraph Augmented Nonnegative Matrix Factorization (SANMF) has been successfully applied to time series data to provide a path to interpret the features well. Therefore, we adopted this approach as the backbone to analyze the patient data. One limitation of the original SANMF method is its poor prediction ability due to its unsupervised nature. To deal with this problem, we proposed a supervised SANMF algorithm by integrating the logistic regression loss function into the NMF framework and solved it with an alternating optimization procedure. We used the simulation data to verify the effectiveness of this method, and then we applied it to ICU mortality risk prediction and demonstrated its superiority over other conventional supervised NMF methods.

A Fuzzy Co-Clustering Algorithm via Modularity Maximization

In this paper we propose a fuzzy co-clustering algorithm via modularity maximization, named MMFCC. In its objective function, we use the modularity measure as the criterion for co-clustering object-feature matrices. After converting into a constrained optimization problem, it is solved by an iterative alternative optimization procedure via modularity maximization. This algorithm offers some advantages such as directly producing a block diagonal matrix and interpretable description of resulting co-clusters, automatically determining the appropriate number of final co-clusters. The experimental studies on several benchmark datasets demonstrate that this algorithm can yield higher quality co-clusters than such competitors as some fuzzy co-clustering algorithms and crisp block-diagonal co-clustering algorithms, in terms of accuracy.

Structured Weak Semantic Space Construction for Visual Categorization.

Visual features have been widely used for image representation and categorization. However, visual features are often inconsistent with human perception. Besides, constructing explicit semantic space is still an open problem. To alleviate these two problems, in this paper, we propose to construct structured weak semantic space for image representation. Exemplar classifier is first trained to separate each training image from other images for weak semantic space construction. However, each exemplar classifier separates one training image from other images, and it only has limited semantic separability. Besides, the outputs of exemplar classifiers are inconsistent with each other. We jointly construct the weak semantic space using structured constraint. This is achieved by imposing low-rank constraint on the outputs of exemplar classifiers with sparsity constraint. An alternative optimization procedure is used to learn the exemplar classifiers. Since the proposed method does not dependent on the initial image representation strategy, we can make use of various visual features for efficient exemplar classifier training (e.g., fisher vector-based methods and convolutional neural networks-based methods). We apply the proposed structured weak semantic space-based image representation method for categorization. The experimental results on several public image data sets prove the effectiveness of the proposed method.

Axiomatic generalization of the membership degree weighting function for fuzzy C means clustering: Theoretical development and convergence analysis

For decades practitioners have been using the center-based partitional clustering algorithms like Fuzzy C Means (FCM), which rely on minimizing an objective function, comprising of an appropriately weighted sum of distances of each data point from the cluster representatives. Numerous generalizations in terms of choice of the distance function have been introduced for FCM since its inception. In a stark contrast to this fact, to the best of our knowledge, there has not been any significant effort to address the issue of convergence of the algorithm under the structured generalization of the weighting function. Here, by generalization we mean replacing the conventional weighting function uijm (where uij indicates the membership of data xi to cluster Cj and m is the real-valued fuzzifier with 1≤ m < ∞) with a more general function g(uij) which can assume a wide variety of forms under some mild assumptions. In this article, for the first time, we present a novel axiomatic development of the general weighting function based on the membership degrees. We formulate the fuzzy clustering problem along with the intuitive justification of the technicalities behind the axiomatic approach. We develop an Alternative Optimization (AO) procedure to solve the main clustering problem by solving the partial optimization problems in an iterative way. The novelty of the article lies in the development of an axiomatic generalization of the weighting function of FCM, formulation of an AO algorithm for the fuzzy clustering problem with the extension to this general class of weighting functions, and a detailed convergence analysis of the proposed algorithm.

Lagrangian relaxations for multiple network alignment

We propose a principled approach for the problem of aligning multiple partially overlapping networks. The objective is to map multiple graphs into a single graph while preserving vertex and edge similarities. The problem is inspired by the task of integrating partial views of a family tree (genealogical network) into one unified network, but it also has applications, for example, in social and biological networks. Our approach, called Flan, introduces the idea of generalizing the facility location problem by adding a non-linear term to capture edge similarities and to infer the underlying entity network. The problem is solved using an alternating optimization procedure with a Lagrangian relaxation. Flan has the advantage of being able to leverage prior information on the number of entities, so that when this information is available, Flan is shown to work robustly without the need to use any ground truth data for fine-tuning method parameters. Additionally, we present three multiple-network extensions to an existing state-of-the-art pairwise alignment method called Natalie. Extensive experiments on synthetic, as well as real-world datasets on social networks and genealogical networks, attest to the effectiveness of the proposed approaches which clearly outperform a popular multiple network alignment method called IsoRankN.

Attributes driven tracklet-to-tracklet person re-identification using latent prototypes space mapping

Most of current person re-identification works identify a person by matching his/her probe image against a galley images set. One feasible way to improve the identification accuracy is the multi-shot re-identification, where the probe includes a small set of images rather than a single image. In this paper, we study the tracklet-to-tracklet identification, where both the probe and the target dataset are composed of small sets of sequential images, i.e., tracklets. To solve this problem and make our algorithm robust under multi-camera setting, we take full advantage of low-level features, attributes and inter-attribute correlations at the same time. Attributes are expected to offer semantic information complementary to low-level features. In order to discover the correlations among attributes, a novel discriminative model is proposed to exploit low-level features and map attributes to a discriminative latent prototypes space. An alternating optimization procedure is designed to perform the learning process. We also devise a number of voting schemes to total up matching scores from images to tracklets. Experiments on four public datasets show that our approach achieves a consistently better performance than existing person re-identification methods.

Maximum margin partial label learning

Partial label learning aims to learn from training examples each associated with a set of candidate labels, among which only one label is valid for the training example. The basic strategy to learn from partial label examples is disambiguation, i.e. by trying to recover the ground-truth labeling information from the candidate label set. As one of the popular machine learning paradigms, maximum margin techniques have been employed to solve the partial label learning problem. Existing attempts perform disambiguation by optimizing the margin between the maximum modeling output from candidate labels and that from non-candidate ones. Nonetheless, this formulation ignores considering the margin between the ground-truth label and other candidate labels. In this paper, a new maximum margin formulation for partial label learning is proposed which directly optimizes the margin between the ground-truth label and all other labels. Specifically, the predictive model is learned via an alternating optimization procedure which coordinates the task of ground-truth label identification and margin maximization iteratively. Extensive experiments on artificial as well as real-world datasets show that the proposed approach is highly competitive to other well-established partial label learning approaches.

Graph modularity maximization as an effective method for co-clustering text data

In this paper we show how the modularity measure can serve as a useful criterion for co-clustering document-term matrices. We present and investigate the performance of CoClus, a novel, effective block-diagonal co-clustering algorithm which directly maximizes this modularity measure. The maximization is performed using an iterative alternating optimization procedure, in contrast to algorithms that use spectral relaxations of the discrete optimization problems. Extensive comparative experiments performed on various document-term datasets demonstrate that this approach is very effective, stable, and outperforms other block-diagonal co-clustering algorithms devoted to the same task. Another important advantage of using modularity in the co-clustering context is that it provides a novel, simple way of determining the appropriate number of co-clusters.Availability: an implementation of CoClus is available as part of the recently released coclust Python package which is available at: https://pypi.python.org/pypi/coclust

Structured regularization for conditional Gaussian graphical models

Conditional Gaussian graphical models are a reparametrization of the multivariate linear regression model which explicitly exhibits (i) the partial covariances between the predictors and the responses, and (ii) the partial covariances between the responses themselves. Such models are particularly suitable for interpretability since partial covariances describe direct relationships between variables. In this framework, we propose a regularization scheme to enhance the learning strategy of the model by driving the selection of the relevant input features by prior structural information. It comes with an efficient alternating optimization procedure which is guaranteed to converge to the global minimum. On top of showing competitive performance on artificial and real datasets, our method demonstrates capabilities for fine interpretation, as illustrated on three high-dimensional datasets from spectroscopy, genetics, and genomics.