Subgradient Descent Algorithm Research Articles

Optimal topology control for grounding protection sensor networks in shipboard electrical systems.

Intelligent monitoring of environmental information in ships is an important prerequisite for reducing and avoiding accidents and is of great importance for safety at sea. The use of a ship environmental monitoring system based on wireless sensor networks has the unique advantages of low cost, stability, and reliability, allowing real-time monitoring of many environmental factors in ships. This article proposes an optimal topology control to maximize collected data satisfaction and reduce energy consumption and transmission latency. First, the cluster head selection method is obtained by initializing the wireless sensor network with a fuzzy clustering algorithm, then the amount of collection data is obtained by a sub-gradient descent algorithm, and a sensor association scheme based on ant colony optimization is proposed to solve the knapsack problem. Finally, simulation results show that the proposed algorithm in this article can effectively reduce the energy consumption and transmission delay of the wireless sensor network while ensuring the monitoring of the grounding protection in the electrical system of the offshore ship.

Image emotion multi-label classification based on multi-graph learning

Images contain rich information and can induce various emotions in the audience. Image emotion classification aims to identify the emotion categories that images can evoke. It is widely used in mental health assessment, human–computer interaction, etc. There are two main problems in existing image emotion classification methods: (1) Most of them only focus on a single emotion label; (2) The global structural relationship among semantic objects in the image is ignored. Therefore, this paper proposes an Image Emotion classification method based on Multi-Graph Multi-Label learning (IE-MGML). In contrast to the existing approaches, the image is transformed into a graph-based representation by extracting the emotional features of semantic objects and calculating the similarity between the features. The local (semantic objects) features and global structure (relationship among semantic objects) features of the image are fused by the relationship between nodes. Furthermore, the graph representation of an image from the perspective of multiple emotional features is pooled and modeled as a graph bag containing multiple graphs (i.e., multi-graph). In multi-graph learning, the graph kernel directly evaluates a graph-label dependency score to avoid the loss of structural information caused by graph-instance degradation. The bag(image)-label dependency score is obtained by aggregating the graph-label dependency score from different perspectives through the aggregation function. The problem of error accumulation in the learning process is handled by proposing a threshold-based ranking loss objective function. Moreover, the non-convex optimization problem is addressed using a subgradient descent algorithm to deal with the required high-dimensional space computation. Experimental results on three general image emotion datasets show that the proposed method outperforms the state-of-the-art methods.

Learning Maximum Margin Channel Decoders

The problem of learning a channel decoder is considered for two channel models. The first model is an additive noise channel whose noise distribution is unknown and nonparametric. The learner is provided with a fixed codebook and a dataset comprised of independent samples of the noise, and is required to select a precision matrix for a nearest neighbor decoder in terms of the Mahalanobis distance. The second model is a non-linear channel with additive white Gaussian noise and unknown channel transformation. The learner is provided with a fixed codebook and a dataset comprised of independent input-output samples of the channel, and is required to select a matrix for a nearest neighbor decoder with a linear kernel. For both models, the objective of maximizing the margin of the decoder is addressed. Accordingly, for each channel model, a regularized loss minimization problem with a codebook-related regularization term and hinge-like loss function is developed, which is inspired by the support vector machine paradigm for classification problems. Expected generalization error bounds for the error probability loss function are provided for both models, under optimal choice of the regularization parameter. For the additive noise channel, a theoretical guidance for choosing the training signal-to-noise ratio is proposed based on this bound. In addition, for the non-linear channel, a high probability uniform generalization error bound is provided for the hypothesis class. For each channel, a stochastic sub-gradient descent algorithm for solving the regularized loss minimization problem is proposed, and an optimization error bound is stated. The performance of the proposed algorithms is demonstrated through several examples.

Federated learning with [formula omitted] regularization

Federated Learning (FL) is a widely adopted deep learning method that does not require the collection of raw training data and solves specific learning tasks by federating distributed devices. Due to the heterogeneous distribution of data across clients, the clients will drift toward the local optimal solutions during local training and result in different local models. The global model after aggregating these different local models may keep away from the global optimal solution. This phenomenon is known as client drift, which often hinders the performance of FL. Parameter regularization methods address this challenge of client drift by controlling the update direction of each client. They consider the global model as both the starting point and a reference for the induction bias in the penalty. However, the existing regularization approaches produce dense solutions so that all parameters need to be updated during local model training. At the same time, we note that some studies on deep learning have found that it is unnecessary to update all parameters at each round. Therefore, in this work, we design a novel FL training approach called Fedℓ1 which can alleviate the performance degradation of FL by updating only part of the parameters at each round. ℓ1 regularization is utilized to control the update direction of each client and avoid unnecessary parameter updates at the same time. To our knowledge, our study is the first to introduce sparse regularization term to correct the local training of individual clients in FL. We design a stochastic subgradient descent algorithm to train the ℓ1-regularized nonsmooth model. The comparison experiments with state-of-the-art baselines verify the superiority of the proposed approach.

Robust multi-view learning with the bounded LINEX loss

Multi-view learning, as a promising direction, emphasizes the consensus principle and the complementarity principle to boost the performance. By exploiting view-consistency or view-discrepancy among different views, numerous successful multi-view support vector machine models have been proposed. However, existing methods face two challenges. Firstly, most multi-view support vector machine models only consider the consensus principle, but ignore the complementarity principle. How to build a novel model with both principles has not been fully considered. Secondly, most multi-view support vector machine models neglect the robustness when the multi-view dataset is contaminated by the noisy samples, error-prone samples and view-inconsistent samples. Considering that the bounded linear-exponential (BLINEX) loss function possesses elegant merits, i.e., asymmetry and boundedness, developing a robust BLINEX-based model is worth exploring. Therefore, in this paper, we propose a BLINEX-based multi-view learning method called MVASY-BX, which explores the consensus and complementarity information with a between-view co-regularization term and importance weights of two views respectively. The mixed BLINEX loss is designed to make the model robust to noisy samples, view-inconsistent samples and error-prone samples. We solve linear and nonlinear MVASY-BX through the stochastic sub-gradient descent algorithm and the alternating direction method of multipliers respectively. Furthermore, we analyze the generalization error bound via Rademacher complexity. The comprehensive experiments confirm that our proposed model is more competitive than other benchmark methods.

Adaptive CSMA scheduling algorithm for queuing delay enhancement and energy optimization

This paper proposes an adaptive carrier sense multiple access (CSMA) algorithm for link scheduling in a network of conflict graph structures. The algorithm optimizes throughput and brings the queue size of packets at a node close to zero as it makes the average service rate strictly larger than the arrival rate of packets. It also minimizes energy consumption of the node owing to the incorporation of sleep–wake up strategy when attempting to access the physical medium. This CSMA scheme utilizes a simple subgradient descent algorithm to obtain its inherent parameters for each node in an iterative and distributed manner. The conducted numerical results corroborate the superiority of the developed CSMA over the state-of-art schedulers in terms of transmission rate, queue length and energy saving.

Synthesis cascade estimation for aircraft system identification

PurposeThe purpose of this paper extends the authors’ previous contributions on aircraft system identification, such as open loop identification or closed loop identification, to cascade system identification. Because the cascade system is one special network system, existing in lots of practical engineers, more unknown systems are needed to identify simultaneously within the statistical environment with the probabilistic noises. Consider this problem of cascade system identification, prediction error method is proposed to identify three unknown systems, which are parameterized by three unknown parameter vectors. Then the cascade system identification is transferred as one parameter identification problem, being solved by the online subgradient descent algorithm. Furthermore, the nonparametric estimation is proposed to consider the general case without any parameterized process. To make up the identification mission, model validation process is given to show the asymptotic interval of the identified parameter. Finally, simulation example confirms the proposed theoretical results.Design/methodology/approachFirstly, aircraft system identification is reviewed through the understanding about system identification and advances in control theory, then cascade system identification is introduced to be one special network system. Secondly, for the problem of cascade system identification, prediction error method and online subgradient decent algorithm are combined together to identify the cascade system with the parameterized systems. Thirdly from the point of more general completeness, another way is proposed to identify the nonparametric estimation, then model validation process is added to complete the whole identification mission.FindingsThis cascade system corresponds to one network system, existing in lots of practice, such as aircraft, ship and robot, so it is necessary to identify this cascade system, paving a way for latter network system identification. Parametric and nonparametric estimations are all studied within the statistical environment. Then research on bounded noise is an ongoing work.Originality/valueTo the best of the authors’ knowledge, research on aircraft system identification only concern on open loop and closed loop system identification, no any identification results about network system identification. This paper considers cascade system identification, being one special case on network system identification, so this paper paves a basic way for latter more advanced system identification and control theory.

Divergent Projection Analysis for Unsupervised Dimensionality Reduction

Principal component analysis (PCA) has been extended to a series of classical methods in dimensionality reduction for unsu¬pervised learning. However, the differences among the projected data samples may not be sufficient by PCA, causing problems both in data visualization and recognition. Though some nonlinear dimensionality reduction methods have been proposed to preserve the samples’ differences, their nonlinear mappings hide the relationships among the data features, e.g., Isometric mapping (Isomap) and Locally Linear Embedding (LLE). In this paper, we propose a linear dimensionality reduction method, called Divergent Projection Analysis (DPA), to preserve the largest differences among the samples. Our DPA projects the sam¬ples into a low-dimensional space such that any two projected samples have a certain distance, resulting in solving a nonconvex optimization problem. The nonconvex problem is solved by the subgradient descent algorithm and particle swarm optimiza¬tion algorithm, and their solution qualities are compared in the experiments. Experiments on a synthetic dataset and three face recognition problems confirm the effectiveness of the proposed method compared with other state-of-the-art methods.

On Online Adaptive Direct Data Driven Control

Based on our recent contributions on direct data driven control scheme, this paper continues to do some new research on direct data driven control, paving another way for latter future work on advanced control theory. Firstly, adaptive idea is combined with direct data driven control, one parameter adjustment mechanism is constructed to design the parameterized controller online. Secondly, to show the input-output property for the considered closed loop system, passive analysis is studied to be similar with stability. Thirdly, to validate whether the designed controller is better or not, another safety controller modular is added to achieve the designed or expected control input with the essence of model predictive control. Finally, one simulation example confirms our proposed theories. More generally, this paper studies not only the controller design and passive analysis, but also some online algorithm, such as recursive parameter identification and online subgradient descent algorithm. Furthermore, safety controller modular is firstly introduced in direct data driven control scheme.

Compressive Sensing SAR Imaging Algorithm for LFMCW Systems

Linear frequency-modulated (LFM) continuous-wave (CW) radar is usually the first choice in synthetic aperture radar (SAR) imaging missions due to its relatively low cost and hardware simplicity. However, the use of continuous-wave unnecessarily introduces the problem of intrapulse motion. Furthermore, a large amount of data generated by the CW imaging process may overburden the onboard communication system with its high streaming data rate. On the other hand, a large amount of data may, indeed, not be necessary for high-resolution SAR imaging. In this article, we took full consideration of the intrapulse motion in LFMCW radar systems and modeled the phase preserving extended frequency scaling algorithm (EFSA) reconstruction process with far fewer data samples as compressive sensing problem and used subgradient descent algorithm with optimal step size to realize compressive sensing reconstruction. Our compressive sensing problem is different from the traditional direct inversion-based problem in that our reconstructed results contain both mainlobe and sidelobes. Comparisons were made with mainstream iterative soft thresholding-based reconstruction algorithm to demonstrate its capability to reconstruct large imaging scenes with high resolution. Both simulations and experiments with measured data have verified our proposed algorithm.

Decentralized Online Convex Optimization With Event-Triggered Communications

Decentralized multi-agent optimization usually relies on information exchange between neighboring agents, which can incur unaffordable communication overhead in practice. To reduce the communication cost, we apply event-triggering technique to the decentralized multi-agent online convex optimization problem, where each agent is associated with a time-varying local loss function and the goal is to minimize the accumulated total loss (the sum of all local loss functions) by choosing appropriate actions sequentially. We first develop an event-triggered decentralized online subgradient descent algorithm for the full information case, where the local loss function is fully revealed to each agent at each time. We establish an upper bound for the regret of each agent in terms of the event-triggering thresholds. It is shown that the regret is sublinear provided that the event-triggering thresholds converge to zero as time goes to infinity. The algorithm and analysis are further extended to the scenario of bandit feedback, where only the values of the local loss function at two random points close to the current action are disclosed to each agent. We show that the two-point bandit feedback does not degrade the performance of the proposed algorithm in order sense and a regret bound similar to the full information case can be established. Finally, numerical results on the problem of decentralized online least squares are presented to validate the proposed algorithms.

QoS Driven Power Allocation in Secure Multicarrier Full-Duplex Relay Systems

In this paper, we propose the quality-of-service (QoS) driven robust power allocation policy with channel uncertainty in the secure multicarrier full-duplex (FD) relay communication system, where the statistical delay QoS is given by the secure effective capacity (SEC). To maximize the SEC under the constraints of both the total system power and the residual loopback interference (LI) power of the FD relay, an optimization problem is formulated and the corresponding power allocation strategy has been presented in the case of perfect channel state information (CSI) and imperfect CSI. Specially, considering the imperfect CSI, we propose a robust power allocation scheme, where the channel uncertainty is modeled by a bounded region. Furthermore, the worst-case method and the Bernstein approximations are utilized to deal with the uncertainty. After the Taylor approximation based simplification of the formulated optimization problem, the optimal solution is obtained by subgradient descent algorithm based on Lagrangian dual method and Karush-Kuhn-Tucker (KKT) conditions. With known error range of CSI, numerical results and analysis illustrate that the proposed robust power allocation strategy can satisfy the statistical delay QoS requirement very well. Furthermore, the trade-off between the SEC and the robustness can be achieved with channel uncertainty.

Performance Analysis and Optimization of Caching Placement in Heterogeneous Wireless Networks

Caching at the network edge is regarded as a promising approach to handle content-dominated wireless traffic congestion problem and improve the quality of experience at mobile users (MUs). In this letter, we consider a heterogeneous wireless network (HetNet) where caches are not only deployed at base stations (BSs) but also at MUs. To study the caching optimization in such an HetNet, we first derive the analytical results of successful transmission probability (STP) using stochastic geometry. Then, we propose a subgradient descent algorithm (SGDA) to optimize the caching placement with an aim to maximize the STP. Simulations indicate that a significant performance gain can be obtained by our proposed solution when compared with existing caching placements.

Lyapunov Optimized Resource Management for Multiuser Mobile Video Streaming

Buffering techniques have been commonly used in mobile video streaming systems to handle the bandwidth fluctuation and to mitigate the impact of the stochastic characteristic of wireless channels on a mobile user’s quality of experience. However, it has been shown by measurement study that users tend to abort when watching videos with mobile devices, which results in a significant wastage of the video data in the buffer. Therefore, one important problem in mobile video streaming is how to manage the buffer at each mobile user. On the other hand, mobile users generally share the wireless media to download the video data, i.e., mobile users compete with each other for the bandwidth to download the video data. Thus, another important problem in mobile video streaming is how to allocate bandwidth among mobile users. In this paper, we propose to optimize the resource management, i.e., to design buffer management strategy at each mobile user and bandwidth allocation strategy among mobile users, for the multiuser mobile video streaming systems. Specifically, we optimize the long-term average total cost of data wastage and quality of experience of mobile users with certain constraints. By introducing virtual queues and employing the Lyapunov optimization theory, we transform the original optimization problem into the drift-plus-penalty minimization problem. Then, we adopt the primal decomposition to decouple the relationship among different mobile users, which decomposes the problem into a master problem with multiple subproblems. A one-dimension full search algorithm is applied to find the global optimal solution to each subproblem, and the subgradient descent algorithm is utilized to update the solution to the master problem. Finally, simulations are conducted to show that the proposed algorithm is effective for the buffer management and bandwidth allocation in a multiuser mobile video streaming system.

Energy‐efficient scheduling for sustainable manufacturing systems with renewable energy resources

AbstractEnvironmentally friendly energy resources open a new opportunity to tackle the problem of energy security and climate change arising from wide use of fossil fuels. This paper focuses on optimizing the allocation of the energy generated by the renewable energy system to minimize the total electricity cost for sustainable manufacturing systems under time‐of‐use tariff by clipping the peak demand. A rolling horizon approach is adopted to handle the uncertainty caused by the weather change. A nonlinear mathematical programming model is established for each decision epoch based on the predicted energy generation and the probability distribution of power demand in the manufacturing plant. The objective function of the model is shown to be convex, Lipchitz‐continuous, and subdifferentiable. A generalized benders decomposition method based on the primal‐dual subgradient descent algorithm is proposed to solve the model. A series of numerical experiments is conducted to show the effectiveness of the solution approach and the significant benefits of using the renewable energy resources.

Event-Triggered Communication and Data Rate Constraint for Distributed Optimization of Multiagent Systems

This paper is concerned with solving a large category of convex optimization problems using a group of agents, each only being accessible to its individual convex cost function. The optimization problems are modeled as minimizing the sum of all the agents’ cost functions. The communication process between agents is described by a sequence of time-varying yet balanced directed graphs which are assumed to be uniformly strongly connected. Taking into account the fact that the communication channel bandwidth is limited, for each agent we introduce a vector-valued quantizer with finite quantization levels to preprocess the information to be exchanged. We exploit an event-triggered broadcasting technique to guide information exchange, further reducing the communication cost of the network. By jointly designing the dynamic event-triggered encoding–decoding schemes and the event-triggered sampling rules (to analytically determine the sampling time instant sequence for each agent), a distributed subgradient descent algorithm with constrained information exchange is proposed. By selecting the appropriate quantization levels, all the agents’ states asymptotically converge to a consensus value which is also the optimal solution to the optimization problem, without committing saturation of all the quantizers. We find that one bit of information exchange across each connected channel can guarantee that the optimiztion problem can be exactly solved. Theoretical analysis shows that the event-triggered subgradient descent algorithm with constrained data rate of networks converges at the rate of ${O}( {\ln t/{\sqrt {t}}})$ . We supply a numerical simulation experiment to demonstrate the effectiveness of the proposed algorithm and to validate the correctness of theoretical results.

Traffic‐aware resource allocation scheme for mMTC in dynamic TDD systems

The asymmetry traffic between downlink (DL) and uplink (UL) in massive machine-type communication (mMTC) systems is so prominent that it makes the traditionally fixed frame protocols insufficient to handle. Meanwhile, the dynamic time-division duplexing (D-TDD) is a promising and attractive technology since its number of time slots for the DL and UL can be asymmetric and adjusted dynamically. In the cellular and mMTC co-existing network, to balance the discrepancy of the UL/DL ratio and alleviate the interference as well, in this study, the authors design a D-TDD-based transmission frame structure to first fulfil the basic transmission requirements of the human-type communication (HTC) users with a low power almost blank subframe. Herein, the stochastic geometry methods are adopted to calculate spectral efficiencies of the HTC user equipment. Then, focusing on the worst queue state in mMTC, they devise the slot allocation problem with the min–max objective of the UL/DL queues and utilise the sub-gradient descent (SGD) method for a solution. Simulation results show that the proposed traffic aware sub-frame configuration is more appropriate for the dynamical asymmetry environment. Meanwhile, the adopted dynamic step size SGD algorithm can achieve a trade-off between the worst-case queue and the network throughput.

Optimal distributed stochastic mirror descent for strongly convex optimization

In this paper we consider convergence rate problems for stochastic strongly-convex optimization in the non-Euclidean sense with a constraint set over a time-varying multi-agent network. We propose two efficient non-Euclidean stochastic subgradient descent algorithms based on the Bregman divergence as distance-measuring function rather than the Euclidean distances that were employed by the standard distributed stochastic projected subgradient algorithms. For distributed optimization of non-smooth and strongly convex functions whose only stochastic subgradients are available, the first algorithm recovers the best previous known rate of O(ln(T)∕T) (where T is the total number of iterations). The second algorithm is an epoch variant of the first algorithm that attains the optimal convergence rate of O(1∕T), matching that of the best previously known centralized stochastic subgradient algorithm. Finally, we report some simulation results to illustrate the proposed algorithms.

Location-Based Parallel Tag Completion for Geo-Tagged Social Image Retrieval

Having benefited from tremendous growth of user-generated content, social annotated tags get higher importance in the organization and retrieval of large-scale image databases on Online Sharing Websites (OSW). To obtain high-quality tags from existing community contributed tags with missing information and noise, tag-based annotation or recommendation methods have been proposed for performance promotion of tag prediction. While images from OSW contain rich social attributes, they have not taken full advantage of rich social attributes and auxiliary information associated with social images to construct global information completion models. In this article, beyond the image-tag relation, we take full advantage of the ubiquitous GPS locations and image-user relationship to enhance the accuracy of tag prediction and improve the computational efficiency. For GPS locations, we define the popular geo-locations where people tend to take more images as Points of Interests (POI), which are discovered by mean shift approach. For image-user relationship, we integrate a localized prior constraint, expecting the completed tag sub-matrix in each POI to maintain consistency with users’ tagging behaviors. Based on these two key issues, we propose a unified tag matrix completion framework, which learns the image-tag relation within each POI. To solve the optimization problem, an efficient proximal sub-gradient descent algorithm is designed. The model optimization can be easily parallelized and distributed to learn the tag sub-matrix for each POI. Extensive experimental results reveal that the learned tag sub-matrix of each POI reflects the major trend of users’ tagging results with respect to different POIs and users, and the parallel learning process provides strong support for processing large-scale online image databases. To fit the response time requirement and storage limitations of Tag-based Image Retrieval (TBIR) on mobile devices, we introduce Asymmetric Locality Sensitive Hashing (ALSH) to reduce the time cost and meanwhile improve the efficiency of retrieval.

Stochastic subgradient descent method for large-scale robust chance-constrained support vector machines

Robust chance-constrained Support Vector Machines (SVM) with second-order moment information can be reformulated into equivalent and tractable Semidefinite Programming (SDP) and Second Order Cone Programming (SOCP) models. However, practical applications involve processing large-scale data sets. For the reformulated SDP and SOCP models, existed solvers by primal-dual interior method do not have enough computational efficiency. This paper studies the stochastic subgradient descent method and algorithms to solve robust chance-constrained SVM on large-scale data sets. Numerical experiments are performed to show the efficiency of the proposed approaches. The result of this paper breaks the computational limitation and expands the application of robust chance-constrained SVM.