Proximal Algorithm Research Articles

Unsupervised spectral reconstruction from RGB images under two lighting conditions.

Unsupervised spectral reconstruction (SR) aims to recover the hyperspectral image (HSI) from corresponding RGB images without annotations. Existing SR methods achieve it from a single RGB image, hindered by the significant spectral distortion. Although several deep learning-based methods increase the SR accuracy by adding RGB images, their networks are always designed for other image recovery tasks, leaving huge room for improvement. To overcome this problem, we propose a novel, to our knowledge, approach that reconstructs the HSI from a pair of RGB images captured under two illuminations, significantly improving reconstruction accuracy. Specifically, an SR iterative model based on two illuminations is constructed at first. By unfolding the proximal gradient algorithm solving this SR model, an interpretable unsupervised deep network is proposed. All the modules in the proposed network have precise physical meanings, which enable our network to have superior performance and good generalization capability. Experimental results on two public datasets and our real-world images show the proposed method significantly improves both visually and quantitatively as compared with state-of-the-art methods.

Proximal-based recursive implementation for model-free data-driven fault diagnosis

We present a novel problem formulation for model-free data-driven fault diagnosis, in which possible faults are diagnosed simultaneously to identifying the linear time-invariant system. This problem is practically relevant for systems whose model cannot be identified reliably prior to diagnosing possible faults, for instance when operating conditions change over time, when a fault is already present before system identification is carried out, or when the system dynamics change due to the presence of the fault. A computationally attractive solution is proposed by solving the problem using unconstrained convex optimization, where the objective function consists of three terms of which two are non-differentiable. An additional recursive implementation based on a proximal algorithm is presented in order to solve the optimization problem online. The numerical results on a buck converter show the application of the proposed solution both offline and online.

CTV-Net: Complex-Valued TV-Driven Network With Nested Topology for 3-D SAR Imaging.

The regularization-based approaches offer promise in improving synthetic aperture radar (SAR) imaging quality while reducing system complexity. However, the widely applied l1 regularization model is hindered by their hypothesis of inherent sparsity, causing unreal estimations of surface-like targets. Inspired by the edge-preserving property of total variation (TV), we propose a new complex-valued TV (CTV)-driven interpretable neural network with nested topology, i.e., CTV-Net, for 3-D SAR imaging. In our scheme, based on the 2-D holography imaging operator, the CTV-driven optimization model is constructed to pursue precise estimations in weakly sparse scenarios. Subsequently, a nested algorithmic framework, i.e., complex-valued TV-driven fast iterative shrinkage thresholding (CTV-FIST), is derived from the theory of proximal gradient descent (PGD) and FIST algorithm, theoretically supporting the design of CTV-Net. In CTV-Net, the trainable weights are layer-varied and functionally relevant to the hyperparameters of CTV-FIST, which aims to constrain the algorithmic parameters to update in a well-conditioned tendency. All weights are learned by end-to-end training based on a two-term cost function, which bounds the measurement fidelity and TV norm simultaneously. Under the guidance of the SAR signal model, a reasonably sized training set is generated, by randomly selecting reference images from the MNIST set and consequently synthesizing complex-valued label signals. Finally, the methodology is validated, numerically and visually, by extensive SAR simulations and real-measured experiments, and the results demonstrate the viability and efficiency of the proposed CTV-Net in the cases of recovering 3-D SAR images from incomplete echoes.

Multi-policy deep reinforcement learning for multi-objective multiplicity flexible job shop scheduling

This study considers the simultaneous minimization of makespan and total tardiness for the multi-objective multiplicity flexible job shop scheduling problem (MOMFJSP). A deep reinforcement learning framework employing a multi-policy proximal policy optimization algorithm (MPPPO) is developed to solve MOMFJSP. The MOMFJSP is treated as a Markov decision process, allowing an intelligent agent to make sequential decisions based on the current production status. This framework involves multiple policy networks with different objective weight vectors. Using MPPPO, these networks are optimized simultaneously to obtain a set of high-quality Pareto-optimal policies. Moreover, a fluid model is introduced to extract state features and devise composite dispatching rules as discrete actions. A multi-policy co-evolution mechanism (MPCEM) is proposed to facilitate collaborative evolution among policy networks, supported by a reward mechanism that considers the objective weights. A training algorithm based on MPPPO is designed for learning across multiple policy networks. The effectiveness and superiority of the proposed method are confirmed through comparisons with composite dispatching rules and other scheduling methods.

A Proximal Algorithm for Distributed Optimization With Nonsmooth Inequality Constraints

A Proximal Algorithm for Distributed Optimization With Nonsmooth Inequality Constraints

A primal-dual data-driven method for computational optical imaging with a photonic lantern.

Optical fibers aim to image in vivo biological processes. In this context, high spatial resolution and stability to fiber movements are key to enable decision-making processes (e.g. for microendoscopy). Recently, a single-pixel imaging technique based on a multicore fiber photonic lantern has been designed, named computational optical imaging using a lantern (COIL). A proximal algorithm based on a sparsity prior, dubbed SARA-COIL, has been further proposed to solve the associated inverse problem, to enable image reconstructions for high resolution COIL microendoscopy. In this work, we develop a data-driven approach for COIL. We replace the sparsity prior in the proximal algorithm by a learned denoiser, leading to a plug-and-play (PnP) algorithm. The resulting PnP method, based on a proximal primal-dual algorithm, enables to solve the Morozov formulation of the inverse problem. We use recent results in learning theory to train a network with desirable Lipschitz properties, and we show that the resulting primal-dual PnP algorithm converges to a solution to a monotone inclusion problem. Our simulations highlight that the proposed data-driven approach improves the reconstruction quality over variational SARA-COIL method on both simulated and real data.

Learn to flap: foil non-parametric path planning via deep reinforcement learning

To optimize flapping foil performance, in the current study we apply deep reinforcement learning (DRL) to plan foil non-parametric motion, as the traditional control techniques and simplified motions cannot fully model nonlinear, unsteady and high-dimensional foil–vortex interactions. Therefore, a DRL training framework is proposed based on the proximal policy optimization algorithm and the transformer architecture, where the policy is initialized from the sinusoidal expert display. We first demonstrate the effectiveness of the proposed DRL-training framework, learning the coherent foil flapping motion to generate thrust. Furthermore, by adjusting reward functions and action thresholds, DRL-optimized foil trajectories can gain significant enhancement in both thrust and efficiency compared with the sinusoidal motion. Last, through visualization of wake morphology and instantaneous pressure distributions, it is found that DRL-optimized foil can adaptively adjust the phases between motion and shedding vortices to improve hydrodynamic performance. Our results give a hint of how to solve complex fluid manipulation problems using the DRL method.

Investigating the Impact of Curriculum Learning on Reinforcement Learning for Improved Navigational Capabilities in Mobile Robots

This paper proposes a method for finding the shortest path of a mobile robot using deep reinforcement learning with utilizing Proximal policy optimization algorithm (PPO) enhanced with curriculum learning. By modelling the environment in 3D space using the Webots simulator, we extend the PPO algorithm's capabilities to handle continuous states from 8 IR sensors and control the velocities of two motors of E-puck robot. Our study uniquely integrates curriculum learning into the PPO framework, aiming to improve adaptability and training efficiency in complex environments. A comparative analysis is conducted between the modified PPO, the original PPO, and the deep deterministic policy gradient algorithm, highlighting the strengths of our approach The results demonstrate that our curriculum-augmented PPO algorithm not only accelerates the training process but also shows superior adaptability and generalization in new environments. This work underscores the significant potential of curriculum learning in enhancing the performance of deep reinforcement learning algorithms for robust and efficient robotic navigation.

PPO-Clip Attains Global Optimality: Towards Deeper Understandings of Clipping

Proximal Policy Optimization algorithm employing a clipped surrogate objective (PPO-Clip) is a prominent exemplar of the policy optimization methods. However, despite its remarkable empirical success, PPO-Clip lacks theoretical substantiation to date. In this paper, we contribute to the field by establishing the first global convergence results of a PPO-Clip variant in both tabular and neural function approximation settings. Our findings highlight the O(1/√T ) min-iterate convergence rate specifically in the context of neural function approximation. We tackle the inherent challenges in analyzing PPO-Clip through three central concepts: (i) We introduce a generalized version of the PPO-Clip objective, illuminated by its connection with the hinge loss. (ii) Employing entropic mirror descent, we establish asymptotic convergence for tabular PPO-Clip with direct policy parameterization. (iii) Inspired by the tabular analysis, we streamline convergence analysis by introducing a two-step policy improvement approach. This decouples policy search from complex neural policy parameterization using a regression-based update scheme. Furthermore, we gain deeper insights into the efficacy of PPO-Clip by interpreting these generalized objectives. Our theoretical findings also mark the first characterization of the influence of the clipping mechanism on PPO-Clip convergence. Importantly, the clipping range affects only the pre-constant of the convergence rate.

Inertial Algorithm with Dry Fraction and Convolutional Sparse Coding for 3D Localization with Light Field Microscopy

Light field microscopy is a high-speed 3D imaging technique that records the light field from multiple angles by the microlens array(MLA), thus allowing us to obtain information about the light source from a single image only. For the fundamental problem of neuron localization, we improve the method of combining depth-dependent dictionary with sparse coding in this paper. In order to obtain higher localization accuracy and good noise immunity, we propose an inertial proximal gradient acceleration algorithm with dry friction, Fast-IPGDF. By preventing falling into a local minimum, our algorithm achieves better convergence and converges quite fast, which improves the speed and accuracy of obtaining the locolization of the light source based on the matching depth of epipolar plane images (EPI). We demonstrate the effectiveness of the algorithm for localizing non-scattered fluorescent beads in both noisy and non-noisy environments. The experimental results show that our method can achieve simultaneous localization of multiple point sources and effective localization in noisy environments. Compared to existing studies, our method shows significant improvements in both localization accuracy and speed.

Reinforcement Learning-Based Simulation of Seal Engraving Robot in the Context of Artificial Intelligence

The rapid development of robotics technology has made people's lives and work more convenient and efficient. The research and simulation of robots combined with reinforcement learning intelligent algorithms have become a hotspot in various fields of robot applications. In view of this, this study is based on deep reinforcement learning convolutional neural networks, combined with point cloud models, proximal strategy optimization algorithms, and flexible action evaluation algorithms. A seal cutting robot based on deep reinforcement learning has been proposed. The final results show that the descent speed of the seal cutting robot with the root mean square difference as the performance standard is about 1% faster than the flexible action evaluation algorithm. About 2% faster than the proximal strategy optimization algorithm. It is about 4% faster than the deep deterministic strategy gradient algorithm. This indicates that the research model has certain advantages in terms of actual accuracy after cutting. The fluctuation of this model is about 10% smaller than the evaluation of flexible actions and about 60% smaller than the gradient of deep deterministic strategies. Therefore, the research model has the highest overall stability without falling into local optima. In addition, compared to the near end strategy optimization algorithm, it falls into local optima, resulting in a low coincidence degree of about 17%. The deep deterministic strategy gradient algorithm has a large fluctuation amplitude during the seal cutting process, and the overall curve is relatively slow, with a final overlap of about 70%. The overlap degree of flexible action evaluation is slightly higher by about 83%. The maximum stability of the model's overlap is best around 90%. Through experiments, it can be found that the seal cutting robot proposed in the study based on deep reinforcement learning maintains certain advantages in performance indicators in various types of tests.

Enhanced deep unrolling networks for snapshot compressive hyperspectral imaging

Snapshot compressive hyperspectral imaging necessitates the reconstruction of a complete hyperspectral image from its compressive snapshot measurement, presenting a challenging inverse problem. This paper proposes an enhanced deep unrolling neural network, called EDUNet, to tackle this problem. The EDUNet is constructed via the deep unrolling of a proximal gradient descent algorithm and introduces two innovative modules for gradient-driven update and proximal mapping reflectivity. The gradient-driven update module leverages a memory-assistant descent approach inspired by momentum-based acceleration techniques, for enhancing the unrolled reconstruction process and improving convergence. The proximal mapping is modeled by a sub-network with a cross-stage spectral self-attention, which effectively exploits the inherent self-similarities present in hyperspectral images along the spectral axis. It also enhances feature flow throughout the network, contributing to reconstruction performance gain. Furthermore, we introduce a spectral geometry consistency loss, encouraging EDUNet to prioritize the geometric layouts of spectral curves, leading to a more precise capture of spectral information in hyperspectral images. Experiments are conducted using three benchmark datasets including KAIST, ICVL, and Harvard, along with some real data, comprising a total of 73 samples. The experimental results demonstrate that EDUNet outperforms 15 competing models across four metrics including PSNR, SSIM, SAM, and ERGAS.

Proximal point type algorithms with relaxed and inertial effects beyond convexity

We show that the recent relaxed-inertial proximal point algorithm due to Attouch and Cabot remains convergent when the function to be minimized is not convex, being only endowed with certain generalized convexity properties. Numerical experiments showcase the improvements brought by the relaxation and inertia features to the standard proximal point method in this setting, too.

A scenario-based robust approach for joint planning of multi-blood product logistics and multi-casualty type evacuation

This paper presents a two-stage hybrid robust programming model that integrates multiple blood products scheduling and multiple casualty types evacuation under facility disruptions risk and casualty number uncertainty. The model seeks to maximize the rescue efficiency and minimize the total operations cost. Under a set of disruption scenarios, we adopt a scenario-based robust method to address disruption risks at temporary medical centres and two robust uncertainty sets to deal with uncertain casualty numbers. We propose a proximal bundle algorithm to solve large-scale instances of the proposed hybrid robust model approximately. Extensive numerical experiments show that: (i) the trade-off between model robustness and solution robustness can help the decision-makers determine an appropriate risk-aversion weight; (ii) compared with the corresponding stochastic and scenario-based robust models, the hybrid robust model can obtain more robust solutions with a slight increase in cost; (iii) the proximal bundle algorithm can produce near-optimal solutions within reasonable computational times; (iv) some model parameters have significant impact on the total cost, which can help decision-makers set the appropriate parameters to achieve the desired outcome. Finally, we also use the real data of the 2008 Wenchuan County Earthquake in Sichuan Province, China, to illustrate the application of the model.

Digital Twin and Deep Reinforcement Learning-Driven Robotic Automation System for Confined Workspaces: A Nozzle Dam Replacement Case Study in Nuclear Power Plants

Robotic automation has emerged as a leading solution for replacing human workers in dirty, dangerous, and demanding industries to ensure the safety of human workers. However, practical implementation of this technology remains limited, requiring substantial effort and costs. This study addresses the challenges specific to nuclear power plants, characterized by hazardous environments and physically demanding tasks such as nozzle dam replacement in confined workspaces. We propose a digital twin and deep-reinforcement-learning-driven robotic automation system with an autonomous mobile manipulator. The study follows a four-step process. First, we establish a simplified testbed for a nozzle dam replacement task and implement a high-fidelity digital twin model of the real-world testbed. Second, we employ a hybrid visual perception system that combines deep object pose estimation and an iterative closest point algorithm to enhance the accuracy of the six-dimensional pose estimation. Third, we use a deep-reinforcement-learning method, particularly the proximal policy optimization algorithm with inverse reachability map, and a centroidal waypoint strategy, to improve the controllability of an autonomous mobile manipulator. Finally, we conduct pre-performed simulations of the nozzle dam replacement in the digital twin and evaluate the system on a robot in the real-world testbed. The nozzle dam replacement with precise object pose estimation, navigation, target object grasping, and collision-free motion generation was successful. The robotic automation system achieved a 92.0%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$92.0\\%$$\\end{document} success rate in the digital twin. Our proposed method can improve the efficiency and reliability of robotic automation systems for extreme workspaces and other perilous environments.

A new proximal heavy ball inexact line-search algorithm

We study a novel inertial proximal-gradient method for composite optimization. The proposed method alternates between a variable metric proximal-gradient iteration with momentum and an Armijo-like linesearch based on the sufficient decrease of a suitable merit function. The linesearch procedure allows for a major flexibility on the choice of the algorithm parameters. We prove the convergence of the iterates sequence towards a stationary point of the problem, in a Kurdyka–Łojasiewicz framework. Numerical experiments on a variety of convex and nonconvex problems highlight the superiority of our proposal with respect to several standard methods, especially when the inertial parameter is selected by mimicking the Conjugate Gradient updating rule.

HP3O algorithm-based all electric ship energy management strategy integrating demand-side adjustment

To tackle the energy management challenge that integrates power generation scheduling and demand-side adjustment for all-electric ship in uncertain marine environment, a hybrid penalized proximal policy optimization algorithm (HP3O)-based energy management strategy is proposed. First, demand-side adjustment, which involves adjusting the power of the ship's electric propulsion motors and flexible service loads, is integrated into the energy management problem. Second, HP3O algorithm is employed to obtain both continuous and discrete variables simultaneously. It utilizes a continuous actor network to obtain continuous variables, such as the generator's power and ship cruising speed, while employing a discrete actor network to determine discrete variables, i.e., the on/off status of the generators. Third, to handle complex constraints reasonably, the energy management problem is formulated as a constrained Markov decision process (CMDP), and an action mask mechanism is also integrated into the energy management framework to make agent's actions more reliable. The simulation results of an all-electric cruise ship validate the effectiveness and superiority of the proposed strategy in achieving near-optimal scheduling while satisfying operation constraints. Furthermore, a case study on a hybrid diesel-electric ferry confirms its generalization performance.

A predictive-reactive strategy for flight test task scheduling with aircraft grounding

In flight test engineering, the flight test duration (FTD) affects the aircraft’s delivery node and directly impacts costs. In the actual flight test process, the environmental status updates frequently, and various uncertain events are often encountered, which affect the flight test progress and project implementation. Therefore, when scheduling flight test tasks, rescheduling should be taken into account. This paper proposes a predictive-reactive strategy based on a deep reinforcement learning approach to solve the flight test task scheduling problem with consideration of aircraft grounding. In the predictive stage, a constructive heuristic algorithm is designed to generate an initial schedule. The rescheduling problem is solved by the appropriate rescheduling method that aims to optimize the FTD deviation, task reallocation, and workload cost simultaneously. The problem is modeled as a Markov decision process, including the well-designed state features, rewards, and actions based on different rescheduling methods. The policy is trained by the proximal policy optimization algorithm. At last, numerical results are provided to demonstrate the effectiveness and superiority of the proposed approach.

A Distributed Proximal Consensus Algorithm for Energy Saving in Ethylene Production.

This article presents a distributed optimization framework in order to solve the plant-wide energy-saving problem of an ethylene plant. First, the ethylene production process is abstracted into a distributed network, and then, a new distributed consensus algorithm is proposed, which is called adaptive step-size-based distributed proximal consensus algorithm (ASS-DPCA). This algorithm can dynamically adjust the step size and automatically abandon the irrational evolutionary route while eliminating the dependence of optimization algorithms on model gradient information. Moreover, the designed algorithm is able to converge to an optimal solution for any convex cost functions and approach to a convex constraint set of agents over an undirected connected graph. Finally, the results of numerical simulation and industrial experiments show that the algorithm can reduce the total energy consumption of an ethylene plant with less computing time and assured consensus.

Energy optimization management of microgrid using improved soft actor-critic algorithm

To tackle the challenges associated with variability and uncertainty in distributed power generation, as well as the complexities of solving high-dimensional energy management mathematical models in mi-crogrid energy optimization, a microgrid energy optimization management method is proposed based on an improved soft actor-critic algorithm. In the proposed method, the improved soft actor-critic algorithm employs an entropy-based objective function to encourage target exploration without assigning signifi-cantly higher probabilities to any part of the action space, which can simplify the analysis process of distributed power generation variability and uncertainty while effectively mitigating the convergence fragility issues in solving the high-dimensional mathematical model of microgrid energy management. The effectiveness of the proposed method is validated through a case study analysis of microgrid energy op-timization management. The results revealed an increase of 51.20%, 52.38%, 13.43%, 16.50%, 58.26%, and 36.33% in the total profits of a microgrid compared with the Deep Q-network algorithm, the state-action-reward-state-action algorithm, the proximal policy optimization algorithm, the ant-colony based algorithm, a microgrid energy optimization management strategy based on the genetic algorithm and the fuzzy inference system, and the theoretical retailer stragety, respectively. Additionally, com-pared with other methods and strategies, the proposed method can learn more optimal microgrid energy management behaviors and anticipate fluctuations in electricity prices and demand.