Online Optimization Research Articles

Digital Twin Technology of Human–Machine Integration in Cross-Belt Sorting System

The Chinese express delivery industry processes nearly 110 billion items in 2022, averaging an annual growth rate of 200%. Among the various types of sorting systems used for handling express items, cross-belt sorting systems stand out as the most crucial. However, despite their high degree of automation, the workload for operators has intensified owing to the surging volume of express items. In the era of Industry 5.0, it is imperative to adopt new technologies that not only enhance worker welfare but also improve the efficiency of cross-belt systems. Striking a balance between efficiency in handling express items and operator well-being is challenging. Digital twin technology offers a promising solution in this respect. A realization method of a human–machine integrated digital twin is proposed in this study, enabling the interaction of biological human bodies, virtual human bodies, virtual equipment, and logistics equipment in a closed loop, thus setting an operating framework. Key technologies in the proposed framework include a collection of heterogeneous data from multiple sources, construction of the relationship between operator fatigue and operation efficiency based on physiological measurements, virtual model construction, and an online optimization module based on real-time simulation. The feasibility of the proposed method was verified in an express distribution center.

Cyber-Physical Systems for High-Performance Machining of Difficult to Cut Materials in I5.0 Era-A Review.

The fifth Industrial revolution (I5.0) prioritizes resilience and sustainability, integrating cognitive cyber-physical systems and advanced technologies to enhance machining processes. Numerous research studies have been conducted to optimize machining operations by identifying and reducing sources of uncertainty and estimating the optimal cutting parameters. Virtual modeling and Tool Condition Monitoring (TCM) methodologies have been developed to assess the cutting states during machining processes. With a precise estimation of cutting states, the safety margin necessary to deal with uncertainties can be reduced, resulting in improved process productivity. This paper reviews the recent advances in high-performance machining systems, with a focus on cyber-physical models developed for the cutting operation of difficult-to-cut materials using cemented carbide tools. An overview of the literature and background on the advances in offline and online process optimization approaches are presented. Process optimization objectives such as tool life utilization, dynamic stability, enhanced productivity, improved machined part quality, reduced energy consumption, and carbon emissions are independently investigated for these offline and online optimization methods. Addressing the critical objectives and constraints prevalent in industrial applications, this paper explores the challenges and opportunities inherent to developing a robust cyber-physical optimization system.

Three-dimensional finite element-based data-driven thermal model of in-situ automated fiber placement

This study proposes a data-driven model for thermal history prediction during in-situ consolidation of thermoplastic composites using Hot Gas Torch (HGT)-assisted Automated Fiber Placement (AFP). Temperature data was generated using a three-dimensional finite element (FE) model that was fitted to and validated with experimentally measured temperatures. Temperature curves for various combinations of heat source velocity and locations through the thickness and width were extracted from the fitted FE model. Three feedforward neural networks were trained to predict the temperature distribution during processing. The work demonstrates the effectiveness and potential of using neural networks for online prediction and optimization.

Online Optimization of Vehicle-to-Grid Scheduling to Mitigate Battery Aging

The penetration of electric vehicles (EVs) in vehicle-to-grid (V2G) interaction can effectively assist the grid in achieving frequency regulation and peak load balancing. However, the customer perceives that participating in V2G services would result in the additional cycling of the battery and the accelerated aging of the EVs’ power battery, which has become a major obstacle to the widespread adoption of V2G services. Most existing methods require long-term cycling data and battery parameters to quantify battery aging, which is not suitable for the V2G scenario with large-scale and short-time intervals. Consequently, the real-time and accurate quantification of battery aging for optimization remains a challenge. This study proposes a charging scheduling method for EVs that can accurately and online quantify battery aging. Firstly, V2G scheduling is formulated as an optimization problem by defining an online sliding window to collect real-time vehicle information on the grid, enabling online optimization. Secondly, battery aging is more accurately quantified by proposing a novel amplitude-based rain-flow cycle counting (MRCC) method, which utilizes the charging information of the battery within a shorter time period. Lastly, an intelligent optimization algorithm is employed to optimize the charging and discharging power of EVs, aiming to minimize grid fluctuations and battery aging. The proposed method is validated using a V2G scenario with 50 EVs with randomly generated behaviors, and the results demonstrate that the proposed online scheduling method not only reduces the EFCC of the battery by 8.4%, but also achieves results close to global optimization.

Risk Early Warning of a Dynamic Ideological and Political Education System Based on LSTM-MLP: Online Education Data Processing and Optimization

Risk Early Warning of a Dynamic Ideological and Political Education System Based on LSTM-MLP: Online Education Data Processing and Optimization

A self‐learning human‐machine cooperative control method based on driver intention recognition

AbstractHuman‐machine cooperative control has become an important area of intelligent driving, where driver intention recognition and dynamic control authority allocation are key factors for improving the performance of cooperative decision‐making and control. In this paper, an online learning method is proposed for human‐machine cooperative control, which introduces a priority control parameter in the reward function to achieve optimal allocation of control authority under different driver intentions and driving safety conditions. Firstly, a two‐layer LSTM‐based sequence prediction algorithm is proposed to recognise the driver's lane change (LC) intention for human‐machine cooperative steering control. Secondly, an online reinforcement learning method is developed for optimising the steering authority to reduce driver workload and improve driving safety. The driver‐in‐the‐loop simulation results show that our method can accurately predict the driver's LC intention in cooperative driving and effectively compensate for the driver's non‐optimal driving actions. The experimental results on a real intelligent vehicle further demonstrate the online optimisation capability of the proposed RL‐based control authority allocation algorithm and its effectiveness in improving driving safety.

Detecting community on social networks with fast and optimal online clustering algorithms

Detecting community on social networks with fast and optimal online clustering algorithms

Dynamic metaheuristic selection via Thompson Sampling for online optimization

It is acknowledged that no single heuristic can outperform all the others in every optimization problem. This has given rise to hyper-heuristic methods for providing solutions to a wider range of problems. In this work, a set of five non-competing low-level heuristics is proposed in a hyper-heuristic framework. The multi-armed bandit problem analogy is efficiently leveraged and Thompson Sampling is used to actively select the best heuristic for online optimization. The proposed method is compared against ten population-based metaheuristic algorithms on the well-known CEC’05 optimizing benchmark consisting of 23 functions of various landscapes. The results show that the proposed algorithm is the only one able to find the global minimum of all functions with remarkable consistency.

Sequential track fusion in multi-sensor networks of unscented Kalman filters: A case of slip estimation in planetary mobile robots

Accurate wheel slip estimation facilitates Wheeled Mobile Robots (WMRs) with improved localization and traversability monitoring which are crucial to their autonomy in challenging planetary environments. Although distributed track-level fusion offers better computational efficiency, often sensor-level fusion is adopted in slip estimators of planetary rovers. Extending upon our earlier work, we develop a novel explicit track-to-track fusion algorithm for multi-sensor networks of unscented Kalman filters that has immediate application to slip ratio estimation in WMRs. The algorithm demonstrates better consistency compared to the existing fusion techniques since it implements a sub-optimal fusion rule to sequentially combine local tracks. It also saves computational power due to the recursive propagation of cross-covariance matrices based on the statistical linearization technique, instead of performing online optimizations. We prove that this recursion only requires the information of the first and last tracks in a sequence if all local tracks are unbiased. We rigorously study various key properties of the developed fusion algorithm and show its superior level of confidence when compared to the two prominent fusion methods of sequential and batch covariance intersection. The slip ratio estimation in a six-wheel WMR is considered as a case study, where the steerable wheel sets act as a network of sensors. The proposed slip estimator works based on the rigid body kinematics and readings of purely proprioceptive sensors, i.e., an inertial measurement unit and encoders. The slip estimator’s performance is evaluated in a high-fidelity software-in-the-loop simulation environment connecting MATLAB and CM Lab’s Vortex Studio software. In a comparison study that considers four rival strategies, we show the superiority of the proposed fusion method offering a balance between consistency, accuracy, and speed in real-time slip estimations.

Percentile Risk-Constrained Budget Pacing for Guaranteed Display Advertising in Online Optimization

Guaranteed display (GD) advertising is a critical component of advertising since it provides publishers with stable revenue and enables advertisers to target specific audiences with guaranteed impressions. However, smooth pacing control for online ad delivery presents a challenge due to significant budget disparities, user arrival distribution drift, and dynamic change between supply and demand. This paper presents robust risk-constrained pacing (RCPacing) that utilizes Lagrangian dual multipliers to fine-tune probabilistic throttling through monotonic mapping functions within the percentile space of impression performance distribution. RCPacing combines distribution drift resilience and compatibility with guaranteed allocation mechanism, enabling us to provide near-optimal online services. We also show that RCPacing achieves O(sqrt(T)) dynamic regret where T is the length of the horizon. RCPacing's effectiveness is validated through offline evaluations and online A/B testing conducted on Taobao brand advertising platform.

Online Submodular Maximization via Online Convex Optimization

We study monotone submodular maximization under general matroid constraints in the online setting. We prove that online optimization of a large class of submodular functions, namely, threshold potential functions, reduces to online convex optimization (OCO). This is precisely because functions in this class admit a concave relaxation; as a result, OCO policies, coupled with an appropriate rounding scheme, can be used to achieve sublinear regret in the combinatorial setting. We also show that our reduction extends to many different versions of the online learning problem, including the dynamic regret, bandit, and optimistic-learning settings.

Online Markov Decision Processes Configuration with Continuous Decision Space

In this paper, we investigate the optimal online configuration of episodic Markov decision processes when the space of the possible configurations is continuous. Specifically, we study the interaction between a learner (referred to as the configurator) and an agent with a fixed, unknown policy, when the learner aims to minimize her losses by choosing transition functions in online fashion. The losses may be unrelated to the agent's rewards. This problem applies to many real-world scenarios where the learner seeks to manipulate the Markov decision process to her advantage. We study both deterministic and stochastic settings, where the losses are either fixed or sampled from an unknown probability distribution. We design two algorithms whose peculiarity is to rely on occupancy measures to explore with optimism the continuous space of transition functions, achieving constant regret in deterministic settings and sublinear regret in stochastic settings, respectively. Moreover, we prove that the regret bound is tight with respect to any constant factor in deterministic settings. Finally, we compare the empiric performance of our algorithms with a baseline in synthetic experiments.

Online delay optimization for MEC and RIS-assisted wireless VR networks

Online delay optimization for MEC and RIS-assisted wireless VR networks

Optimized energy management for plug-in hybrid vehicles with predicted driving cycles1

The traditional rule-based energy management strategy for plug-in hybrid vehicles has issues, such as difficulty in online correction and limited online optimization capabilities. In addition, the global optimization energy management strategy cannot be applied online or in real-time. Considering the above difficulties, this study proposes a real-time optimization energy management strategy based on the Markov chain for driving condition prediction and online optimization with the minimum principle. To verify the proposed control strategy, the plug-in hybrid vehicle dynamics model, driving condition prediction model, and online optimization control model were first established. The initial value of the battery state of charge was set to 0.4 under the UDDS (Urban Dynamometer Driving Schedule) standard cycle. The simulation results showed that the comprehensive fuel consumption cost was 1.66 yuan, which was 8.28% better than the energy economy of the traditional rule-based energy management strategy. At the same time, a complete vehicle test was also conducted based on a sample vehicle test platform. The experimental results indicated that the energy management strategy proposed herein exhibits better fuel economy compared to that exhibited by the traditional rule-based energy management strategy. Simulations and experiments have verified the effectiveness of the proposed control strategy in this study.

A Hybrid Approach for Home Energy Management With Imitation Learning and Online Optimization

A Hybrid Approach for Home Energy Management With Imitation Learning and Online Optimization

Double-Layer RRT* Objective Bias Anytime Motion Planning Algorithm

This paper proposes a double-layer structure RRT* algorithm based on objective bias called DOB-RRT*. The algorithm adopts an initial path with an online optimization structure for motion planning. The first layer of RRT* introduces a feedback-based objective bias strategy with segment forward pruning processing to quickly obtain a smooth initial path. The second layer of RRT* uses the heuristics of the initial tree structure to optimize the path by using reverse maintenance strategies. Compared with conventional RRT and RRT* algorithms, the proposed algorithm can obtain the initial path with high quality, and it can quickly converge to the progressive optimal path during the optimization process. The performance of the proposed algorithm is effectively evaluated and tested in real experiments on an actual wheeled robotic vehicle running ROS Kinetic in a real environment.

Optimization-Based Ramping Reserve Allocation of BESS for AGC Enhancement

The transient behavior of Automatic Generation Control (AGC) systems is a critical aspect of power system operation. Therefore, fully extracting the potential of Battery Energy Storage Systems (BESSs) for AGC enhancement is of paramount importance. In light of the challenges posed by diverse resource interconnections and the variability associated, we propose an online optimization scheme that can adapt to changes in an unknown and variable environment. To leverage the synergy between BESSs and Conventional Generators (CGs), we devise a variant of the Area Injection Error (AIE) as a measure to quantify the ramping needs. Based on this measure, we develop a distributed optimization algorithm with adaptive learning rates for the allocation of the ramping reserve. The algorithm restores a larger learning rate for compliance with the ramping needs upon detecting a potentially destabilizing event. We demonstrate the effectiveness and scalability of the proposed scheme through comprehensive case studies. It is shown that the proposed scheme can improve the transient behavior of the AGC system by bridging the gap in ramping capability.

Representation Matters: Trust in Digital Health Information Among Black Patients With Prostate Cancer.

Although the majority of US adults obtain health information on the internet, the quality of information about prostate cancer is highly variable. Black adults are underrepresented in online content about prostate cancer despite a higher incidence of and mortality from the disease. The goal of this study was to explore the perspectives of Black patients with prostate cancer on the importance of racial representation in online content and other factors influencing trust. We conducted 7 virtual focus groups with Black patients with prostate cancer in 2022 and 2023. Participants completed an intake questionnaire with demographics followed by a group discussion, including feedback on purposefully selected online content. Transcripts were independently analyzed by 2 investigators experienced in qualitative research using a constant comparative method. Most participants use online sources to look for prostate cancer information. Racial representation is an important factor affecting trust in the content. A lack of Black representation has consequences, including misperceptions about a lower risk of prostate cancer and discouraging further information-seeking. Other key themes affecting trust in online content included the importance of a reputable source of information, professional website structure, and soliciting money. Underrepresentation of Black adults in prostate cancer content has the potential to worsen health disparities. Optimal online communications should include racially diverse representation and evidence-based information in a professional format from reputable sources without financial conflict.

Online Joint Optimization of Virtual Network Function Deployment and Trajectory Planning for Virtualized Service Provision in Multiple-Unmanned-Aerial-Vehicle Mobile-Edge Networks

The multiple-unmanned-aerial-vehicle (multi-UAV) mobile edge network is a promising networking paradigm that uses multiple resource-limited and trajectory-planned unmanned aerial vehicles (UAVs) as edge servers, upon which on-demand virtual network functions (VNFs) are deployed to provide low-delay virtualized network services for the requests of ground users (GUs), who often move randomly and have difficulty accessing the Internet. However, VNF deployment and UAV trajectory planning are both typical NP-complete problems, and the two operations have a strong coupling effect: they affect each other. Achieving optimal virtualized service provision (i.e., maximizing the number of accepted GU requests under a given period T while minimizing the energy consumption and the cost of accepting the requests in all UAVs) is a challenging issue. In this paper, we propose an improved online deep reinforcement learning (DRL) scheme to tackle this issue. First, we formulate the joint optimization of the two operations as a nonconvex mixed-integer nonlinear programming problem, which can be viewed as a sequence of one-frame joint VNF deployment and UAV-trajectory-planning optimization subproblems. Second, we propose an online DRL based on jointly optimizing discrete (VNF deployment) and continuous (UAV trajectory planning) actions to solve each subproblem, whose key idea is establishing and achieving the coupled influence of discrete and continuous actions. Finally, we evaluate the proposed scheme through extensive simulations, and the results demonstrate its effectiveness.

Smart Gait: A Gait Optimization Framework for Hexapod Robots

The current gait planning for legged robots is mostly based on human presets, which cannot match the flexible characteristics of natural mammals. This paper proposes a gait optimization framework for hexapod robots called Smart Gait. Smart Gait contains three modules: swing leg trajectory optimization, gait period & duty optimization, and gait sequence optimization. The full dynamics of a single leg, and the centroid dynamics of the overall robot are considered in the respective modules. The Smart Gait not only helps the robot to decrease the energy consumption when in locomotion, mostly, it enables the hexapod robot to determine its gait pattern transitions based on its current state, instead of repeating the formalistic clock-set step cycles. Our Smart Gait framework allows the hexapod robot to behave nimbly as a living animal when in 3D movements for the first time. The Smart Gait framework combines offline and online optimizations without any fussy data-driven training procedures, and it can run efficiently on board in real-time after deployment. Various experiments are carried out on the hexapod robot LittleStrong. The results show that the energy consumption is reduced by 15.9% when in locomotion. Adaptive gait patterns can be generated spontaneously both in regular and challenge environments, and when facing external interferences.