Heuristic Control Research Articles

Dynamic Robot Assignment for Flexible Serial Production Systems

This letter aims at modeling and real-time control of a flexible manufacturing system (FMS) that is operated by multi-skilled mobile robots. By introducing the idea of a unique ideal clean configuration and the effective disruption event, the concepts of Opportunity Window (OW) and Permanent Production Loss (PPL) are extended to the FMS. The control considered in this letter is a robot assignment problem where individual robots are dynamically assigned to each workstation in real time to improve performance. The problem is formulated as a Markov Decision Process (MDP) and solved using the Double Deep Q-Network (DDQN) reinforcement learning algorithm. PPL is used as the reward setting for training and a space discretization is used between each workstation to capture real time movement of the robots on the plant floor. The effectiveness of the control strategy is then demonstrated by comparing the results to several heuristic control schemes.

Calibrating a Real-time Energy Management for a Heavy-Duty Fuel Cell Electrified Truck towards Improved Hydrogen Economy

<div class="section abstract"><div class="htmlview paragraph">Fuel cell electrified powertrains are currently a promising technology towards decarbonizing the heavy-duty transportation sector. In this context, extensive research is required to thoroughly assess the hydrogen economy potential of fuel cell heavy-duty electrification. This paper proposes a real-time capable energy management strategy (EMS) that can achieve improved hydrogen economy for a fuel cell electrified heavy-duty truck. The considered heavy-duty truck is modelled first in Simulink® environment. A baseline heuristic map-based controller is then retained that can instantaneously control the electrical power split between fuel cell system and the high-voltage battery pack of the heavy-duty truck. Particle swarm optimization (PSO) is consequently implemented to optimally tune the parameters of the considered EMS. For the aim of this study, the calibration optimization objective involves minimizing the hydrogen consumption estimated by simulating the heavy-duty truck in the Simulink® model. Simulations entail different driving missions, some of which have been generated by using the VECTO software, i.e. the tool used in Europe to certify the CO2 emissions of new heavy-duty vehicles. Furthermore, dynamic programming (DP) is implemented as an off-line reference EMS approach that can identify the global optimal control trajectory over time by knowing the entire driving mission in advance. The real-time EMS calibrated by means of PSO is demonstrated achieving remarkable hydrogen saving potential, which results being only around 5% worse compared with the global optimal benchmark provided by DP.</div></div>

Site demonstration and performance evaluation of MPC for a large chiller plant with TES for renewable energy integration and grid decarbonization

Thermal energy storage (TES) for a cooling plant is a crucial resource for load flexibility. Traditionally, simple, heuristic control approaches, such as the storage priority control which charges TES during the nighttime and discharges during the daytime, have been widely used in practice, and shown reasonable performance in the past benefiting both the grid and the end-users such as buildings and district energy systems. However, the increasing penetration of renewables changes the situation, exposing the grid to a growing duck curve, which encourages the consumption of more energy in the daytime, and volatile renewable generation which requires dynamic planning. The growing pressure of diminishing greenhouse gas emissions also increases the complexity of cooling TES plant operations as different control strategies may apply to optimize operations for energy cost or carbon emissions. This paper presents a model predictive control (MPC), site demonstration and evaluation results of optimal operation of a chiller plant, TES and behind-meter photovoltaics for a campus-level district cooling system. The MPC was formulated as a mixed-integer linear program for better numerical and control properties. Compared with baseline rule-based controls, the MPC results show reductions of the excess PV power by around 25%, of the greenhouse gas emission by 10%, and of peak electricity demand by 10%.

Learning-Based Control Policy and Regret Analysis for Online Quadratic Optimization With Asymmetric Information Structure.

In this article, we propose a learning approach to analyze dynamic systems with an asymmetric information structure. Instead of adopting a game-theoretic setting, we investigate an online quadratic optimization problem driven by system noises with unknown statistics. Due to information asymmetry, it is infeasible to use the classic Kalman filter nor optimal control strategies for such systems. It is necessary and beneficial to develop an admissible approach that learns the probability statistics as time goes forward. Motivated by the online convex optimization (OCO) theory, we introduce the notion of regret, which is defined as the cumulative performance loss difference between the optimal offline-known statistics cost and the optimal online-unknown statistics cost. By utilizing dynamic programming and linear minimum mean square biased estimate (LMMSUE), we propose a new type of online state-feedback control policy and characterize the behavior of regret in a finite-time regime. The regret is shown to be sublinear and bounded by O(lnT) . Moreover, we address an online optimization problem with output-feedback control policy and propose a heuristic online control policy.

System Identification and Heuristic Control of Segmented Ailerons for Enhanced Stability of Fixed Wing UAVs

Different from conventional aircraft, an investigation on system identification and control design has been carried out on a small fixed wing unmanned aerial vehicle (UAV) with a multi-segment ailerons. The multi-segment aileron setup is configured as a multi-input and single-output system and each segment is modeled as a control input. Experiments are conducted in a wind tunnel to determine the frequency responses of the system and the corresponding transfer functions. Multiple PID controllers are designed and implemented in a cascaded form for each control surface. Furthermore, a heuristic switching control strategy is implemented for the aircraft where the multi-segment ailerons perform as a single-segment aileron in a normal flight condition, and adapts to multi-segment control when encountering severe turbulence or a large angle reference change. Experimental results reveal that although each control surface has the capability for stabilization of the aircraft, the proposed control strategy by combining the multiple actuation surfaces reduces the mean squared errors for the roll angle up to $37$ percent in the highly turbulent environment providing superior disturbance rejection properties to the aircraft.

Robust microgrid energy trading and scheduling under budgeted uncertainty

The advent of smart grids came with several technological developments including new electricity market rules and regulation mechanisms. Microgrids can trade energy with the main grid to either sell its production surplus (from renewable energy sources) or buy an additional amount to support local consumers’ demand, which includes flexible loads, such as smart appliances and electric vehicles. In this scenario, smart control devices are important elements, executing real-time energy scheduling according to fluctuations in production and consumption. As we might expect, the main grid’s power generation and supply becomes more unscheduled and risky as energy trading quantities oscillate over time. This work studies a flexible energy contract subscription framework, coupled with a real-time command strategy, suited for energy scheduling of microgrids with uncertainty in both production and consumption. Our main contributions are a Robust Optimization model under budgeted uncertainty for contract subscription and a set of heuristic control strategies for the real-time energy scheduling. The robust model is capable of providing solutions for multi-period-ahead trading of energy, while minimizing the worst-case cost. We run an extensive computational case study on a real microgrid instance to confirm the efficacy of our solution approach.

Assessing the Usability of a Clinical Decision Support System: Heuristic Evaluation.

BackgroundPoor usability is a primary cause of unintended consequences related to the use of electronic health record (EHR) systems, which negatively impacts patient safety. Due to the cost and time needed to carry out iterative evaluations, many EHR components, such as clinical decision support systems (CDSSs), have not undergone rigorous usability testing prior to their deployment in clinical practice. Usability testing in the predeployment phase is crucial to eliminating usability issues and preventing costly fixes that will be needed if these issues are found after the system’s implementation.ObjectiveThis study presents an example application of a systematic evaluation method that uses clinician experts with human-computer interaction (HCI) expertise to evaluate the usability of an electronic clinical decision support (CDS) intervention prior to its deployment in a randomized controlled trial.MethodsWe invited 6 HCI experts to participate in a heuristic evaluation of our CDS intervention. Each expert was asked to independently explore the intervention at least twice. After completing the assigned tasks using patient scenarios, each expert completed a heuristic evaluation checklist developed by Bright et al based on Nielsen’s 10 heuristics. The experts also rated the overall severity of each identified heuristic violation on a scale of 0 to 4, where 0 indicates no problems and 4 indicates a usability catastrophe. Data from the experts’ coded comments were synthesized, and the severity of each identified usability heuristic was analyzed.ResultsThe 6 HCI experts included professionals from the fields of nursing (n=4), pharmaceutical science (n=1), and systems engineering (n=1). The mean overall severity scores of the identified heuristic violations ranged from 0.66 (flexibility and efficiency of use) to 2.00 (user control and freedom and error prevention), in which scores closer to 0 indicate a more usable system. The heuristic principle user control and freedom was identified as the most in need of refinement and, particularly by nonnursing HCI experts, considered as having major usability problems. In response to the heuristic match between system and the real world, the experts pointed to the reversed direction of our system’s pain scale scores (1=severe pain) compared to those commonly used in clinical practice (typically 1=mild pain); although this was identified as a minor usability problem, its refinement was repeatedly emphasized by nursing HCI experts.ConclusionsOur heuristic evaluation process is simple and systematic and can be used at multiple stages of system development to reduce the time and cost needed to establish the usability of a system before its widespread implementation. Furthermore, heuristic evaluations can help organizations develop transparent reporting protocols for usability, as required by Title IV of the 21st Century Cures Act. Testing of EHRs and CDSSs by clinicians with HCI expertise in heuristic evaluation processes has the potential to reduce the frequency of testing while increasing its quality, which may reduce clinicians’ cognitive workload and errors and enhance the adoption of EHRs and CDSSs.

MPC solution for optimal load shifting for buildings with ON/OFF staged packaged units: Experimental demonstration, and lessons learned

Small and medium-sized commercial buildings (SMCB) are significant demand response resources, and it is important to develop grid-responsive control algorithms that exploit those resources and create financial benefits for building owners and HVAC service providers. Furthermore, unlike large-sized commercial buildings, there is an opportunity to have universally applicable control solutions for many SMCBs since those buildings have a consistent HVAC system configuration: SMCBs are commonly served by multiple-staged air conditioning units controlled by their own thermostats. Despite the demand response potential and scalability, however, very few control solutions are available for SMCBs. Typical model predictive control (MPC) and heuristic control approaches for cooling load shifting that lower thermostat setpoints before an electric price jump are suitable mainly for large-sized commercial buildings where a continuous capacity modulation is possible, e.g., via dampers in variable air volume terminal units. However, those approaches can cause undesired, high peaks for SMCBs due to the nature of ON/OFF unit staging and narrow thermostat deadbands. This could discourage the use of advanced grid-responsive controls for SMCBs due to the concern of high demand charges, and has to be resolved. This paper presents a MPC solution that overcomes this challenge. It has a hierarchical MPC structure where an upper level MPC is responsible for electrical load shifting in response to an electric price signal while a lower level MPC is responsible for coordinating compressor stages to eliminate unnecessary peaks and follows the setpoints determined by the upper level MPC. Two one-month, comprehensive laboratory tests have been carried out to demonstrate load shifting and cost savings for the algorithm. Interesting trade-offs between energy efficiency and load flexibility were observed and are discussed, and lessons learned for applying MPCs for SMCBs are also presented.

Efficient simulation for an open-pit mine

This paper presents an approach towards computationally efficient discrete-event-simulation designed to take advantage of high-performance computing architecture. We demonstrate how computational efficiency combined with large computing capacity enables quantification and mapping of stochastic system performance across a computationally challenging multidimensional parameter space requiring several million individual DES instances. An open-pit mine bulk materials handling system case study is presented to demonstrate the approach. A dispatching heuristic controlling the operation of the trucking system can be optimised through the selection of values for four control parameters. The multidimensional space represented by these control parameters is exhaustively mapped and visualised to provide insight into the relationships between combinations of parameters and system performance. The extensive data generated is ultimately used within a hyper-heuristic to determine near-optimal values for the dispatching heuristic control parameters.

Modelling and condition-based control of a flexible and hybrid disassembly system with manual and autonomous workstations using reinforcement learning

Remanufacturing includes disassembly and reassembly of used products to save natural resources and reduce emissions. While assembly is widely understood in the field of operations management, disassembly is a rather new problem in production planning and control. The latter faces the challenge of high uncertainty of type, quantity and quality conditions of returned products, leading to high volatility in remanufacturing production systems. Traditionally, disassembly is a manual labor-intensive production step that, thanks to advances in robotics and artificial intelligence, starts to be automated with autonomous workstations. Due to the diverging material flow, the application of production systems with loosely linked stations is particularly suitable and, owing to the risk of condition induced operational failures, the rise of hybrid disassembly systems that combine manual and autonomous workstations can be expected. In contrast to traditional workstations, autonomous workstations can expand their capabilities but suffer from unknown failure rates. For such adverse conditions a condition-based control for hybrid disassembly systems, based on reinforcement learning, alongside a comprehensive modeling approach is presented in this work. The method is applied to a real-world production system. By comparison with a heuristic control approach, the potential of the RL approach can be proven simulatively using two different test cases.

A Comprehensive Decomposition Based Hierarchical Heuristic Control of Multimicrogrids

This paper proposes a novel decomposition based hierarchical heuristic control of multimicrogrids to maximize the utilization of renewable energy sources and load satisfaction. The proposed control is capable of providing comprehensive online dispatch of active and reactive powers in steady state modes of operation according to IEEE 2030.7 while keeping in view the restrictions of the participants (microsources and/or microgrids). The proposed solution is generalized to be adopted with respect to different types of participants and grid structures, modular with respect to the number of its participants and capable of protecting the privacy of its participants to maximum possible extent. The proposed heuristic control decomposes the control problem at four levels of hierarchy namely microsource, unit, microgrid and multimicrogrid level. All the distributed energy sources have their own microsource controllers and all the distributed energy sources of the same type are put under one unit controller. All the unit controllers are put under a microgrid controller and all the microgrid controllers are put under a multimicrogrid controller. Information flows from lower to upper levels while dispatch commands flow vice versa. This provides a comprehensive, generic, modular and secure control of a multimicrogrid system. Test cases are simulated to verify the validity of the proposed control scheme.

Evaluation of the performance of ventilative cooling systems with different control parameters

To evaluate the performance and energy savings of different ventilative cooling systems, three mixed forms of ventilative cooling were described and the most promising rule-based heuristic control criteria were used as applicable control strategies based on various control parameters. Parametric analyses including the ventilative cooling set-points and control frequencies have been conducted. EnergyPlus was used to perform the aforementioned analyses based on a typical office building. Case studies verified that the use of different control parameters led to different indoor air environment and energy performance in a certain climatic zone. Then, the percentages outside the range indexes (POR) of the three mixed forms of ventilative cooling with various control parameters in five representative climatic zones in China were calculated, the results demonstrated that the ventilative cooling performance was mainly related to the ventilative cooling form in HSCW, HSWW, and temperate climatic zones. While in cold and severe cold climatic zones, it was mainly affected by selection of control parameters and its corresponding set-points. It can provide guidance for the selection of ventilative cooling scheme capable of maintaining indoor environment level with a low energy consumption in different climatic zones of China.

A Heuristic Strategy for Cognitive State-based Feedback Control to Accelerate Human Learning

Autonomous systems are increasingly being used for the purpose of training humans to attain new skills or perform new tasks. In these contexts, autonomous systems should be responsive to, and guide, human behavior such that skill or task performance is maximized. These systems generally rely on human performance to determine if assistance is needed. However, it is recognized that these systems should also respond to human cognitive factors, such as self–confidence, that are relevant for human learning. We propose and experimentally validate a heuristic control strategy, based on both a user's performance and self-reported self-confidence as they, that determines whether or not they receive automated assistance in learning how to land a quadrotor in a simulated environment. Through a human subject study involving a benchmark strategy that is solely performance-based, we show that the proposed strategy not only successfully calibrates the self-confidence of the participants, but also leads to statistically significant improvements in participants’ task performance and consistency after 20 trials relative to outcomes for participants who experience the benchmark strategy.

Verification and improvement of flexible mathematical procedures for co-optimizing design and control of fuel cell hybrid vehicles

A study on the usefulness of flexible mathematical tools for determining the optimal architecture of fuel cell hybrid vehicles is presented. Starting from a pre-existing powertrain and control strategies co-optimization tool, the technological (especially in terms of lithium battery type) search domain was first expanded by including an updated battery model. Afterward, the availability of specification independent control strategies was exploited in such a way as to enable two optimization tasks: one relying on previous heuristic control rules and the other based on newly optimized control strategies. The results evidenced negligible differences, in terms of key control variable trends, objective (i.e., fuel economy), and design parameters (i.e., fuel cell system size and battery energy density), thus further proving the tool versatility. Moreover, optimal configurations exhibit appreciable fuel economies and acceleration performance on the WLTP driving cycle, while proposing potentially cost-effective solutions in terms of fuel cell system size.

A robust approach to establish tolerance limits for the gamma passing rate-based patient-specific quality assurance using the heuristic control charts.

Establishing the tolerance limits of patient-specific quality assurance (PSQA) processes based on the gamma passing rate (GPR) by using normal statistical process control (SPC) methods involves certain problems. The aim of this study was threefold: (a) to show that the heuristic SPC method can replace the quantile method for establishing tolerance limits in PSQA processes and is more robust, (b) to introduce an iterative procedure of "Identify-Eliminate-Recalculate" for establishing the tolerance limits in PSQA processes with unknown states based on retrospective GPRs, and (c) to recommend a workflow to define tolerance limits based on actual clinical retrospective GPRs. A total of 1671 volumetric-modulated arc therapy (VMAT) pretreatment plans were measured on four linear accelerators (linacs) and analyzed by treatment sites using the GPRs under the 2%/2mm, 3%/2mm, and 3%/3mm criteria. Normality testing was performed using the Anderson-Darling (AD) statistic and the optimal distributions of GPRs were determined using the Fitter Python package. The iterative "Identify-Eliminate-Recalculate" procedure was used to identify the PSQA outliers. The tolerance limits of the initial PSQAs, remaining PSQAs after elimination, and in-control PSQAs after correction were calculated using the conventional Shewhart method, two transformation methods, three heuristic methods, and two quantile methods. The tolerance limits of PSQA processes with different states for the respective methods, linacs, and treatment sites were comprehensively compared and analyzed. It was found that 75% of the initial PSQA processes and 63% of the in-control processes were non-normal (AD test, p<0.05). The optimal distributions of GPRs for the initial and in-control PSQAs varied with different linacs and treatment sites. In the implementation of the "Identify-Eliminate-Recalculate" procedure, the quantile methods could not identify the out-of-control PSQAs effectively due to the influence of outliers. The tolerance limits of the in-control PSQAs, calculated using the quantile of optimal fitting distributions, represented the ground truth. The tolerance limits of the in-control PSQAs and remaining PSQAs after elimination calculated using the heuristic methods were considerably close to the ground truth (the maximum average absolute deviations were 0.50 and 1.03%, respectively). Some transformation failures occurred under both transformation methods. For the in-control PSQAs at 3%/2mm gamma criteria, the maximum differences in the tolerance limits for four linacs and different treatment sites were 3.10 and 5.02%, respectively. The GPR distributions of PSQA processes vary with different linacs and treatment sites but most are skewed. In applying SPC methodologies to PSQA processes, heuristic methods are robust. For in-control PSQA processes, the tolerance limits calculated by heuristic methods are in good agreement with the ground truth. For unknown PSQA processes, the tolerance limits calculated by the heuristic methods after the iterative "Identify-Eliminate-Recalculate" procedure are closest to the ground truth. Setting linac- and treatment site-specific tolerance limits for PSQA processes is necessary for clinical applications.

Utilizing commercial heating, ventilating, and air conditioning systems to provide grid services: A review

The modern power grid faces multiple challenges due to an increase in the adoption of renewable generation, such as dynamically balancing supply and demand at different time scales. Demand side management in buildings plays a vital role in achieving this balance because buildings can provide grid services through a variety of building assets. However, the development of grid-interactive, efficient buildings is still in its infancy, and a systematic and holistic understanding of grid service delivery strategies in terms of energy efficiency, load shifting, load shedding and load modulating is still limited. This paper is a comprehensive review of the development and application of building-level control strategies for utilizing heating, ventilating, and air conditioning systems to provide grid services. These strategies have been investigated through numerical and experimental studies. Control algorithms, such as heuristic rule-based control and model-based control, have been used to enable the automatic control delivery of grid services. The advantages and disadvantages of the strategies are summarized and discussed. Research trends are also identified, which include considering predicted mean vote-based and occupant-based thermal comfort, modeling of occupant behavior, integrating power grid operations with building control, and combining different demand flexibility modes in the control design.

Robust control of fed-batch high-cell density cultures: a simulation-based assessment

High-cell density cultures (HCDC), prone to metabolic overflow, are typically operated in fed-batch mode to maximize productivity. We developed a simulation-based procedure for assessing advanced control strategies for HCDC, aiming to avoid metabolic overflow. We tested a heuristic controller, a nonlinear model predictive controller (NMPC), and two adaptive controllers under realistic conditions. As a case study, we considered the growth of a carotenoid-producing Saccharomyces cerevisiae strain developed in our laboratory. Our control simulations were consistent with experimental results reported in the literature with equivalent control strategies. All tested controllers could cope well with measurement noise and model parameter variations, reaching low ethanol concentrations (≤ 1 g/L), albeit with substantially different biomass productivities. The NMPC algorithm yielded the best performance even under parametric uncertainty, achieving high biomass concentrations (200 gDCW/L) and cell productivities (5 gDCW/L/h). The presented systematic procedure helps in the evaluation of novel control strategies before their implementation in practice.

Model experiments with different cycloidal propeller algorithms using same electric controller

This article presents a methodology of controlling the cycloidal propeller electronically in model scale by an experimental approach. It establishes the function of single-unit controlled multiple propeller models such as Kirsten-Boeing and Voith-Schneider propeller. It also deals with the execution of different maneuvering cases like straight acceleration, autopilot, zigzag, crash stop. Blade motors independently control the pitching of the propeller-blades unlike the mechanism of the conventional cycloidal propeller. PID control and heuristic control are investigated for this purpose. Mathematical models for system identification, pulse generation, position, and speed control of propeller blades, and disc and synchronizations are demonstrated. Based on model experimental results, advantages/disadvantages of PID and heuristic control are discussed.

Human Belief State-Based Exploration and Exploitation in an Information-Selective Symmetric Reversal Bandit Task

Humans often face sequential decision-making problems, in which information about the environmental reward structure is detached from rewards for a subset of actions. In the current exploratory study, we introduce an information-selective symmetric reversal bandit task to model such situations and obtained choice data on this task from 24 participants. To arbitrate between different decision-making strategies that participants may use on this task, we developed a set of probabilistic agent-based behavioral models, including exploitative and explorative Bayesian agents, as well as heuristic control agents. Upon validating the model and parameter recovery properties of our model set and summarizing the participants’ choice data in a descriptive way, we used a maximum likelihood approach to evaluate the participants’ choice data from the perspective of our model set. In brief, we provide quantitative evidence that participants employ a belief state-based hybrid explorative-exploitative strategy on the information-selective symmetric reversal bandit task, lending further support to the finding that humans are guided by their subjective uncertainty when solving exploration-exploitation dilemmas.

Performance Evaluation of Heuristics and Meta-Heuristics Traffic Control Strategies Using the UTNSim Traffic Simulator

In the past few decades, intelligent traffic controllers have been developed to responsively cope with the increasing traffic demands and congestions in urban traffic networks. Various studies to compare and evaluate the performance of traffic controllers have been conducted to investigate its effect on traffic performances such as its ability to reduce delay time, stops, throughputs and queues within a traffic network. In this paper, the authors aim to present another comparative study on heuristics versus meta-heuristics traffic control methods. To our knowledge, such comparison has not been conducted and could provide insights into a purely heuristic controller compared to meta-heuristics. The study aims to answer the research question “Can heuristics traffic control strategies outperformed meta-heuristics in terms of performance and computational costs?” For this purpose, a heuristics model-based control strategy (MCS) which was previously developed by the authors is compared to genetic algorithms (GA) and evolution strategy (ES) respectively on a nine intersections symmetric network. These control strategies were implemented via simulations on a traffic simulator called UTNSim for three different types of traffic scenarios. Performance indices such as average delays, vehicle throughputs and the computational time of these controllers were evaluated. The results revealed that the heuristic MCS outperformed GA and ES with superior performance in average delays whereas vehicle throughputs were in close agreement. The computation time of the MCS is also feasible for real-time application compared to GA and ES that has longer convergent time.