Safe Set Research Articles

Conflict-Aware Safe Reinforcement Learning: A Meta-Cognitive Learning Framework

In this paper, a data-driven conflict-aware safe reinforcement learning (CAS-RL) algorithm is presented for control of autonomous systems. Existing safe RL results with predefined performance functions and safe sets can only provide safety and performance guarantees for a single environment or circumstance. By contrast, the presented CAS-RL algorithm provides safety and performance guarantees across a variety of circumstances that the system might encounter. This is achieved by utilizing a bilevel learning control architecture: A higher meta-cognitive layer leverages a data-driven receding-horizon attentional controller (RHAC) to adapt relative attention to different system's safety and performance requirements, and, a lower-layer RL controller designs control actuation signals for the system. The presented RHAC makes its meta decisions based on the reaction curve of the lower-layer RL controller using a meta-model or knowledge. More specifically, it leverages a prediction meta-model (PMM) which spans the space of all future meta trajectories using a given finite number of past meta trajectories. RHAC will adapt the system's aspiration towards performance metrics (e.g., performance weights) as well as safety boundaries to resolve conflicts that arise as mission scenarios develop. This will guarantee safety and feasibility (i.e., performance boundness) of the lower-layer RL-based control solution. It is shown that the interplay between the RHAC and the lower-layer RL controller is a bilevel optimization problem for which the leader (RHAC) operates at a lower rate than the follower (RL-based controller) and its solution guarantees feasibility and safety of the control solution. The effectiveness of the proposed framework is verified through a simulation example.

TEST OF PHYSIOLOGICAL PERFORMANCE: RATIONALE AND FEASIBILITY

Rigorous clinical evaluation of the physiological performance is currently performed with complex and long procedures which need expensive technology and skilled operators. In a wide range of situations (frail patients, daily clinical practice, etc.), these approaches are difficult to be applied and simpler tests, with a lack of scientific background, are mandatory. To avoid these problems, we propose a test (test of physiological performance (TOPP)) to evaluate the physiological behavior of a subject, in a really easy and safe clinical setting, measuring only the heart rate. The subject is submitted to an active standing-up test and then two submaximal exercises (with a low power load) on a cycle-ergometer. The heart rate modifications due to each submaximal step are analyzed by exponential interpolation to calculate the ascending and descending time constants and evaluate the way each subject adapts his heart rate to work. The standard deviation of the RR for each stationary phase (warm-up, load, recovery) was calculated as an index of short-term variability. Then a standard Fourier analysis of the stationary periods of the standing-up procedures allows to quickly and easily evaluate the autonomic nervous activation. We tested the protocol on five healthy subjects to verify the feasibility and the acceptance of the procedure. The five subjects demonstrated a good tolerance of the entire procedure. The standing-up showed a behavior of the autonomic system consistent with the physiology (with an increase in sympathetic activation in the passage to standing position). The analysis of the two submaximal steps highlights how younger and trained subjects present lower heart rates (both in the ascending phase and in the recovery) with a quicker adaptation ability (smaller time constants) consistent with what is expected. The short-term variability of heart rate is greater in young and trained subjects, thus confirming how the sympatho-vagal balance, in these subjects, is more dynamic. The proposed test is well tolerated by the subjects and the results, albeit in a small cohort of healthy volunteers, are consistent with what is expected from physiology and is already present in the literature. Our work aims to be a proposal with a feasibility check of a method for evaluating performance. The work to be done for the clinical validation of the TOPP is still long, but we are aware that it can give important results and that the TOPP can become an effective tool for the assessment of the physiological performance even of fragile subjects.

Process and Systems: Experience of using a regional network of hospitals in the care of older inpatients with COVID-19 in spring 2020

Frail, older patients with COVID-19 have an increased risk of hospital admission and death. We studied a regional model of care used for older patients with COVID-19 in spring 2020 across three settings: an acute teaching hospital, a district general hospital and a temporary emergency hospital. We also studied demographic and outcome data for these patients. Increasing bed capacity in non-acute sites freed up beds in acute hospitals. Strict admission criteria and multidisciplinary team involvement allowed for the safe delivery of care in step-down sites. This model of care allowed for patient flow out of acute sites following the acute stage of their illness allowing for an increase in bed capacity while providing a safe setting for ongoing management.

Co-Desistance From Crime: Engaging the Pro-Social Dimensions of Co-Offending

Abstract Drawing on primary data from a unique crime prevention initiative in Australia, this article examines how a small group of co-offenders harnessed the pro-social dimensions of their offending to co-desist from crime. We focus particularly on the evolving nature of group dynamics and the roles played by policing (including non-policing), innovative ‘24/7’ casework, and the provision of a culturally safe divergent setting in the emergence of co-desistance scenarios. We show how initial apprehensions around co-offenders associating in the community were unfounded and in fact proved key to the success of such an approach. Building on the supported desistance literature, the article positions co-desistance as a related but distinct concept worthy of further study in its own right.

Model reference safety‐critical adaptive control for nonlinear and switched systems

SummaryThis paper studies the problem of model reference safety‐critical adaptive control for a class of nonlinear and switched systems with nonlinear reference models. An adaptive law is designed to estimate unknown parameters, while an energy function is proposed to guarantee the boundedness and safety of nonlinear systems. Also, a model reference safety‐critical adaptive control framework is established to ensure that the state trajectory of nonlinear systems is always within the safe set in the presence of unknown parameters. Moreover, the results obtained are extended to switched uncertain nonlinear systems under arbitrary switchings. A switched adaptive controller is designed to reduce the conservativeness caused by adoption of a common update law for all subsystems. Also, the safety of switched systems under arbitrary switchings is guaranteed as well as the boundedness of the error states between the switched nonlinear systems and the switched reference models by utilizing the common Lyapunov function method. Finally, two illustrative numerical examples are given to demonstrate the effectiveness of the proposed control architecture.

Stochastic integrated model-based protocol for volume-controlled ventilation setting

Background and objectiveMechanical ventilation (MV) is the primary form of care for respiratory failure patients. MV settings are based on general clinical guidelines, intuition, and experience. This approach is not patient-specific and patients may thus experience suboptimal, potentially harmful MV care. This study presents the Stochastic integrated VENT (SiVENT) protocol which combines model-based approaches of the VENT protocol from previous works, with stochastic modelling to take the variation of patient respiratory elastance over time into consideration.MethodsA stochastic model of Ers is integrated into the VENT protocol from previous works to develop the SiVENT protocol, to account for both intra- and inter-patient variability. A cohort of 20 virtual MV patients based on retrospective patient data are used to validate the performance of this method for volume-controlled (VC) ventilation. A performance evaluation was conducted where the SiVENT and VENT protocols were implemented in 1080 instances each to compare the two protocols and evaluate the difference in reduction of possible MV settings achieved by each.ResultsFrom an initial number of 189,000 possible MV setting combinations, the VENT protocol reduced this number to a median of 10,612, achieving a reduction of 94.4% across the cohort. With the integration of the stochastic model component, the SiVENT protocol reduced this number from 189,000 to a median of 9329, achieving a reduction of 95.1% across the cohort. The SiVENT protocol reduces the number of possible combinations provided to the user by more than 1000 combinations as compared to the VENT protocol.ConclusionsAdding a stochastic model component into a model-based approach to selecting MV settings improves the ability of a decision support system to recommend patient-specific MV settings. It specifically considers inter- and intra-patient variability in respiratory elastance and eliminates potentially harmful settings based on clinically recommended pressure thresholds. Clinical input and local protocols can further reduce the number of safe setting combinations. The results for the SiVENT protocol justify further investigation of its prediction accuracy and clinical validation trials.

Training in the Dark: Using Target Training for Non-Invasive Application and Validation of Accelerometer Devices for an Endangered Primate (Nycticebus bengalensis).

Simple SummaryRecent advances in technology allow for the study of animal behaviours through indirect observations. This facilitates research on cryptic animals for which direct observations may miss a considerable portion of their activity. The validity of accelerometers in obtaining accurate animal behaviours, however, needs to be tested before collecting data in the wild. Modern zoos offer excellent opportunities for researchers to test field techniques in a safe setting. Here, we describe a non-invasive training program to attach an accelerometer to an individual Bengal slow loris at the Shaldon Wildlife Trust. This training took 39 15-min sessions and allowed for the attachment of the accelerometer for validation with reduced stress for the animal. We also collected videos to associate to accelerometer data to estimate the accuracy of accelerometers in identifying the behaviours of Bengal slow loris. The accuracy was above 80% with some of the behaviours that were clearly identified (e.g., resting: 99.8%), while others were more difficult to discern (e.g., suspensory walk, a locomotion behaviour, was discerned only 60.3% of times from other behaviours). The non-invasive training and accelerometer validation can be used on similar species before using accelerometers in the wild.Accelerometers offer unique opportunities to study the behaviour of cryptic animals but require validation to show their accuracy in identifying behaviours. This validation is often undertaken in captivity before use in the wild. While zoos provide important opportunities for trial field techniques, they must consider the welfare and health of the individuals in their care and researchers must opt for the least invasive techniques. We used positive reinforcement training to attach and detach a collar with an accelerometer to an individual Bengal slow loris (Nycticebus bengalensis) at the Shaldon Wildlife Trust, U.K. This allowed us to collect accelerometer data at different periods between January–June 2020 and January–February 2021, totalling 42 h of data with corresponding video for validation. Of these data, we selected 54 min where ten behaviours were present and ran a random forest model. We needed 39 15-min sessions to train the animal to wear/remove the collar. The accelerometer data had an accuracy of 80.7 ± SD 9.9% in predicting the behaviours, with 99.8% accuracy in predicting resting, and a lower accuracy (but still >75% for all of them apart from suspensory walk) for the different types of locomotion and feeding behaviours. This training and validation technique can be used in similar species and shows the importance of working with zoos for in situ conservation (e.g., validation of field techniques).

Antimicrobial Stewardship and Infection Prevention and Control in Low- and Middle-income Countries: Current Status and Best Practices.

Antimicrobial Stewardship and Infection Prevention and Control in Low- and Middle-income Countries: Current Status and Best Practices.

Survivors of domestic abuse and sexual violence within sexual health services, how we responded, who is missing and what should be done?

Reports of domestic abuse (DA) and sexual violence (SV) continue to rise across the UK, and sexual health services are well situated to offer a safe setting for disclosure. We performed a retrospective analysis of all safeguarding proformas completed between April and September 2019 across three sexual health clinics in London. 158 patients reported current and/or historical DA and/or SV during the collection period. 70% of cohort identified as female, 28% as male and 2% as transgender. 25% of the cohort identified as gay men. Black and Asian attendees were disproportionately represented in the survivor cohort. 86% were offered a same-day appointment with the health advisor team and assessment for onward referral to specialist services. Almost half of the cohort were asymptomatic at the time of attendance and disclosure. With changes to access due to ongoing COVID-19 restrictions, asymptomatic survivors may not receive opportunities to disclose as they are directed to online platforms. Healthcare services must adapt routine enquiry practices to ensure safe disclosure is possible to make all contacts count and expand these practices into other healthcare specialities to provide further disclosure opportunities for survivors not attending sexual health services.

Beyond partial control: controlling chaotic transients with the safety function

Partial control is a technique used in systems with transient chaos. The aim of this control method is to avoid the escape of the orbits from a region Q of the phase space where the transient chaotic dynamics takes place. This technique is based on finding a special subset of Q called the safe set. The chaotic orbit can be sustained in the safe set with a minimum amount of control. In this work, we develop a control strategy to gradually lead any chaotic orbit in Q to the safe set by using the safety function. With the technique proposed here, the safe set can be converted into a global attractor of Q.

Gaussian Control Barrier Functions: Non-Parametric Paradigm to Safety

Inspired by the success of control barrier functions (CBFs) in addressing safety, and the rise of data-driven techniques for modeling functions, we propose a non-parametric approach for online synthesis of CBFs using Gaussian Processes (GPs). A dynamical system is defined to be safe if a subset of its states remains within the prescribed set, also called the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">safe set</i> . CBFs achieve safety by designing a candidate function a priori. However, designing such a function can be challenging. Consider designing a CBF in a disaster recovery scenario where safe and navigable regions need to be determined. The decision boundary for safety here is unknown and cannot be designed a priori. Moreover, CBFs employ a parametric design approach and cannot handle arbitrary changes to the safe set in practice. In our approach, we work with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">safety samples</i> to construct the CBF online by assuming a flexible GP prior on these samples, and term our formulation as a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Gaussian</i> CBF. GPs have favorable properties such as analytical tractability and robust uncertainty estimation. This allows realizing the posterior with high safety guarantees while also computing associated partial derivatives analytically for safe control. Moreover, Gaussian CBFs can change the safe set arbitrarily based on sampled data, thus allowing non-convex safe sets. We validated experimentally on a quadrotor by demonstrating safe control for 1) arbitrary safe sets, 2) collision avoidance with online safe set synthesis, 3) and juxtaposed Gaussian CBFs with CBFs in the presence of noisy states. The experiment video link is: https://youtu.be/HX6uokvCiGk.

Learning Discrete-Time Uncertain Nonlinear Systems With Probabilistic Safety and Stability Constraints

This paper presents a discrete-time dynamical system model learning method from demonstration while providing probabilistic guarantees on the safety and stability of the learned model. The controlled dynamic model of a discrete-time system with a zero-mean Gaussian process noise is approximated using an Extreme Learning Machine (ELM) whose parameters are learned subject to chance constraints derived using a discrete-time control barrier function and discrete-time control Lyapunov function in the presence of the ELM reconstruction error. To estimate the ELM parameters a quadratically constrained quadratic program (QCQP) is developed subject to the constraints that are only required to be evaluated at sampled points. Simulations validate that the system model learned using the proposed method can reproduce the demonstrations inside a prescribed safe set while converging to the desired goal location starting from various different initial conditions inside the safe set. Furthermore, it is shown that the learned model can adapt to changes in goal location during reproductions without violating the stability and safety constraints.

Abstraction-Based Symbolic Control Barrier Functions for Safety-Critical Embedded Systems

A control barrier function is commonly used for a safety-critical system. It enforces forward invariance with respect to a given safe set. On the other hand, abstraction-based synthesis with robustness has been investigated. In this letter, we introduce the notion of a “symbolic control barrier function” for the abstraction-based synthesis. The characteristics of the proposed method is that a conservative safe set is considered with an abstracted plant model. It is proved that both forward invariance and robustness are enforced on the physical plant.

Guaranteed Safe Spacecraft Docking With Control Barrier Functions

This paper presents a strategy for control of a spacecraft docking with a non-maneuvering target in the presence of safety constraints and bounded disturbances. The presence of disturbances prevents convergence to a unique docking state, so in our formulation, docking is defined as occurring within a set constructed using prescribed tolerances. Safety is ensured via application of Robust Control Barrier Functions to render a designated safe set forward invariant for any allowable disturbance. However, this safety strategy necessarily presumes a worst-case disturbance, and thus restricts trajectories to a subset of the safe set when a worst-case disturbance is not present. The presented controller accounts for this restriction, and guarantees that the spacecraft both remains safe and achieves docking in finite time for any allowable disturbance. The controller is then validated in simulation for a spacecraft landing on an asteroid, and two spacecraft docking in low Earth orbit.

A Formal Characterization of Black-Box System Safety Performance With Scenario Sampling

A typical scenario-based evaluation framework seeks to characterize a black-box system's safety performance (e.g., failure rate) through repeatedly sampling initialization configurations (scenario sampling) and executing a certain test policy for scenario propagation (scenario testing) with the black-box system involved as the test subject. In this letter, we first present a novel safety evaluation criterion that seeks to characterize the actual operational domain within which the test subject would remain safe indefinitely with high probability. By formulating the black-box testing scenario as a dynamic system, we show that the presented problem is equivalent to finding a certain "almost" robustly forward invariant set for the given system. Second, for an arbitrary scenario testing strategy, we propose a scenario sampling algorithm that is provably asymptotically optimal in obtaining the safe invariant set with arbitrarily high accuracy. Moreover, as one considers different testing strategies (e.g., biased sampling of safety-critical cases), we show that the proposed algorithm still converges to the unbiased approximation of the safety characterization outcome if the scenario testing satisfies a certain condition. Finally, the effectiveness of the presented scenario sampling algorithms and various theoretical properties are demonstrated in a case study of the safety evaluation of a control barrier function-based mobile robot collision avoidance system.

The sandbox approach to FinTech regulation: a case study of China

Finance has become increasingly enabled by innovations and technologies after the global financial crisis, calling for profound reforms regarding the regulatory regime. Regulation has lifted requirements to deal with systemic risk in the post-crisis era, but being slow to address the rise of FinTech activities. Such a mismatch requires more accommodative regulatory treatments, bringing about the sandbox among others. As far as Mainland China is concerned, it has been very active in applying technologies to finance, but its regulators and policy makers have struggled to manage market dynamics. This article is aimed to explore how to balance between ongoing regulatory reforms and a sharp rise of FinTech activities via the sandbox approach in China. As an innovative regulatory tool, the sandbox is a safe space set to test innovation activities without harming consumers’ interests and fair competition. Since 2015, a growing number of sandboxes have been introduced to both developed and developing countries. This article selects some of them to make a comparative study. As presented by such cases, convergences co-exist with divergences, and the key elements can be summarized accordingly. The sandbox has been further examined as a form of principles-based regulation, alongside risk-based approach. It is not the only option or best practice to boost innovations, but it is beneficial as long as to get it designed carefully. Then this article reviews newly-established sandboxes in China by drawing insights from the comparative study, proposing a “to-reform” list to Chinese reformers. As the conclusion, it highlights that the sandbox, carrying critical limitations, might face extra challenges in China.

Control Barrier Functions in Sampled-Data Systems

This paper presents conditions for ensuring forward invariance of safe sets under sampled-data system dynamics with piecewise-constant controllers and fixed time-steps. First, we introduce two different metrics to compare the conservativeness of sufficient conditions on forward invariance under piecewise-constant controllers. Then, we propose three approaches for guaranteeing forward invariance, two motivated by continuous-time barrier functions, and one motivated by discrete-time barrier functions. All proposed conditions are control affine, and thus can be incorporated into quadratic programs for control synthesis. We show that the proposed conditions are less conservative than those in earlier studies, and show via simulation how this enables the use of barrier functions that are impossible to implement with the desired time-step using existing methods.

A Linear Programming Approach to Computing Safe Sets for Software Rejuvenation

Software rejuvenation was born to fix operating system faults by periodically refreshing the run-time code and data. This mechanism has been extended to protect control systems from cyber-attacks. This letter proposes a software rejuvenation design method in discrete-time where invariant sets for the safety and mission controllers are designed to schedule the timing of software refreshes. To compute a minimal robust positively invariant (min-RPI) set and the bounded time between software refreshes to ensure system safety, an LP based approach is proposed for stable and unstable systems. Finally, the designed approach is illustrated by the case study of a simulated lab-scale microgrid.

Vehicle Lateral Stability Regions for Control Applications

This article presents an improved method of obtaining lateral stability regions for road vehicles, considering the influence of steering angle, center of gravity, longitudinal speed, and tire-road friction coefficient on the vehicle dynamics. Comprehensive stability regions are obtained for a wide range of such parameters. Moreover, conservative stability regions are proposed, in the case of control applications that demand robust or safety-critical control actions. Next, a steering-angle-dependent region is used to implement a safety-critical electronic stability control system with active front steering as actuation. The resulting control system extends safety-critical control based on control-dependent barrier functions, introducing a control-dependent Lyapunov function to improve its steady-state behavior. Finally, we identify and propose workarounds for problems that arise from the number of inputs being less than the dimension of the desired safe set. The conservative stability regions and the extended safety-critical control system are validated by means of simulation results based on a nonlinear lateral stability model.

Converse Barrier Functions via Lyapunov Functions

We prove a robust converse barrier function theorem via the converse Lyapunov theory. While the use of a Lyapunov function as a barrier function is straightforward, the existence of a converse Lyapunov function as a barrier function for a given safety set is not. We establish this link by a robustness argument. We show that the closure of the forward reachable set of a robustly safe set must be robustly asymptotically stable under mild technical assumptions. As a result, all robustly safe dynamical systems must admit a robust barrier function in the form of a Lyapunov function for set stability. We present the results in both continuous-time and discrete-time settings and remark on connections with various barrier function conditions.