Real-world Constraints Research Articles

Planning for potential: efficient safe reinforcement learning

Deep reinforcement learning (DRL) has shown remarkable success in artificial domains and in some real-world applications. However, substantial challenges remain such as learning efficiently under safety constraints. Adherence to safety constraints is a hard requirement in many high-impact application domains such as healthcare and finance. These constraints are preferably represented symbolically to ensure clear semantics at a suitable level of abstraction. Existing approaches to safe DRL assume that being unsafe leads to low rewards. We show that this is a special case of symbolically constrained RL and analyze a generic setting in which total reward and being safe may or may not be correlated. We analyze the impact of symbolic constraints and identify a connection between expected future reward and distance towards a goal in an automaton representation of the constraints. We use this connection in an algorithm for learning complex behaviors safely and efficiently. This algorithm relies on symbolic reasoning over safety constraints to improve the efficiency of a subsymbolic learner with a symbolically obtained measure of progress. We measure sample efficiency on a grid world and a conversational product recommender with real-world constraints. The so-called Planning for Potential algorithm converges quickly and significantly outperforms all baselines. Specifically, we find that symbolic reasoning is necessary for safety during and after learning and can be effectively used to guide a neural learner towards promising areas of the solution space. We conclude that RL can be applied both safely and efficiently when combined with symbolic reasoning.

Reconsidering the “War on Rats”: What We Know From Over a Century of Research Into Municipal Rat Management

To sustainably control urban rat infestations, management efforts need to encompass large areas of urban centers. Therefore, the objective of this review and narrative synthesis was to collate what is known about municipal-scale rat management. We explored the management frameworks that have been used at a large scale in cities and we describe the expectations of experts who have designed and implemented these frameworks. We found that there has been a persistent “war on rats” paradigm driving this literature since the early 1900s. Not only was there little quantitative evidence to support this paradigm and associated methodologies, but together, they failed to meet the expectations of those who designed and implemented them due to real-world constraints (i.e., limited resources). To improve the field of municipal management, we identify two distinct options. First, stakeholders may continue to wage the “war on rats” while improving existing strategies within this paradigm. Key pathways forward include developing evaluation metrics aligned with program objectives, establishing the cost-effectiveness of methodologies, and improving control efficacy. Second, we suggest a new paradigm, one that considers that rat management is a complex system that must be approached by first mapping its complexity to highlight and prioritize the many complex upstream determinants of infestations. This perspective highlights how rat management may be a wicked problem for which there is no overarching solution and instead can only be managed by making incremental gains in individual components of the problem over time. Importantly, we propose an alternative management objective that places a priority on improving the overall health of the community, instead of on eliminating rats, and we stress that management goals must address stakeholders’ goals as their investment is essential for a sustained program.

DiploSim: A Flexible Framework for Diplomatic Simulations in International Relations

AbstractAlthough educators increasingly appreciate the pedagogical benefits of active learning techniques including simulations, many still see implementing them in their own classroom as a daunting task. The formidable time investment required often deters instructors from designing new simulations, and many find published simulations to be an imperfect fit. This article seeks to reduce the barriers to entry for instructors who are interested in designing personalized simulations yet hesitant in the face of real-world constraints. It does so by introducing a flexible framework for diplomatic simulations (DiploSim) that is firmly rooted in the design principles of drama, immersion, and reflection yet also easily customizable to fit instructors’ preferred thematic content, negotiating format, schedule, and class size. By combining research and role play within a straightforward and pedagogically sound structure, DiploSim offers instructors a useful gateway into the world of simulations.

Using Action Design Research for Developing and Implementing a Method for Risk Assessment and Continuity Management

This paper presents the results from a longitudinal study aiming at developing and implementing a method integrating principles from Risk Management and Business Continuity Management in its naturalistic organisational setting. Despite the fact that a myriad of real-world constraints and challenges typically arise in the transition from method development to full implementation and use of the method in its intended practical context, systematic and longitudinal investigation of how to attain practical viability and usefulness of such methods is limited and often restricted to small-scale demonstrations in the scientific literature. Research that strives at generating both outcome with high practical relevance and generalisable theoretical insights calls for a methodology that pays equal attention to scientific rigour and practical value. Action Design Research (ADR) represents such methodology, serving the dual mission of advancing theory while at the same time generating knowledge about how to solve organisational problems in practice. In this paper, an ADR approach to method development in a public sector organisation during a period of three and a half years is described, based on data collected through participant observation and focus group discussions. Starting off by formulating design principles guiding the method development, a number of iterative cycles of intervention in the organisational context and evaluation of the use of the method in its intended practical setting were undertaken. The paper outlines the challenges and lessons learned from this process, and shows how the final method was shaped by organisational needs and limitations to ensure practical relevance and use.

Planning of Hydrogen Refueling Stations in Urban Setting While Considering Hydrogen Redistribution

Electrified transportation systems and renewable energy resources have been recognized as effective environmental-friendly technologies against global warming contributed by greenhouse gas (GHG). Remarkably, hydrogen fuel cell-powered electric vehicles can outperform battery-powered electric vehicles largely in the sense of the driving range and the refueling time. However, both of them require a better coordination of infrastructure system and renewable energy resources to achieve a significant reduction of GHG emissions. This article aims to maximize the long-term profitability for the planning model of hydrogen refueling stations, where the capacitated flow refueling location model is leveraged for maximal traffic flow coverage. Furthermore, we discuss various real-world constraints, such as traffic network constraints, distribution network constraints, hydrogen balance constraints, and energy constraints for electric vehicles, to make the planning model more practical. By considering the uncertainty of the short-term refueling demand across the city, an approach for geographically redistributing hydrogen among the stations is also presented where the minimal cost of redistribution is modeled by one-commodity pickup-and-delivery traveling salesman problem. A real-life case study of Western Sydney is adopted to testify the efficiency of the planning model under current and future cost levels. Finally, a numerical simulation is utilized to demonstrate the validity of the hydrogen redistribution method.

Network control by a constrained external agent as a continuous optimization problem

Social science studies dealing with control in networks typically resort to heuristics or solely describing the control distribution. Optimal policies, however, require interventions that optimize control over a socioeconomic network subject to real-world constraints. We integrate optimisation tools from deep-learning with network science into a framework that is able to optimize such interventions in real-world networks. We demonstrate the framework in the context of corporate control, where it allows to characterize the vulnerability of strategically important corporate networks to sensitive takeovers, an important contemporaneous policy challenge. The framework produces insights that are relevant for governing real-world socioeconomic networks, and opens up new research avenues for improving our understanding and control of such complex systems.

Finding an optimised infrastructure for electricity distribution networks in rural areas - A comparison of different approaches

We introduce two novel evolutionary algorithms for the problem of optimising electricity distribution networks in remote areas. Traditionally, electricity networks have been built using a centralised network infrastructure. In remote areas these typically include long distances between sparsely distributed consumers that lead to high costs and several risks that accompany these long connection lines. Therefore, alternative distributed infrastructures including potential microgrids or stand-alone power systems can be beneficial. Given a set of electricity consumers, their loads and locations, the aim is to find an optimal electricity network topology that might not be interconnected. Previously, this problem has been addressed using different standard clustering approaches comparing the total network costs of each solution. We propose a problem-specific evolutionary algorithm that alters the network structure directly as well as a novel quadtree-based evolutionary algorithm. The algorithms minimise a non-linear objective function representing the cost of the network while approximating real-world constraints. They are tested on a large set of various problem instances and compared to the standard clustering algorithms that have been used previously and an intuitive baseline method. The problem-specific evolutionary algorithm outperforms most standard clustering methods on a range of different test cases while the quadtree-based algorithm is shown to be effective in certain settings.

Tackling the rich vehicle routing problem with nature-inspired algorithms

In the last decades, the classical Vehicle Routing Problem (VRP), i.e., assigning a set of orders to vehicles and planning their routes has been intensively researched. As only the assignment of order to vehicles and their routes is already an NP-complete problem, the application of these algorithms in practice often fails to take into account the constraints and restrictions that apply in real-world applications, the so called rich VRP (rVRP) and are limited to single aspects. In this work, we incorporate the main relevant real-world constraints and requirements. We propose a two-stage strategy and a Timeline algorithm for time windows and pause times, and apply a Genetic Algorithm (GA) and Ant Colony Optimization (ACO) individually to the problem to find optimal solutions. Our evaluation of eight different problem instances against four state-of-the-art algorithms shows that our approach handles all given constraints in a reasonable time.

Property Taxes in the Real World

Property taxes are regarded by economists as good taxes for local governments. Yet property tax revenues rarely exceed 3 percent of gross domestic product in any country and usually amount to much less than that. Moreover, jurisdictions that levy property taxes rarely do so in a way that adheres to economic principles. Political pressure to maintain the tax burden at or near its current level for the sake of stability or to favour one group of taxpayers over another often overrides the economics principles of efficiency and equity. The author asserts that no matter how good the property tax is in theory, we need to consider the real-world constraints on the tax. The author sets out what an ideal property tax would look like, contrasting that ideal with what exists around the world, then describes the unique characteristics of the tax that help to explain the gap between theory and practice. Some promising strategies for reform are proposed, along with cautions regarding problematic approaches. Policy makers are well advised to link property tax reform to broader reforms in public sector management aimed at improving public services and governance. Public support for the reform is also very important. Successful implementation requires sustained political will, technical capacity, systems and procedures, financial resources, and time, but it is well worth the effort.

Deep neural network models of sound localization reveal how perception is adapted to real-world environments

Mammals localize sounds using information from their two ears. Localization in real-world conditions is challenging, as echoes provide erroneous information, and noises mask parts of target sounds. To better understand real-world localization we equipped a deep neural network with human ears and trained it to localize sounds in a virtual environment. The resulting model localized accurately in realistic conditions with noise and reverberation. In simulated experiments, the model exhibited many features of human spatial hearing: sensitivity to monaural spectral cues and interaural time and level differences, integration across frequency, biases for sound onsets, and limits on localization of concurrent sources. But when trained in unnatural environments without either reverberation, noise, or natural sounds, these performance characteristics deviated from those of humans. The results show how biological hearing is adapted to the challenges of real-world environments and illustrate how artificial neural networks can reveal the real-world constraints that shape perception.

A simulation-optimization approach for solving the forestry logistics problem

Solving the forestry logistic problem is involved with multiple short- and long-term operational decisions such as truck routing, inventory management, and scheduling. Although these problems are separately studied in the literature, but in many situations, a near-optimal solution of the “comprehensive” problem suffices the managerial needs and directions. In this paper, we consider the forestry logistic and use a simulation-optimization method to reach a near-optimal solution. This method enables us to incorporate different uncertainties such as stochastic travel time and occurrence of external factors. We use real-world data to evaluate our model and measure the complexity of real-world constraints.

Integration of phasing on multi-objective optimization of building stock energy retrofit

Decision making for the energy retrofit of building stock is key to improve the energy efficiency and comfort in buildings. This article present a methodology developed in order to help the building stock or real estate manager to improve the efficiency of his buildings. The methodology is based on the multi-objective optimization of the retrofit, and specifically of the walls and windows of the buildings. The paper presents the integration of a temporal dimension with the phasing of the construction work in the retrofit strategies on multiple buildings in order to get closer to real world practice in the construction sites. The case study is carried out on a small building stock with the NSGA-II genetic algorithm. The search space of the optimization are vertical and horizontal walls and windows of each building; objective functions are heating demand, price of the proposed retrofit and an overheating indicator based on adaptive comfort. Overheating and heating demand are evaluated using EnergyPlus simulations. Phasing is implemented directly into the optimization formalisation with a separated chromosome describing temporality. As a result, different retrofit strategies were obtained by integrating phasing than with a standard optimisation with no temporal planning of operations. The main difference lies into the which building is selected to be retrofitted first, and in the improved performance of each retrofit material proposed: it led to an increase of performance of 10% in the overheating while thermal performance decreased for 2% in the retrofit strategies possible in reality when including the phasing. These results highlight the importance of formulating the optimization problem as close as possible to the real world constraints in construction in order to have accurate retrofit strategies.

Is There Opportunity Without Stakeholders? A Stakeholder Theory Critique and Development of Opportunity-Actualization

How can stakeholder theory contribute to opportunity theory? We suggest that stakeholder theory affords appropriate theoretical lenses for grounding the opportunity-actualization perspective more firmly within the real-world constraints of business venturing. Actualization departs from a strong focus on entrepreneurial agency to conceptualize how pre-existing environmental conditions determine what entrepreneurial action can achieve. We explain that stakeholder theory can strengthen the outward-looking orientation of actualization by (1) bringing the entirety of stakeholders centre-stage, beyond a narrow focus on market stakeholders, and (2) stressing the importance of noneconomic considerations for the actualization of economic opportunities. Our theorization culminates in the concept of ‘strategic opportunity thinking’ (SOT). We conceptualize SOT as a way of protecting entrepreneurs from the blind-to-stakeholders mindset that either sleepwalks them into the territory of non-opportunity or prevents them from the actualization of real yet difficult-to-actualize opportunities in the absence of stakeholder-centric thinking.

Examining the perspectives of using manure from livestock farms as fertilizer to crop fields based on a realistic simulation

Intensive livestock production has a negative environmental impact by producing large amounts of animal dejections, which, if not properly managed, can contaminate nearby water bodies with nutrient excess. However, if the animal manure could be transferred efficiently to nearby crops and used as a fertilizer for the plants, pollution/contamination would be mitigated, transforming manure from a waste to a resource. This valorization of manure from waste to a resource falls within the circular economy principles, but the transportation of manure also comes at an environmental and economic cost. It is a single-objective optimization problem regarding finding the best solution for the logistics process of satisfying nutrient crops needs through livestock manure. This paper uses a centralized optimal algorithm (COA) to solve the problem, based on a realistic simulator that considers numerous real-world constraints that related work has not yet addressed. Implementation and evaluation of this method have been carried out based on extensive geolocalized data from Catalonia (Spain), one of the densest European farming regions, as a case study. The findings show that the use of treatment units in pig farms is not profitable, while applying treatment units on selected cow farms for composting manure has its merits, under an intelligent choice of cow farms. Finally, a comparison of our findings with those of two similar studies in Hangzhou, China and Minnesota, USA, are performed.

Making a few talk for the many – Modeling driver behavior using synthetic populations generated from experimental data

Understanding driver behavior is the basis for the development of many advanced driver assistance systems, and experimental studies are indispensable tools for constructing appropriate driver models. However, the high cost associated with testing is a serious obstacle in collecting large amounts of experimental data. This paper presents a methodology that can improve the reliability of results from experimental studies with a limited number of participants by creating a virtual population. Specifically, a methodology based on Bayesian inference has been developed, that generates synthetic cases that adhere to various real-world constraints and represent possible variations of the observed experimental data. The application of the framework is illustrated using data collected during a test-track experiment where truck drivers performed a right turn maneuver, with and without a cyclist crossing the intersection. The results show that, based on the speed profiles of the dataset and physical constraints, the methodology can produce synthetic speed profiles during braking that mimic the original curves but extend to other realistic braking patterns that were not directly observed. The models obtained from the proposed methodology have applications for the design of active safety systems and automated driving, demonstrating thereby that the developed framework has great promise for the automotive industry.

현실 제약 조건을 반영한 강화학습 기반 교통 신호 제어

교통 신호 제어는 도로의 한정된 용량을 효율적으로 사용하기 위해 중요한 역할을 한다. 전통적인 교통 신호 제어 방법은 미리 설정된 신호를 기반으로 운영되기 때문에 수시로 변하는 교통 상황에 대응하기 어렵다. 최근에 복잡한 문제를 해결하기 위한 방법으로 강화학습이 주목을 받으면서 효율적인 교통 신호 제어를 위해 강화학습을 사용하는 연구가 진행되고 있다. 전통적인 방법과 비교하여 대기 시간이나 여행 시간이 개선되는 것을 시뮬레이션을 통해 입증하였으나 대부분의 연구가 실제 신호의 제약조건을 반영하지 않아 현실에 적용하기에는 부적합하게 설계되어 있다. 본 논문에서는 현실에 존재하는 신호 운영체계의 제약조건을 반영하여 비교적 현실 적용 가능하며 교통량 변화에 대응할 수 있는 강화학습 기반의 신호 제어 방법을 제안한다.

Self-Directed Exergaming for Stroke Upper Limb Impairment Increases Exercise Dose Compared to Standard Care

Background. One of the strongest modifiable determinants of rehabilitation outcome is exercise dose. Technologies enabling self-directed exercise offer a pragmatic means to increase dose, but the extent to which they achieve this in unselected cohorts, under real-world constraints, is poorly understood. Objective. Here we quantify the exercise dose achieved by inpatient stroke survivors using an adapted upper limb (UL) exercise gaming (exergaming) device and compare this with conventional (supervised) therapy. Methods. Over 4 months, patients presenting with acute stroke and associated UL impairment were screened at a single stroke centre. Participants were trained in a single session and provided with the device for unsupervised use during their inpatient admission. Results. From 75 patients referred for inpatient UL therapy, we recruited 30 (40%), of whom 26 (35%) were able to use the device meaningfully with their affected UL. Over a median enrolment time of 8 days (IQR: 5–14), self-directed UL exercise duration using the device was 26 minutes per day (median; IQR: 16–31), in addition to 25 minutes daily conventional UL therapy (IQR: 12–34; same cohort plus standard care audit; joint n = 50); thereby doubling total exercise duration (51 minutes; IQR: 32–64) relative to standard care (Z = 4.0, P <.001). The device enabled 104 UL repetitions per day (IQR: 38–393), whereas conventional therapy achieved 15 UL repetitions per day (IQR: 11–23; Z = 4.3, P <.001). Conclusion. Self-directed adapted exergaming enabled participants in our stroke inpatient cohort to increase exercise duration 2-fold, and repetitions 8-fold, compared to standard care, without requiring additional professional supervision.

Future Directions in Research to Improve Care Transitions From Hospital Discharge.

Future Directions in Research to Improve Care Transitions From Hospital Discharge.

Leak Detection in Smart Water Grids Using EPANET and Machine Learning Techniques

Water supply networks are liable to leakages, resulting in loss of large quantities of water. Hence, it is required to implement a leak detection and localization system through water network simulations and IoT enabled sensors which will enable faster leak detection and repairs to leaks such as underground leaks, which would otherwise go undetected for long durations. This study presents a clear and detailed procedure to set up the water network and leak simulation in EPANET, to generate the dataset, to apply machine learning classifiers to predict leakage and to apply the proposed system to a real-world leak detection system. Using extended period analysis in EPANET, emitters have been used to model leakages of various sizes and at different locations, while keeping in mind the real-world constraints. Leaks cause changes in the dynamic network parameters such as pressure and flow. This fact has been utilized for leak prediction implementation. Once the simulated model is validated, the trained machine learning classifiers can be used to predict and localize the leaks in real time.

Gaze During Locomotion in Virtual Reality and the Real World.

How vision guides gaze in realistic settings has been researched for decades. Human gaze behavior is typically measured in laboratory settings that are well controlled but feature-reduced and movement-constrained, in sharp contrast to real-life gaze control that combines eye, head, and body movements. Previous real-world research has shown environmental factors such as terrain difficulty to affect gaze; however, real-world settings are difficult to control or replicate. Virtual reality (VR) offers the experimental control of a laboratory, yet approximates freedom and visual complexity of the real world (RW). We measured gaze data in 8 healthy young adults during walking in the RW and simulated locomotion in VR. Participants walked along a pre-defined path inside an office building, which included different terrains such as long corridors and flights of stairs. In VR, participants followed the same path in a detailed virtual reconstruction of the building. We devised a novel hybrid control strategy for movement in VR: participants did not actually translate: forward movements were controlled by a hand-held device, rotational movements were executed physically and transferred to the VR. We found significant effects of terrain type (flat corridor, staircase up, and staircase down) on gaze direction, on the spatial spread of gaze direction, and on the angular distribution of gaze-direction changes. The factor world (RW and VR) affected the angular distribution of gaze-direction changes, saccade frequency, and head-centered vertical gaze direction. The latter effect vanished when referencing gaze to a world-fixed coordinate system, and was likely due to specifics of headset placement, which cannot confound any other analyzed measure. Importantly, we did not observe a significant interaction between the factors world and terrain for any of the tested measures. This indicates that differences between terrain types are not modulated by the world. The overall dwell time on navigational markers did not differ between worlds. The similar dependence of gaze behavior on terrain in the RW and in VR indicates that our VR captures real-world constraints remarkably well. High-fidelity VR combined with naturalistic movement control therefore has the potential to narrow the gap between the experimental control of a lab and ecologically valid settings.