Deterministic Scheme Research Articles

Collaborative Decision-Making Method of Emergency Response for Highway Incidents

With the continuous increase in highway mileage and vehicles in China, highway accidents are also increasing year by year. However, the on-site disposal procedures of highway accidents are complex, which makes it difficult for the emergency department to fully observe the accident scene, resulting in the lack of sufficient communication and cooperation between multiple emergency departments, making the rescue efficiency low and wasting valuable rescue time, and causing unnecessary injury or loss of life due to the lack of timely assistance. Thus, this paper proposes a multi-agent-based collaborative emergency-decision-making algorithm for traffic accident on-site disposal. Firstly, based on the analysis and abstraction of highway surveillance videos obtained from the Shaanxi Provincial Highway Administration, this paper constructs an emergency disposal model based on Petri net to simulate the emergency on-site disposal procedures. After transforming the emergency disposal model into a Markov game model and applying it to the multi-agent deep deterministic strategy gradient (MADDPG) algorithm proposed in this paper, the multiple agents can optimize the emergency-decision-making and on-site disposal procedures through interactive learning with the environment. Finally, the proposed algorithm is compared with the typical algorithm and the actual processing procedure in the simulation experiment of an actual Shaanxi highway traffic accident. The results show that the proposed emergency-decision-making method could greatly improve collaboration efficiency among emergency departments and effectively reduce emergency response time. This algorithm is not only superior to other decision-making algorithms such as genetic algorithm (EA), evolutionary strategy (ES), and deep Q network (DQN), but also reduces the disposal processes by 28%, 28%, and 42%, respectively, compared with the actual disposal process in three emergency disposal cases. In summary, with the continuous development of information technology and highway management systems, the multi-agent-based collaborative emergency-decision-making algorithm will contribute to the actual emergency response process and emergency disposal in the future, improving rescue efficiency and ensuring the safety of individuals.

Assessment of an Optimal Design Method for a High-Energy Ultrasonic Igniter Based on Multi-Objective Robustness Optimization

The current deterministic optimization design method ignores uncertainties in the material properties and potential machining error which could lead to unreliable or unstable designs. To improve the design efficiency and anti-jamming ability of a high-energy ultrasonic igniter, a Six Sigma multi-objective robustness optimization design method based on the response surface model and the design of the experiment has been proposed. In this paper, the initial structural dimensions of a high-energy ultrasonic igniter have been obtained by employing one-dimensional longitudinal vibration theory. The finite element simulation method of COMSOL Multiphysics software has been verified by the finite element simulation results of ANSYS Workbench software. The optimal igniter design has been achieved by using the proposed method, which is based on the finite element method, the Optimal Latin Hypercube Design method, Grey Relational Analysis, the response surface model, the non-dominated sorting genetic algorithm, and the mean value method. Considering the influence of manufacturing errors on the igniter’s performance, the Six Sigma method was used to optimize the robustness of the igniter. The Eigenfrequency analysis and the vibration velocity ratio calculation were conducted to verify the design’s effectiveness. The results reveal that the longitudinal resonant frequency of the deterministic optimization scheme and the robustness optimization scheme are closer to the design’s target frequency. The relative error is less than 0.1%. Compared with the deterministic optimization scheme, the vibration velocity ratio of the robustness optimization scheme is 2.8, which is about 15.7% higher than that of the deterministic optimization scheme, and the quality level of the design targets is raised to above Six Sigma. The proposed method can provide an efficient and accurate optimal design for developing a new special piezoelectric transducer.

Intelligent Pursuit–Evasion Game Based on Deep Reinforcement Learning for Hypersonic Vehicles

As defense technology develops, it is essential to study the pursuit–evasion (PE) game problem in hypersonic vehicles, especially in the situation where a head-on scenario is created. Under a head-on situation, the hypersonic vehicle’s speed advantage is offset. This paper, therefore, establishes the scenario and model for the two sides of attack and defense, using the twin delayed deep deterministic (TD3) gradient strategy, which has a faster convergence speed and reduces over-estimation. In view of the flight state–action value function, the decision framework for escape control based on the actor–critic method is constructed, and the solution method for a deep reinforcement learning model based on the TD3 gradient network is presented. Simulation results show that the proposed strategy enables the hypersonic vehicle to evade successfully, even under an adverse head-on scene. Moreover, the programmed maneuver strategy of the hypersonic vehicle is improved, transforming it into an intelligent maneuver strategy.

Deterministic learning-based neural network control with adaptive phase compensation

Under the persistent excitation (PE) condition, the real dynamics of the nonlinear system can be obtained through the deterministic learning-based radial basis function neural network (RBFNN) control. However, in this scheme, the learning speed and accuracy are limited by the tradeoff between the PE levels and the approximation capabilities of the neural network (NN). Inspired by the frequency domain phase compensation of linear time-invariant (LTI) systems, this paper presents an adaptive phase compensator employing the pure time delay to improve the performance of the deterministic learning-based adaptive feedforward control with the reference input known a priori. When the adaptive phase compensation is applied to the hidden layer of the RBFNN, the nonlinear approximation capability of the RBFNN is effectively improved such that both the learning performance (learning speed and accuracy) and the control performance of the deterministic learning-based control scheme are improved. Theoretical analysis is conducted to prove the stability of the proposed learning control scheme for a class of systems which are affine in the control. Simulation studies demonstrate the effectiveness of the proposed phase compensation method.

HETEROGENEOUS TRADING STRATEGY ENSEMBLING FOR INTRADAY TRADING ALGORITHMS

Since the inception of algorithmic trading during the mid-1970s, considerable resources and time have been committed by the financial sector to the development of trading algorithms in the hope of obtaining a competitive advantage over human contenders. A plethora of trading algorithms has been proposed in the literature; each algorithm is unique in its design, but little emphasis has been placed on heterogeneous trading strategy ensembling. In this paper we propose a trading strategy ensemble method for combining three different domain-specific trading strategies: a deterministic strategy, a probabilistic strategy, and a machine-learning strategy. The objective of the trading strategy ensemble is to find an appropriate trade-off between the levels of return and the risk exposure of a trader. We implement our strategy across different historical forex currency pair data in a bid to validate the trading strategy ensemble, and we analyse the results by invoking appropriate return and risk performance measures.

Utilizing Deep Reinforcement Learning and QLearning algorithms for Improved Ethereum Cybersecurity

The purpose of the research is to explore and develop Deep Reinforcement Learning and Q-Learning algorithms in order to improve Ethereum cybersecurity in contract vulnerabilities, the smart contract market and research leadership in the area. Deep Reinforcement Learning (Deep RL) is gaining popularity among AI researchers due to its ability to handle complex, dynamic, and particularly high-dimensional cyber protection problems. The benchmark of RL is goal-oriented behavior that increases rewards and decreases penalties or losses, and enhances real-time interaction between an agent and its surroundings. The research paper examines the three major cryptocurrencies (Bitcoin, Litecoin and Ethereum) and the role played by cyber-attacks.The Design Science Research Paradigm as applied in Information Systems research was used in this research, as it is hinged on the idea that information and understanding of a design problem and its solution are attained in the crafting of an artefact. The proposed constructs were in the form of Deep Reinforcement Learning and Q-Learning algorithms designed to improve Ethereum cybersecurity. Smart contracts on the Ethereum blockchain can automatically enforce contracts made between two unknown parties. Blockchain (BC) and artificial intelligence (AI) are used together to strengthen one another's skills and complement one another. Consensus algorithms (CAs) of BC and deep reinforcement learning (DRL) in ETS were thoroughly reviewed. In order to integrate many DCRs and provide grid services, this article suggests an effective incentive-based autonomous DCR control and management framework. This framework simultaneously adjusts the grid's active power with accuracy, optimizes DCR allocations, and increases profits for all prosumers and system operators. The best incentives in a continuous action space to persuade prosumers to reduce their energy consumption were found using a model-free deep deterministic policy gradient-based strategy. Extensive experimental experiments were carried out utilizing real-world data to show the framework's efficacy.

Evolutionary dynamics in networked trust games with diverse investment patterns

Most previous works study the evolution of trust by commonly assuming that investors adopt a deterministic investment strategy. In this work, we propose a mechanism of diverse investment in the trust game model on social networks, where each investor adopts a probabilistic strategy by considering the trustworthiness level in the local group to decide whether to invest or not. Extensive simulation results suggest that the proposed mechanism inhibits the untrustworthy behavior and limits its spread, thus stabilizing the cooperative cluster of investors and trustworthy trustees. Therefore, the trust level and global wealth are greatly enhanced comparing to the traditional setup with homogeneous investment pattern. The strong investment diversity can even eliminate untrustworthiness completely despite the fact that the severe temptation condition is disadvantageous to the evolution of trust. We also investigate the impact of investment diversity on trust game model embedding in different network structures with different initial conditions, where we observe similarly positive evolutionary outcomes. We hope these observations can provide valuable insights into further exploring the improvement of trust in real life.

Research on the robotic arm modeling algorithm based on deep learning

We propose a Deep learning which is to be aimed at the method of robotic arm modeling algorithm. Firstly, the movement of the mechanical arm, raise the traditional intelligent grasping system, improve the SSD original model, and enhance the backbone network performance selectively, improve the depth of the applicability of deterministic strategy gradient (DDPG) algorithm for further research, to shorten the mechanical arm model debugging time to reach the goal of avoiding obstacles according to research on the mechanical arm dynamics modeling.This will make the robotic arm to have a high ability to adapt to the environment, and can provide researchers with ideas to solve the theoretical research and engineering implementation in this field after training and learning.

Memory-two strategies forming symmetric mutual reinforcement learning equilibrium in repeated prisoners’ dilemma game

We investigate symmetric equilibria of mutual reinforcement learning when both players alternately learn the optimal memory-two strategies against the opponent in the repeated prisoners’ dilemma game. We provide a necessary condition for memory-two deterministic strategies to form symmetric equilibria. We then provide three examples of memory-two deterministic strategies which form symmetric mutual reinforcement learning equilibria. We also prove that mutual reinforcement learning equilibria formed by memory-two strategies are also mutual reinforcement learning equilibria when both players use reinforcement learning of memory-n strategies with n>2.

Tuneable Gaussian entanglement in levitated nanoparticle arrays

Optically levitated nanoparticles emerged as an interesting platform for probing fundamental physics. Quantum control of their motion (including potential shaping) predisposes them for emulating various physical systems and studying quantum phenomena with massive objects. Extending these capabilities to quantum many-body systems requires feasible strategies to couple and entangle nanoparticles directly or via an optical bus. We propose a variable, deterministic scheme to generate Gaussian entanglement in the motional steady state of levitated nanoparticles using coherent scattering. Coupling multiple nanoparticles to a common cavity mode allows cooling of a collective Bogoliubov mode; cooling multiple Bogoliubov modes (by trapping each nanoparticle in multiple tweezers, each scattering into a separate cavity mode) removes most thermal noise, leading to strong entanglement. Numerical simulations for three nanoparticles show great tuneability of entanglement with realistic experimental parameters. Our proposal paves the way towards complex motional quantum states for advanced quantum sensing protocols and many-body quantum simulations.

A hybrid ensemble deep reinforcement learning model for locomotive axle temperature using the deterministic and probabilistic strategy

Abstract This paper proposes a hybrid deep reinforcement learning framework for locomotive axle temperature by combining the wavelet packet decomposition (WPD), long short-term memory (LSTM), gated recurrent unit (GRU) reinforcement learning and generalized autoregressive conditional heteroskedasticity (GARCH) algorithms. The WPD is utilized to decompose the raw nonlinear series into subseries. Then the deep learning predictors LSTM and GRU are established to predict the future axle temperatures in each subseries. The Q-learning could generate optimal ensemble weights to integrate the predictors to finish the deterministic forecasting and GARCH is used to conduct the deterministic forecasting based on the deterministic forecasting residual. These parts of the hybrid ensemble structure contributed to optimal modelling accuracy and provided effective support in the real-time monitoring and fault diagnosis of transportation.

Convergence and error analysis of compressible fluid flows with random data: Monte Carlo method

The goal of this paper is to study convergence and error estimates of the Monte Carlo method for the Navier–Stokes equations with random data. To discretize in space and time, the Monte Carlo method is combined with a suitable deterministic discretization scheme, such as a finite volume (FV) method. We assume that the initial data, force and the viscosity coefficients are random variables and study both the statistical convergence rates as well as the approximation errors. Since the compressible Navier–Stokes equations are not known to be uniquely solvable in the class of global weak solutions, we cannot apply pathwise arguments to analyze the random Navier–Stokes equations. Instead, we have to apply intrinsic stochastic compactness arguments via the Skorokhod representation theorem and the Gyöngy–Krylov method. Assuming that the numerical solutions are bounded in probability, we prove that the Monte Carlo FV method converges to a statistical strong solution. The convergence rates are discussed as well. Numerical experiments illustrate theoretical results.

Deep-learning-based parallel single-pixel imaging for efficient 3D shape measurement in the presence of strong interreflections by using sampling Fourier strategy

Strong interreflections cause intractable systematic errors to traditional 3D shaping methods, for example, fringe projection profilometry. Parallel single-pixel imaging (PSI) captures the light transport coefficients between the camera projector pair, thus overcoming the influence of interreflections. However, PSI requires a large number of measurements, causing poor capturing efficiency. To achieve efficient 3D shape measurement in the presence of strong interreflections, we present a deep-learning-based parallel single-pixel imaging method (dlPSI), featuring adopting a deep-learning network to achieve accurate light transport coefficient reconstruction and using sampling Fourier strategy to reduce measurements. The deep-learning network is designed to suppress the noise caused by undersampling the Fourier domains. Experiments prove that our dlPSI with the characteristic-based deterministic Fourier sampling strategy achieves high-quality 3D shape measurement while saving 94% of measurements.

USING MACHINE LEARNING ON DEPTH MAPS AND IMAGES FOR TUNNEL EQUIPMENT SURVEYING

Abstract. In this contribution, we illustrate how to use high-resolution 3D data and images to model the global shape of a tunnel and to survey its equipment, in a semi-automatic way using pattern recognition and machine learning techniques. We first implement a robust B-spline fitting algorithm, based on a parametric family of M-estimators that allows an efficient deterministic optimization strategy, to accurately model the tunnel lining using range data, despite the presence of acquisition artifacts and significant perturbations related to the equipment and some surface defects. The residual maps from the robust fit can be exploited to segment the equipment in an unsupervised manner using clustering algorithms, but at the cost of post-processing which makes the method rather ineffective for routine use. However, we deploy it to annotate data for the supervised learning of a deep learning model, namely Mask R-CNN.We comment on the first results, obtained on a still limited number of examples, and from image data only, and discuss the possibilities of improving the method, in the immediate or longer term.

Information Leakage in Index Coding With Sensitive and Nonsensitive Messages

Index coding can be viewed as a compression problem with multiple decoders with side information. In such a setup, an encoder compresses a number of messages into a common codeword such that every decoder can decode its requested messages with the help of knowing some other messages as side information. In this paper, we study how much information is leaked to a guessing adversary observing the codeword in index coding, where some messages in the system are sensitive and others are not. The non-sensitive messages can be used by the encoder in a manner similar to secret keys to mitigate leakage of the sensitive messages to the adversary. We first characterize the optimal information leakage rate of a given index coding problem by the optimal compression rate of a related problem, which is constructed by adding an extra decoder with certain parameters to the original problem. Both the achievability and converse of the characterization are derived from a graph-theoretic perspective based on confusion graphs (Alon et al. 2008). In particular, the achievable coding scheme is a randomized mapping exploiting certain symmetrical properties of the confusion graph. Our second main result is a practical deterministic linear coding scheme, developed from the rank minimization method based on fitting matrices (Bar-Yossef et al. 2011). The linear scheme leads to an upper bound on the optimal leakage rate, which is proved to be tight over all deterministic scalar linear codes. While it is shown through an example that simultaneously achieving optimal compression and leakage rates is not always possible, time-sharing between different schemes could be used to balance the compression and leakage rates. Finally, we show how our results can be applied to different variants of index coding.

Distributed Channel Access for Control Over Unknown Memoryless Communication Channels

We consider the distributed channel access problem for a system consisting of multiple control subsystems that close their loop over a shared wireless network. We propose a distributed method for providing deterministic channel access without requiring explicit information exchange between the subsystems. This is achieved by utilizing timers for prioritizing channel access with respect to a local cost which we derive by transforming the control objective cost to a form that allows its local computation. This property is then exploited for developing our distributed deterministic channel access scheme. A framework to verify the stability of the system under the resulting scheme is then proposed. Next, we consider a practical scenario in which the channel statistics are unknown. We propose learning algorithms for learning the parameters of imperfect communication links for estimating the channel quality and, hence, define the local cost as a function of this estimation and control performance. We establish that our learning approach results in collision-free channel access. The behavior of the overall system is exemplified via a proof-of-concept illustrative example, and the efficacy of this mechanism is evaluated for large-scale networks via simulations.

Power Allocation in Ultra-Dense Networks Through Deep Deterministic Policy Gradient

The existing reinforcement learning-based downlink power allocation (PA) schemes mostly consider the power optimization space as a discrete value space, however, their results will deviate from the optimal result in ultra-dense networks, and the deviation grows as the network size increases. This letter proposed a PA model based on deep deterministic policy gradient (DDPG), where policy-based power selection assisted with value-based evaluation is leveraged to explore the optimal result from a continuous power space. Specifically, this model uses two CNNs named actors to formulate continuous deterministic PA strategy function instead of discrete power distribution sampling, and designs another two CNNs named critics for PA strategy evaluation and actor CNNs’ update supervision. Additionally, to reduce the interference, a tunable service base station set is designed for each user and is considered for the model training. Experiments demonstrate the proposed DDPG-based PA model respectively reaches 116.2% and 95.9% sum-rate relative to the iterative algorithm in small and large-scale networks.

Multi-objective control of isolated power systems under different uncertainty approaches

This paper proposes and compares a set of multi-objective supervisory controllers for an isolated power system including a gas turbine operating in load following mode as a dominant source of generation, a battery energy storage system, and stochastic renewable generation. It analyzes their capability to coordinate the gas turbine and energy storage to provide isochronous speed control, achieving fast frequency regulation for the local grid, while the gas turbine can still operate with minimum deviation from its optimal loading and the storage system can follow a pre-scheduled reference state of charge trajectory. In more detail, we consider and compare different control strategies to handle model and disturbance uncertainties, including stochastic model predictive control under various parametrizations of the scenario approach, and robust control under the H ∞ paradigm. The various controllers are compared against a benchmark, i.e., a deterministic predictive control strategy. Instead of point-to-point comparisons for some arbitrary cases (i.e., worst-case or expected), empirical distributions of the controller’s performance for the whole probability spectrum are derived, leading to more accurate and representative comparisons. The analyzed performance indicators are nominal dynamic performance, constraint violation probabilities, and expected system operation performance. The results indicate the clear superiority of stochastic control over both robust and deterministic control in dealing with both parametric and disturbance uncertainty. Moreover, choosing an appropriate parametrization is shown to be essential to achieve both good nominal performance and lower violations probability, indicating that the superiority of stochastic control comes with the drawback of needing user-defined tuning from the designer. • Multi-objective optimal MIMO control of isolated power systems with energy storage. • Simultaneous grid frequency regulation and state-of-charge schedule tracking. • Performance metrics for robust and predictive control assessment under uncertainty. • Comparison of control parametrizations for scenario-based stochastic control.

The influence of maximum (s,t)-cuts on the competitiveness of deterministic strategies for the Canadian Traveller Problem

We study the k-Canadian Traveller Problem, where the objective is to lead a traveller in an undirected weighted graph G from a source s to a target t, knowing that at most k edges of the graph are blocked and cannot be traversed. The locations of blockages are unknown at the beginning of the walk but a blocked edge is revealed when the traveller visits one of its endpoints. There exist graphs for which the competitive ratio of any deterministic strategies cannot be smaller than 2k+1. Conversely, there exists a very simple strategy, reposition, which achieves this ratio 2k+1. It consists in successively traversing shortest (s,t)-paths and coming back to s when the traveller is blocked.We refine this analysis by detecting families of graphs for which a smaller competitive ratio can be obtained. This paper produces a global analysis to understand the impact of the size of the maximum (s,t)-cuts of G on the competitiveness of deterministic strategies. We design deterministic strategies achieving a ratio ρk+O(λ), with ρ<2, for two different cut parameters λ. In particular, we propose a strategy called detour with a competitive ratio 2k+O(μmaxE), where μmaxE is the size of the maximum edge (s,t)-cut. Another contribution is a strategy called bypass with a competitive ratio 234k+O(λmaxV), where λmaxV is the size of the maximum vertex (s,t)-cut of all subgraphs of G. This produces an efficient algorithm for outerplanar graphs, which verify λmaxV≤2.

From deterministic to stochastic: an interpretable stochastic model-free reinforcement learning framework for portfolio optimization.

As a fundamental problem in algorithmic trading, portfolio optimization aims to maximize the cumulative return by continuously investing in various financial derivatives within a given time period. Recent years have witnessed the transformation from traditional machine learning trading algorithms to reinforcement learning algorithms due to their superior nature of sequential decision making. However, the exponential growth of the imperfect and noisy financial data that is supposedly leveraged by the deterministic strategy in reinforcement learning, makes it increasingly challenging for one to continuously obtain a profitable portfolio. Thus, in this work, we first reconstruct several deterministic and stochastic reinforcement algorithms as benchmarks. On this basis, we introduce a risk-aware reward function to balance the risk and return. Importantly, we propose a novel interpretable stochastic reinforcement learning framework which tailors a stochastic policy parameterized by Gaussian Mixtures and a distributional critic realized by quantiles for the problem of portfolio optimization. In our experiment, the proposed algorithm demonstrates its superior performance on U.S. market stocks with a 63.1% annual rate of return while at the same time reducing the market value max drawdown by 10% when back-testing during the stock market crash around March 2020.