Neural Networks For Function Approximation Research Articles

A Unified Perspective on Value Backup and Exploration in Monte-Carlo Tree Search

Monte-Carlo Tree Search (MCTS) is a class of methods for solving complex decisionmaking problems through the synergy of Monte-Carlo planning and Reinforcement Learning (RL). The highly combinatorial nature of the problems commonly addressed by MCTS requires the use of efficient exploration strategies for navigating the planning tree and quickly convergent value backup methods. These crucial problems are particularly evident in recent advances that combine MCTS with deep neural networks for function approximation. In this work, we propose two methods for improving the convergence rate and exploration based on a newly introduced backup operator and entropy regularization. We provide strong theoretical guarantees to bound convergence rate, approximation error, and regret of our methods. Moreover, we introduce a mathematical framework based on the use of the α-divergence for backup and exploration in MCTS. We show that this theoretical formulation unifies different approaches, including our newly introduced ones, under the same mathematical framework, allowing to obtain different methods by simply changing the value of α. In practice, our unified perspective offers a flexible way to balance between exploration and exploitation by tuning the single α parameter according to the problem at hand. We validate our methods through a rigorous empirical study from basic toy problems to the complex Atari games, and including both MDP and POMDP problems.

Mesoscopic V2X simulation framework to enhance simulation performance

The rapid evolution of vehicular communication has led to numerous new algorithms and applications based on this technology. Neglecting issues arising from wireless communication, such as the loss of information and delays, can result in problems such as reduced performance or compromised safety. However, while simulating V2X demands significant computational resources, it proves unsuitable for complex testing setups, including mixed-reality testing. This paper enhances V2X simulation by relying on an ecosystem based on SUMO, OMNeT++, Veins, and INET simulation tools. The proposed novel method introduces mesoscopic simulation in Vehicular Ad-hoc Networks to increase simulation performance to a level where real-time behavior is achievable. Meanwhile, it can also be beneficial in the acceleration of regular simulations. The presented solution introduces Meso nodes that are capable of aggregating communication across an entire traffic area, facilitated by a neural network function approximator. Results showed substantial performance gain while simulation accuracy was preserved.

UAV-assisted fair communications for multi-pair users: A multi-agent deep reinforcement learning method

Unmanned aerial vehicle (UAV) plays an important role in scenarios like search and rescue, remote communication relay, battlefield mobile networks, etc. In this paper, we investigate multiple UAV relays providing real-time transmission for multi-pair ground users (GUs) in the absence of ground-based stations (GBSs). Due to the limited load capacities and energy resources on-board, fairness between multi-pair users, throughput maximization, as well as the connectivity maintenance between UAVs are jointly considered. We formulate the energy-efficient fair throughput objective function, which turns to be non-convex with hybrid variables. To solve this intractable problem, we utilize the power of neural networks in function approximation and propose a multi-agent deep reinforcement learning (MADRL)-based algorithm. Different from traditional MADRL algorithms, we utilize the method, named independent proximal policy optimization (IPPO), which allows agents to update according to their own observations memory and encourages more explorations to some extend, as the base of our solution. Simulation results demonstrate that our algorithm outperforms some baselines in terms of fairness, throughput as well as energy consumption.

Safety-critical control for autonomous underwater vehicles with unknown disturbance using function approximator

Safety and stability are two primary factors that affect the task execution performance of autonomous underwater vehicles (AUVs), especially when they operate in a disturbed environment. The unknown disturbances and unmodeled dynamics will severely degrade the performance, resulting in unsafe and unstable behaviors. This paper presents a safety-critical control framework for AUVs in the presence of unknown disturbances using a neural network function approximator. The high-order control barrier functions (CBFs) are utilized to address the safe constraints of relative degree two. A nominal controller is designed in which the unknown lumped disturbances are compensated by a radial basis function neural network, thus ensuring that the tracking errors are uniformly ultimately bounded. Besides, the function approximation and the associated bounded approximation errors are adopted to estimate the unknown lumped disturbance terms in the second-order derivative of CBFs, which alleviates the mismatch of the forward invariance condition for the safety set. The minimally invasive quadratic program (QP) is formulated to make minimal sacrifice on the performance of nominal controller. The proposed method encompasses a straightforward structure for disturbance estimation, alongside a highly efficient explicit estimating rule that promotes rapid convergence. Integrated with computationally efficient QP, the proposed method demonstrates substantial promise for real-time implementation. Comparative numerical results with disturbance observer and input-to-state safety-based robust methods reveal an improved trade-off between safety and control performance.

Active Steering Controller for Driven Independently Rotating Wheelset Vehicles Based on Deep Reinforcement Learning

This paper proposes an active steering controller for Driven Independently Rotating Wheelset (DIRW) vehicles based on deep reinforcement learning (DRL). For the two-axle railway vehicles equipped with Independently Rotating Wheelsets (IRWs), each wheel connected to a wheel-side motor, the Ape-X DDPG controller, an enhanced version of the Deep Deterministic Policy Gradient (DDPG) algorithm, is adopted. Incorporating Distributed Prioritized Experience Replay (DPER), Ape-X DDPG trains neural network function approximators to obtain a data-driven DIRW active steering controller. This controller is utilized to control the input torque of each wheel, aiming to improve the steering capability of IRWs. Simulation results indicate that compared to the existing model-based H∞ control algorithm and data-driven DDPG control algorithm, the Ape-X DDPG active steering controller demonstrates better curving steering performance and centering ability in straight tracks across different running conditions and significantly reduces wheel–rail wear. To validate the proposed algorithm’s efficacy in real vehicles, a 1:5 scale model of the DIRW vehicle and its digital twin dynamic model were designed and manufactured. The proposed control algorithm was deployed on the scale vehicle and subjected to active steering control experiments on a scaled track. The experimental results reveal that under the active steering control of the Ape-X DDPG controller, the steering performance of the DIRW scale model on both straight and curved tracks is significantly enhanced.

Approximating smooth and sparse functions by deep neural networks: Optimal approximation rates and saturation

Constructing neural networks for function approximation is a classical and longstanding topic in approximation theory. In this paper, we aim at constructing deep neural networks with three hidden layers using a sigmoidal activation function to approximate smooth and sparse functions. Specifically, we prove that the constructed deep nets with controllable magnitude of free parameters can reach the optimal approximation rate in approximating both smooth and sparse functions. In particular, we prove that neural networks with three hidden layers can avoid the phenomenon of saturation, i.e., the phenomenon that for some neural network architectures, the approximation rate stops improving for functions of very high smoothness.

Learning to Transmit Fresh Information in Energy Harvesting Networks

We study age of information (AoI) minimization in an ad hoc network consisting of energy harvesting transmitters that are scheduled to send status updates to their intended receivers. The transmission scheduling with power allocation problem over a communication session is first studied assuming apriori knowledge of channel state information, harvested energy, and update packet arrivals, i.e., the offline setting. The global optimal scheduling policy in this case is the solution of a mixed integer linear program which is known to be computationally hard. We propose a supervised-learning-based algorithm to mitigate the high computational complexity. A bidirectional recurrent neural network that interprets user scheduling as a time-series classification problem is trained and tested to achieve near-optimal AoI. Next, we consider online scheduling and power allocation with causal knowledge of the system state, which is an infinite-state Markov decision problem. In this case, the related reinforcement learning problem is solved by a model-free on-policy deep reinforcement learning, where the actor-critic algorithm with deep neural network function approximation is implemented. Comparable AoI to the optimal is demonstrated and faster runtime of learning solvers is observed, verifying the efficacy of learning in terms of both optimality and computational energy efficiency for AoI-focused scheduling and resource allocation problems in wireless networks.

Robust Adaptive Finite-Time Synergetic Tracking Control of Delta Robot Based on Radial Basis Function Neural Networks

This paper presents a robust proportional derivative adaptive nonsingular finite-time synergetic tracking control (PDAFS) for a parallel Delta robot system. First, a finite-time synergetic controller combined with a proportional derivative (PD) control is constructed based on an object-oriented model to fulfill the robust tracking control of the robot. Then, an adaptive radial basis function approximation neural network (RBF) is designed to compensate for the effects of uncertainty parameters and external disturbances. Second, a second-order sliding mode (SOSM) differentiator is implemented to reduce the chattering noises due to the low-resolution encoders. Third, the stability theorems of the proposed control scheme are provided, where the Lyapunov stability theory is used to prove the theorems. Then, simulations of the helix trajectory tracking and the pick-and-place task are demonstrated on the Delta robot to validate the advantages of the proposed control scheme. Based on the advances, an implementing control system of the proposed controller is performed to improve the Delta robot’s performance in the experiments.

Best neural approximation for 2-quasi inner product spaces

Many researchers studied two-normed spaces for objects of function approximation. Also, neural networks are great instruments for function approximation from function spaces, especially, Lebesque spaces (Lp ). In this paper, the aim is to approximate Lebesque integrable functions via two-quasi inner product that is defined here in terms of two-norm. Moreover, there is a definition of a special type of neural network with special weights to estimate the approximation error, which is equivalent to the modulus of smoothness of functions of study. Estimates of infimum and supremum bounds so that the modulus of smoothness is involved in each bound. It is interesting to investigate more work related to what we presented in this study, not only for those fields of neural network function approximation, but also to approximate functions by various types of operators.

Deep reinforcement learning for automated design of reinforced concrete structures

AbstractThis study proposes a novel concept of reinforcement learning (RL) framework to facilitate automated structural design, with a particular focus on reinforced concrete (RC) beam design as case studies. We trained an RL agent called deep deterministic policy gradient (DDPG) with a convolutional neural network function approximators. The RL agent was successfully trained to design RC beams subject to American Concrete Institute (318) provisions without any hand‐labeled dataset. A python‐based RC beam design environment was developed and used to simulate RC beam designs with customized reward functions that encouraged the agent to minimize material cost by maximizing reward. The DDPG agent self‐learned cost‐effective RC beam design through 100,000 randomly generated design cases during the training procedure. The trained agent was able to design an RC beam in a cost‐effective way while taking both flexural and shear reinforcement arrangements into consideration. The trained agent generated near‐optimal design parameters without the need for unnecessary iterations over various design conditions. The performance of the agent was validated with 100 design cases and comparative studies, showing a great promise for automated RC beam design.

Monte Carlo Tree Search Methods for the Earth-Observing Satellite Scheduling Problem

This work explores on-board planning for the single spacecraft, multiple ground station Earth-observing satellite scheduling problem through artificial neural network function approximation of state–action value estimates generated by Monte Carlo tree search (MCTS). An extensive hyperparameter search is conducted for MCTS on the basis of performance, safety, and downlink opportunity utilization to determine the best hyperparameter combination for data generation. A hyperparameter search is also conducted on neural network architectures. The learned behavior of each network is explored, and each network architecture’s robustness to orbits and epochs outside of the training distributions is investigated. Furthermore, each algorithm is compared with a genetic algorithm, which serves to provide a baseline for optimality. MCTS is shown to compute near-optimal solutions in comparison to the genetic algorithm. The state–action value networks are shown to match or exceed the performance of MCTS in six orders of magnitude less execution time, showing promise for execution on board spacecraft.

A modified batch intrinsic plasticity method for pre-training the random coefficients of extreme learning machines

In extreme learning machines (ELM) the hidden-layer coefficients are randomly set and fixed, while the output-layer coefficients of the neural network are computed by a least squares method. The randomly-assigned coefficients in ELM are known to influence its performance and accuracy significantly. In this paper we present a modified batch intrinsic plasticity (modBIP) method for pre-training the random coefficients in the ELM neural networks. The current method is devised based on the same principle as the batch intrinsic plasticity (BIP) method, namely, by enhancing the information transmission in every node of the neural network. It differs from BIP in two prominent aspects. First, modBIP does not involve the activation function in its algorithm, and it can be applied with any activation function in the neural network. In contrast, BIP employs the inverse of the activation function in its construction, and requires the activation function to be invertible (or monotonic). The modBIP method can work with the often-used non-monotonic activation functions (e.g. Gaussian, swish, Gaussian error linear unit, and radial-basis type functions), with which BIP breaks down. Second, modBIP generates target samples on random intervals with a minimum size, which leads to highly accurate computation results when combined with ELM. The combined ELM/modBIP method is markedly more accurate than ELM/BIP in numerical simulations. Ample numerical experiments are presented with shallow and deep neural networks for function approximation and boundary/initial value problems with partial differential equations. They demonstrate that the combined ELM/modBIP method produces highly accurate simulation results, and that its accuracy is insensitive to the random-coefficient initializations in the neural network. This is in sharp contrast with the ELM results without pre-training of the random coefficients.

Machine learning time-local generators of open quantum dynamics

In the study of closed many-body quantum systems one is often interested in the evolution of a subset of degrees of freedom. On many occasions it is possible to approach the problem by performing an appropriate decomposition into a bath and a system. In the simplest case the evolution of the reduced state of the system is governed by a quantum master equation with a time-independent, i.e. Markovian, generator. Such evolution is typically emerging under the assumption of a weak coupling between the system and an infinitely large bath. Here, we are interested in understanding to which extent a neural network function approximator can predict open quantum dynamics - described by time-local generators - from an underlying unitary dynamics. We investigate this question using a class of spin models, which is inspired by recent experimental setups. We find that indeed time-local generators can be learned. In certain situations they are even time-independent and allow to extrapolate the dynamics to unseen times. This might be useful for situations in which experiments or numerical simulations do not allow to capture long-time dynamics and for exploring thermalization occurring in closed quantum systems.

A Radial Base Localized Distribution for Deterministic Function Approximation Neural Networks.(Dept.E)

A Radial Base Localized Distribution for Deterministic Function Approximation Neural Networks.(Dept.E)

Noise-Boosted Backpropagation Learning of Feedforward Threshold Neural Networks for Function Approximation

Aiming to ensure the feasibility of the backpropagation training of feedforward threshold neural networks, each hidden unit layer is designed to be composed of a sufficiently large number of hard-limiting activation functions that are excited by mutually independent external noise components and the weighted inputs simultaneously. The application of noise to nondifferentiable activation functions enables a proper definition of the gradients, and the injected noise is treated as a network parameter that can be adaptively updated by a stochastic gradient descent learning rule. This noise-boosted backpropagation learning process is found to converge to a nonzero optimized level of noise, indicating that the injected noise is beneficial both for the learning and for the ensuing retrieval phase. For minimizing the total error energy of the function approximation in the designed threshold neural network, the proposed noise-boosted backpropagation learning method is proven to be better than directly injecting noise into network inputs or weight coefficients. The Lipschitz continuous property of the noise-smoothed activation function in the hidden unit layer is demonstrated to guarantee the local convergence of the learning process. Beyond the Gaussian injected noise, the optimal noise type is also numerically solved for training the designed threshold neural network. Test experiments for approximating nonlinear functions and real-world datasets verify the feasibility of this noise-boosted backpropagation algorithm in the threshold neural network. These results not only extend the analysis of the beneficial effects of noise similar to stochastic resonance and exploited here to the universal approximation capabilities of threshold neural networks, but also allow backpropagation training of neural networks with a much wider family of nondifferentiable activation functions.

NNs-Based Adaptive Control for Genetic Regulatory Networks With Sensor Faults

This article considers an adaptive control problem of genetic regulatory networks, where unknown sensor faults are considered. By using the function approximation capability of neural networks, a neural-networks-based gene circuit control method is designed, where the unknown sensor faults are compensated. Comparing with the existing results where regulatory functions meet known SUM logic, the regulatory functions considered in this article are unknown and do not satisfy SUM logic. Furthermore, the fault negative influence on neural network function approximation, which is caused by state sensor faults, has been compensated. In sense of Lyapunov stability theory, the closed-loop system is asymptotically bounded and all the signals in the system converge to an adjustable neighborhood of the origin. Finally, some simulation results are given to show the effectiveness of the design method.

Optimization of FDM process parameters for dual extruder 3d printer using Artificial Neural network

Abstract Fused Deposition Modelling is one of the most widely used processes of additive manufacturing or 3D printing. The FDM process of 3D printing deposits material in the form of a continuous flow, layer-by-layer to make objects. As the FDM-based products are used in various fields it becomes important to look after the mechanical aspects, part quality, and the economical aspect of FDM 3D printing and hence optimize the necessary process parameters. In this study, critical process parameters like layer thickness, air gap, raster width, build orientation, raster angle, and the number of contours is optimized for enhancing the properties of FDM printed part such as tensile strength surface roughness, and build time. The material used for 3D printing is polylactic acid (PLA). The task of training the data sets and optimizing them was accomplished by using function approximation of Artificial Neural Network. ANN can predict experimental data with a coefficient of correlation R = 0.9981,0.9984,0.99837 subsequently for tensile strength, Build time, and surface roughness and root mean square error as 0.5543, 0.578 and 0.241 for three outputs. Further, it is revealed that build orientation is the most important parameter for optimum results.

Data-based reduced-order modeling of nonlinear two-time-scale processes

This work focuses on data-based reduced-order modeling of nonlinear processes that exhibit time-scale multiplicity. Using time-series data from all the state variables of a nonlinear process, an approach that involves nonlinear principal component analysis and neural network function approximators is employed to identify the fast and slow process state variables as well as compute a nonlinear slow manifold function approximation in which the fast variables are “slaved” in terms of the slow variables. Subsequently, a nonlinear sparse identification approach is employed to calculate a dynamic model of nonlinear first-order ordinary differential equations that describe the temporal evolution of the slow process states. The method is applied to two chemical process examples, and the advantages of the proposed method over using only sparse identification for the original two-time-scale process are discussed. The approach can be thought of as a data-based analogue of the classical singular perturbation modeling of two-time-scale processes where explicit time-scale separation is assumed (and expressed in terms of a small positive parameter ε multiplying the time derivative of the fast states), and the reduced-order slow subsystem is calculated analytically.

Reinforcement Learning-Based School Energy Management System

Energy efficiency is a key to reduced carbon footprint, savings on energy bills, and sustainability for future generations. For instance, in hot climate countries such as Qatar, buildings are high energy consumers due to air conditioning that resulted from high temperatures and humidity. Optimizing the building energy management system will reduce unnecessary energy consumptions, improve indoor environmental conditions, maximize building occupant’s comfort, and limit building greenhouse gas emissions. However, lowering energy consumption cannot be done despite the occupants’ comfort. Solutions must take into account these tradeoffs. Conventional Building Energy Management methods suffer from a high dimensional and complex control environment. In recent years, the Deep Reinforcement Learning algorithm, applying neural networks for function approximation, shows promising results in handling such complex problems. In this work, a Deep Reinforcement Learning agent is proposed for controlling and optimizing a school building’s energy consumption. It is designed to search for optimal policies to minimize energy consumption, maintain thermal comfort, and reduce indoor contaminant levels in a challenging 21-zone environment. First, the agent is trained with the baseline in a supervised learning framework. After cloning the baseline strategy, the agent learns with proximal policy optimization in an actor-critic framework. The performance is evaluated on a school model simulated environment considering thermal comfort, CO2 levels, and energy consumption. The proposed methodology can achieve a 21% reduction in energy consumption, a 44% better thermal comfort, and healthier CO2 concentrations over a one-year simulation, with reduced training time thanks to the integration of the behavior cloning learning technique.

New method of traffic flow forecasting based on quantum particle swarm optimization strategy for intelligent transportation system

SummaryTraffic flow forecasting is one of the essential means to realize smart cities and smart transportation. The accurate and effective prediction will provide an important basis for decision‐making in smart transportation systems. This paper proposes a new method of traffic flow forecasting based on quantum particle swarm optimization (QPSO) strategy for intelligent transportation system (ITS). We establish a corresponding model based on the characteristics of the traffic flow data. The genetic simulated annealing algorithm is applied to the quantum particle swarm algorithm to obtain the optimized initial cluster center, and is applied to the parameter optimization of the radial basis neural network prediction model. The function approximation of radial basis neural network is used to obtain the required data. In addition, in order to compare the performance of the algorithms, a comparison study with other related algorithms such as QPSO radial basis function (QPSO‐RBF) is also performed. Simulation results show that compared with other algorithms, the proposed algorithm can reduce prediction errors and get better and more stable prediction results.