Real-time Learning Capability Research Articles

Application of Hybrid Deep Reinforcement Learning for Managing Connected Cars at Pedestrian Crossings: Challenges and Research Directions

The autonomous vehicle is an innovative field for the application of machine learning algorithms. Controlling an agent designed to drive safely in traffic is very complex as human behavior is difficult to predict. An individual’s actions depend on a large number of factors that cannot be acquired directly by visualization. The size of the vehicle, its vulnerability, its perception of the environment and weather conditions, among others, are all parameters that profoundly modify the actions that the optimized model should take. The agent must therefore have a great capacity for adaptation and anticipation in order to drive while ensuring the safety of users, especially pedestrians, who remain the most vulnerable users on the road. Deep reinforcement learning (DRL), a sub-field that is supported by the community for its real-time learning capability and the long-term temporal aspect of its objectives looks promising for AV control. In a previous article, we were able to show the strong capabilities of a DRL model with a continuous action space to manage the speed of a vehicle when approaching a pedestrian crossing. One of the points that remains to be addressed is the notion of discrete decision-making intrinsically linked to speed control. In this paper, we will present the problems of AV control during a pedestrian crossing, starting with a modelization and a DRL model with hybrid action space adapted to the scalability of a vehicle-to-pedestrian (V2P) encounter. We will also present the difficulties raised by the scalability and the curriculum-based method.

Rogue AI: Cautionary Cases in Neuroradiology and What We Can Learn From Them.

Introduction In recent years, artificial intelligence (AI) in medical imaging has undergone unprecedented innovation and advancement, sparking a revolutionary transformation in healthcare. The field of radiology is particularly implicated, as clinical radiologists are expected to interpret an ever-increasing number of complex cases in record time. Machine learning software purchased by our institution is expected to help our radiologists come to a more prompt diagnosis by delivering point-of-care quantitative analysis of suspicious findings and streamlining clinical workflow. This paper explores AI's impact on neuroradiology, an area accounting for a substantial portion of recent radiology studies. We present a case series evaluating an AI software's performance in detecting neurovascular findings, highlighting five cases where AI interpretations differed from radiologists' assessments. Our study underscores common pitfalls of AI in the context of CT head angiograms, aiming to guide future AI algorithms. Methods We conducted a retrospective case series study at Stony Brook University Hospital, a large medical center in Stony Brook, New York, spanning from October 1, 2021 to December 31, 2021, analyzing 140 randomly sampled CT angiograms using AI software. This software assessed various neurovascular parameters, and AI findings were compared with neuroradiologists' interpretations. Five cases with divergent interpretations were selected for detailed analysis. Results Five representative cases in which AI findings were discordant with radiologists' interpretations are presented with diagnoses including diffuse anoxic ischemic injury, cortical laminar necrosis, colloid cyst, right superficial temporal artery-to-middle cerebral artery(STA-MCA) bypass, and subacute bilateral subdural hematomas. Discussion The errors identified in our case series expose AI's limitations in radiology. Our case series reveals that AI's incorrect interpretations can stem from complexities in pathology, challenges in distinguishing densities, inability to identify artifacts, identifying post-surgical changes in normal anatomy, sensitivity limitations, and insufficient pattern recognition. AI's potential for improvement lies in refining its algorithms to effectively recognize and differentiate pathologies. Incorporating more diverse training datasets, multimodal data, deep-reinforcement learning, clinical context, and real-time learning capabilities are some ways to improve AI's performance in the field of radiology. Conclusion Overall, it is apparent that AI applications in radiology have much room for improvement before becoming more widely integrated into clinical workflows. While AI demonstrates remarkable potential to aid in diagnosis and streamline workflows, our case series highlights common pitfalls that underscore the need for continuous improvement. By refining algorithms, incorporating diverse datasets, embracing multimodal information, and leveraging innovative machine learning strategies, AI's diagnostic accuracy can be significantly improved.

Real-Time Machine Learning for Multi-User Massive MIMO: Symbol Detection Using Multi-Mode StructNet

In this paper, we develop a learning-based symbol detection algorithm for massive MIMO-OFDM systems. To exploit the structure information inherited in the received signals from massive antenna array, multi-mode reservoir computing is adopted as the building block to facilitate over-the-air training in time domain. In addition, alternating recursive least square optimization method, and decision feedback mechanism are utilized in our algorithm to achieve the real-time learning capability. That is, the neural network is trained purely online with its weights updated on an OFDM symbol basis to promptly and adaptively track the dynamic environment. Furthermore, an online learning-based module is devised to compensate the nonlinear distortion caused by RF circuit components. On top of that, a learning-efficient classifier named StructNet is introduced in frequency domain to further improve the symbol detection performance by utilizing the QAM constellation structural pattern. Evaluation results demonstrate that our algorithm achieves substantial gain over traditional model-based approach and state-of-the-art learning-based techniques under dynamic channel environment and RF circuit nonlinear distortion. Moreover, empirical result reveals our NN model is robust to training label error, which benefits the decision feedback mechanism.

An Extended Spatiotemporal Contextual Learning and Memory Network Model for Hardware Implementation

A spatiotemporal contextual learning and memory network (STCLMN) model was proposed based on the hippocampal CA1 network using the difference equations suitable for digital computer simulations. However, the inherent capability of handling and processing of continuous-time asynchronous signals (spikes) cannot be fully employed in the STCLMN model. Furthermore, digital circuit implementation of the STCLMN is not suitable for real-time edge applications, wherein small size, low power consumption, and real-time learning capability are necessary. Therefore, this study extends the original STCLMN model using a set of differential equations to overcome the above problems encountered by STCLMN for asynchronous circuit implementation. The minimum accuracy required for a compact and low-power hardware implementation of the proposed model is then investigated. The validity and feasibility of the proposed model are confirmed through numerical simulations and experiments with a small mixed analog/digital circuit based on the investigated accuracy. In this study, LIF neurons are used because of their ease of circuit implementation. The proposed eSTCLMN model would enable an efficient circuit implementation of an asynchronous spiking neural network with spatiotemporal contextual one-shot learning and memory.

Memristive device based learning for navigation in robots

Biomimetic robots have gained attention recently for various applications ranging from resource hunting to search and rescue operations during disasters. Biological species are known to intuitively learn from the environment, gather and process data, and make appropriate decisions. Such sophisticated computing capabilities in robots are difficult to achieve, especially if done in real-time with ultra-low energy consumption. Here, we present a novel memristive device based learning architecture for robots. Two terminal memristive devices with resistive switching of oxide layer are modeled in a crossbar array to develop a neuromorphic platform that can impart active real-time learning capabilities in a robot. This approach is validated by navigating a robot vehicle in an unknown environment with randomly placed obstacles. Further, the proposed scheme is compared with reinforcement learning based algorithms using local and global knowledge of the environment. The simulation as well as experimental results corroborate the validity and potential of the proposed learning scheme for robots. The results also show that our learning scheme approaches an optimal solution for some environment layouts in robot navigation.

5th International Conferences on Geological,Geographical, Aerospaces and Earth Sciences 2017 (5th AeroEarth 2017)

1. Professor Dr Ah-Hwee TanProfessor of Computer Science Associate Chair (Research) School of Computer Science and Engineering Nanyang Technological University - SingaporeTitle: Biologically-inspired Machine Learning Theory for Knowledge Discovery:Machine learning and knowledge discovery are two critical intertwined functions in next generation intelligent information systems. This talk will present a family of self-organizing neural networks, collectively known as fusion Adaptive Resonance Theory (fusion ART), for building intelligent knowledge-based systems with real-time learning capabilities. By extending the Adaptive Resonance Theory (ART) models, consisting of a single input pattern field, into a multi-channel architecture, fusion ART unifies a number of important neural network designs developed over the past decades. Based on a universal set of neural encoding and adaptation principles, fusion AT supports a myriad of machine learning paradigms, notably unsupervised learning, supervised learning, and reinforcement learning. In addition, domain knowledge in the form of symbolic rules can be inserted into fusion ART and subsequently refined as part of the network's dynamics, which maximizes exploitation of the existing knowledge while retaining the plasticity of exploring new solutions. Case studies will be presented, illustrating how such self-adaptive intelligent systems may be used as autonomous Non-Player Characters (NPC) in first-person shooting games, Computer Generated Forces (CGF) in air combat simulation, and human-like avatars in 3D virtual environment.Biography:Dr Ah-Hwee Tan received Ph.D. in Cognitive and Neural Systems from Boston University, Master of Science and Bachelor of Science (First Class Honors) in Computer and Information Science from the National University of Singapore. He is currently a Professor of Computer Science and the Associate Chair (Research) at the School of Computer Science and Engineering (SCE), Nanyang Technological University. Prior to joining NTU, he was a Research Manager at the A*STAR Institute for Infocomm Research (I2R), heading the Text Mining and Intelligent Agents research programmes. His current research interests include cognitive and neural systems, brain-inspired intelligent agents, machine learning, and text mining. Dr. Tan has published ten edited books/proceeding volumes and over 200 technical papers in major international journals and conferences of his fields. He holds two US patents, five Singapore patents, and has spearheaded several A*STAR projects in commercializing a suite of knowledge management and text mining software. He serves as Associate Editor/Editorial Board Member of several journals, including IEEE Access, IEEE Transactions on Neural Networks and Learning, and IEEE Transactions on SMC Systems. He is a Senior Member of IEEE, a Member of Web Intelligence (WI) Technical Committee and Web Intelligence (WI) Conference Steering Committee, and Vice Chair of IEEE CIS Task Force on Towards Human-Like Intelligence.2. Professor Dr. Ida Ayu Dwi GiriantariTitle: The role of Geothermal Energy in Indonesia: Geothermal energy is heat energy generated and stored in the Earth. Thermal energy is the energy that determines the temperature of matter. The geothermal energy of the Earth's crust originates from the original formation of the planet and from radioactive decay of materials (in currently uncertain but possibly roughly equal proportions). The geothermal gradient, which is the difference in temperature between the core of the planet and its surface, drives a continuous conduction of thermal energy in the form of heat from the core to the surface.In the 20th century, demand for electricity led to the consideration of geothermal power as a generating source. Indonesia Power Electricity tested the first geothermal power generator on 4 July 1978, at the same dry steam field where geothermal acid extraction began. It successfully lit four light bulbs. Later, in 1990, the Indonesia's first commercial geothermal power plant was built.Short Bio:Ida Ayu Dwi Giriantari is Professor in Electrical Engineering Udayana University. She hold PhD from The University of New South Wales in 2003. Her research areas are in Power Plant, Renewable Energy and Power Transformers.She is Head of Magister Program of Electrical Engineering, Udayana University.

An Integrated System for UAV Control Using a Neural Network Implemented in a Prototyping Board

Modern aerospace vehicles are expected to have non-conventional flight envelopes and then, in order to operate in uncertain environments, they must guarantee a high level of robustness and adaptability. A Neural Network (NN) controller, with real-time learning capability, can be used in applications with manned or unmanned aerial vehicles. In this paper a novel real-time control system, based on a NN model, in order to control the trajectories of a hexacopter is proposed. The proposed NN is optimized by the analytical calculation of the embedding parameters. The paper shows a performance evaluation, through a real experimental testbed, of the proposed approach in terms of error measures and computation of the angular velocities of the hexacopter.

The Coordinating Role of Language in Real-Time Multimodal Learning of Cooperative Tasks

One of the defining characteristics of human cognition is our outstanding capacity to cooperate. A central requirement for cooperation is the ability to establish a “shared plan”-which defines the interlaced actions of the two cooperating agents-in real time, and even to negotiate this shared plan during its execution. In the current research we identify the requirements for cooperation, extending our earlier work in this area. These requirements include the ability to negotiate a shared plan using spoken language, to learn new component actions within that plan, based on visual observation and kinesthetic demonstration, and finally to coordinate all of these functions in real time. We present a cognitive system that implements these requirements, and demonstrate the system's ability to allow a Nao humanoid robot to learn a nontrivial cooperative task in real-time. We further provide a concrete demonstration of how the real-time learning capability can be easily deployed on a different platform, in this case the iCub humanoid. The results are considered in the context of how the development of language in the human infant provides a powerful lever in the development of cooperative plans from lower-level sensorimotor capabilities.

Multilevel Darwinist Brain (MDB): Artificial Evolution in a Cognitive Architecture for Real Robots

The multilevel Darwinist brain (MDB) is a cognitive architecture that follows an evolutionary approach to provide autonomous robots with lifelong adaptation. It has been tested in real robot on-line learning scenarios obtaining successful results that reinforce the evolutionary principles that constitute the main original contribution of the MDB. This preliminary work has lead to a series of improvements in the computational implementation of the architecture so as to achieve realistic operation in real time, which was the biggest problem of the approach due to the high computational cost induced by the evolutionary algorithms that make up the MDB core. The current implementation of the architecture is able to provide an autonomous robot with real time learning capabilities and the capability for continuously adapting to changing circumstances in its world, both internal and external, with minimal intervention of the designer. This paper aims at providing an overview or the architecture and its operation and defining what is required in the path towards a real cognitive robot following a developmental strategy. The design, implementation and basic operation of the MDB cognitive architecture are presented through some successful real robot learning examples to illustrate the validity of this evolutionary approach.

Real-time learning capability of neural networks.

In some practical applications of neural networks, fast response to external events within an extremely short time is highly demanded and expected. However, the extensively used gradient-descent-based learning algorithms obviously cannot satisfy the real-time learning needs in many applications, especially for large-scale applications and/or when higher generalization performance is required. Based on Huang's constructive network model, this paper proposes a simple learning algorithm capable of real-time learning which can automatically select appropriate values of neural quantizers and analytically determine the parameters (weights and bias) of the network at one time only. The performance of the proposed algorithm has been systematically investigated on a large batch of benchmark real-world regression and classification problems. The experimental results demonstrate that our algorithm can not only produce good generalization performance but also have real-time learning and prediction capability. Thus, it may provide an alternative approach for the practical applications of neural networks where real-time learning and prediction implementation is required.

Enhancing the universality and transferability of freeway incident detection using a Bayesian-based neural network

Development of a universal freeway incident detection algorithm is a task that remains unfulfilled despite the promising approaches that have been recently explored. Incident detection researchers are realizing that an operationally successful detection framework needs to fulfill a full set of recognized needs. In this paper we attempt to define one possible set of universality requirements. Among the set of requirements, a freeway incident detection algorithm needs to be operationally accurate and transferable. Guided by the envisioned requirements, we introduce a new algorithm with potential for enhanced performance. The algorithm is a modified form of the Bayesian-based Probabilistic Neural Network (PNN) that utilizes the concept of statistical distance. The paper is divided into three main sections. The first section is a detailed definition of the attributes and capabilities that a potentially universal freeway incident detection framework should possess. The second section discusses the training and testing of the PNN. In the third section, we evaluate the PNN relative to the universality template previously defined. In addition to a large set of simulated incidents, we utilize a fairly large real incident databases from the I-880 freeway in California and the I-35W in Minnesota to comparatively evaluate the performance and transferability of different algorithms, including the PNN. Experimental results indicate that the new PNN-based algorithm is competitive with the Multi Layer Feed Forward (MLF) architecture, which was found in previous studies to yield superior incident detection performance, while being significantly faster to train. In addition, results also point to the possibility of utilizing the real-time learning capability of this new architecture to produce a transferable incident detection algorithm without the need for explicit off-line retraining in the new site. In this respect, and unlike existing algorithms, the PNN has been found to markedly improve in performance with time in service as it retrains itself on captured incident data, verified by the Traffic Management Center (TMC) operator. Moreover, the overall PNN-based framework promises potential enhancements towards the envisioned universality requirements.

Neural networks in nonlinear aircraft flight control

This paper describes an approach for incorporating a neural network with real-time learning capability in a flight control architecture. The architecture is also applicable, in general, for the control of processes described by nonlinear differential equations of motion in which there exists a control for each degree of freedom. The main features are that the defining equations of motion for the process to be controlled are poorly known with respect to their functional forms, and that the functional forms, themselves, may undergo sudden and unexpected variation. It is well known that such systems are difficult to control, particularly when the effect of the control action enters nonlinearly. Numerical results based on 6DOF simulations of a high performance aircraft are presented to illustrate the potential benefits of incorporating neural networks as a part of a flight control system architecture.