A continual causal learning architecture with multiscale graph attention for robust mold level fluctuation prediction in smart continuous casting systems

Continual learning in medical devices: FDA's action plan and beyond

Deep reinforcement learning has achieved significant success in various domains. However, it still faces a huge challenge when learning multiple tasks in sequence. This is because the interaction in a complex setting involves continual learning that results in the change in data distributions over time. A continual learning system should ensure that the agent acquires new knowledge without forgetting the previous one. However, catastrophic forgetting may occur as the new experience can overwrite previous experience due to limited memory size. The dual experience replay algorithm which retains previous experience is widely applied to reduce forgetting, but it cannot be applied in scalable tasks when the memory size is constrained. To alleviate the constrained by the memory size, we propose a new continual reinforcement learning algorithm called Self-generated Long-term Experience Replay (SLER). Our method is different from the standard dual experience replay algorithm, which uses short-term experience replay to retain current task experience, and the long-term experience replay retains all past tasks’ experience to achieve continual learning. In this paper, we first trained an environment sample model called Experience Replay Mode (ERM) to generate the simulated state sequence of the previous tasks for knowledge retention. Then combined the ERM with the experience of the new task to generate the simulation experience all previous tasks to alleviate forgetting. Our method can effectively decrease the requirement of memory size in multiple tasks, reinforcement learning. We show that our method in StarCraft II and the GridWorld environments performs better than the state-of-the-art deep learning method and achieve a comparable result to the dual experience replay method, which retains the experience of all the tasks.

With the advancement of artificial intelligence technologies in recent years, research on intelligent robots has progressed. Robots are required to understand human intentions and communicate more smoothly with humans. Since gestures can have a variety of meanings, gesture recognition is one of the essential issues in communication between robots and humans. In addition, robots need to learn new gestures as humans grow. Moreover, individual gestures vary. Because catastrophic forgetting occurs in training new data in traditional gesture recognition approaches, it is necessary to preserve the prepared data and combine it with further data to train the model from scratch. We propose a Multi-scopic Cognitive Memory System (MCMS) that mimics the lifelong learning process of humans and can continuously learn new gestures without forgetting previously learned gestures. The proposed system comprises a two-layer structure consisting of an episode memory layer and a semantic memory layer, with a topological map as its backbone. The system is designed with reference to conventional continuous learning systems in three ways: (i) using a dynamic architecture without setting the network size, (ii) adding regularization terms to constrain learning, and (iii) generating data from the network itself and performing relearning. The episode memory layer clusters the data and learns their spatiotemporal representation. The semantic memory layer generates a topological map based on task-related inputs and stores them as longer-term episode representations in the robot's memory. In addition, to alleviate catastrophic forgetting, the memory replay function can reinforce memories autonomously. The proposed system could mitigate catastrophic forgetting and perform continuous learning by using both machine learning benchmark datasets and real-world data compared to conventional methods.

Causal Cognitive Architecture 3: A solution to the binding problem

The surface quality of the continuous casting strands is closely related to the initial solidification of liquid steel in the vicinity of the mold meniscus, and thus the clear understanding of the behavior of molten steel initial solidification would be of great importance for the control of the quality of final slab. With the development of the mold simulator techniques, the complex interrelationship between the solidified shell surface profile, heat flux, shell thickness, mold level fluctuation, and the infiltrated slag film was well illustrated in our previous study. As the second part, this article investigated the effect of the mold oscillation frequency, stroke, and mold level fluctuation on the initial solidification of the molten steel through the conduction of five different experiments. Results suggested that in the case of the stable mold level, the oscillation marks (OMs) exhibit equally spaced horizon depressions on the shell surface, where the heat flux at the meniscus area raises rapidly during negative strip time (NST) period and the presence of each OMs on the shell surface is corresponding to a peak value of the heat flux variation rate. Otherwise, the shell surface is poorly defined by the existence of wave-type defects, such as ripples or deep depressions, and the heat flux variation is irregular during NST period. The rising of the mold level leads to the longer-pitch and deeper OMs formation; conversely, the falling of mold level introduces shorter-pitch and shallower OMs. With the increase of the mold oscillation frequency, the average value of the low-frequency heat flux at the meniscus increases; however, it decreases when the mold oscillation stroke increases. Additionally, the variation amplitude of the high-frequency temperature and the high-frequency heat flux decreases with the increase of the oscillation frequency and the reduction of the oscillation stroke.

In the continuous casting of steel, mold level fluctuation caused by unsteady bulging of the solidifying shell affects the surface quality of the product and stable operation of the continuous casting process. To clarify this problem, inter-roll bulging and unsteady bulging in experimental casting machines and commercial continuous casting machines have been measured by various methods in a number of studies. In this study, the fluctuation of inter-roll bulging with time in a commercial continuous casting machine was measured by an ultrasonic range finder using a water column. In these measurements, the fluctuation of the segment was also considered. The validity of the measured data was estimated by comparison with the mold level. The results showed that both inter-roll bulging and the mold level fluctuated with the cycle calculated from the roll pitch and casting speed, and the amplitude of the mold level fluctuation converted from the amount of fluctuation of inter-roll bulging corresponded to the actual mold level fluctuation. Therefore, the cycle and absolute amount of inter-roll bulging fluctuation measured in this study were considered reasonable. These results also revealed that the value measured in this study corresponded directly to the fluctuation of inter-roll bulging as such.

This paper is concerned with a class of continuous time uncertain systems which satisfy a certain Integral Quadratic Constraint. The problems of robust filtering, robust prediction, and robust smoothing for such systems are defined, and nonconservative solutions are given in terms of Riccati differential equations. This paper also addresses a problem of robust observability for this class of uncertain systems.

Continual learning systems will interact with humans, with each other, and with the physical world through time - and continue to learn and adapt as they do. An important open problem for continual learning is a large-scale benchmark which enables realistic evaluation of algorithms. In this paper, we study a natural setting for continual learning on a massive scale. We introduce the problem of personalized online language learning (POLL), which involves fitting personalized language models to a population of users that evolves over time. To facilitate research on POLL, we collect massive datasets of Twitter posts. These datasets, Firehose10 M and Firehose100 M, comprise 100 million tweets, posted by one million users over six years. Enabled by the Firehose datasets, we present a rigorous evaluation of continual learning algorithms on an unprecedented scale. Based on this analysis, we develop a simple algorithm for continual gradient descent (ConGraD) that outperforms prior continual learning methods on the Firehose datasets as well as earlier benchmarks. Collectively, the POLL problem setting, the Firehose datasets, and the ConGraD algorithm enable a complete benchmark for reproducible research on web-scale continual learning.

This study sheds light on a new generation of Swedish food producers, market gardeners, who are attracting attention in terms of food system sustainability, prompted by increasing consumer awareness about the value of healthy and locally produced food. Market gardening is part of a global agroecological movement opposed to industrialized agriculture and its negative impacts on the environment and rural communities. These food producers challenge the incumbent agri-food regime through the building of alternative food networks. This case-based study involving 14 young vegetable producers showed that young people who engage in market gardening are strongly motivated by dual incentives, namely entrepreneurship and transformation to sustainability. Six main competences were identified as important for market gardeners: practical skills related to growing vegetables, business management, innovation and continuous learning, systems thinking, pioneering, and networking. Individuals develop their skills through continuous experiential learning and gain knowledge through peer-to-peer learning using social media. However, they need to acquire certain skills relating to their daily work in the field and to managing a business. Market gardeners currently face a number of barriers erected by the sociopolitical environment, in particular regarding access to research-based knowledge, extension services, and business support.

The continuous casting system is the most important to solidify the liquid steel in the steel industry. Steel is the backbone of civilization and modernization. So, there is a need to optimize the performance of continuous casting system of steel industry. Continuous casting system has six subsystems: “Pouring turret ladle”, “Tundish”, “Mold”, “Water spray chamber”, “Support roller” and “Torch cutter”. Series configuration is used to arrange these subsystems. The subsystem “Pouring turret ladle” is having three similar units. These units are operating in parallel. The subsystems “Tundish”, “Mold”, “Water spray chamber” and “Support roller” have a single unit. The subsystem “Torch cutter” contains two identical units: one is operative and other keep in cold standby. For all subsystems, the distribution of repair rates and failure rates of continuous casting system are taken as arbitrary distributions. Analysis of continuous casting system has been done by using supplementary variable technique. The numerical results of reliability measure of continuous casting system in terms of availability and profit have been computed by assuming exponential, Rayleigh and Weibull distributions.

Objective: the rapid expansion of the use of telemedicine in clinical practice and the increasing use of Artificial Intelligence has raised many privacy issues and concerns among legal scholars. Due to the sensitive nature of the data involved particular attention should be paid to the legal aspects of those systems. This article aimed to explore the legal implication of the use of Artificial Intelligence in the field of telemedicine, especially when continuous learning and automated decision-making systems are involved; in fact, providing personalized medicine through continuous learning systems may represent an additional risk. Particular attention is paid to vulnerable groups, such as children, the elderly, and severely ill patients, due to both the digital divide and the difficulty of expressing free consent.Methods: comparative and formal legal methods allowed to analyze current regulation of the Artificial Intelligence and set up its correlations with the regulation on telemedicine, GDPR and others.Results: legal implications of the use of Artificial Intelligence in telemedicine, especially when continuous learning and automated decision-making systems are involved were explored; author concluded that providing personalized medicine through continuous learning systems may represent an additional risk and offered the ways to minimize it. Author also focused on the issues of informed consent of vulnerable groups (children, elderly, severely ill patients).Scientific novelty: existing risks and issues that are arising from the use of Artificial Intelligence in telemedicine with particular attention to continuous learning systems are explored.Practical significance: results achieved in this paper can be used for lawmaking process in the sphere of use of Artificial Intelligence in telemedicine and as base for future research in this area as well as contribute to limited literature on the topic.

Learning new tasks and skills in succession without overwriting or interfering with prior learning (i.e., “catastrophic forgetting”) is a computational challenge for both artificial and biological neural networks, yet artificial systems struggle to achieve even rudimentary parity with the performance and functionality apparent in biology. One of the processes found in biology that can be adapted for use in artificial systems is sleep, in which the brain deploys numerous neural operations relevant to continual learning and ripe for artificial adaptation. Here, we investigate how modeling three distinct components of mammalian sleep together affects continual learning in artificial neural networks: (1) a veridical memory replay process observed during non-rapid eye movement (NREM) sleep; (2) a generative memory replay process linked to REM sleep; and (3) a synaptic downscaling process which has been proposed to tune signal-to-noise ratios and support neural upkeep. To create this tripartite artificial sleep, we modeled NREM veridical replay by training the network using intermediate representations of samples from the current task. We modeled REM by utilizing a generator network to create intermediate representations of samples from previous tasks for training. Synaptic downscaling, a novel con-tribution, is modeled utilizing a size-dependent downscaling of network weights. We find benefits from the inclusion of all three sleep components when evaluating performance on a continual learning CIFAR-100 image classification benchmark. Maximum accuracy improved during training and catastrophic forgetting was reduced during later tasks. While some catastrophic forget-ting persisted over the course of network training, higher levels of synaptic downscaling lead to better retention of early tasks and further facilitated the recovery of early task accuracy during subsequent training. One key takeaway is that there is a trade-off at hand when considering the level of synaptic downscaling to use - more aggressive downscaling better protects early tasks, but less downscaling enhances the ability to learn new tasks. Intermediate levels can strike a balance with the highest overall accuracies during training. Overall, our results both provide insight into how to adapt sleep components to enhance artificial continual learning systems and highlight areas for future neuroscientific sleep research to further such systems.

Continual learning for scene analysis is a continuous process to incrementally learn distinct events, actions, and even noise models from past experiences using different sensory modalities. In this paper, an Auditory Scene Analysis (ASA) approach based on a continual learning system is developed to incrementally learn the acoustic events in a dynamically-changing domestic environment. The events being salient sound sources are localized by a Sound Source Localization (SSL) method to robustly process the signals of the localized sound source in the domestic scene where multiple sources can co-exist. For real-time ASA, audio patterns are segmented from the acoustic signal stream of the localized source for extraction of the audio features, and construction of a feature set for each pattern. The continual learning is employed via a time-series algorithm, Hidden Markov Model (HMM), on these feature sets from acoustic signals stemming from the sources. The learning process is investigated by conducting a variety of experiments to evaluate the performance of Unknown Event Detection (UED), Acoustic Event Recognition (AER), and continual learning using a Hierarchical HMM algorithm. The Hierarchical HMM consists of two layers: 1) a lower layer in which AER is performed using an HMM for each event and the event-wise likelihood thresholds; and 2) an upper layer in which UED is achieved by one HMM with a suspicion threshold through the audio features with their proto symbols stemming from the lower layer HMMs. We verified the effectiveness of the proposed system capable of continual learning, AER and UED in terms of False-Positive Rates, True-Positive Rates, recognition accuracy and computational time to meet the demands in a learning task of multiple events in real-time. The effectiveness of the AER system has been verified with high accuracy, and a short retraining time in real-time ASA having nine different sounds.

In statistics, Box-Jenkins Time Series is a linear method widely used to forecasting. The linearity makes the method inadequate to forecast real time series, which could present irregular behavior. On the other hand, in artificial intelligence FeedForward Artificial Neural Networks and Continuous Machine Learning Systems are robust handlers of data in the sense that they are able to reproduce nonlinear relationships. Their main disadvantage is the selection of adequate inputs or attributes better related with the output or category. In this paper, we present a methodology that employs Box-Jenkins Time Series as feature selector to Feedforward Artificial Neural Networks inputs and Continuous Machine Learning Systems attributes. We also apply this methodology to forecast some real time series collected in a power plant. It is shown that Feedforward Artificial Neural Networks performs better than Continuous Machine Learning Systems, which in turn performs better than Box-Jenkins Time Series.

Real-time continual learning which could continually learn a series of computer vision tasks, has received more attention on robotic platforms and embedded systems with on-device training manner. However, most existing real-time continual learning systems need more training time when learning new visual perception tasks; the class imbalance issues are neglected amongst most real-time continual learning systems, which could degrade the generalization performance among past learned tasks, as well as the new tasks. To address these challenges above, we in this work propose to preserve and reuse the learned knowledge to achieve real-time continual learning. To be specific, when encountering a new visual perception task, we choose to freeze the learned backbone weights of all the past tasks, which could speedup the training process for new tasks. Moreover, we also store several activations volumes from some intermediate layer, which could further reduce the computational cost. For the improved perception performance, a focal loss is employed to guide the attention for poorly-identified sample categories and mitigate the class imbalance issue. Experimental results on popularly-used continual learning justify the efficiency and effectiveness of our proposed method.