Adaptive Deep Learning Research Articles

An adaptive deep reinforcement learning framework enables curling robots with human-like performance in real-world conditions

The game of curling can be considered a good test bed for studying the interaction between artificial intelligence systems and the real world. In curling, the environmental characteristics change at every moment, and every throw has an impact on the outcome of the match. Furthermore, there is no time for relearning during a curling match due to the timing rules of the game. Here, we report a curling robot that can achieve human-level performance in the game of curling using an adaptive deep reinforcement learning framework. Our proposed adaptation framework extends standard deep reinforcement learning using temporal features, which learn to compensate for the uncertainties and nonstationarities that are an unavoidable part of curling. Our curling robot, Curly, was able to win three of four official matches against expert human teams [top-ranked women's curling teams and Korea national wheelchair curling team (reserve team)]. These results indicate that the gap between physics-based simulators and the real world can be narrowed.

Group-based local adaptive deep multiple kernel learning with lp norm.

The deep multiple kernel Learning (DMKL) method has attracted wide attention due to its better classification performance than shallow multiple kernel learning. However, the existing DMKL methods are hard to find suitable global model parameters to improve classification accuracy in numerous datasets and do not take into account inter-class correlation and intra-class diversity. In this paper, we present a group-based local adaptive deep multiple kernel learning (GLDMKL) method with lp norm. Our GLDMKL method can divide samples into multiple groups according to the multiple kernel k-means clustering algorithm. The learning process in each well-grouped local space is exactly adaptive deep multiple kernel learning. And our structure is adaptive, so there is no fixed number of layers. The learning model in each group is trained independently, so the number of layers of the learning model maybe different. In each local space, adapting the model by optimizing the SVM model parameter α and the local kernel weight β in turn and changing the proportion of the base kernel of the combined kernel in each layer by the local kernel weight, and the local kernel weight is constrained by the lp norm to avoid the sparsity of basic kernel. The hyperparameters of the kernel are optimized by the grid search method. Experiments on UCI and Caltech 256 datasets demonstrate that the proposed method is more accurate in classification accuracy than other deep multiple kernel learning methods, especially for datasets with relatively complex data.

An Adaptive Deep Learning Framework for Shipping Container Code Localization and Recognition

Shipping containers play an important role in global transportation. As container codes are the unique identifiers for shipping containers, recognizing these codes is an essential step to manage the containers and logistics. The conventional code localization methods can easily be interfered by varied noises and cannot identify the best regions for code recognition. In this article, we propose an adaptive deep learning framework for shipping container code localization and recognition. In the framework, the noisy text regions will be removed by an adaptive score aggregation (ASA) algorithm. The code region boundaries are identified by the average-to-maximum suppression range (AMSR) algorithm. Thus, the predicted locations can be adjusted within this range to fit the code recognition model to achieve higher accuracy. The experimental results on the comparative study with the state-of-the-art models, including EAST, PSENet, GCRNN, and MaskTextSpotter, demonstrated that the proposed framework achieved better localization performance and obtained 93.33% recognition accuracy. The processing speed reaches 1.13 frames/s, which is sufficient to meet the operational requirements. Thus, the proposed solution will facilitate the digital transformation of shipping container management and logistics at ports.

Deep reinforcement learning based AGVs real-time scheduling with mixed rule for flexible shop floor in industry 4.0

Driven by the recent advances in industry 4.0 and industrial artificial intelligence, Automated Guided Vehicles (AGVs) has been widely used in flexible shop floor for material handling. However, great challenges aroused by the high dynamics, complexity, and uncertainty of the shop floor environment still exists on AGVs real-time scheduling. To address these challenges, an adaptive deep reinforcement learning (DRL) based AGVs real-time scheduling approach with mixed rule is proposed to the flexible shop floor to minimize the makespan and delay ratio. Firstly, the problem of AGVs real-time scheduling is formulated as a Markov Decision Process (MDP) in which state representation, action representation, reward function, and optimal mixed rule policy, are described in detail. Then a novel deep q-network (DQN) method is further developed to achieve the optimal mixed rule policy with which the suitable dispatching rules and AGVs can be selected to execute the scheduling towards various states. Finally, the case study based on a real-world flexible shop floor is illustrated and the results validate the feasibility and effectiveness of the proposed approach.

Deep learning based projector defocus compensation in single-pixel imaging.

Fourier single-pixel imaging (FSI) uses a digital projector to illuminate the target with Fourier basis patterns, and captures the back-scattered light with a photodetector to reconstruct a high-quality target image. Like other single-pixel imaging (SPI) schemes, FSI requires the projector to be focused on the target for best performance. In case the projector lens is defocused, the projected patterns are blurred and their interaction with the target produces a low-quality image. To address this problem, we propose a fast, adaptive, and highly-scalable deep learning (DL) approach for projector defocus compensation in FSI. Specifically, we employ a deep convolutional neural network (DCNN), which learns to offset the effects of projector defocusing through training on a large image set reconstructed with varying defocus parameters. The model is further trained on experimental data to make it robust against system bias. Experimental results demonstrate the efficacy of our method in reconstructing high-quality images at high projector defocusing. Comparative results indicate the superiority of our method over conventional FSI and existing projector defocus rectification method. The proposed work can also be extended to other SPI methods influenced by projector defocusing, and open avenues for applying DL to correct optical anomalies in SPI.

Spatio-temporal feature fusion for dynamic taxi route recommendation via deep reinforcement learning

Dynamic taxi route recommendation aims at recommending cruising routes to vacant taxis such that they can quickly find and pick up new passengers. Given citizens’ giant but unbalancing riding demand and the very limited taxis in a city, dynamic taxi route recommendation is essential for its ability to alleviate the waiting time of passengers and increase the earning of taxi drivers. Thus, in this paper we study the dynamic taxi route recommendation problem as a sequential decision-making problem and we design an effective two-step method to tackle it. First, we propose to consider and extract multiple real-time spatio-temporal features, which are related with the easiness degree of vacant taxis picking up new passengers. Second, we design an adaptive deep reinforcement learning method, which learns a carefully designed deep policy network to better fuse the extracted spatio-temporal features such that effective route recommendation can be done. Extensive experiments using real-world data from San Francisco and New York are conducted. Comparing with the state-of-the-arts, our method can increase at least 15.8% of average earning for taxi drivers and reduce at least 29.6% of average waiting time for passengers.

A novel adaptive deep learning model of Covid-19 with focus on mortality reduction strategies.

We employ deep learning to propose an Artificial Neural Network (ANN) based and data stream guided real-time incremental learning algorithm for parameter estimation of a non-intrusive, intelligent, adaptive and online analytical model of Covid-19 disease. Modeling and simulation of such problems pose an additional challenge of continuously evolving training data in which the model parameters change over time depending upon external factors. Our main contribution is that in a scenario of continuously evolving training data, unlike typical deep learning techniques, this non-intrusive algorithm eliminates the need to retrain or rebuild the model from scratch every time a new training data set is received. After validating the model, we use it to study the impact of different strategies for epidemic control. Finally, we propose and simulate a strategy of controlled natural immunization through risk-based population compartmentalization (PC) wherein the population is divided in Low Risk (LR) and High Risk (HR) compartments based on risk factors (like comorbidities and age) and subjected to different disease transmission dynamics by isolating the HR compartment while allowing the LR compartment to develop natural immunity. Upon release from the preventive isolation, the HR compartment finds itself surrounded by enough number of immunized individuals to prevent the spread of infection and thus most of the deaths occurring in this group are avoided.

Intelligent prognostics of machining tools based on adaptive variational mode decomposition and deep learning method with attention mechanism

In the modern manufacturing industry, remaining useful life (RUL) prediction of the machining tools plays a significant role in promoting machining efficiency, ensuring product quality and reducing production costs. In recent years, many data-driven prognostic approaches have been developed for machining tools, but few of them have considered the operating conditions such as spindle load and rotating speed that may have great impact on the degradation behavior and sensor signals. It may give rise to more uncertainty and lead to an obvious decrease in prediction accuracy when operating condition changes. Besides, feature extraction from the raw signals that are nonstationary, nonlinear, and mixed with abundant noise is essential but quite challenging. To address these issues, this paper proposes a novel prognostic approach for machining tools under dynamic operating condition with varying spindle load. In the proposed approach, an adaptive variational mode decomposition (VMD) is newly developed to adaptively search the optimal parameters for processing the raw vibration data, then several components with good trendability and noise robustness are obtained for feature extraction. Furthermore, a deep learning model combining one-dimensional convolutional long short-term memory (LSTM) with attention mechanism is constructed to perform RUL prediction. Numerical experiments on a real-world case study show the effectiveness and superiority of the proposed approach in comparison with other baseline data-driven approaches.

Context Aware Energy Disaggregation Using Adaptive Bidirectional LSTM Models

Energy disaggregation, or Non-Intrusive Load Monitoring (NILM), describes various processes aiming to identify the individual contribution of appliances, given the aggregate power signal. In this paper, a non-causal adaptive context-aware bidirectional deep learning model for energy disaggregation is introduced. The proposed model, CoBiLSTM, harnesses the representational power of deep recurrent Long Short-Term Memory (LSTM) neural networks, while fitting two basic properties of NILM problem which state of the art methods do not appropriately account for: non-causality and adaptivity to contextual factors (e.g., seasonality). A Bayesian-optimized framework is introduced to select the best configuration of the proposed regression model, driven by a self-training adaptive mechanism. Furthermore, the proposed model is structured in a modular way to address multi-dimensionality issues that arise when the number of appliances increases. Experimental results indicate the proposed method’s superiority compared to the current state of the art.

Traffic Light Detection in Autonomous Driving Systems

Vision-only systems are currently very popular for use in autonomous driving systems and advanced driver-assistance systems. These systems operate by using input images from a camera and no other sensors. One of the tasks these systems needs to perform is the detection and understanding of traffic lights in a traffic environment, by localizing all relevant traffic lights in an image received from an on board camera mounted on the vehicle. This article proposes a traffic light recognition system where adaptive thresholding and deep learning are used for region proposal and traffic light localization, respectively. The LISA open-source dataset is used along with custom augmentation methods in order to increase the number of available data samples. Performance of the developed system is presented in the form of true and false positive rates obtained on the test data. The classification part of the algorithm gives a total of 89.60% true detection rate, while the regression part of the model produced a correct location of the traffic light in 92.67% of cases.

Wind speed prediction based on singular spectrum analysis and neural network structural learning

High precision wind speed prediction is an important technology to improve the operation and maintenance efficiency of wind power. The wind is intermittent and the characteristics of wind speed data can change with time; however, the majority of previous investigations have focused on the hybrid models with fixed structures without considering the adaptive structural learning. In this paper, an adaptive structural deep learning model is proposed to capture the dynamic characteristics of the wind speed. The proposed model involves two parts: the singular spectrum analysis and the modified adaptive structural learning of neural network. The first part is used for wind speed data feature extraction; the second part, designed by combining the adaptive structural learning of neural networks with the long short-term memory network and the newly-designed mix-way structure layer, is used for forecasting. Four simulation tests and several comparison models are used to analyze the prediction accuracy and stability of the established model. The simulation results show that the established model has the advantages of excellent data-dependent learning ability and prediction performance compared to the benchmark models in wind speed prediction.

A Software Defined Network Based Fuzzy Normalized Neural Adaptive Multipath Congestion Control for the Internet of Things

Multipath Transmission Control Protocol (MPTCP) enables multi-homed devices to establish multiple simultaneous routes for data transmission. Congestion Control (CC) is a fundamental mechanism for implementing and designing MPTCP. The Internet of Things (IoT) networks generate a massive volume of heterogeneous traffic with high dimensional states and diverse QoS characteristics. The existing MPTCP CC algorithms are unable to perform efficiently under highly mobile and dynamic IoT environments. We propose a novel model-free SDN-based adaptive actor-critic deep reinforcement learning framework based on a fuzzy normalized neural network to address the issue of CC for MPTCP in the IoT networks. In the proposed method, an agent can learn efficiently and better approximate the state-action value function of the actor and the action function of the critic to adjust the sub-flows congestion windows size adaptively according to the dynamic condition of a network. Simulation results show that the proposed scheme outperforms the state of the art schemes in terms of the goodput under highly-dynamic IoT environments.

Choroidal thickness estimation from colour fundus photographs by adaptive binarisation and deep learning, according to central serous chorioretinopathy status

This study was performed to estimate choroidal thickness by fundus photography, based on image processing and deep learning. Colour fundus photography and central choroidal thickness examinations were performed in 200 normal eyes and 200 eyes with central serous chorioretinopathy (CSC). Choroidal thickness under the fovea was measured using optical coherence tomography images. The adaptive binarisation method was used to delineate choroidal vessels within colour fundus photographs. Correlation coefficients were calculated between the choroidal vascular density (defined as the choroidal vasculature appearance index of the binarisation image) and choroidal thickness. The correlations between choroidal vasculature appearance index and choroidal thickness were −0.60 for normal eyes (p < 0.01) and −0.46 for eyes with CSC (p < 0.01). A deep convolutional neural network model was independently created and trained with augmented training data by K-Fold Cross Validation (K = 5). The correlation coefficients between the value predicted from the colour image and the true choroidal thickness were 0.68 for normal eyes (p < 0.01) and 0.48 for eyes with CSC (p < 0.01). Thus, choroidal thickness could be estimated from colour fundus photographs in both normal eyes and eyes with CSC, using imaging analysis and deep learning.

Efficient complex ISAR object recognition using adaptive deep relation learning

Complex inverse synthetic aperture radar (ISAR) object recognition is a critical and challenging problem in computer vision tasks. An efficient complex object recognition method for ISAR images is proposed based on adaptive deep relation learning. (i) An adaptive multimodal mechanism is proposed to greatly improve the multimodal sampling and transformation capabilities of convolutional neural networks and increase the resolutions of the feature maps. In particular, an adaptive ranking and selection strategy for related regions are proposed. (ii) Deep graphical learning is proposed to jointly estimate large numbers of heterogeneous attributes and leverage the relations among features and attributes. Extensive experiments performed on real-world datasets show that the proposed method outperforms other state-of-the-art methods.

Adaptive Deep Cascade Broad Learning System and Its Application in Image Denoising

This article proposes a novel regularization deep cascade broad learning system (DCBLS) architecture, which includes one cascaded feature mapping nodes layer and one cascaded enhancement nodes layer. Then, the transformation feature representation is easily obtained by incorporating the enhancement nodes and the feature mapping nodes. Once such a representation is established, a final output layer is constructed by implementing a simple convex optimization model. Furthermore, a parallelization framework on the new method is designed to make it compatible with large-scale data. Simultaneously, an adaptive regularization parameter criterion is adopted under some conditions. Moreover, the stability and error estimate of this method are discussed and proved mathematically. The proposed method could extract sufficient available information from the raw data compared with the standard broad learning system and could achieve compellent successes in image denoising. The experiments results on benchmark datasets, including natural images as well as hyperspectral images, verify the effectiveness and superiority of the proposed method in comparison with the state-of-the-art approaches for image denoising.

An adaptive deep reinforcement learning approach for MIMO PID control of mobile robots

Intelligent control systems are being developed for the control of plants with complex dynamics. However, the simplicity of the PID (proportional–integrative–derivative) controller makes it still widely used in industrial applications and robotics. This paper proposes an intelligent control system based on a deep reinforcement learning approach for self-adaptive multiple PID controllers for mobile robots. The proposed hybrid control strategy uses an actor–critic structure and it only receives low-level dynamic information as input and simultaneously estimates the multiple parameters or gains of the PID controllers. The proposed approach was tested in several simulated environments and in a real time robotic platform showing the feasibility of the approach for the low-level control of mobile robots. From the simulation and experimental results, our proposed approach demonstrated that it can be of aid by providing with behavior that can compensate or even adapt to changes in the uncertain environments providing a model free unsupervised solution. Also, a comparative study against other adaptive methods for multiple PIDs tuning is presented, showing a successful performance of the approach.

Adaptive Deep Learning Aided Digital Predistorter Considering Dynamic Envelope

Memory effects of radio frequency power amplifiers (PAs) can interact with dynamic transmitting signals, dynamic operations, and dynamic environment, resulting in complicated nonlinear problems of the PAs. Recently, deep learning based schemes have been proposed to deal with the memory effects. Although these schemes are powerful in constructing complex nonlinear structures, they are still direct learning-based and are relatively static. In this paper, we propose an adaptive deep learning aided digital predistortion (DL-DPD) model by optimizing a deep regression neural network. Thanks to the sequence structure of the proposed DL-DPD, we then make the linearization architecture more adaptive by using multiple sub-DPD modules and an ensemble predicting process. The results show the effectiveness of the proposed adaptive DL-DPD, and reveals that the online system handovers the sub-DPD modules more frequently than expected.

Deep Learning-Based Autoencoder for m-User Wireless Interference Channel Physical Layer Design

Deep learning (DL) based autoencoder (AE) has been proposed recently as a promising, and potentially disruptive approach to design the physical layer of beyond-5G communication systems. Compared to a traditional communication system with a multiple-block structure, the DL based AE approach provides a new paradigm to physical layer design with a pure data-driven and end-to-end learning based solution. In this article, we address the dynamic interference in a multi-user Gaussian interference channel. We show that standard constellation are not optimal for this context, in particular, for a high interference condition. We propose a novel adaptive DL based AE to overcome this problem. With our approach, dynamic interference can be learned and predicted, which updates the learning processing for the decoder. Compared to other machine learning approaches, our method does not rely on a fixed training function, but is adaptive and applicable to practical systems. In comparison with the conventional system using $n$ -psk or $n$ -QAM modulation schemes with zero force (ZF) and minimum mean square error (MMSE) equalizer, the proposed adaptive deep learning (ADL) based AE demonstrates a significant achievable BER in the presence of interference, especially in strong and very strong interference scenarios. The proposed approach has laid the foundation of enabling adaptable constellation for 5G and beyond communication systems, where dynamic and heterogeneous network conditions are envisaged.

A Novel Signal Detection Scheme Based on Adaptive Ensemble Deep Learning Algorithm in SC-FDE Systems

Reliable signal detection plays an essential role in enhancing the quality of signal transmission in wireless communication systems. In this paper, we combine signal detection theory with a deep learning model and propose a novel signal detection scheme based on adaptive ensemble long short term memory (AE-LSTM) neural network to handle wireless single carrier frequency domain equalization (SC-FDE) systems in an end-to-end manner. The feature information used for offline training of the deep learning model is extracted from the received signal containing channel state information (CSI) after the multi-path channel and fast Fourier transform (FFT), and the labels are assigned according to the constellation map adopted at the transmitter. To improve the adaptability of the system, we utilize the received power under different delays as the adaptive factor to integrate the output of each sub-network. Then the original data generated by the channel model is recovered by using the trained model instead of channel estimation and frequency domain equalization. Comparative experiments on SC-FDE symbol detection demonstrate that the proposed scheme achieves better performance in terms of reliability than the traditional scheme and the similar deep learning scheme.

Dynamic MLML-tree based adaptive object detection using heterogeneous data distribution

We propose a robust object-detector ensemble by introducing a dynamic multi-layer multi-label (MLML)-tree–based adaptive deep learning framework. In many heterogeneous data distributions, deep attributes show latent hierarchical clustering properties. Object detector performance can be enhanced using the dynamic MLML-tree, which can adjust the ambiguities between inter-class nodes and variations between sub-class nodes. In the MLML-tree, dynamic multi-label (DML) trees are configured in two layers and adapt to using a sparse working dataset. First, coarse object clusters are built using an outlier-aware soft-clustering algorithm. Each coarse cluster is denoted by an inter-class node and is associated with an adaptive object detector in DML tree layer 1. It is built by recursively partitioning inter-class nodes until homogeneous object-class leaves are built. DML tree layer 2 is built for each object-class node, which is associated with a convolutional neural network detector, recursively. A novel sub-class can be learned automatically in DML tree layer 2 by applying semi-supervised learning. Extensive experiments show that the proposed method is superior to state-of-the-art techniques using PASCAL Visual Object Classes (VOC) 2007, VOC 2012, and the Microsoft Common Objects in Context (COCO) datasets.