Conventional Learning Algorithms Research Articles

Lorentz attractor excitation-based structural damage identification using state space curvature reconstruction- enhanced transformer

Abstract Vibration-based structural damage identification has been widely investigated. Different from previous studies that analyze vibrational responses in time and frequency domains, a new Lorentz attractor excitation-based damage identification is becoming a novel strategy with the advantage of capturing the structure’s nonlinear dynamic effects. In this study, Lorentz attractor-based chaotic signals were employed as excitation signals for the structural damage identification of a frame structure. Nonlinear responses were recorded and damages of bolt looseness at different locations were considered. The structural damages could be revealed in the state-space plot of the responses. A state space curvature reconstruction method was introduced to enhance the key features of the nonlinear responses. A small-sample damage identification is performed using a deep learning algorithm – a Transformer with an accuracy of 92.38%. The advantages of the proposed method over conventional deep learning algorithms were validated. The proposed method can be applied to health conditions identification of buildings, bridges, and trusses.

Achieving Robust Learning Outcomes in Autonomous Driving with DynamicNoise Integration in Deep Reinforcement Learning

The advancement of autonomous driving technology is becoming increasingly vital in the modern technological landscape, where it promises notable enhancements in safety, efficiency, traffic management, and energy use. Despite these benefits, conventional deep reinforcement learning algorithms often struggle to effectively navigate complex driving environments. To tackle this challenge, we propose a novel network called DynamicNoise, which was designed to significantly boost the algorithmic performance by introducing noise into the deep Q-network (DQN) and double deep Q-network (DDQN). Drawing inspiration from the NoiseNet architecture, DynamicNoise uses stochastic perturbations to improve the exploration capabilities of these models, thus leading to more robust learning outcomes. Our experiments demonstrated a 57.25% improvement in the navigation effectiveness within a 2D experimental setting. Moreover, by integrating noise into the action selection and fully connected layers of the soft actor–critic (SAC) model in the more complex 3D CARLA simulation environment, our approach achieved an 18.9% performance gain, which substantially surpassed the traditional methods. These results confirmed that the DynamicNoise network significantly enhanced the performance of autonomous driving systems across various simulated environments, regardless of their dimensionality and complexity, by improving their exploration capabilities rather than just their efficiency.

Differentiation of tuberculous and brucellar spondylitis using conventional MRI-based deep learning algorithms

PurposeTo investigate the feasibility of deep learning (DL) based on conventional MRI to differentiate tuberculous spondylitis (TS) from brucellar spondylitis (BS). MethodsA total of 383 patients with TS (n = 182) or BS (n = 201) were enrolled from April 2013 to May 2023 and randomly divided into training (n = 307) and validation (n = 76) sets. Sagittal T1WI, T2WI, and fat-suppressed (FS) T2WI images were used to construct single-sequence DL models and combined models based on VGG19, VGG16, ResNet18, and DenseNet121 network. The area under the receiver operating characteristic curve (AUC) was used to assess the classification performance. The AUC of DL models was compared with that of two radiologists with different levels of experience. ResultsThe AUCs based on VGG19, ResNet18, VGG16, and DenseNet121 ranged from 0.885 to 0.973, 0.873 to 0.944, 0.882 to 0.929, and 0.801 to 0.933, respectively, and VGG19 models performed better. The diagnostic efficiency of combined models outperformed single-sequence DL models. The combined model of T1WI, T2WI, and FS T2WI based on VGG19 achieved optimal performance, with an AUC of 0.973. In addition, the performance of all combined models based on T1WI, T2WI, and FS T2WI was better than that of two radiologists (P<0.05). ConclusionThe DL models have potential guiding value in the diagnosis of TS and BS based on conventional MRI and provide a certain reference for clinical work.

CLESR: Context-Based Label Knowledge Enhanced Span Recognition for Named Entity Recognition

Named entity recognition (NER) stands as a pivotal task in natural language processing, bearing profound implications for diverse downstream applications. Recent scholarship underscores the substantial performance gains attainable by integrating label knowledge into token representations, effectively treating named entity recognition as a machine reading comprehension task. Nevertheless, extant methodologies often inadequately leverage the potential of label knowledge. In response to this limitation, we introduce the Context-based Label Knowledge Enhanced Span Recognition (CLESR) architecture, designed to augment label knowledge through the assimilation of contextual information. We formulate an annotation paradigm tailored to nested scenarios, concurrently training external context-based label knowledge using conventional word association learning algorithms. The ensuing context-based label knowledge is seamlessly integrated into the model via a dedicated label attention module, thereby fortifying label learning capabilities during training. To adeptly manage both flat and nested entities, we implement a global pointer as our decoding strategy, enabling direct predictions of the positions and corresponding categories of named entities. Rigorous experimentation across six widely recognized benchmarks substantiates the efficacy of CLESR in both flat and nested tasks. Impressively, our model achieves performance enhancements surpassing those of the baseline BERT-MRC model, with gains of + 0.29%, + 0.77%, + 0.39%, + 1.9%, and + 0.47% on English CoNLL2003, English OntoNotes 4.0, Chinese MSRA, English ACE2004, and English ACE2005, respectively.

Sample-efficient reinforcement learning with knowledge-embedded hybrid model for optimal control of mining industry

The froth flotation process is a cost-effective and widely employed method for mineral separation. A core challenge is to maximize mineral recovery while maintaining a specified minimum concentrate grade. This requires precise control of aeration and slurry level and overcoming disturbances. The rapid advancements in reinforcement learning present a promising approach for controlling froth flotation. However, reinforcement learning suffers from sample inefficiency, which significantly hampers its application to real-world problems. In this research, we proposed an innovative sample-efficient model-based reinforcement learning algorithm to enhance flotation performance by directly leveraging the dynamics of the slurry phase. Our proposed algorithm involves the construction of a hybrid model, comprising a physical model and a data residual model, which effectively learns the underlying dynamics of the flotation process during the interaction between the reinforcement learning agent and process. The physical model integrates domain knowledge into the hybrid model to expedite the training process and enhance the interpretability and generalizability of the hybrid model. To capture uncertainty and address unmodeled aspects of the physical model, we employ an ensemble data residual model. Actor and critic are augmented with fuzzy representation modules based on fuzzy logic inference to improve the learning capacity of the networks and the multi-head critic is applied to reduce overestimation biases and enhance training stability. Case studies demonstrate that, compared to the baseline algorithm, our proposed algorithm requires merely 86.8% of the samples employed by conventional model-based reinforcement learning algorithm and improves the long-term return by at least 12.0%, underscoring the role of domain knowledge in facilitating efficient reinforcement learning.

Learning with sparse reward in a gap junction network inspired by the insect mushroom body.

Animals can learn in real-life scenarios where rewards are often only available when a goal is achieved. This 'distal' or 'sparse' reward problem remains a challenge for conventional reinforcement learning algorithms. Here we investigate an algorithm for learning in such scenarios, inspired by the possibility that axo-axonal gap junction connections, observed in neural circuits with parallel fibres such as the insect mushroom body, could form a resistive network. In such a network, an active node represents the task state, connections between nodes represent state transitions and their connection to actions, and current flow to a target state can guide decision making. Building on evidence that gap junction weights are adaptive, we propose that experience of a task can modulate the connections to form a graph encoding the task structure. We demonstrate that the approach can be used for efficient reinforcement learning under sparse rewards, and discuss whether it is plausible as an account of the insect mushroom body.

Federated Variational Inference: Towards Improved Personalization and Generalization

Conventional federated learning algorithms train a single global model by leveraging all participating clients’ data. However, due to heterogeneity in client generative distributions and predictive models, these approaches may not appropriately approximate the predictive process, converge to an optimal state, or generalize to new clients. We study personalization and generalization in stateless cross-device federated learning setups assuming heterogeneity in client data distributions and predictive models. We first propose a hierarchical generative model and formalize it using Bayesian Inference. We then approximate this process using Variational Inference to train our model efficiently. We call this algorithm Federated Variational Inference (FedVI). We use PAC-Bayes analysis to provide generalization bounds for FedVI. We evaluate our model on FEMNIST and CIFAR-100 image classification and show that FedVI beats the state-of-the-art on both tasks.

Finding community structure in Bayesian networks by heuristic K-standard deviation method

When constructing a Bayesian network for high-dimensional data, due to the complex relationships among distinct nodes, the difficulty in detecting the community structure will directly restrict the feasibility of the divide-and-conquer learning algorithm. This study attempts to solve this problem from the shortest path perspective and proposes the heuristic K-standard deviation algorithm. Firstly, we design a novel heuristic function and drive the A* algorithm to find the shortest paths between nodes in the Bayesian network. In addition, the new heuristic function is theoretically proved to be admissible and consistent. Experiments on different synthetic datasets and benchmark Bayesian networks verify that the proposed heuristic K-standard deviation algorithm generally gets better clustering performance than other representative algorithms and improves the efficiency and accuracy of the conventional Bayesian network structure learning algorithms, especially for high-dimensional data.

Decoupled Optimisation for Long-Tailed Visual Recognition

When training on a long-tailed dataset, conventional learning algorithms tend to exhibit a bias towards classes with a larger sample size. Our investigation has revealed that this biased learning tendency originates from the model parameters, which are trained to disproportionately contribute to the classes characterised by their sample size (e.g., many, medium, and few classes). To balance the overall parameter contribution across all classes, we investigate the importance of each model parameter to the learning of different class groups, and propose a multistage parameter Decouple and Optimisation (DO) framework that decouples parameters into different groups with each group learning a specific portion of classes. To optimise the parameter learning, we apply different training objectives with a collaborative optimisation step to learn complementary information about each class group. Extensive experiments on long-tailed datasets, including CIFAR100, Places-LT, ImageNet-LT, and iNaturaList 2018, show that our framework achieves competitive performance compared to the state-of-the-art.

Empty container repositioning problem using a reinforcement learning framework with multi-weight adaptive reward function

ABSTRACT In logistics networks, empty container congestion and scarcity often stem from trade imbalance and supply-demand mismatch. This paper focuses on the problem of empty container repositioning in maritime logistics and proposes a reinforcement learning framework that integrates a self-adaptive mechanism for adjusting the weights of a multi-objective reward function. The objective is to enhance container utilization and reduce scheduling costs. By reviewing the development of the empty container repositioning problem and analysing the advantages of using reinforcement learning to address the temporal and spatial complexity, the problem is modelled as a Markov decision process and tackled using reinforcement learning techniques. To achieve the optimization objectives, which involve reducing resource shortages at various locations and minimizing resource repositioning costs, a multi-objective reward function is introduced to capture the mutually constrained preferences. The weights of the reward function are dynamically adjusted to account for the potential time-varying preferences of the agent, mitigating the issue of poor generalization performance associated with fixed-weight reward functions. Comparative experimental analysis against conventional reinforcement learning algorithms demonstrates the superior performance of the proposed approach in problem solving. Based on the results and practical requirements of the case study, relevant recommendations for addressing empty container repositioning are presented.

Deep Reinforcement Learning-Based Security-Constrained Battery Scheduling in Home Energy System

The home energy system today involves multiple renewable energy sources and battery energy storage systems, which can be considered as a microgrid. The battery energy storage system is a key component in the home energy system for the sake of filling the gap between the user demand and volatile energy supplies to maximize the techno and economic performances. However, the battery scheduling must suffer the stochastic nature of renewable energy resources and loads, which results in an intractable multi-period stochastic optimization problem with security constraints. An improved actor-critic-based reinforcement learning is proposed for this issue, where a distributional critic net is applied for faster and more accurate reward estimation under uncertainties, and a policy net incorporating protective secondary control is adopted to satisfy security constraints, preventing the unsafe state of batteries during the trial-and-error process. Numerical tests show that the proposed approach outperforms conventional reinforcement learning algorithms, as well as the rule-based battery scheduling approach while guaranteeing safe operation. The robustness and adaptability of the proposed method are also verified in case studies with different optimization tasks.

Exploiting sparsity of hyperspectral image: A novel approach for compressive hyperspectral image reconstruction using deep learning

Compressive hyperspectral imaging is an emerging technology that captures compressed two-dimensional (2D) measurements and subsequently reconstructs three-dimensional (3D) hyperspectral images, capitalizing on the sparsity prior. Numerous end-to-end networks have been proposed, yielding notable improvements in reconstruction speed and quality compared to traditional algorithms. However, these networks fall short in exploiting the sparsity of hyperspectral images in their direct mapping from 2D measurements to 3D hyperspectral images. To address this limitation, this paper introduces a novel approach for compressive hyperspectral image reconstruction using deep learning. The deep learning network comprises two components: the sparse coefficient recovery net and the dictionary training net. The dictionary training net incorporates a pre-trained dictionary, whose parameters can also be updated during the backpropagation process. Concurrently, the sparse coefficient recovery net generates sparse coefficients under this dictionary. Our approach draws inspiration from conventional dictionary learning algorithms, enabling effective utilization of signal sparsity. In comparison to several state-of-the-art methods, our method achieves superior reconstruction quality and demonstrates enhanced robustness against noise. The proposed approach is envisioned to offer fresh insights into hyperspectral images reconstruction using deep learning models.

Improving Out-of-Distribution Generalization in SAR Image Scene Classification with Limited Training Samples

For practical maritime SAR image classification tasks with special imaging platforms, scenes to be classified are often different from those in the training sets. The quantity and diversity of the available training data can also be extremely limited. This problem of out-of-distribution (OOD) generalization with limited training samples leads to a sharp drop in the performance of conventional deep learning algorithms. In this paper, a knowledge-guided neural network (KGNN) model is proposed to overcome these challenges. By analyzing the saliency features of various maritime SAR scenes, universal knowledge in descriptive sentences is summarized. A feature integration strategy is designed to assign the descriptive knowledge to the ResNet-18 backbone. Both the individual semantic information and the inherent relations of the entities in SAR images are addressed. The experimental results show that our KGNN method outperforms conventional deep learning models in OOD scenarios with varying training sample sizes and achieves higher robustness in handling distributional shifts caused by weather conditions, terrain type, and sensor characteristics. In addition, the KGNN model converges within many fewer epochs during training. The performance improvement indicates that the KGNN model learns representations guided by beneficial properties for ODD generalization with limited training samples.

A novel subject-wise dictionary learning approach using multi-subject fMRI spatial and temporal components

The conventional dictionary learning (DL) algorithms aim to adapt the dictionary/sparse code to individual functional magnetic resonance imaging (fMRI) data. Thus, lacking the capability to consolidate the spatiotemporal diversities offered by other subjects. Considering that subject-wise (sw) data matrix can be decomposed into the sparse linear combination of multi-subject (MS) time courses and MS spatial maps, two new algorithms, sw sequential DL (swsDL) and sw block DL (swbDL), have been proposed. They are based on the novel framework, defined by the mixing model, where base matrices prepared by operating a computationally fast sparse spatiotemporal blind source separation method over multiple subjects are employed to adapt the mixing matrices to sw training data. They solve the optimization models formulated using l_0/l_1-norm penalization/constraints through dictionary/sparse code pair update and alternating minimization approach. They are unique because no existing sparse DL method can incorporate MS spatiotemporal components while updating sw atoms/sparse codes, which can eventually be assembled using neuroscience knowledge to extract group-level dynamics. Various fMRI datasets are used to evaluate and compare the performance of the proposed algorithms with existing state-of-the-art algorithms. Specifically, overall, a 14% increase in the mean correlation value and 39% reduction in the mean computation time exhibited by swsDL and swbDL, respectively, over the adaptive consistent sequential dictionary algorithm.

Computation Offloading and Beamforming Optimization for Energy Minimization in Wireless-Powered IRS-Assisted MEC

Intelligent reflecting surface (IRS) has been recently exploited as a symbiotic radio (SR) technology to improve energy-and spectral-efficiencies in wireless systems. In this paper, we consider a symbiotic IRS-assisted mobile edge computing (MEC) system that allows edge users to first harvest RF power from a hybrid access point (HAP) and then offload its computational workload to the MEC server associated with the HAP. We aim to minimize the HAP’s energy consumption by jointly optimizing the users’ offloading schemes, the HAP’s active beamforming, and the IRS’s passive beamforming strategies. We propose an optimization-driven hierarchical deep deterministic policy gradient (OH-DDPG) framework to decompose the energy minimization problem into the optimization and the learning sub-problems, respectively. The outer-loop DDPG learning method adapts the IRS’s passive beamforming strategy, while the inner-loop optimization deals with the other control variables with reduced dimensionality. Moreover, to improve the learning efficiency, we extend OH-DDPG to the multi-agent scenario. In particular, the HAP first estimates the users’ offloading strategy by the inner-loop optimization and shares it with all user agents. Then, each user agent refines its offloading decision using the DDPG algorithm independently. This can avoid signaling overhead among users and improve the multi-user learning efficiency. Simulation results show that the proposed OH-DDPG and the multi-user extension can achieve significant performance gains compared to the conventional model-free learning algorithms.

Dynamic resource management in integrated NOMA terrestrial–satellite networks using multi-agent reinforcement learning

The integration of terrestrial and satellite wireless communication networks offers a practical solution to enhance network coverage, connectivity, and cost-effectiveness. Moreover, in today’s interconnected world, connectivity’s reliable and widespread availability is increasingly important across various domains. This is especially more crucial for applications like the Internet of Things (IoT), remote sensing, disaster management, and bridging the digital divide. However, allocating the limited network resources efficiently and ensuring seamless handover between satellite and terrestrial networks present significant challenges. Therefore, this study introduces a resource allocation framework for integrated satellite–terrestrial networks to address these challenges. The framework leverages local cache pool deployments and non-orthogonal multiple access (NOMA) to reduce time delays and improve energy efficiency. Our proposed approach utilizes a multi-agent enabled deep deterministic policy gradient algorithm (MADDPG) to optimize user association, cache design, and transmission power control, resulting in enhanced energy efficiency. The approach comprises two phases: User Association and Power Control, where users are treated as agents, and Cache Optimization, where the satellite (Bs) is considered the agent. Through extensive simulations, we demonstrate that our approach surpasses conventional single-agent deep reinforcement learning algorithms in addressing cache design and resource allocation challenges in integrated terrestrial–satellite networks. Specifically, our proposed approach achieves significantly higher energy efficiency and reduced time delays compared to existing methods. This research highlights the importance and addresses the need for efficient resource allocation and cache design in integrated terrestrial–satellite networks, paving the way for enhanced connectivity and improved network performance in various applications.

Development of intelligent methodologies perceiving microstructure and mechanical properties of hot rolled steels

Traditional metallographic characterization of steel microstructure can only roughly determine average grain size and phase fractions because it relies too much on researchers’ personal experiences. As a result, when the microstructure-property relationships are to be established, only the rough microstructural features can be taken into accounts, which is usually incapable of grasping the correct strengthening and toughening mechanisms. To solve this long-existed problem, an integrated system with the microstructural perception by deep learning (DL) and prediction of mechanical properties by machine learning (ML) was developed, which can quickly and precisely recognize typical ferrite-pearlite microstructures in the most used hot rolled plain carbon steels and make the predictions of their mechanical properties. In the microstructural perception, a U-shaped residual network was proposed to implement deep learning of metallographic images to obtain digital information such as the size and distribution of grains and the lengths of grain and second phase boundaries. As compared to the conventional deep learning algorithm, the pixel accuracy (PA) and frequency weighted intersection over union (FWIoU) of predictions were improved by 2.84% and 5.27%, respectively. On this basis, the relationships between mechanical properties and the detailed microstructural features were established by using the random forest (RF) algorithm. Compared with the predictions by rough features of average grain size and phase fractions, the root mean square errors (RMSE) of the yield strength (YS), tensile strength (TS), and elongation (EL) based on the test data were reduced by 71.53%, 67.51% and 53.4%, respectively. By using the accurate perceptron for microstructure and properties, changes of strengths and elongations with processing parameters and chemical compositions were analyzed in good agreement with physical laws. Also, this perceptron can be extended to high strength steels with microstructures comprised of ferrite and bainite or complex phases if the data of metallographic images and properties are available.

Fault estimation and active fault‐tolerant control for a class of nonlinear systems with actuator and sensor faults based on unknown input iterative learning

AbstractIn this paper, fault estimation and active fault‐tolerant control are studied for a class of nonlinear systems with simultaneous actuator and sensor faults, as well as unknown external disturbances. Firstly, the state equation of a class of nonlinear systems is transformed into an augmented system state equation by extending the sensor fault as an auxiliary state. Then, a novel fault estimation observer based on iterative learning with unknown inputs is designed to estimate the system state, as well as actuator and sensor faults. Subsequently, by using the fault estimation information, a dynamic output feedback active fault‐tolerant control scheme is proposed to compensate for the influence of faults on the system. Lyapunov stability theory is used to prove the stability of the closed‐loop system and the convergence of the fault estimation observer. The gain matrices of the fault estimation observer and fault‐tolerant controller are obtained by solving linear matrix inequalities. Furthermore, the paper avoids the use of the norm in the convergence proof of the conventional iterative learning algorithm, which reduces the amount of calculation in the derivation process. Finally, the effectiveness and accuracy of the proposed method are verified through simulation of the DC motor angular velocity system.

Analysis of educational data enabled by deep learning to increase student success

A key component of improving educational quality is identifying pupils who are at a high risk of doing poorly academically as early as feasible. To accomplish this, most studies now in existence have used conventional Deep learning (DL) algorithms to forecast students' academic progress based on their behavior data, from which behavior elements are manually identified owing to the professional expertise and knowledge. Nevertheless, it has become increasingly difficult to recognize finely constructed handcrafted traits as a result of a rise in the types and quantities of behavioral data. The Enriched Plant Growth Optimized Artificial Neural Network (EPGO-ANN) technique enabled data analysis of educational data which is a viable tactic that may be used to improve student accomplishment in educational settings as we suggested in this research. The optimization predicts the academic success by autonomously extracting characteristics from student behavior data from several heterogeneous sources. This model's novelty employs recording the in-built time-series data for each kind of activity and uses EPGO-ANN to extract correlation features between various behaviors. The results of the experiments show that the suggested deep model approach outperforms several DL methods. The results of experiments show that the EPGO-ANN technique is superior to other DL methods.

An innovative deep neural network coordinating with percussion-based technique for automatic detection of concrete cavity defects

Cavity defects are usually invisible in concrete, failure to detect and repair them in time could significantly weaken the load-bearing capacity of engineering structures, potentially threatening the structural safety during the construction and operation periods. Therefore, it is essential to explore efficient approaches to detect concrete cavity defects. In this study, we propose a novel method to detect concrete cavity defects based on percussion and deep learning (DL) techniques. First, the percussion sound dataset was collected from concrete specimens with different types of precast cavity defects. Next, an innovative DL framework named the multi-scale convolutional bidirectional memory network (MCBMNet) was developed for classifying these percussion sound signals associated with specified concrete cavity defects. The MCBMNet utilizes the multi-scale convolutional operation to generate various scale features directly from raw percussion sounds, and then employs the bidirectional memory units to capture intrinsic feature connections and enhance feature separability, thus improving prediction accuracy and robustness. Finally, two experimental cases, including water-free and water-filled conditions, were designed to test the feasibility of the proposed method considering the influences of humidity environment in reality. Experimental results well demonstrate the excellent performances of the proposed MCBMNet for concrete cavity defect detection, with its average accuracy over 98% in both cases. Additionally, the anti-noise capacity and adaptability of MCBMNet were thoroughly investigated, which outperforms conventional learning algorithms.