Network Learning Research Articles

A cache-aware congestion control mechanism using deep reinforcement learning for wireless sensor networks

In Wireless Sensor Networks (WSN) communication protocols, rule-based approaches have been traditionally used for managing caching and congestion control. These approaches rely on explicitly defined, unchanging models. Recently, a trend has been toward incorporating adaptive methods that leverage machine learning (ML), including its subset deep learning (DL), during network congestion conditions. However, an adaptive cache-aware congestion control mechanism using Deep Reinforcement Learning (DRL) in WSN has not yet been explored. Therefore, this study developed a DRL-based adaptive cache-aware congestion control mechanism called DRL-CaCC to alleviate WSN during congestion scenarios. The DRL-CaCC uses intermediate caching parameters as its state space and adaptively moves the congestion window as its action space through the Rapid Start and DRL algorithms. The mechanism aims to find the optimal congestion window movement to avoid further network congestion while ensuring maximum cache utilization. Results show that DRL-CaCC achieved an average improvement gain between 20% and 40% compared to its baseline protocol, RT-CaCC. Finally, DRL-CaCC outperformed other caching-based and DRL-based congestion control protocols in terms of cache utilization, throughput, end-to-end delay, and packet loss metrics, with improvement gains between 10% and 30% in various congestion scenarios in WSN.

Deep learning network for indoor point cloud semantic segmentation with transferability

Semantic segmentation is crucial for interpreting point cloud data and plays a fundamental role in automating the creation of as-built BIM. Existing neural network models for semantic segmentation often heavily rely on the training dataset, resulting in a significant performance drop when applied to new datasets. This paper presents AttTransNet, a neural network model for automated point cloud semantic segmentation. Its attention-based pooling module improves local feature extraction from point clouds while reducing computational costs. The transfer learning framework enhances segmentation accuracy with minimal training on target datasets. Comparative experiments show that AttTransNet reduces training time by 80 % and improves segmentation accuracy by over 20 % compared with other SOTA methods. Cross-dataset experiments reveal that the transfer learning framework increases accuracy on new datasets by 150 %. By adding semantic information to point clouds, AttTransNet aids BIM modelers with direct reference, encouraging broader application of automated point cloud segmentation in the industry.

A model-based deep learning approach to interpretable impact force localization and reconstruction

Model-based and deep learning-based methods have been widely studied for force identification. However, model-based methods usually have high computational complexity and face challenges in parameter setting, while the generic network structures in deep learning methods are mostly designed empirically. In this paper, a model-based deep learning network is proposed to simultaneously localize and reconstruct impact f[orces. The proposed network, dubbed NSC-Net, is obtained by unrolling the iterative shrinkage-thresholding algorithm (ISTA) for the non-negative sparse coding (NSC) model. NSC-Net combines the interpretability of the model-based ISTA with the powerful parameter learning capability of deep learning. The transfer matrix is embedded into the network as physical information through learnable scaling. The structure of NSC-Net is based on explicit theoretical analysis, which enhances its interpretability in terms of structural design. In contrast to ISTA, the parameters in NSC-Net can be trained end-to-end from the training data without manual setting. Simulations and experiments on composite panels are conducted to validate the performance of the proposed method. The results demonstrate that NSC-Net accurately localizes and reconstructs impact forces, outperforming both ISTA and learned iterative shrinkage-thresholding algorithm (LISTA) network. Additionally, NSC-Net exhibits excellent noise immunity. Furthermore, the systematic approach of constructing an algorithm unrolling network for impact force identification is summarized, aiming to facilitate further related research.

Knowledge enhanced multi-task learning for simultaneous optimization of human parsing and pose estimation

Human parsing and pose estimation are both closely related to the human body structure, and it is advantageous to deal with both together in order to assist each other’s learning. Nevertheless, the two tasks have their own unique characteristics, and it is therefore difficult to efficiently and simultaneously obtain a good performance for both tasks in one single multi-task learning (MTL) network. This paper proposes a new MTL network (named “KDNet”) to enhance the communication between human parsing and pose estimation through knowledge distillation so as to improve the simultaneous learning of both tasks. In the proposed KDNet, a consistent representation module is used to enhance the related information sharing between the tasks, and a single-task model-based loss weighting method is developed to balance the loss levels. The proposed KDNet has been verified across three benchmark datasets, and in comparison to other state-of-the-art methods it achieves substantial performance gains without increasing model parameters during inference. The codes are shared at https://github.com/Yhdian/KDNet.git.

Memoryless Multimodal Anomaly Detection via Student–Teacher Network and Signed Distance Learning

Memoryless Multimodal Anomaly Detection via Student–Teacher Network and Signed Distance Learning

NAS FD Lung: A novel lung assist diagnostic system based on neural architecture search

In the detection and recognition of lung nodules, pulmonary nodules vary in size and shape and contain many similar tissues and organs around them, leading to the problems of both missed detection and false detection in existing detection algorithms. Designing proprietary detection and recognition networks manually requires substantial professional expertise. This process is time-consuming and labour-intensive and leads to issues like parameter redundancy and improper feature selection. Therefore, this paper proposes a new pulmonary CAD (computer-aided diagnosis) system for pulmonary nodules, NAS FD Lung (Using the NAS approach to search deep FPN and DPN networks), that can automatically learn and generate a deep learning network tailored to pulmonary nodule detection and recognition task requirements. NAS FD Lung aims to use automatic search to generate deep learning networks in the auxiliary diagnosis of pulmonary nodules to replace the manual design of deep learning networks. NAS FD Lung comprises two automatic search networks: BM NAS-FPN (Using NAS methods to search for deep FPN structures with Binary operation and Matrix multiplication fusion methods) network for nodule detection and NAS-A-DPN (Using the NAS approach to search deep DPN networks with attention mechanism) for nodule identification. The proposed technique is tested on the LUNA16 dataset, and the experimental results show that the model is superior to many existing state-of-the-art approaches. The detection accuracy of lung nodules is 98.23%. Regarding the lung nodules classification, the accuracy, specificity, sensitivity, and AUC values achieved 96.32%,97.14%,95.82%, and 98.33%, respectively.

Learning Networks: How Family Medicine Departments Are Meeting the Requirement.

In 2023, the Accreditation Council for Graduate Medical Education added participation within a "learning collaborative" or "learning network" (LN)as a requirement for family medicine residencies. The structure and scope of what makes an acceptable LN was only vaguely defined. The purpose of this study was to learn how many family medicine residencies associated with departments already belong to LNs, the purpose and funding of these existing LNs, and barriers to entering LNs. An online survey was sent to family medicine department chairs through a Council of Academic Family Medicine Educational Research Alliance omnibus study from August to September 2023. Survey questions explored the purpose, structure, and funding of LNs that associated residency programs already belonged to as well as the chairs' beliefs and knowledge about LNs. Of the 227 chairs, 119 completed the survey (50.2%). About 53% reported that their department was part of an LN, with more than one-third belonging for 5 years or less; 47% had a low understanding of what an LN is; and 71% had little to no concern that collaborating in an LN would negatively affect residency recruitment. The purpose of most LNs was a mix of research, education, and clinical activities. Faculty's lack of knowledge about LNs and lack of time were the top barriers identified to joining an LN. Funding was varied, and departmental funding was positively associated with administrative control of the LN. About half of the residency programs associated with family medicine departments already belong to LNs. Wide variation among existing LNs may lead to significantly disparate outcomes for residents and residencies as they navigate this new requirement.

Predicting OCD severity from religiosity and personality: A machine learning and neural network approach

Obsessive-compulsive disorder (OCD) affects a significant portion of the United States population. The present study investigated the complex relationships among OCD severity, personality traits, religiosity, and spirituality with a dataset of 229 participants. We applied advanced machine and deep learning techniques to identify key predictors of OCD severity, uncovering nuanced relationships and unexpected findings. Notably, item-level features were more influential than aggregate scores, challenging traditional analytical approaches. Moreover, a neural network model, despite not surpassing a linear regression in predictive accuracy, provided a more comprehensive understanding of OCD’s heterogeneity and of the nonlinear relationships between our variables. The inclusion of demographic factors provided further explanatory power for predicting OCD severity, emphasizing the multifaceted nature of the disorder. Our results show that machine learning models can nearly match traditional linear models in predictive power while retaining nonlinear relationships essential to understanding OCD. Our study advocates for the adoption of sophisticated predictive models in examining complex psychological phenomena, encouraging a reevaluation of conventional analytical approaches when prediction is central to research questions.

Flow prediction via adaptive dynamic graph with spatio-temporal correlations

Flow prediction has shown its significance in optimizing and smoothing resource allocation. However, it still faces challenges due to inherent temporal variability and dynamic spatial correlations of flow data. Although many methods have been introduced to solve the problem, they often rely on predefined graphs to capture shared patterns, which is likely to overlook the critical spatiotemporal dependencies that are necessary for mid-term and long-term prediction tasks. To address the issue, a novel flow prediction model, termed Dynamic Graph Multi-Step Prediction via Long Short-Term Memory Network and Adaptive Learning (DMLSA), is proposed. DMLSA employs Node Adaptive Learning (NAL) to learn node-specific patterns without relying on predefined static adjacency matrices. Moreover, DMLSA utilizes Graph Adaptive Generation (GAG) to extract dynamic features from node attributes and automatically infer interdependencies among flow sequences, thus enhancing the ability to predict complex flow dynamics. Additionally, DMLSA integrates multi-head self-attention mechanisms with gated recurrent units, effectively capturing both long-term and short-term node dependencies, therefore allowing for more precise identification of spatiotemporal patterns at the individual node level. Extensive experiments on five datasets demonstrate that DMLSA outperformed existing baseline models in terms of accuracy and stability for multi-step prediction tasks.

Multimodal Dataset Construction and Validation for Driving-Related Anger: A Wearable Physiological Conduction and Vehicle Driving Data Approach

Anger impairs a driver’s control and risk assessment abilities, heightening traffic accident risks. Constructing a multimodal dataset during driving tasks is crucial for accurate anger recognition. This study developed a multimodal physiological -vehicle driving dataset (DPV-MFD) based on drivers’ self-reported anger during simulated driving tasks. In Experiment 1, responses from 624 participants to anger-inducing videos and driving scenarios were collected via questionnaires to select appropriate materials. In Experiments 2 and 3, multimodal dynamic data and self-reported SAM emotion ratings were collected during simulated and real-vehicle tasks, capturing physiological and vehicle responses in neutral and anger states. Spearman’s correlation coefficient analysis validated the DPV-MFD’s effectiveness and explored the relationships between multimodal data and emotional dimensions. The CNN-LSTM deep learning network was used to assess the emotion recognition performance of the DPV-MFD across different time windows, and its applicability in real-world driving scenarios was validated. Compared to using EEG data alone, integrating multimodal data significantly improved anger recognition accuracy, with accuracy and F1 scores rising by 4.49% and 9.14%, respectively. Additionally, real-vehicle data closely matched simulated data, confirming the dataset’s effectiveness for real-world applications. This research is pivotal for advancing emotion-aware human–machine- interaction and intelligent transportation systems.

A Fast and Robust Range Alignment Method for ISAR Imaging Based on a Deep Learning Network and Regional Multi-Scale Minimum Entropy Method

In inverse synthetic aperture radar (ISAR) imaging, range alignment (RA) is crucial for translational compensation. To address the need for rapidity and accuracy in the RA process, a fast and robust range alignment method is proposed based on a deep learning network and minimum entropy (ME) method. The proposed method consists primarily of two components: the CNN-RNN attention mechanism network (CRAN) architecture and the regional multi-scale minimum entropy (RMSME) method. The main distinction of this method from existing approaches lies in its utilization of a deep learning network for rapid coarse alignment, followed by the search for minimum entropy within local regions at multiple scales. The integration strategy effectively addresses the current challenges of poor generalization in deep learning networks and low efficiency in the traditional ME method. The experimental results of simulation data indicate that the proposed method achieves the best range alignment performance compared to RNN, CRAN, and the traditional ME method. The experimental results of the measured data further validate the practicality of the proposed method. This research provides reference significance for the joint application of deep learning and traditional methods in the RA process.

Multi-view multi-behavior interest learning network and contrastive learning for multi-behavior recommendation

The recommendation system aims to recommend items to users by capturing their personalized interests. Traditional recommendation systems typically focus on modeling target behaviors between users and items. However, in practical application scenarios, various types of behaviors (e.g., click, favorite, purchase, etc.) occur between users and items. Despite recent efforts in modeling various behavior types, multi-behavior recommendation still faces two significant challenges. The first challenge is how to comprehensively capture the complex relationships between various types of behaviors, including their interest differences and interest commonalities. The second challenge is how to solve the sparsity of target behaviors while ensuring the authenticity of information from various types of behaviors. To address these issues, a multi-behavior recommendation framework based on Multi-View Multi-Behavior Interest Learning Network and Contrastive Learning (MMNCL) is proposed. This framework includes a multi-view multi-behavior interest learning module that consists of two submodules: the behavior difference aware submodule, which captures intra-behavior interests for each behavior type and the correlations between various types of behaviors, and the behavior commonality aware submodule, which captures the information of interest commonalities between various types of behaviors. Additionally, a multi-view contrastive learning module is proposed to conduct node self-discrimination, ensuring the authenticity of information integration among various types of behaviors, and facilitating an effective fusion of interest differences and interest commonalities. Experimental results on three real-world benchmark datasets demonstrate the effectiveness of MMNCL and its advantages over other state-of-the-art recommendation models. Our code is available at https://github.com/sujieyang/MMNCL.

MultiNet 2.0: A lightweight attention-based deep learning network for stenosis measurement in carotid ultrasound scans and cardiovascular risk assessment

BackgroundCardiovascular diseases (CVD) cause 19 million fatalities each year and cost nations billions of dollars. Surrogate biomarkers are established methods for CVD risk stratification; however, manual inspection is costly, cumbersome, and error-prone. The contemporary artificial intelligence (AI) tools for segmentation and risk prediction, including older deep learning (DL) networks employ simple merge connections which may result in semantic loss of information and hence low in accuracy. MethodologyWe hypothesize that DL networks enhanced with attention mechanisms can do better segmentation than older DL models. The attention mechanism can concentrate on relevant features aiding the model in better understanding and interpreting images. This study proposes MultiNet 2.0 (AtheroPoint, Roseville, CA, USA), two attention networks have been used to segment the lumen from common carotid artery (CCA) ultrasound images and predict CVD risks. ResultsThe database consisted of 407 ultrasound CCA images of both the left and right sides taken from 204 patients. Two experts were hired to delineate borders on the 407 images, generating two ground truths (GT1 and GT2). The results were far better than contemporary models. The lumen dimension (LD) error for GT1 and GT2 were 0.13±0.08 and 0.16±0.07 mm, respectively, the best in market. The AUC for low, moderate and high-risk patients’ detection from stenosis data for GT1 were 0.88, 0.98, and 1.00 respectively. Similarly, for GT2, the AUC values for low, moderate, and high-risk patient detection were 0.93, 0.97, and 1.00, respectively.The system can be fully adopted for clinical practice in AtheroEdge™ model by AtheroPoint, Roseville, CA, USA.

Dynamic collaborative learning with heterogeneous knowledge transfer for long-tailed visual recognition

Solving the long-tailed visual recognition with deep convolutional neural networks is still a challenging task. As a mainstream method, multi-experts models achieve SOTA accurary for tackling this problem, but the uncertainty in network learning and the complexity in fusion inference constrain the performance and practicality of the multi-experts models. To remedy this, we propose a novel dynamic collaborative learning with heterogeneous knowledge transfer model (DCHKT) in this paper, in which experts with different expertise collaborate to make predictions. DCHKT consists of two core components: dynamic adaptive weight adjustment and heterogeneous knowledge transfer learning. First, the dynamic adaptive weight adjustment is designed to shift the focus of model training between the global expert and domain experts via dynamic adaptive weight. By modulating the trade-off between the learning of features and classifier, the dynamic adaptive weight adjustment can enhance the discriminative ability of each expert and alleviate the uncertainty of model learning. Then, heterogeneous knowledge transfer learning, which measures the distribution differences between the fusion logits of multiple experts and the predicted logits of each expert with different specialties, can achieve message passing between experts and enhance the consistency of ensemble prediction in model training and inference to promote their collaborations. Finally, extensive experimental results on public long-tailed datasets: CIFAR-LT, ImageNet-LT, Place-LT and iNaturalist2018, demonstrate the effectiveness and superiority of our DCHKT.

Structured pruning of neural networks for constraints learning

In recent years, the integration of Machine Learning (ML) models with Operation Research (OR) tools has gained popularity in applications such as cancer treatment, algorithmic configuration, and chemical process optimization. This integration often uses Mixed Integer Programming (MIP) formulations to represent the chosen ML model, that is often an Artificial Neural Networks (ANNs) due to their widespread use. However, ANNs frequently contain a large number of parameters, resulting in MIP formulations impractical to solve. In this paper we showcase the effectiveness of a ANN pruning, when applied to models prior to their integration into MIPs. We discuss why pruning is more suitable in this context than other ML compression techniques, and we highlight the potential of appropriate pruning strategies via experiments on MIPs used to construct adversarial examples to ANNs. Our results demonstrate that pruning offers remarkable reductions in solution times without hindering the quality of the final decision, enabling the resolution of previously unsolvable instances.

Adaptive control of recurrent neural networks using conceptors.

Recurrent neural networks excel at predicting and generating complex high-dimensional temporal patterns. Due to their inherent nonlinear dynamics and memory, they can learn unbounded temporal dependencies from data. In a machine learning setting, the network's parameters are adapted during a training phase to match the requirements of a given task/problem increasing its computational capabilities. After the training, the network parameters are kept fixed to exploit the learned computations. The static parameters, therefore, render the network unadaptive to changing conditions, such as an external or internal perturbation. In this paper, we demonstrate how keeping parts of the network adaptive even after the training enhances its functionality and robustness. Here, we utilize the conceptor framework and conceptualize an adaptive control loop analyzing the network's behavior continuously and adjusting its time-varying internal representation to follow a desired target. We demonstrate how the added adaptivity of the network supports the computational functionality in three distinct tasks: interpolation of temporal patterns, stabilization against partial network degradation, and robustness against input distortion. Our results highlight the potential of adaptive networks in machine learning beyond training, enabling them to not only learn complex patterns but also dynamically adjust to changing environments, ultimately broadening their applicability.

Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey

Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey

ITIMCA: Image-Text Information and Cross-Attention for Multi-Modal Cassava Leaf Disease Classification Based on a Novel Multi-Modal Dataset in Natural Environments

With artificial intelligence and deep learning development, crop disease recognition methods leverage deep networks for automatic feature learning but rely on the volume of training data. Addressing the scarcity of data in agriculture, few-shot learning (FSL) and multi-modal learning have become focal points. However, existing methods are confined to a single modality or insufficiently exploit cross-modal features. To address this, we propose a multi-modal contrastive learning approach integrating images and text to tackle the problem of small-sample recognition. This method combines CLIP multi-modal pre-training with cross-attention, termed ITIMCA. Experimental validation demonstrates the effectiveness of our approach in cassava leaf disease recognition tasks under natural conditions. Experimental results show that the proposed model achieved an accuracy of 78.00%, a precision of 88.48%, a recall rate of 80.00%, and an F1-Score of 79.00% on the cassava leaf disease identification and classification dataset. These results suggest that the proposed network effectively identifies cassava leaf diseases.

Multi-level discriminator based contrastive learning for multiplex networks

Graph embedding is a technique for obtaining low-dimensional representations of nodes across diverse networks, which may then be used for various downstream tasks and applications. When it applies to heterogeneous networks, it is hard to handle heterogeneous networks because they usually contain different types of nodes and edges with more semantic and structural information. Recently, contrastive learning has developed as the preferred strategy for dealing with unsupervised heterogeneous graph embedding to reduce the cost of human label annotation. However, most multi-view contrastive learning approaches calculate the model’s loss only based on the mutual dependence between the node representation and graph representation. These approaches ignore that both node attributes and node clustering contain discriminative content. To solve this issue, we propose a model called Multi-Level Discriminator-based Contrastive Learning for Multiplex Networks (MLDCL). This model adopts a multi-level multi-discriminator-based approach that can simultaneously learn the global-level structural information, node-level attribute information, and local-level clustering information. Moreover, an augmentation strategy in the contrast learning process from the spectral domain is proposed to improve the representation and discriminative ability of MLDCL. Numerous tests with node clustering and classification tasks on widely used datasets demonstrate the efficacy of the proposed approach.

Engineering flexible machine learning systems by traversing functionally invariant paths

Contemporary machine learning algorithms train artificial neural networks by setting network weights to a single optimized configuration through gradient descent on task-specific training data. The resulting networks can achieve human-level performance on natural language processing, image analysis and agent-based tasks, but lack the flexibility and robustness characteristic of human intelligence. Here we introduce a differential geometry framework—functionally invariant paths—that provides flexible and continuous adaptation of trained neural networks so that secondary tasks can be achieved beyond the main machine learning goal, including increased network sparsification and adversarial robustness. We formulate the weight space of a neural network as a curved Riemannian manifold equipped with a metric tensor whose spectrum defines low-rank subspaces in weight space that accommodate network adaptation without loss of prior knowledge. We formalize adaptation as movement along a geodesic path in weight space while searching for networks that accommodate secondary objectives. With modest computational resources, the functionally invariant path algorithm achieves performance comparable with or exceeding state-of-the-art methods including low-rank adaptation on continual learning, sparsification and adversarial robustness tasks for large language models (bidirectional encoder representations from transformers), vision transformers (ViT and DeIT) and convolutional neural networks.