Network Learning Research Articles

Removing nonrigid refractive distortions for underwater images using an attention-based deep neural network

Refractive distortions in underwater images usually occur when these images are captured through a dynamic refractive water surface, such as unmanned aerial vehicles capturing shallow underwater scenes from the surface of water or autonomous underwater vehicles observing floating platforms in the air. We propose an end-to-end deep neural network for learning to restore real scene images for removing refractive distortions. This network adopts an encoder-decoder architecture with a specially designed attention module. The use of the attention image and the distortion field generated by the proposed deep neural network can restore the exact distorted areas in more detail. Qualitative and quantitative experimental results show that the proposed framework effectively eliminates refractive distortions and refines image details. We also test the proposed framework in practical applications by embedding it into an NVIDIA JETSON TX2 platform, and the results demonstrate the practical value of the proposed framework.

A high-efficiency local and global detector for diatom-based drowning diagnosis

Detection and classification of diatoms has been considered as the most effective method for the drowning diagnosis in forensic practice. However, challenges (such as limited samples, complex background interference, and detecting multiple objects) remain during detection and classification. To address these challenges, we propose a MDMS (multi-scale dynamic multi-head self-attention)-based deep learning network for the diatom-based drowning diagnosis. This framework uses TFFT (two-stage full fine-tuning training strategy) to solve the challenges of limited diatom samples and detecting multiple objects, and builds a multi-scale dynamic multi-head self-attention module to extract the local and global features of diatom images. In the meantime, we introduce an online hard example mining strategy to attenuate the complex background interference. The experimental results show that the proposed framework can effectively reduce the missing and false detection rates of diatom objects, with the mAP (mean Average Precision) reaching 92.434%, which is better than the result using the mainstream methods.

Enhanced deep learning based decision support system for kidney tumour detection

This study presents a high-accuracy deep learning-based decision support system for kidney cancer detection. The research utilizes a relatively large dataset of 10,000 CT images, including both healthy and tumour-detected kidney scans. After data preprocessing and optimization, various deep learning models were evaluated, with DenseNet-201 emerging as the top performer, achieving an accuracy of 99.75 %. The study compares multiple deep learning architectures, including AlexNet, EfficientNet, Darknet-53, Xception, and DenseNet-201, across different learning rates. Performance metrics such as accuracy, precision, sensitivity, F1-score, and specificity are analysed using confusion matrices. The proposed system outperforms different deep learning networks, demonstrating superior accuracy in kidney cancer detection. The improvement is attributed to effective data engineering and hyperparameter optimization of the deep learning networks. This research contributes to the field of medical image analysis by providing a robust decision support tool for early and rapid diagnosis of kidney cancer. The high accuracy and efficiency of the proposed system make it a promising aid for healthcare professionals in clinical settings.

Detection of bone cancer based on a four-phase framework generative deep belief neural network in deep learning

The medical image processing plays an important role in analyzing and detecting tumors and detecting cancer cells in their early stages. Microscopic images were used as a deep learning approach to recognize bone features. These images were acquired using microscopy, a time-consuming and labor-intensive process that required repeating. Cancer cells cannot be identified by conventional techniques because of the noise in the images. To address these challenges, this paper introduces the Gated Convolutional Neural Network (GCNN) techniques used to identify multiclass bone cancer, including non-tumor, non-viable, and viable tumors. As a first stage, the most basic part of image Preprocessing is to deposing without interrupting the diagnostics information during the removal of noise. The second stage is edge detection, used for image brightness discontinuity detection based on Scaled Region Edge Detector (SRED). It is used for image segmentation and data extraction within regions based on the Screen Cluster Area Segmentation (SCAS) to segment the Region of Interest (RoI) based on Relative Scalar Color Histogram (RSCH). The fourth step is Feature extraction is to find out the features and match the image feature from the Bone Cancer Detection object based on the maximum threshold weights using GCNN with Maximum Standard External Regions (MSER) used for features extracting applied to improve feature quality analysis and Feature support rate. This results in higher classification accuracy, Precision, recall under F-measure, and lower error rates. This enables a more achieved classification than other existing methods.

Efficient face anti-spoofing via head-aware transformer based knowledge distillation with 5 MB model parameters

Although face recognition technology has been applied in many scenarios, it still suffers from many types of presentation attacks, so face anti-spoofing (FAS) becomes a hot topic in computer vision. Recently, vision transformer is recognized as the mainstream architecture for FAS, which always relies on auxiliary information, sophisticated tricks and huge model parameters. Considering that face based identity authentication usually takes place on mobile-like devices, therefore how to design an effective and lightweight model is of great significance. Inspired by the powerful global modeling ability of self-attention and the model compression ability of knowledge distillation, a simple yet effective knowledge distillation approach is proposed for FAS under transformer framework. Our primary idea is to leverage the rich knowledge of a teacher network pre-trained on large-scale face data to guide the learning of a lightweight student network. The main contributions of our method are threefold: (1) Feature- and logits-level distillation are combined to transfer the rich knowledge of teacher to student. (2) A head-aware strategy is proposed to deal with the dimension mismatching issue of middle encoder layers between teacher and student networks, in which a novel attention head correlation matrix is introduced. (3) Our method can bridge the performance gap between teacher and student, and the resulting student network is extremely lightweight with only 5 MB parameters. Extensive experiments are conducted on three public face-spoofing datasets, CASIA-FASD, Replay-Attack and OULU-NPU, the results demonstrate that our method can obtain performance on par with or superior to most FAS methods and outperform many knowledge distillation methods. Meanwhile, the distilled student network achieves excellent performance with 17× fewer parameters and 9× faster inference time compared to the teacher network. The code will be publicly available at https://github.com/Maricle-zhangjun/HaTFAS.

An efficient energy supply policy and optimized self-adaptive data aggregation with deep learning in heterogeneous wireless sensor network

An efficient energy supply policy and optimized self-adaptive data aggregation with deep learning in heterogeneous wireless sensor network

Use of machine learning methods in predicting the main components of essential oils: Laurus nobilis L.

Laurus nobilis L. is an export plant that contributes to the industry with its medicinal and aromatic properties, as well as the use of its leaves or fruits in the food, pharmaceutical, and cosmetic industries. This study aims to predict the first three main components of bay laurel essential oil with different machine learning methods as a result of GC-MS analysis, taking into account a wide range of parameters used before and after the analysis of the essential oil in pharmacognostic studies. In this study, the K-Means algorithm was used to detect unknown relationships (chemical or physical properties) between the data obtained as a result of the analysis, and Artificial Neural Network (ANN) and Decision Tree machine learning methods were used to perform prediction operations. Based on this, considering the R2 values obtained from ANN analyses, R2 values of 0.9992 for testing, 1 for training, 1 for validation value, and 0.99992 for all were obtained. The ANN model produced results close to the real values with an accuracy rate of 98.97% in predicting the three main components. The Decision Tree algorithm achieved classification with a 96.23% accuracy rate in predicting the three main components. K-Means clustering was performed with an accuracy rate of 97.83% and significant relationships between the features of the Laurus nobilis dataset were detected. This comprehensive study serves as a guide for machine learning research on other essential oils.

Subject-Independent Wearable P300 Brain-Computer Interface Based on Convolutional Neural Network and Metric Learning.

The calibration procedure for a wearable P300 brain-computer interface (BCI) greatly impact the user experience of the system. Each user needs to spend additional time establishing a decoder adapted to their own brainwaves. Therefore, achieving subject independent is an urgent issue for wearable P300 BCI needs to be addressed. A dataset of electroencephalogram (EEG) signals was constructed from 100 individuals by conducting a P300 speller task with a wearable EEG amplifier. A framework is proposed that initially improves cross-subject consistency of EEG features through a common feature extractor. Subsequently, a simple and compact convolutional neural network (CNN) architecture is employed to learn an embedding sub-space, where the mapped EEG features are maximally separated, while pursuing the minimum distance within the same class and the maximum distance between different classes. Finally, the model's generalization capability was further optimized through fine-tuning. The proposed method significantly boosts the average accuracy of wearable P300 BCI to 73.23 ±7.62% without calibration and 78.75±6.37% with fine-tuning. The results demonstrate the feasibility and excellent performance of our dataset and framework. A calibration-free wearable P300 BCI system is feasible, suggesting significant potential for practical applications of the wearable P300 BCI system.

Co-Expression Network and Machine Learning Analysis of Transcriptomics Data Identifies Distinct Gene Signatures and Pathways in Lesional and Non-Lesional Atopic Dermatitis

Background: Atopic dermatitis (AD) is a common inflammatory skin condition with complex origins. Current treatments often yield suboptimal results due to an incomplete understanding of its underlying mechanisms. This study aimed to identify pathway and gene signatures that distinguish between lesional AD, non-lesional AD, and healthy skin. Method: We conducted differential gene expression and co-expression network analyses to identify differentially co-expressed genes (DCEGs) in lesional AD vs. healthy skin, lesional vs. non-lesional AD, and non-lesional AD vs. healthy skin. Modules associated with lesional and non-lesional AD were identified based on the correlation coefficients between module eigengenes and clinical phenotypes (|R| ≥ 0.5, p-value < 0.05). Subsequently, we employed Ingenuity Pathway Analysis (IPA) on the identified DCEGs, followed by machine learning (ML) analysis within the pathway expression framework. The ML analysis of pathway expressions, selected by IPA and derived from gene expression data, identified relevant pathway signatures, which were validated using an independent dataset and correlated with AD severity measures (EASI and SCORAD). Results: We identified 975, 441, and 40 DCEGs in lesional vs. healthy skin, lesional vs. non-lesional, and non-lesional vs. healthy skin, respectively. IPA and ML analyses revealed 25 relevant pathway signatures, including wound healing, glucocorticoid receptor signaling, and S100 gene family signaling pathways. Validation confirmed the significance of 10 pathway signatures, which were correlated with the AD severity measures. DCEGs such as MMP12 and S100A8 demonstrated high diagnostic efficacy (AUC > 0.70) in both the discovery and validation datasets. Conclusions: Differential gene expression, co-expression networks and ML analyses of pathway expression have unveiled relevant pathways and gene signatures that distinguish between lesional, non-lesional, and healthy skin, providing valuable insights into AD pathogenesis.

Innovative load identification with Res-UNet: Integrating phase space reconstruction and physics-informed deep learning

This study developed a specialized load identification model for a specific ocean platform that integrates advanced deep learning techniques with the unique physical characteristics of marine engineering. Key achievements include high-dimensional mapping of data features using phase space reconstruction (PSR) techniques to enhance the predictive capacity of the model through chaos system theory; optimization of ResNet and UNet deep learning networks with attention mechanisms to boost performance; calculation of stiffness, mass, and damping matrices using finite element software; establishment of corresponding state-space equations; and integration of structural dynamics equations with the deep learning network loss function to significantly improve load identification accuracy. In addition, proprietary structural equations for ocean platform structures were developed, underscoring the uniqueness and applicability of the algorithm. The proposed physics-informed Res-UNet model exhibited excellent performance in identifying both periodic and random loads. These improvements were validated through training with finite element simulated data and subsequent experimental applications, demonstrating the effectiveness of the proposed innovative methods.

Multimodal Deep Learning Network for Differentiating Between Benign and Malignant Pulmonary Ground Glass Nodules.

This study aimed to establish a multimodal deep-learning network model to enhance the diagnosis of benign and malignant pulmonary ground glass nodules [GGNs]. Retrospective data on pulmonary GGNs were collected from multiple centers across China, including North, Northeast, Northwest, South, and Southwest China. The data were divided into a training set and a validation set in an 8:2 ratio. In addition, a GGN dataset was also obtained from our hospital database and used as the test set. All patients underwent chest computed tomography [CT], and the final diagnosis of the nodules was based on postoperative pathological reports. The Residual Network [ResNet] was used to extract imaging data, the Word2Vec method for semantic information extraction, and the Self Attention method for combining imaging features and patient data to construct a multimodal classification model. Then, the diagnostic efficiency of the proposed multimodal model was compared with that of existing ResNet and VGG models and radiologists. The multicenter dataset comprised 1020 GGNs, including 265 benign and 755 malignant nodules, and the test dataset comprised 204 GGNs, with 67 benign and 137 malignant nodules. In the validation set, the proposed multimodal model achieved an accuracy of 90.2%, a sensitivity of 96.6%, and a specificity of 75.0%, which surpassed that of the VGG [73.1%, 76.7%, and 66.5%] and ResNet [78.0%, 83.3%, and 65.8%] models in diagnosing benign and malignant nodules. In the test set, the multimodal model accurately diagnosed 125 [91.18%] malignant nodules, outperforming radiologists [80.37% accuracy]. Moreover, the multimodal model correctly identified 54 [accuracy, 80.70%] benign nodules, compared to radiologists' accuracy of 85.47%. The consistency test comparing radiologists' diagnostic results with the multimodal model's results in relation to postoperative pathology showed strong agreement, with the multimodal model demonstrating closer alignment with gold standard pathological findings [Kappa=0.720, P<0.01]. The multimodal deep learning network model exhibited promising diagnostic effectiveness in distinguishing benign and malignant GGNs and, therefore, holds potential as a reference tool to assist radiologists in improving the diagnostic accuracy of GGNs, potentially enhancing their work efficiency in clinical settings.

Research on priority scheduling strategy for smoothing power fluctuations of microgrid tie‐lines based on PER‐DDPG algorithm

AbstractThe variability of renewable energy within microgrids (MGs) necessitates the smoothing of power fluctuations through the effective scheduling of internal power equipment. Otherwise, significant power variations on the tie‐line connecting the MG to the main power grid could occur. This study introduces an innovative scheduling strategy that utilizes a data‐driven approach, employing a deep reinforcement learning algorithm to achieve this smoothing effect. The strategy prioritizes the scheduling of MG's internal power devices, taking into account the stochastic charging patterns of electric vehicles. The scheduling optimization model is initially described as a Markov decision process with the goal of minimizing power fluctuations on the interconnection lines and operational costs of the MG. Subsequently, after preprocessing the historical operational data of the MG, an enhanced scheduling strategy is developed through a neural network learning process. Finally, the results from four scheduling scenarios demonstrate the significant impact of the proposed strategy. Comparisons of reward curves before and after data preprocessing underscore its importance. In contrast to optimization results from deep deterministic policy gradient, soft actor‐critic, and particle swarm optimization algorithms, the superiority of the deep deterministic policy gradient algorithm with the addition of a priority experience replay mechanism is highlighted.

Transdisciplinary and systemic construction of a framework for assessing the sustainability of alternative food networks in Mexico City

ABSTRACT This paper describes and analyzes the process and results of the construction of a framework to assess the sustainability of alternative food networks in Mexico City, using a transdisciplinary and systemic approach. We generated a practical tool with twenty-two evaluation criteria linked to five themes: “care for nature and waste reduction;” “equity and economic viability;” “quality, diversity and local products;” “social equity, transparency and democracy,” and “learning and collaboration.” Our method allowed us to develop common languages and context-appropriate tools for sustainability assessment; identify trade-offs and leverage points; and promote collective learning in alternative food networks.

Transfer Learning and Deep Neural Networks for Robust Intersubject Hand Movement Detection from EEG Signals

In this research, five systems were developed to classify four distinct motor functions—forward hand movement (FW), grasp (GP), release (RL), and reverse hand movement (RV)—from EEG signals, using the WAY-EEG-GAL dataset where participants performed a sequence of hand movements. During preprocessing, band-pass filtering was applied to remove artifacts and focus on the mu and beta frequency bands. The initial system, a preliminary study model, explored the overall framework of EEG signal processing and classification, utilizing time-domain features such as variance and frequency-domain features such as alpha and beta power, with a KNN model for classification. Insights from this study informed the development of a baseline system, which innovatively combined the common spatial patterns (CSP) method with continuous wavelet transform (CWT) for feature extraction and employed a GoogLeNet classifier with transfer learning. This system classified six unique pairs of events derived from the four motor functions, achieving remarkable accuracy, with the highest being 99.73% for the GP–RV pair and the lowest 80.87% for the FW–GP pair in intersubject classification. Building on this success, three additional systems were developed for four-way classification. The final model, ML-CSP-OVR, demonstrated the highest intersubject classification accuracy of 78.08% using all combined data and 76.39% for leave-one-out intersubject classification. This proposed model, featuring a novel combination of CSP-OVR, CWT, and GoogLeNet, represents a significant advancement in the field, showcasing strong potential as a general system for motor imagery (MI) tasks that is not dependent on the subject. This work highlights the prominence of the research contribution by demonstrating the effectiveness and robustness of the proposed approach in achieving high classification accuracy across different motor functions and subjects.

Correction to: Analysis of Internet Financial Risks Based on Deep Learning and BP Neural Network

Correction to: Analysis of Internet Financial Risks Based on Deep Learning and BP Neural Network

Enhancing classification of lung diseases by optimizing training hyperparameters of the deep learning network

Enhancing classification of lung diseases by optimizing training hyperparameters of the deep learning network

Advancing healthcare practice and education via data sharing: demonstrating the utility of open data by training an artificial intelligence model to assess cardiopulmonary resuscitation skills.

Health professional education stands to gain substantially from collective efforts toward building video databases of skill performances in both real and simulated settings. An accessible resource of videos that demonstrate an array of performances - both good and bad-provides an opportunity for interdisciplinary research collaborations that can advance our understanding of movement that reflects technical expertise, support educational tool development, and facilitate assessment practices. In this paper we raise important ethical and legal considerations when building and sharing health professions education data. Collective data sharing may produce new knowledge and tools to support healthcare professional education. We demonstrate the utility of a data-sharing culture by providing and leveraging a database of cardio-pulmonary resuscitation (CPR) performances that vary in quality. The CPR skills performance database (collected for the purpose of this research, hosted at UK Data Service's ReShare Repository) contains videos from 40 participants recorded from 6 different angles, allowing for 3D reconstruction for movement analysis. The video footage is accompanied by quality ratings from 2 experts, participants' self-reported confidence and frequency of performing CPR, and the demographics of the participants. From this data, we present an Automatic Clinical Assessment tool for Basic Life Support that uses pose estimation to determine the spatial location of the participant's movements during CPR and a deep learning network that assesses the performance quality.

A Dual-Stream Deep Learning-Based Acoustic Denoising Model to Enhance Underwater Information Perception

Estimating the line spectra of ship-radiated noise is a crucial remote sensing technique for detecting and recognizing underwater acoustic targets. Improving the signal-to-noise ratio (SNR) makes the low-frequency components of the target signal more prominent. This enhancement aids in the detection of underwater acoustic signals using sonar. Based on the characteristics of low-frequency narrow-band line spectra signals in underwater target radiated noise, we propose a dual-stream deep learning network with frequency characteristics transformation (DS_FCTNet) for line spectra estimation. The dual streams predict amplitude and phase masks separately and use an information exchange module to swap learn features between the amplitude and phase spectra, aiding in better phase information reconstruction and signal denoising. Additionally, a frequency characteristics transformation module is employed to extract convolutional features between channels, obtaining global correlations of the amplitude spectrum and enhancing the ability to learn target signal features. Through experimental analysis on ShipsEar, a dataset of underwater acoustic signals by hydrophones deployed in shallow water, the effectiveness and rationality of different modules within DS_FCTNet are verified.Under low SNR conditions and with unknown ship types, the proposed DS_FCTNet model exhibits the best line spectrum enhancement compared to methods such as SEGAN and DPT_FSNet. Specifically, SDR and SSNR are improved by 14.77 dB and 13.58 dB, respectively, enabling the detection of weaker target signals and laying the foundation for target localization and recognition applications.

MyThisYourThat for interpretable identification of systematic bias in federated learning for biomedical images

Distributed collaborative learning is a promising approach for building predictive models for privacy-sensitive biomedical images. Here, several data owners (clients) train a joint model without sharing their original data. However, concealed systematic biases can compromise model performance and fairness. This study presents MyThisYourThat (MyTH) approach, which adapts an interpretable prototypical part learning network to a distributed setting, enabling each client to visualize feature differences learned by others on their own image: comparing one client’s 'This’ with others’ 'That’. Our setting demonstrates four clients collaboratively training two diagnostic classifiers on a benchmark X-ray dataset. Without data bias, the global model reaches 74.14% balanced accuracy for cardiomegaly and 74.08% for pleural effusion. We show that with systematic visual bias in one client, the performance of global models drops to near-random. We demonstrate how differences between local and global prototypes reveal biases and allow their visualization on each client’s data without compromising privacy.

Application of high performance computing and deep neural network learning in intelligent city mechanical product recommendation

Application of high performance computing and deep neural network learning in intelligent city mechanical product recommendation