Network Learning Research Articles

Prediction of Cutting Force for Different Tools Based on Transfer Learning and Neural Networks

Prediction of Cutting Force for Different Tools Based on Transfer Learning and Neural Networks

Surface defect detection on industrial drum rollers: Using enhanced YOLOv8n and structured light for accurate inspection.

Drum roller surface defect detection is of great research significance for control production quality. Aiming at solving the problems that the traditional light source visual imaging system, which does not clearly reflect defect features, the defect detection efficiency is low, and the accuracy is not enough, this paper designs an image acquisition system based on line fringe structured light and proposes an improved deep learning network model based on YOLOv8n to achieve efficient detection of defects on the rolling surface of a drum roller. In the aspect of image acquisition, this paper selected the line fringe structured light as the system light source, which made up for the problem that the traditional light source does not reflect the defect characteristics. In terms of algorithms, firstly, using deformable convolution instead of standard convolution to enhance the feature extraction ability of the backbone network. Then, a new feature fusion module was proposed to enable the fusion network to learn additional original information. Finally, Wise-IoU was applied to replace CIoU in the loss function, so that the network pays more attention to the high-quality samples. The experimental results show that the improved YOLOv8n algorithm has a certain improvement in detection accuracy. The main average accuracy (mAP) is 97.2%, and the detection time is 4.3ms. The system and algorithm designed in this paper can better ensure the production quality of drum rollers. While effective, the model's standard rectangular bounding boxes may limit precision for elongated defects. Future work could explore rotated bounding boxes and broader dataset diversity to enhance generalization in real-world applications.

Screening of multi deep learning-based de novo molecular generation models and their application for specific target molecular generation

Traditional virtual screening methods need to explore expanse and vast chemical spaces and need to be based on existing chemical libraries. With the development of deep learning techniques for the de novo generation of molecules, also known as inverse molecular design, the increasingly widespread application of various types of deep learning algorithms has led to revolutionary changes in de novo molecular generation research. In particular, the emergence of a novel natural language processing (NLP) architecture called the transformer has improved the state-of-the-art performance of existing AI technologies. In this study, we modified one top-performing molecular generation model on the basis of the generative pretraining transformer (GPT) architecture in three directions. Moreover, we propose an integrated end-to-end neural network learning framework based on one complete encoder-decoder architecture transformer model: Transfer Text-to-Text Transformer (T5), by learning the embedding vector representation space of conditional molecular properties to encode and guide the vector representation of SMILES sequences, resulting in the output of the final decoder block with a softmax output (maximum likelihood objective). Moreover, we evaluated the performance of these NLP-based generation models and another new model architecture based on a selective state space and selected the best approach jointing a transfer learning strategy for de novo drug discovery to target L858R/T790M/C797S-mutant EGFR in non-small cell lung cancer.

3D-HoloNet: fast, unfiltered, 3D hologram generation with camera-calibrated network learning

Computational holographic displays typically rely on time-consuming iterative computer-generated holographic (CGH) algorithms and bulky physical filters to attain high-quality reconstruction images. This trade-off between inference speed and image quality becomes more pronounced when aiming to realize 3D holographic imagery. This work presents 3D-HoloNet, a deep neural network-empowered CGH algorithm for generating phase-only holograms (POHs) of 3D scenes, represented as RGB-D images, in real time. The proposed scheme incorporates a learned, camera-calibrated wave propagation model and a phase regularization prior into its optimization. This unique combination allows for accommodating practical, unfiltered holographic display setups that may be corrupted by various hardware imperfections. Results tested on an unfiltered holographic display reveal that the proposed 3D-HoloNet can achieve 30 fps at full HD for one color channel using a consumer-level GPU while maintaining image quality comparable to iterative methods across multiple focused distances.

Optimised Continually Evolved Classifier for Few-Shot Learning Overcoming Catastrophic Forgetting

Catastrophic forgetting (CF) poses the most important challenges in neural networks for continual learning. During training, numerous current approaches replay precedent data that deviate from the constraints of an optimal continual learning system. In this work, optimisation-enabled continually evolved classifier is used for the CF. Moreover, a few-shot continual learning model is exploited to mitigate CF. Initially, images undergo pre-processing using Contrast Limited Adaptive Histogram Equalisation (CLAHE). Then; the pre-processed outputs are utilised in the classification through Continually Evolved Classifiers based on few-shot incremental learning. Here, the initial training is done by using the CNN (Convolutional Neural Network) model and then the pseudo incremental learning phase is performed. Furthermore, to enhance the performance, an optimisation approach, called Serial Exponential Sand Cat Swarm Optimisation (SExpSCSO), is developed. SExpSCSO algorithm modifies Sand Cat Swarm Optimisation by incorporating the serial exponential weighted moving average concept. The proposed SExpSCSO is applied to train the continually evolved classifier by optimising weights and thus, improves the classifiers performance. Finally, the experimentation analysis reveals that the adopted system acquired the maximal accuracy of 0.677, maximal specificity of 0.592, maximal precision of 0.638, recall of 0.716 and F-measure of 0.675.

Bayesian Network in Machine Learning: An Empirical Investigation to Assess the Price Clustering Model During Crises

Bayesian Network in Machine Learning: An Empirical Investigation to Assess the Price Clustering Model During Crises

Gene mutation estimations via mutual information and Ewens sampling based CNN & machine learning algorithms

We conduct gene mutation rate estimations via developing mutual information and Ewens sampling based convolutional neural network (CNN) and machine learning algorithms. More precisely, we develop a systematic methodology through constructing a CNN. Meanwhile, we develop two machine learning algorithms to study protein production with target gene sequences and protein structures. The core of the CNN and machine learning approach is to address a two-stage optimization problem to balance gene mutation rates during protein production. To wit, we try to optimally coordinate the consistency between the given input DNA sequences and the given (or optimally computed) target ones through controlling their intermediate gene mutation rates. The purposes in doing so are aimed to conduct gene editing and protein structure prediction. For example, after the gene mutation rates are estimated, the computing complexity of protein structure prediction will be reduced to a reasonable degree. Our developed CNN numerical optimization scheme consists of two newly designed machine learning algorithms. The stochastic gradients for the two algorithms are designed according to the Kuhn-Tucker conditions with boundary constraints and with the support of Ewens sampling, multi-input multi-output (MIMO) mutual information, and codon optimization techniques. The associated learning rate bounds are explicitly derived from the method and the two algorithms are numerically implemented. The convergence and optimality of the algorithms are mathematically proved. To illustrate the usage of our study, we also conduct a real-world data implementation.

Enhancing Gas-Liquid Two-Phase Stratified Flow Imaging with EIGEN-NET: A Fusion of Deep Learning and Measurement Matrices

Abstract Gas-liquid two-phase stratified flow is a common occurrence in various industries, demanding real-time monitoring and precise measurement for safety and operational efficiency. Current methods primarily rely on sensors or limited imaging technologies, leaving room for improvement. In this paper, we introduce the EIGEN-Net framework, specifically designed for real-time monitoring of gas-liquid
stratified flow within pipelines, utilizing Electrical Impedance Tomography(EIT). EIT, a noninvasive imaging method, shows potential despite challenges from gas affecting voltage stability in image reconstruction. EIGEN-Net capitalizes on measurement matrices and deep learning to enhance measurement accuracy. It extracts crucial physical properties characterizing liquid fractions and employs acquired voltage data to pre-reconstruct a tensor. By fusing these tensors with prior information, it significantly improves imaging outcomes, effectively addressing data insufficiency issues. The incorporation of adaptive information into deep learning networks eliminates the necessity for data classification. Moreover, this approach accommodates scenarios of continuous flow variation, enhancing imaging resolution. EIGEN-Net’s versatility extends to other two-phase flow scenarios and sensor measurement matrices, holding promise for applications in various industrial inspection domains. 
EIGEN-Net: Deep Learning Fusion Network Guided by Leading Eigenvalues of Measurement Matrix

Accurate Neural Network Fine-Tuning Approach for Transferable Ab Initio Energy Prediction across Varying Molecular and Crystalline Scales.

Existing machine learning models attempt to predict the energies of large molecules by training small molecules, but eventually fail to retain high accuracy as the errors increase with system size. Through an orbital pairwise decomposition of the correlation energy, a pretrained neural network model on hundred-scale data containing small molecules is demonstrated to be sufficiently transferable for accurately predicting large systems, including molecules and crystals. Our model introduces a residual connection to explicitly learn the pairwise energy corrections, and employs various low-rank retraining techniques to modestly adjust the learned network parameters. We demonstrate that with as few as only one larger molecule retraining the base model originally trained on only small molecules of (H2O)6, the MP2 correlation energy of the large liquid water (H2O)64 in a periodic supercell can be predicted at chemical accuracy. Similar performance is observed for large protonated clusters and periodic poly glycine chains. A demonstrative application is presented to predict the energy ordering of symmetrically inequivalent sublattices for distinct hydrogen orientations in the ice XV phase. Our work represents an important step forward in the quest for cost-effective, highly accurate and transferable neural network models in quantum chemistry, bridging the electronic structure patterns between small and large systems.

Enabling high-throughput quantitative wood anatomy through a dedicated pipeline.

Throughout their lifetime, trees store valuable environmental information within their wood. Unlocking this information requires quantitative analysis, in most cases of the surface of wood. The conventional pathway for high-resolution digitization of wood surfaces and segmentation of wood features requires several manual and time consuming steps. We present a semi-automated high-throughput pipeline for sample preparation, gigapixel imaging, and analysis of the anatomy of the end-grain surfaces of discs and increment cores. The pipeline consists of a collaborative robot (Cobot) with sander for surface preparation, a custom-built open-source robot for gigapixel imaging (Gigapixel Woodbot), and a Python routine for deep-learning analysis of gigapixel images. The robotic sander allows to obtain high-quality surfaces with minimal sanding or polishing artefacts. It is designed for precise and consistent sanding and polishing of wood surfaces, revealing detailed wood anatomical structures by applying consecutively finer grits of sandpaper. Multiple samples can be processed autonomously at once. The custom-built open-source Gigapixel Woodbot is a modular imaging system that enables automated scanning of large wood surfaces. The frame of the robot is a CNC (Computer Numerical Control) machine to position a camera above the objects. Images are taken at different focuspoints, with a small overlap between consecutive imagesin the X-Y plane, and merged by mosaic stitching, into a gigapixel image. Multiple scans can be initiated through the graphical application, allowing the system to autonomously image several objects and large surfaces. Finally, a Python routine using a trained YOLOv8 deep learning network allows for fully automated analysis of the gigapixel images, here shown as a proof-of-concept for the quantification of vessels and rays on full disc surfaces and increment cores. We present fully digitized beech discs of 30-35cm diameter at a resolution of 2.25 m, for which we automatically quantified the number of vessels (up to 13 million) and rays. We showcase the same process for five 30cm length beech increment cores also digitized at a resolution of 2.25 m, and generated pith-to-bark profiles of vessel density. This pipeline allows researchers to perform high-detail analysis of anatomical features on large surfaces, test fundamental hypotheses in ecophysiology, ecology, dendroclimatology, and many more with sufficient sample replication.

Phantom-metasurface cooperative system trained by deep learning network driven by bound state for magnetic-resonance-enhanced system

Phantom-metasurface cooperative system trained by deep learning network driven by bound state for magnetic-resonance-enhanced system

Adaptive Congestion Control Protocol based on Meta-Reinforcement Learning for Data Communication Networks

Adaptive Congestion Control Protocol based on Meta-Reinforcement Learning for Data Communication Networks

Artificial Intelligence in the Heart of Medicine: Transforming Arrhythmia Care with Intelligent Systems.

At a critical juncture in the ongoing fight against cardiovascular disease (CVD), healthcare professionals are striving for more informed and expedited decisionmaking. Artificial Intelligence (AI) promises to be a guiding light in this endeavor. The diagnosis of coronary artery disease has now become non-invasive and convenient, while wearable devices excel at promptly detecting life-threatening arrhythmias and treatments for heart failure. This study aimed to highlight the applications of AI in cardiology with a particular focus on arrhythmias and its potential impact on healthcare for all through careful implementation and constant research efforts. An extensive search strategy was implemented. The search was conducted in renowned electronic medical databases, including Medline, PubMed, Cochrane Library, and Google Scholar. Artificial Intelligence, cardiovascular diseases, arrhythmias, machine learning, and convolutional neural networks in cardiology were used as keywords for the search strategy. A total of 6876 records were retrieved from different electronic databases. Duplicates (N = 1356) were removed, resulting in 5520 records for screening. Based on predefined inclusion and exclusion criteria, 4683 articles were excluded. Following the full-text screening of the remaining 837 articles, a further 637 were excluded. Ultimately, 200 studies were included in this review. AI represents not just a development but a cutting-edge force propelling the next evolution of cardiology. With its capacity to make precise predictions, facilitate non-invasive diagnosis, and personalize therapies, AI holds the potential to save lives and enhance healthcare quality on a global scale.

Hybrid neural networks for continual learning inspired by corticohippocampal circuits

Current artificial systems suffer from catastrophic forgetting during continual learning, a limitation absent in biological systems. Biological mechanisms leverage the dual representation of specific and generalized memories within corticohippocampal circuits to facilitate lifelong learning. Inspired by this, we develop a corticohippocampal circuits-based hybrid neural network (CH-HNN) that emulates these dual representations, significantly mitigating catastrophic forgetting in both task-incremental and class-incremental learning scenarios. Our CH-HNNs incorporate artificial neural networks and spiking neural networks, leveraging prior knowledge to facilitate new concept learning through episode inference, and offering insights into the neural functions of both feedforward and feedback loops within corticohippocampal circuits. Crucially, CH-HNN operates as a task-agnostic system without increasing memory demands, demonstrating adaptability and robustness in real-world applications. Coupled with the low power consumption inherent to SNNs, our model represents the potential for energy-efficient, continual learning in dynamic environments.

Energy Scheduling of Hydrogen Hybrid UAV Based on Model Predictive Control and Deep Deterministic Policy Gradient Algorithm

Energy scheduling for hybrid unmanned aerial vehicles (UAVs) is of critical importance to their safe and stable operation. However, traditional approaches, predominantly rule-based, often lack the dynamic adaptability and stability necessary to address the complexities of changing operational environments. To overcome these limitations, this paper proposes a novel energy scheduling framework that integrates the Model Predictive Control (MPC) with a Deep Reinforcement Learning algorithm, specifically the Deep Deterministic Policy Gradient (DDPG). The proposed method is designed to optimize energy management in hydrogen-powered UAVs across diverse flight missions. The energy system comprises a proton exchange membrane fuel cell (PEMFC), a lithium-ion battery, and a hydrogen storage tank, enabling robust optimization through the synergistic application of MPC and DDPG. The simulation results demonstrate that the MPC effectively minimizes electric power consumption under various flight conditions, while the DDPG achieves convergence and facilitates efficient scheduling. By leveraging advanced mechanisms, including continuous action space representation, efficient policy learning, experience replay, and target networks, the proposed approach significantly enhances optimization performance and system stability in complex, continuous decision-making scenarios.

Optimization of sparse-view CT reconstruction based on convolutional neural network.

Sparse-view CT shortens scan time and reduces radiation dose but results in severe streak artifacts due to insufficient sampling data. Deep learning methods can now suppress these artifacts and improve image quality in sparse-view CT reconstruction. The quality of sparse-view CT reconstructed images can still be improved. Additionally, the interpretability of deep learning-based optimization methods for these reconstruction images is lacking, and the role of different network layers in artifact removal requires further study. Moreover, the optimization capability of these methods for reconstruction images from various sparse views needs enhancement. This study aims to improve the network's optimization ability for sparse-view reconstructed images, enhance interpretability, and boost generalization by establishing multiple network structures and datasets. In this paper, we developed a sparse-view CT reconstruction images improvement network (SRII-Net) based on U-Net. We added a copy pathway in the network and designed a residual image output block to boost the network's performance. Multiple networks with different connectivity structures were established using SRII-Net to analyze the contribution of each layer to artifact removal, improving the network's interpretability. Additionally, we created multiple datasets with reconstructed images of various sampling views to train and test the proposed network, investigating how these datasets from different sampling views affect the network's generalization ability. The results show that the proposed method outperforms current networks, with significant improvements in metrics like PSNR and SSIM. Image optimization time is at the millisecond level. By comparing the performance of different network structures, we've identified the impact of various hierarchical structures. The image detail information learned by shallow layers and the high-level abstract feature information learned by deep layers play a crucial role in optimizing sparse-view CT reconstruction images. Training the network with multiple mixed datasets revealed that, under a certain amount of data, selecting the appropriate categories of sampling views and their corresponding samples can effectively enhance the network's optimization ability for reconstructing images with different sampling views. The network in this paper effectively suppresses artifacts in reconstructed images with different sparse views, improving generalization. We have also created diverse network structures and datasets to deepen the understanding of artifact removal in deep learning networks, offering insights for noise reduction and image enhancement in other imaging methods.

Cancer drug resistance as learning of signaling networks.

Drug resistance is a major cause of tumor mortality. Signaling networks became useful tools for driving pharmacological interventions against cancer drug resistance. Signaling datasets now cover the entire human cell. Recently, network adaptation became understood as a learning process. We review rapidly increasing evidence showing that the development of cancer drug resistance can be described as learning of signaling networks. During drug adaptation, the network forgets drug-affected pathways by desensitization and relearns by strengthening alternative pathways. Thus, resistant cancer cells develop a drug resistance memory. We show that all key players of cellular learning (i.e., IDPs, protein translocation, microRNAs/lncRNAs, scaffolding proteins and epigenetic/chromatin memory) have important roles in the development of cancer drug resistance. Moreover, all of them are central components of the epithelial-mesenchymal transition leading to metastases and resistance. Phenotypic plasticity was recently listed as a hallmark of cancer. We review how network plasticity induces rare, pre-existent drug-resistant cells in the absence of drug treatment. Key network methods assessing the development of drug resistance and network pharmacological interventions against drug resistance are summarized. Finally, we highlight the class of cellular memory drugs affecting cellular learning and forgetting, and we summarize current challenges to prevent or break drug resistance using network models.

Swin-MSTP: Swin transformer with multi-scale temporal perception for continuous sign language recognition

Continuous sign language recognition (CSLR) aims to recognize and interpret sequences of sign language gestures in videos. Currently, most CSLR frameworks combine spatial feature extractors based on convolutional neural networks (CNNs) with temporal convolutional networks (TCNs) for sequence learning. However, CNN-based spatial feature extractors apply the same convolutional kernel uniformly across all regions of an image, which limits their capacity to extract complex details such as fingers and facial features, which are essential for CSLR. In addition, sign languages include signs of varying lengths that cannot be accurately modeled using a fixed-size TCN. To address these issues, we present the Swin multiscale temporal perception (Swin-MSTP) framework, in which the Swin Transformer is utilized as the spatial feature extractor, capable of capturing fine spatial details and providing a stronger contextual understanding between sign language elements in video frames. The Swin Transformer was integrated with the MSTP module to extract time-wise features. Experimental results show that our single-modality system outperformed existing methods on the CSL dataset, including multimodal frameworks. The model also achieved competitive performance on the Phoenix2014, Phoenix2014-T, and CSL-Daily datasets. The code is available at https://github.com/Hamzah-Luqman/SWIN-MSTP.

PRNet: Parallel Reinforcement Network for two-view correspondence learning

PRNet: Parallel Reinforcement Network for two-view correspondence learning

Multi-compartment neuron and population encoding powered spiking neural network for deep distributional reinforcement learning.

Inspired by the brain's information processing using binary spikes, spiking neural networks (SNNs) offer significant reductions in energy consumption and are more adept at incorporating multi-scale biological characteristics. In SNNs, spiking neurons serve as the fundamental information processing units. However, in most models, these neurons are typically simplified, focusing primarily on the leaky integrate-and-fire (LIF) point neuron model while neglecting the structural properties of biological neurons. This simplification hampers the computational and learning capabilities of SNNs. In this paper, we propose a brain-inspired deep distributional reinforcement learning algorithm based on SNNs, which integrates a bio-inspired multi-compartment neuron (MCN) model with a population coding approach. The proposed MCN model simulates the structure and function of apical dendritic, basal dendritic, and somatic compartments, achieving computational power comparable to that of biological neurons. Additionally, we introduce an implicit fractional embedding method based on population coding of spiking neurons. We evaluated our model on Atari games, and the experimental results demonstrate that it surpasses the vanilla FQF model, which utilizes traditional artificial neural networks (ANNs), as well as the Spiking-FQF models that are based on ANN-to-SNN conversion methods. Ablation studies further reveal that the proposed multi-compartment neuron model and the quantile fraction implicit population spike representation significantly enhance the performance of MCS-FQF while also reducing power consumption.