Learning Process Of Network Research Articles

Unified Flowing Normality Learning for Rotating Machinery Anomaly Detection in Continuous Time-Varying Conditions.

Intelligent anomaly detection (AD) methods have achieved much successes in machinery condition monitoring. However, the underlying independent and identically distributed assumption restricts their application scopes to steady operating conditions. False and missing alarms would occur when machines operate under time-varying circumstances. In this work, a more challenging time-varying setting is studied, where the working conditions are continuously changing, such that few or no samples are available for model training at one single condition. To tackle this issue, we propose a unified flowing normality learning (UFNL) framework, which aims to capture the flowing normal conditional distribution of time-varying samples and assigns dynamic decision boundary for AD. Specifically, a manifold-based probability density estimation is utilized to guide the adversarial learning process of generative adversarial networks, where adjacent samples are aggregated to approximate the conditional distribution by a conditional generator. Then, a latent normality inversion is proposed to extract the manifold structure from the pretrained generator and to map it into the latent space via a conditional encoder. The reconstruction errors from the encoder and generator can reveal the deviation of signals to the flowing normality. Finally, a condition-aware adaptive threshold selection strategy is proposed, where different thresholds are adaptively assigned for different conditions. Experiments are carried out under two typical continuous time-varying scenarios. The results demonstrate that the proposed framework can realize accurate fault detection at any operating condition within continuously changing environments.

Neural Solving Uninterpreted Predicates with Abstract Gradient Descent

Uninterpreted predicate solving is a fundamental problem in formal verification, including loop invariant and constrained horn clauses predicate solving. Existing approaches have been mostly in symbolic ways. While achieving sustainable progress, they still suffer from inefficiency and seem unable to leverage the ever-increasing computility, such as GPU. Recently, neural relaxation has been proposed to tackle this problem. They treat the uninterpreted predicate-solving task as an optimization problem by relaxing the discrete search process into a learning process of neural networks. However, two bottlenecks keep them from being valid. First, relaxed neural networks cannot match the original semantics of predicates rigorously; second, the neural networks are difficult to train to reach global optimization. Therefore, this article presents a novel discrete neural architecture with the Abstract Gradient Decent (AGD) algorithm to directly solve uninterpreted predicates in the discrete hypothesis space. The abstract gradient is for discrete neurons whose calculation rules are designed in an abstract domain. Our approach conforms to the original semantics of predicates, and the proposed AGD algorithm can achieve global optimization satisfactorily. We implement the tool Dasp in the Boxes abstract domain to solve uninterpreted predicates in the QF-NIA SMT theory. In the experiments, Dasp has outperformed seven state-of-the-art tools across three predicate synthesis tasks.

Quantum-inspired semantic matching based on neural networks with the duality of density matrices

Social media text can be semantically matched in different ways, viz paraphrase identification, answer selection, community question answering, and so on. The performance of the above semantic matching tasks depends largely on the ability of language modeling. Neural network based language models and probabilistic language models are two main streams of language modeling approaches. However, few prior work has managed to unify them in a single framework on the premise of preserving probabilistic features during the neural network learning process. Motivated by recent advances of quantum-inspired neural networks for text representation learning, we fill the gap by resorting to density matrices, a key concept describing a quantum state as well as a quantum probability distribution. The state and probability views of density matrices are mapped respectively to the neural and probabilistic aspects of language models. Concretizing this state-probability duality to the semantic matching task, we build a unified neural-probabilistic language model through a quantum-inspired neural network. Specifically, we take the state view to construct a density matrix representation of sentence, and exploit its probabilistic nature by extracting its main semantics, which form the basis of a legitimate quantum measurement. When matching two sentences, each sentence is measured against the main semantics of the other. Such a process is implemented in a neural structure, facilitating an end-to-end learning of parameters. The learned density matrix representation reflects an authentic probability distribution over the semantic space throughout the training process. Experiments show that our model significantly outperforms a wide range of prominent classical and quantum-inspired baselines.

An optimized multilayer perceptron-based network intrusion detection using Gray Wolf Optimization

The exponential growth in the use of network services through the design of various network infrastructures, has led to increased complexities and challenges in the network. A major problem in computer networks is privacy and security breach. Cyber attackers exploit loopholes to infiltrate and disrupt the operation of the network through various attacks. Anomaly-based intrusion detection often employs Artificial Neural Network techniques like Multi-layer Perceptron (MLP) to classify malicious and legitimate traffic. Nevertheless, these techniques are vulnerable to overfitting and require extensive labeled data and computational resources. Consequently, this reduces the accuracy of intrusion detection systems and increases the error detection rate. To minimize the error detection rate of the intrusion detection system, it is necessary to optimize the connection parameters of the MLP neural network such as weights and biases. To this end, we proposed an optimized MLP-based Intrusion Detection using Gray Wolf Optimization (GWOMLP-IDS) to optimize the learning process of the MLP neural network by optimizing weights and biases. GWO aims to select an optimal connection parameter during the learning process to minimize the error rate of intrusion detection. Extensive simulations in Python reveal the effectiveness of the proposed approach in terms of designated performance metrics.

Achieving view-distance and -angle invariance in motion prediction using a simple network

Recently, human motion prediction has gained significant attention and achieved notable success. However, current methods primarily rely on training and testing with ideal datasets, overlooking the impact of variations in the viewing distance and viewing angle, which are commonly encountered in practical scenarios. In this study, we address the issue of model invariance by ensuring robust performance despite variations in view distances and angles. To achieve this, we employed Riemannian geometry methods to constrain the learning process of neural networks, enabling the prediction of invariances using a simple network. Furthermore, this enhances the application of motion prediction in various scenarios. Our framework uses Riemannian geometry to encode motion into a novel motion space to achieve prediction with an invariant viewing distance and angle using a simple network. Specifically, the specified path transport square-root velocity function is proposed to aid in removing the view-angle equivalence class and encode motion sequences into a flattened space. Motion coding by the geometry method linearizes the optimization problem in a non-flattened space and effectively extracts motion information, allowing the proposed method to achieve competitive performance using a simple network. Experimental results on Human 3.6M and CMU MoCap demonstrate that the proposed framework has competitive performance and invariance to the viewing distance and viewing angle.

Individual Buffalo Identification Through Muzzle Dermatoglyphics Images using Deep Learning Approaches

Animal identification is essential for routine farm operations, residue traceback, insurance, and ownership management. Owing to their uniqueness, incorrigible nature, tamperproof over time, environment-friendly, and pain-free, visual biometrics-based animal identification has recently gained momentum over traditional animal identification methods. Among visual biometrics-based cues, muzzle identification is a simple and relatively low-cost method. Therefore, to address the inherent significant limitations of conventional animal identification systems, we undertook this investigation to collect a database of digital images of muzzles that works as a benchmark, to apply deep learning frameworks to identify individual buffaloes from their muzzle images, and to compare their accuracy in terms of their identification capabilities. Muzzle images of 198 Surti buffaloes were subjected to transfer learning and fine-tuning processes in deep-learning neural networks. The performance was recorded for each pre-train model (ResNet50, InceptionV3, VGG16, AlexNet) with different hyperparameters of the epoch, batch size, and learning rate. A perusal of the data revealed that ResNet50 has the highest train accuracy (99.8%) and test accuracy (99.69%) among all four models used. AlexNet has the lowest train accuracy (90.8%) among the models. The findings concluded that all these four models could be applied to identify individual buffaloes; however, ResNet50 had the highest accuracy, and deep learning applications have great potential for individual buffalo identification and are promising tools for precision livestock farming

Research on priority scheduling strategy for smoothing power fluctuations of microgrid tie‐lines based on PER‐DDPG algorithm

AbstractThe variability of renewable energy within microgrids (MGs) necessitates the smoothing of power fluctuations through the effective scheduling of internal power equipment. Otherwise, significant power variations on the tie‐line connecting the MG to the main power grid could occur. This study introduces an innovative scheduling strategy that utilizes a data‐driven approach, employing a deep reinforcement learning algorithm to achieve this smoothing effect. The strategy prioritizes the scheduling of MG's internal power devices, taking into account the stochastic charging patterns of electric vehicles. The scheduling optimization model is initially described as a Markov decision process with the goal of minimizing power fluctuations on the interconnection lines and operational costs of the MG. Subsequently, after preprocessing the historical operational data of the MG, an enhanced scheduling strategy is developed through a neural network learning process. Finally, the results from four scheduling scenarios demonstrate the significant impact of the proposed strategy. Comparisons of reward curves before and after data preprocessing underscore its importance. In contrast to optimization results from deep deterministic policy gradient, soft actor‐critic, and particle swarm optimization algorithms, the superiority of the deep deterministic policy gradient algorithm with the addition of a priority experience replay mechanism is highlighted.

Agent-Based Model for Situational Awareness in the Workplace: Enhancing Neural Networks with Direct Feedback Alignment

The integration of direct feedback alignment (DFA) into neural networks presents a paradigm shift in computational models, enhancing situational awareness within professional settings. This paper explores the application of DFA in agent-based computational models, demonstrating its efficiency and biological plausibility over traditional backpropagation methods. The research highlights the significant impact of direct feedback alignment on reinforcing situational awareness, evidenced through comprehensive simulations that show marked improvements in agents' ability to effectively navigate and comprehend complex work environments. The study suggests that direct feedback alignment revolutionizes neural network learning processes and significantly enhances cognitive aspects critical to employee safety and operational efficiency. It lays the groundwork for future investigations into the integration of neural computation techniques with organizational psychology and behavior, offering a new perspective on fostering safer, more aware, and efficient workplace environments. The potential application of direct feedback alignment across various professional scenarios opens new avenues for research in computational neuroscience, cognitive psychology, and organizational behavior, with a focus on optimizing human-environment interactions in complex systems.

A modified physics-informed neural network to fatigue life prediction of deck-rib double-side welded joints

Deck-rib welded joint is an important part of orthotropic steel decks, which are susceptible to fatigue cracks under various load cycles. Therefore, it is essential to use efficient predicting methods to assess the fatigue performance of deck-rib welded joints with new geometry and form in order to ensure a reliable operation of the bridge. A modified physics-informed neural network (PINN) is proposed for the fatigue life prediction of the novel deck-rib double-side welded joints. The modified PINN incorporates implicit physical model as an activation function into the hidden neurons, and explicit physical constraints into the loss function. Additionally, the influence of deck thickness is integrated using limited test sample data based on the observational approach. The 62 experimental data were taken from the literature to compare the performance of the artificial neural network (ANN), traditional PINN and the modified PINN. The results demonstrate that the modified PINN enhances the learning process of the neural network through the joint learning of physical knowledge by activation function and loss function. Furthermore, the fatigue life prediction of the deck-rib double-side welded joints under the modified PINN exhibits physical consistency and accuracy when compared with the traditional ANN and PINN.

Soft computing-driven infrared and visible image fusion network for security application service

The high-level visual tasks (HLVT)-specific infrared and visible image fusion networks aim to guide the fusion network to focus on specific feature regions. The resulting fused images enhance the performance of HLVT in adverse conditions. However, cross-layer visual task fusion may diminish the fusion network’s adaptability to diverse data. This deficiency results in a lack of robustness and security in the fusion network. Consequently, the fusion network is rendered insufficient to fulfill the requirements of HLVT security applications, including object detection and security systems. To address these challenges, we propose a soft computing-driven infrared and visible image fusion network based on layer separation and dual-stream adversarial learning, referred to as LsDSGFuse. Firstly, we employ a layer segmentation network based on a high-resolution network to create salient object segmentation masks for the original image. This helps mitigate the impact of task-specific guidance on the fusion network learning process, preventing overfitting to HLVT. Secondly, we use the original image after information separation as the input of the generator. The separated network inputs clearly define foreground objects and background environments, aiding the generator in better fusing foreground and background. Lastly, we adopt a no-loading training strategy to balance adversarial learning, prevent discriminator overfitting, and reduce resource waste. Experimental results demonstrate the outstanding performance of our proposed image fusion method in terms of fusion quality, surpassing several representative approaches. Furthermore, experiments in HLVT indicate that our method facilitates HLVT in better responding to potential threats and unknown variables. This improvement significantly enhances HLVT’s security performance.

Automatic segmentation of intraluminal thrombosis of abdominal aortic aneurysms from CT angiography using a mixed-scale-driven multiview perception network (M2Net) model

Intraluminal thrombosis (ILT) plays a critical role in the progression of abdominal aortic aneurysms (AAA). Understanding the role of ILT can improve the evaluation and management of AAAs. However, compared with highly developed automatic vessel lumen segmentation methods, ILT segmentation is challenging. Angiographic contrast agents can enhance the vessel lumen but cannot improve boundary delineation of the ILT regions; the lack of intrinsic contrast in the ILT structure significantly limits the accurate segmentation of ILT. Additionally, ILT is not evenly distributed within AAAs; its sparsity and scattered distributions in the imaging data pose challenges to the learning process of neural networks. Thus, we propose a multiview fusion approach, allowing us to obtain high-quality ILT delineation from computed tomography angiography (CTA) data. Our multiview fusion network is named Mixed-scale-driven Multiview Perception Network (M2Net), and it consists of two major steps. Following image preprocessing, the 2D mixed-scale ZoomNet segments ILT from each orthogonal view (i.e., Axial, Sagittal, and Coronal views) to enhance the prior information. Then, the proposed context-aware volume integration network (CVIN) effectively fuses the multiview results.Using contrast-enhanced computed tomography angiography (CTA) data from human subjects with AAAs, we evaluated the proposed M2Net. A quantitative analysis shows that the proposed deep-learning M2Net model achieved superior performance (e.g., DICE scores of 0.88 with a sensitivity of 0.92, respectively) compared with other state-of-the-art deep-learning models. In closing, the proposed M2Net model can provide high-quality delineation of ILT in an automated fashion and has the potential to be translated into the clinical workflow.

SPD Manifold Deep Metric Learning for Image Set Classification.

By characterizing each image set as a nonsingular covariance matrix on the symmetric positive definite (SPD) manifold, the approaches of visual content classification with image sets have made impressive progress. However, the key challenge of unhelpfully large intraclass variability and interclass similarity of representations remains open to date. Although, several recent studies have mitigated the two problems by jointly learning the embedding mapping and the similarity metric on the original SPD manifold, their inherent shallow and linear feature transformation mechanism are not powerful enough to capture useful geometric features, especially in complex scenarios. To this end, this article explores a novel approach, termed SPD manifold deep metric learning (SMDML), for image set classification. Specifically, SMDML first selects a prevailing SPD manifold neural network (SPDNet) as the backbone (encoder) to derive an SPD matrix nonlinear representation. To counteract the degradation of structural information during multistage feature embedding, we construct a Riemannian decoder at the end of the encoder, trained by a reconstruction error term (RT), to induce the generated low-dimensional feature manifold of the hidden layer to capture the pivotal information about the visual data describing the imaged scene. We demonstrate through theory and experiments that it is feasible to replace the Riemannian metric with Euclidean distance in RT. Then, the ReCov layer is introduced into the established Riemannian network to regularize the local statistical information within each input feature matrix, which enhances the effectiveness of the learning process. The theoretical analysis of the activation function used in the ReCov layer in terms of continuity and conditions for generating positive definite matrices is beneficial for network design. Inspired by the fact that the single cross-entropy loss used for training is unable to effectively parse the geometric distribution of the deep representations, we finally endow the suggested model with a novel metric learning regularization term. By explicitly incorporating the encoding and processing of the data variations into the network learning process, this term can not only derive a powerful Riemannian representation but also train an effective classifier. The experimental results show the superiority of the proposed approach on three typical visual classification tasks.

Development of a simulation model and adjustment of hyperparameters of a neural network for predicting possible states of a network of small space carriages

The purpose of the research is to develop a simulation model and tune the hyperparameters of a neural network to predict possible states of a network of small spacecraft.The research methods are based on the concepts of the theory of artificial intelligence to control a group of small spacecraft (SSV) – the use of adaptive methods and tools to make decisions, similar to the mechanisms of human thinking. In relation to space communication systems with a heterogeneous structure, artificial intelligence methods and technologies are aimed at predicting the state of communication channels between network nodes and automatically reconfiguring a network of devices based on neural network learning processes. One of the most important functions of network software for the application of cognitive algorithms is to predict the quality of communication between pairs of SSVs.Results. A method has been developed for using the Transformer architecture neural network to predict possible states of the SSV network, which provides aggregation and time synchronization of data on the state of the SSV network, their use for training the neural network, as well as using the neural network to predict the quality of communication. The data format for the training sample has been created, based on the representation of the state of the SSV network, which ensures the generation of the initial state of the network, modeling the proactive mode of its operation, collecting SSV network state markers to generate training data sets in the form of chronological sequences grouped into frames, and allowing to reduce the amount of data transferred between SSVs when creating a training set. A simulation model of the SSV network has been developed, which provides generation of the initial state of the network, modeling of the proactive mode of its operation, and collection of information about the state of the SSV network to generate sets of synthetic training data.Conclusion. The article develops a simulation model of the SSV network for generating synthetic training data and predicting possible states of the SSV network, as well as a method for using a neural network to predict possible states of the SSV network.

KaTaGCN: Knowledge-Augmented and Time-Aware Graph Convolutional Network for efficient traffic forecasting

Dynamic spatio-temporal dependencies and temporal patterns in traffic series are critical factors affecting traffic forecasting accuracy. Due to the intrinsic challenges of incorporating explicit, logical knowledge into the implicit black-box learning process of neural networks, only a few methods effectively use prior knowledge to improve the learning of traffic forecasting. To tackle this problem, we introduce a new approach called Knowledge-augmented and Time-aware Graph Convolutional Network, namely KaTaGCN. At its core, we have created a knowledge-augmented module that boosts the diffusion weights between closely related adjacent nodes in graph learning. This is achieved by implementing a new loss function. Then, to learn the periodic implicit relationship between these timestamps and traffic signals, the weights and biases are chosen adaptively to be trained based on the timestamps of each sample. Finally, a gated spatio-temporal mapping module regresses high-dimensional embedded features from spatial and temporal dimensions. KaTaGCN is structured without any attention mechanisms or recurrent neural networks. Extensive experimental results on six real-world public traffic datasets demonstrate that KaTaGCN achieves an average improvement of 4.29% in forecasting performance compared with suboptimal results.

Programmable Retention Characteristics in MoS2-Based Atomristors for Neuromorphic and Reservoir Computing Systems.

In this study, we investigate the coexistence of short- and long-term memory effects owing to the programmable retention characteristics of a two-dimensional Au/MoS2/Au atomristor device and determine the impact of these effects on synaptic properties. This device is constructed using bilayer MoS2 in a crossbar structure. The presence of both short- and long-term memory characteristics is proposed by using a filament model within the bilayer transition-metal dichalcogenide. Short- and long-term properties are validated based on programmable multilevel retention tests. Moreover, we confirm various synaptic characteristics of the device, demonstrating its potential use as a synaptic device in a neuromorphic system. Excitatory postsynaptic current, paired-pulse facilitation, spike-rate-dependent plasticity, and spike-number-dependent plasticity synaptic applications are implemented by operating the device at a low-conductance level. Furthermore, long-term potentiation and depression exhibit symmetrical properties at high-conductance levels. Synaptic learning and forgetting characteristics are emulated using programmable retention properties and composite synaptic plasticity. The learning process of artificial neural networks is used to achieve high pattern recognition accuracy, thereby demonstrating the suitability of the use of the device in a neuromorphic system. Finally, the device is used as a physical reservoir with time-dependent inputs to realize reservoir computing by using short-term memory properties. Our study reveals that the proposed device can be applied in artificial intelligence-based computing applications by utilizing its programmable retention properties.

A maize seed variety identification method based on improving deep residual convolutional network.

Seed quality and safety are related to national food security, and seed variety purity is an essential indicator in seed quality detection. This study established a maize seed dataset comprising 5877 images of six different types and proposed a maize seed recognition model based on an improved ResNet50 framework. Firstly, we introduced the ResStage structure in the early stage of the original model, which facilitated the network's learning process and enabled more efficient information propagation across the network layers. Meanwhile, in the later residual blocks of the model, we introduced both the efficient channel attention (ECA) mechanism and depthwise separable (DS) convolution, which reduced the model's parameter cost and enabled the capturing of more precise and detailed features. Finally, a Swish-PReLU mixed activation function was introduced globally to improve the overall predictive power of the model. The results showed that our model achieved an impressive accuracy of 91.23% in corn seed classification, surpassing other related models. Compared with the original model, our model improved the accuracy by 7.07%, reduced the loss value by 0.19, and decreased the number of parameters by 40%. The research suggested that this method can efficiently classify corn seeds, holding significant value in seed variety identification.

Physics-Constraint Variational Neural Network for Wear State Assessment of External Gear Pump.

Most current data-driven prognosis approaches suffer from their uncontrollable and unexplainable properties. To address this issue, this article proposes a physics-constraint variational neural network (PCVNN) for wear state assessment of the external gear pump. First, a response model of the pressure pulsation of the gear pump is constructed via a spectral method, and a compound neural network is utilized to extract features from the pressure pulsation signal. Then, the response model is formulated into an objective function to softly constrain the learning process of the neural network, forcing the learned features to have explicit physics meaning. Meanwhile, to characterize the system uncertainty, the variational inference is utilized to extend a Kullback-Leibler (KL) divergence into the objective function. Finally, the wear state is evaluated based on the distance of learned physics features. Experimental results on an external gear pump validate the merits of the proposed method in explainable representation learning and system uncertainty estimation. It also offers a controllable and explainable perspective to understand the dynamic behavior of the system.

RECURSIVE NEURAL NETWORK AS A HIGH-SPEED PLATE COLLISION EMULATOR

Based on a database obtained using a high-speed plate impact model that relates impact parameters and material model parameters to the free surface velocity profile, the study compares the learning process and accuracy of a feedforward artificial neural network and a recursive neural network. A recursive neural network provides a significantly greater accuracy and requires less training time. Using a recursive neural network as a fast model emulator and Bayesian calibration can make it possible to solve the inverse problem of determining the substance model parameters from the free surface velocity profile with a greater accuracy.

Contrastive Multiscale Transformer for Image Dehazing.

Images obtained in an unfavorable environment may be affected by haze or fog, leading to fuzzy image details, low contrast, and loss of important information. Recently, significant progress has been achieved in the realm of image dehazing, largely due to the adoption of deep learning techniques. Owing to the lack of modules specifically designed to learn the unique characteristics of haze, existing deep neural network-based methods are impractical for processing images containing haze. In addition, most networks primarily focus on learning clear image information while disregarding potential features in hazy images. To address these limitations, we propose an innovative method called contrastive multiscale transformer for image dehazing (CMT-Net). This method uses the multiscale transformer to enable the network to learn global hazy features at multiple scales. Furthermore, we introduce feature combination attention and a haze-aware module to enhance the network's ability to handle varying concentrations of haze by assigning more weight to regions containing haze. Finally, we design a multistage contrastive learning loss incorporating different positive and negative samples at various stages to guide the network's learning process to restore real and non-hazy images. The experimental findings demonstrate that CMT-Net provides exceptional performance on established datasets and exhibits superior visual outcomes.

Phase transitions in the mini-batch size for sparse and dense two-layer neural networks

The use of mini-batches of data in training artificial neural networks is nowadays very common. Despite its broad usage, theories explaining quantitatively how large or small the optimal mini-batch size should be are missing. This work presents a systematic attempt at understanding the role of the mini-batch size in training two-layer neural networks. Working in the teacher-student scenario, with a sparse teacher, and focusing on tasks of different complexity, we quantify the effects of changing the mini-batch size m. We find that often the generalization performances of the student strongly depend on m and may undergo sharp phase transitions at a critical value mc , such that for m<mc the training process fails, while for m>mc the student learns perfectly or generalizes very well the teacher. Phase transitions are induced by collective phenomena firstly discovered in statistical mechanics and later observed in many fields of science. Observing a phase transition by varying the mini-batch size across different architectures raises several questions about the role of this hyperparameter in the neural network learning process.