Network Training Research Articles

Quantitative modeling of signaling in aggressive B cell lymphoma unveils conserved core network.

B cell receptor (BCR) signaling is required for the survival and maturation of B cells and is deregulated in B cell lymphomas. While proximal BCR signaling is well studied, little is known about the crosstalk of downstream effector pathways, and a comprehensive quantitative network analysis of BCR signaling is missing. Here, we semi-quantitatively modelled BCR signaling in Burkitt lymphoma (BL) cells using systematically perturbed phosphorylation data of BL-2 and BL-41 cells. The models unveiled feedback and crosstalk structures in the BCR signaling network, including a negative crosstalk from p38 to MEK/ERK. The relevance of the crosstalk was verified for BCR and CD40 signaling in different BL cells and confirmed by global phosphoproteomics on ERK itself and known ERK target sites. Compared to the starting network, the trained network for BL-2 cells was better transferable to BL-41 cells. Moreover, the BL-2 network was also suited to model BCR signaling in Diffuse large B cell lymphoma cells lines with aberrant BCR signaling (HBL-1, OCI-LY3), indicating that BCR aberration does not cause a major downstream rewiring.

YOLO-V4 Based Real-Time Face Mask Detection via Unmanned Aerial Vehicle

COVID-19, which started in Wuhan city of China's Hubei province and then affected the whole world, continues to spread despite the measures taken. One of the most important of these measures is to use a mask. In some countries, while wearing a mask is mandatory in crowded environments, it is just as difficult to control it. Failure to detect individuals violating the mask causes the virus to spread, resulting in an increase in the number of cases and an increase in the number of deaths. Therefore, detecting the mask and taking action against it is an extremely important issue. In this study, in addition to making mask detection easily and quickly, mask detection is made by using unmanned aerial vehicles (UAVs) using images taken from different angles and different heights. For mask detection from UAV images, training and validation processes were applied using the YOLO-v4 algorithm on a public dataset containing 1510 masked and unmasked human face images. As a result of the training on this dataset, mean-average precision (mAP) was achieved with a success rate of 92.06%. Then, real-time mask detection was performed on the images taken with the DJI Ryze Tello quadcopter using the trained network. The results showed that the UAV is applicable in crowded environments required for autonomous mask detection and gives successful results.

МЕТОД ОБУЧЕНИЯ ИНТЕЛЛЕКТУАЛЬНОГО АГЕНТА С ПОМОЩЬЮ СЕТЕЙ DOUBLE DQN, ПУТЕВЫХ ТОЧЕК И ФУНКЦИИ ВОЗНАГРАЖДЕНИЯ

The problems of increasing the control efficiency of autonomous vehicles are considered. The problem of reducing the required computational resources for the intelligent vehicle control module is highlighted. A neural network training algorithm for Double DQN architecture with modified reward functions is proposed. The basis of the proposed solution is the use of lane segmentation, reward function and the use of additional waypoints in training. A software model has been developed and simulation of the learning process has been performed. The results obtained from a comparative analysis with known solutions show a stable increase in episode duration, and effective training in a realistic urban simulation. The study points to the possibility of reducing the need for high computing power, which will enable the use of central processing units (CPUs) for basic functions of unmanned vehicles instead of graphics processing units (GPUs).

Adaptive pre-processing methods for increasing the accuracy of segmentation of dental X-RAY images

The subject of research in the article is the effectiveness of adaptive methods of preprocessing of medical images, in particular adaptive bilateral filter and modified CLAHE, in the tasks of segmentation of dental X-ray images. These methods make it possible to preserve important image details and effectively reduce noise, even in cases of high variability of images coming from different sources. The goal of the work is to study the impact of adaptive preprocessing methods on increasing the accuracy of segmentation of medical images and to determine the optimal combination of methods that provide the best results in segmentation tasks. The article addresses the following tasks: experimental comparison of adaptive preprocessing methods with traditional approaches, evaluation of segmentation efficiency using metrics such as Dice Score, Jacquard Coefficient (IoU Score), Precision and Sensitivity/Completeness (Recall)), as well as analysis of the effect of pre-processing on the quality of segmentation. The following methods are used: mathematical modeling, neural network training based on the U-Net model with a pre-trained timm-resnest101e encoder, image scaling to 512x512 pixels, training with a fixed learning rate of 0.001. The following results were obtained: the combined use of the adaptive bilateral filter and the modified CLAHE provided the highest segmentation quality indicators (Dice Score 0.9603 and Jacquard Coefficient (IoU Score) 0.94501), surpassing traditional methods. This proves the advantage of adaptive approaches in preserving the contours of objects and reducing noise. Conclusions: the application of adaptive preprocessing methods significantly improves the accuracy of segmentation of medical images. The combined approach including the adaptive bilateral filter and the modified CLAHE is the most effective for medical imaging tasks, which increases the accuracy of diagnosis and the reliability of automated decision support systems.

Efficient semi-supervised hierarchical training for segmenting choroidal vessels and other structures on OCT images of multiple sclerosis patients

Optical coherence tomography (OCT) is a non-invasive imaging technique used to diagnose ocular and systemic diseases. Recently, several clinical studies have linked changes in different ocular layers to the development of multiple sclerosis (MS), so accurate segmentation of these structures has become an essential task. Unfortunately, segmenting the entire set of structures involved is a very difficult task, due to their large number and variability. These two factors hinder the labelling of images and therefore severely restrict the ability to achieve a large dataset with all structures manually annotated, limiting the use of a standard supervised approach. In this paper, we propose a semi-supervised learning methodology to robustly segment ocular structures in OCT images using a limited number of partially labeled images. Our methodology maximizes the information we can extract from labeled images through hierarchical learning, where multiple decoders are used to extract segmented structures progressively. We use a reconstruction loss function to provide structural coherence to the segmentation and a teacher-student strategy to effectively leverage the information present in the set of unlabeled images. In addition to the segmentation of labelled structures, this hierarchical approach allows segmenting structures that are not labelled in the dataset such as the choroidal vessels. To validate the proposed methodology, we have carried out extensive experimentation using two datasets with different characteristics. These experiments have demonstrated a great potential of this methodology to train networks efficiently with partially labeled images, which allows to accurately extract the main biomarkers linked to the development of MS.

Practical Approach to Studying Evolutionary Methods for Setting Weight Coefficients of Artificial Neural Networks

The article describes the problems of developing neurocontrollers for controlling dynamic objects, including the complexity of forming training data sets. It is indicated that one of the known methods for training an artificial neural network controlling an object is the neuroevolutionary approach, which involves using a genetic algorithm to adjust the synaptic weighting coefficients of an artificial neural network. The idea of using a means of demonstrating the evolutionary approach to adjusting the weighting coefficients of an artificial neural network for practical training of students in the basics of the neuroevolutionary approach is proposed. Software has been developed to demonstrate the neuroevolutionary approach using the example of the evolution of an artificial neural network of a given structure intended to control a simplified computer model of an autonomous vehicle. A method for resolving the problem of stagnation when using the evolutionary approach to training an artificial neural network is described. Options for using the developed software in teaching students the basics of artificial intelligence technologies and evolutionary methods of multicriteria optimization are proposed.

IIS-FVIQA: Finger Vein Image Quality Assessment with intra-class and inter-class similarity

In recent years, Finger Vein Image Quality Assessment (FVIQA) has been recognized as an effective solution to the problem of erroneous recognition resulting from low image quality due to false and missing information in finger vein images, and has become an important part of finger vein recognition systems. Compared to traditional FVIQA methods that rely on domain knowledge, newer methods that reject low-quality images have been favored for their independence from human interference. However, these methods only consider intra-class similarity information and ignore valuable information from inter-class distribution, which is also an important factor in evaluating the performance of recognition systems. In this work, we propose a novel FVIQA approach, named IIS-FVIQA, which concurrently takes into account the intra-class similarity density and inter-class similarity distribution distance within recognition systems. Specifically, our method generates quality scores for finger vein images by combining the information entropy of intra-class similarity distribution and Wasserstein distance of inter-class distribution. Then, we train a regression network for quality prediction using training images and corresponding quality scores. When a new image enters the recognition system, the trained regression network directly predicts the quality score of the image, making it easier for the system to select the corresponding operation based on the quality score of the image. Extensive experiments conducted on benchmark datasets demonstrate that the IIS-FVIQA method proposed in this paper consistently achieves top performance across multiple public datasets. After filtering out 10% of low-quality images predicted by the quality regression network, the recognition system’s performance improves by 43.96% (SDUMLA), 32.23% (MMCBNU_6000), and 21.20% (FV-USM), respectively. Furthermore, the method exhibits strong generalizability across different recognition algorithms (e.g., LBP, MC, and Inception V3) and datasets (e.g., SDUMLA, MMCBNU_6000, and FV-USM).

Deep-learning based single-shot 3D reconstruction with simulated color-crosstalk and randomized extrinsics

In recent years, many single-frame 3D reconstruction schemes based on fringe projection profilometry (FPP) have been proposed. However, most single-frame reconstruction schemes still face the following three issues: (1) obtaining large datasets is very time-consuming, (2) focusing only on achieving single-frame reconstruction for white objects, and (3) requiring fixing the camera-projector positional relationship, increasing operational difficulty. By building a virtual FPP simulation system, our method can quickly render the required datasets, avoiding cumbersome manual operations. When rendering the datasets, we simulate adverse factors such as color channel crosstalk, system extrinsic parameter variations, and object surface colors to guide the training of the neural network. Ultimately, from a single three-frequency color image, the corresponding three-frequency three-step phase-shift images are predicted, achieving single-frame 3D reconstruction of colored objects and allowing some variation in system extrinsic parameters. Real-world experiments demonstrate that the network trained with the diverse data generated by our virtual system has good accuracy, providing valuable guidance for the practical application of deep learning methods.

Flotation separation of lithium–ion battery electrodes predicted by a long short-term memory network using data from physicochemical kinetic simulations and experiments

Anode and cathode active materials from spent lithium–ion batteries may be recovered and potentially used in new batteries to promote recycling and resource circulation. Froth flotation was applied to pristine active materials and the black mass obtained from pretreated spent batteries. Flotation kinetics was simulated with the use of computational fluid dynamics and surface chemistry. Bubble surface coverage and entrainment in the flotation kinetics model were selected and optimized by systematic fitting to experimental data. Entrainment influences the recovery and grade of the active materials. The optimized flotation kinetics model was used for generating additional data that, along with the fitted data, were used to train a deep learning neural network. The trained network was validated using anode–cathode and black mass flotation experiments, and its predictions showed a maximum residual error of 0.18 ± 0.11 recovery. The simulation framework was developed into a desktop application that predicts the flotation behavior of active materials. It provides information for estimating results following operational and physicochemical changes and for optimizing flotation processes. Finally, recovered anode active materials from black mass were selected for coin cell tests. The coulombic efficiency of these coin cells was initially lower (86.8 %) than that of cells made with pristine graphite particles (98.4 %).

Fractional-Order Modeling of Heat and Moisture Transfer in Anisotropic Materials Using a Physics-Informed Neural Network.

Mathematical models of heat and moisture transfer for anisotropic materials, based on the use of the fractional calculus of integro-differentiation, are considered because such two-factor fractal models have not been proposed in the literature so far. The numerical implementation of mathematical models for determining changes in heat exchange and moisture exchange is based on the adaptation of the fractal neural network method, grounded in the physics of processes. A fractal physics-informed neural network architecture with a decoupled structure is proposed, based on loss functions informed by the physical process under study. Fractional differential formulas are applied to the expressions of non-integer operators, and finite difference schemes are developed for all components of the loss functions. A step-by-step method for network training is proposed. An algorithm for the implementation of the fractal physics-informed neural network is developed. The efficiency of the new method is substantiated by comparing the obtained numerical results with numerical approximation by finite differences and experimental data for particular cases.

3D Point Cloud Semantic Segmentation based on Multi-scale Dense Nested Networks

Aiming at the problem that the relationship between geometric features and semantic features is ignored in the point cloud data downsampling process, which leads to inaccurate segmentation of object boundaries and structural details, this paper proposes a 3D point cloud semantic segmentation network based on multi-scale dense nested type. Firstly, a dense nested network architecture is constructed by nesting multiple multi-scale feature fusion modules to fuse multi-scale features of different directions between encoder-decoder paths, so as to effectively propagate local ge-ometric context information and enhance the ability of cross-scale information interaction. Secondly, a local feature aggregation unit is constructed in the multi-scale feature fusion module, which strengthens the structural awareness within the local point set based on graph convolution and attention mechanism, and promotes the complementarity of local geometric features and abstract semantic information. Then, the cross-layer multi-loss supervision module is combined to further optimize the multi-scale feature propagation, which makes the network training more stable and improves the point cloud segmentation accuracy. Finally, this paper verified the proposed network on the S3DIS dataset. The experimental results show that the proposed network has the mean intersection over Union of 71.2% and the overall accuracy of 88.7%, which is 1.2 and 0.7 percentage points higher than RandLA-Net, respectively, which proves that the proposed network can effectively improve the accuracy of 3D point cloud semantic segmentation.

Edge-featured multi-hop attention graph neural network for intrusion detection system

With the development of the Internet, the application of computer technology has rapidly become widespread, driving the progress of Internet of Things (IoT) technology. The attacks present on networks have become more complex and stealthy. However, traditional network intrusion detection systems with singular functions are no longer sufficient to meet current demands. While some machine learning-based network intrusion detection systems have emerged, traditional machine learning methods cannot effectively respond to the complex and dynamic nature of network attacks. Intrusion detection systems utilizing deep learning can better enhance detection capabilities through diverse data learning and training. To capture the topological relationships in network data, using graph neural networks (GNNs) is most suitable. Most existing GNNs for intrusion detection use multi-layer network training, which may lead to over-smoothing issues. Additionally, current intrusion detection solutions often lack efficiency. To mitigate the issues mentioned above, this paper proposes an Edge-featured Multi-hop Attention Graph Neural Network for Intrusion Detection System (EMA-IDS), aiming to improve detection performance by capturing more features from data flows. Our method enhances computational efficiency through attention propagation and integrates node and edge features, fully leveraging data characteristics. We carried out experiments on four public datasets, which are NF-CSE-CIC-IDS2018-v2, NF-UNSW-NB15-v2, NF-BoT-IoT, and NF-ToN-IoT. Compared with existing models, our method demonstrated superior performance.

A Highly Accurate Adverse Drug Reactions (ADR) Detection from Medical Forum Comments Using Long Short-Term Memory Networks

Purpose: Adverse Drug Reactions (ADR) classification is useful in modern medical diagnostics and related applications. ADR is an example of how medical information is frequently accessible on social media platforms for healthcare, where people can share their experiences with treatments on desktop computers and mobile devices. Many researchers are interested in gathering valuable medical data from social media for the ADR system training and classification process. Materials and Methods: This research explores the effects of three aspects on recognizing ADR mentions in social media for the medical field and proposes a deep neural network of Long Short-Term Memory (LSTM) neural networks to do so. The comments are collected from various social media platforms to implement the ADR system with proper training and testing processes. The texts from the dataset are initially preprocessed by using a data filtering and clustering process to remove the input data's redundant information to increase the training process's quality. Characteristic features, such as semantic features and text statistics, are extracted from the input text using the American Standard Code for Information Interchange (ASCII) array. Further, the features are converted and fed to LSTM networks for training and validation. Results and Conclusion: This work is evaluated using two datasets, CODEC, and ADR Corpus datasets are used to evaluate the performance of the proposed ADR technique via multiple angles. Via extensive experiments, this work achieved 99.79 accuracy, 98.37 sensitivity, 97.63 specificity, 99.72 precision, 98.39 recall, 97.62 F1-score for the CODEC dataset, 98.16 for accuracy, 99.19 for sensitivity, 98.49 for specificity, 99.49 for precision, 96.72 for recall, and 93.16 for F1-score for ADR corpus, respectively.

Multi-Dimensional Adaptive Learning Rate Gradient Descent Optimization Algorithm for Network Training in Magneto-Optical Defect Detection

Multi-Dimensional Adaptive Learning Rate Gradient Descent Optimization Algorithm for Network Training in Magneto-Optical Defect Detection

Constructed encoded data based coded distributed DNN training for edge computing scenario

Deep learning in unmanned aerial vehicle (UAV) deployment encounters two problems: 1, One UAV may fail to store and execute large Deep Neural Network (DNN) model. 2, One UAV fails to accomplish real time services. One possible solution is the group of UAVs (nodes) collectively generate swarm intelligence in the form of edge computing scenario, namely in the distributed computing (DC) mode with clever arrangement, say Coded Distributed Computing (CDC). In CDC systems, the redundant computation introduced by linear coding can compensate stragglers. However, since linear property cannot pass the nonlinear activation function in Deep Neural Network (DNN) training, coding/decoding for CDC need to be applied layer by layer, which slows down the training. To avoid layer-by-layer coding/decoding, we propose a novel DNN training scheme based on constructing encoded data. This construction process lies before the training process (can be done before training without any impact on training efficiency). Based on both the original data and the newly constructed encoded data, the training phase can take advantage of the (n,k) property (Wait for the first k returned data) and hence improve the training speed. The training process does not require encoding/decoding operations, and hence significantly improves the training speed. Experimental results show that the training scheme based on constructed encoded data can achieve prediction accuracy approximating that of the centralized one and significantly reduce the latency compared to the layer-by-layer linear encoding and decoding scheme.

Convolutional neural networks and seismogram fingerprints as a tool for recognizing the nature of seismic events

In the presented article, two facts are convincingly demonstrated based on the results of numerous experiments. Firstly, fingerprints are informative enough representations of signal seismograms, despite their compression, to carry information about the nature of a seismic event. Secondly, the study showed that it is practically possible to design and train an artificial neural network capable of classifying events by origin based on their fingerprints with high accuracy. Fingerprints are a tenthousandfold compressed representation of the original seismogram obtained using the onedimensional wavelet transform and the twodimensional Haar wavelet transform. They carry information about all significant frequencytime phenomena contained in the primary seismogram. Convolutional neural networks were chosen as a class of neural network classifier based on the conducted review of publications on this topic. They have proven themselves to be excellent in recognizing objects and persons in raster images. And the fingerprints used in this study are binary images measuring 64×64 pixels. The convolutional neural network prepared to work with them has one of the simplest architectures for this type of network and a very small number of adjustable parameters. By means of it, the classification accuracy of 95% was easily achieved. To prove that this result is not accidental, a strategy for modeling the architecture of convolutional neural networks using specially developed software, the Trova system, is demonstrated. This software allows the researcher to easily and conveniently operate seismograms, obtain binary fingerprints from them, correctly augment data, create, train and test neural networks. An important feature of the Trova system is the advanced functionality of the graphical representation of multidimensional modeling results.

Device Specifications for Neural Network Training with Analog Resistive Cross‐Point Arrays Using Tiki‐Taka Algorithms

Recently, specialized training algorithms for analog cross‐point array‐based neural network accelerators have been introduced to counteract device non‐idealities such as update asymmetry and cycle‐to‐cycle variation, achieving software‐level performance in neural network training. However, a quantitative analysis of how these algorithms affect the relaxation of device specifications is yet to be conducted. This study provides a detailed analysis by elucidating the device prerequisites for training with the Tiki‐Taka algorithm versions 1 (TTv1) and 2 (TTv2), which leverage the dynamics between multiple arrays to compensate for device non‐idealities. A multiparameter simulation is conducted to assess the impact of device non‐idealities, including asymmetry, retention, number of pulses, and cycle‐to‐cycle variation, on neural network training. Using pattern‐recognition accuracy as a performance metric, the required device specifications for each algorithm are revealed. The results demonstrate that the standard stochastic gradient descent algorithm requires stringent device specifications. Conversely, TTv2 permits more lenient device specifications than the TTv1 across all examined non‐idealities. The analysis provides guidelines for the development, optimization, and utilization of devices for high‐performance neural network training using Tiki‐Taka algorithms.

Use of deep learning to improve seismic data quality analysis

Seismic data are influenced by various types of noise, which are typically categorized into two primary classes: anthropogenic and environmental. However, the detection of instrumental noise or malfunctioning stations also plays a crucial role in ensuring the data quality and the efficiency of a seismic network. The visual inspection of seismic spectral diagrams (e.g. power spectral density) enables us to identify issues that could potentially compromise data quality, thereby affecting subsequent calculations such as Magnitude or Peak Ground Acceleration (PGA). However, this process is time-consuming and demands significant human expertise due to the complexity of the diagrams, compounded by the sheer number of stations requiring examination. Therefore, in this paper, we explore the feasibility of transferring human expertise into an artificial intelligence system to create an automated system capable of rapidly performing such detection. More specifically, in the first part of this paper, we use Probability Density Function (PDF) diagrams, enabling an initial assessment of station performance via visual inspection. We describe this plot type and provide examples that reveal whether a station is functioning correctly or if technical issues exist. A table containing the main evaluation criteria is provided. In the second part of this paper, we demonstrate that these plots can serve as input for a neural network, allowing the development of the aforementioned automated system. Through extensive testing under various conditions, we have observed that the trained network consistently achieves an accuracy rate exceeding 85% across all four conductedtests. In the latest and most significant test, the achieved accuracy is approximately 87%.

Machine learning and domain decomposition methods - a survey

Hybrid algorithms, which combine black-box machine learning methods with experience from traditional numerical methods and domain expertise from diverse application areas, are progressively gaining importance in scientific machine learning and various industrial domains, especially in computational science and engineering. In the present survey, several promising avenues of research will be examined which focus on the combination of machine learning (ML) and domain decomposition methods (DDMs). The aim of this survey is to provide an overview of existing work within this field and to structure it into domain decomposition for machine learning and machine learning-enhanced domain decomposition, including: domain decomposition for classical machine learning, domain decomposition to accelerate the training of physics-aware neural networks, machine learning to enhance the convergence properties or computational efficiency of DDMs, and machine learning as a discretization method in a DDM for the solution of PDEs. In each of these fields, we summarize existing work and key advances within a common framework and, finally, discuss ongoing challenges and opportunities for future research.

Seismic property prediction using deep learning in LN area, Tarim Basin, China

Abstract Stratigraphy and faults provide the framework for subsurface sequences and structural information, forming the basis for high-precision reservoir parameter prediction. However, reservoir parameter prediction often relies on sparse stratigraphic interpretations based on existing manual interpretations. There is a certain connection between stratigraphic and fault interpretations and reservoir parameter predictions. Therefore, this paper proposes a fully automated workflow from stratigraphic and fault interpretation to reservoir modeling. First, using known sparse expert stratigraphic interpretation results as constraints, three-dimensional implicit model construction is achieved through deep learning networks in the working area. That model incorporates all stratigraphic and fault information, thereby enabling detailed stratigraphic framework interpretation. Second, based on this implicit model, an initial reservoir parameter model is built by integrating well-logging information. Finally, using this model and seismic data as input, with target reservoir parameters as output, a two-dimensional deep learning training network is established to achieve the three-dimensional characterization of reservoir parameters. Applying this method in the LN area of Tarim Basin, based on automatically interpreting fault and stratigraphic horizons results, the predicted results of reservoir parameters such as gamma and impedance guided by this stratigraphic construction model are generally consistent at blind wells. This study demonstrates that the automated workflow can achieve results consistent with geological understanding in practical applications.