Two-dimensional Neural Network Research Articles

Deep Learning for Detecting and Subtyping Renal Cell Carcinoma on Contrast-Enhanced CT Scans Using 2D Neural Network with Feature Consistency Techniques

Abstract Objective The aim of this study was to explore an innovative approach for developing deep learning (DL) algorithm for renal cell carcinoma (RCC) detection and subtyping on computed tomography (CT): clear cell RCC (ccRCC) versus non-ccRCC using two-dimensional (2D) neural network architecture and feature consistency modules. Materials and Methods This retrospective study included baseline CT scans from 196 histopathologically proven RCC patients: 143 ccRCCs and 53 non-ccRCCs. Manual tumor annotations were performed on axial slices of corticomedullary phase images, serving as ground truth. After image preprocessing, the dataset was divided into training, validation, and testing subsets. The study tested multiple 2D DL architectures, with the FocalNet-DINO demonstrating highest effectiveness in detecting and classifying RCC. The study further incorporated spatial and class consistency modules to enhance prediction accuracy. Models' performance was evaluated using free-response receiver operating characteristic curves, recall rates, specificity, accuracy, F1 scores, and area under the curve (AUC) scores. Results The FocalNet-DINO architecture achieved the highest recall rate of 0.823 at 0.025 false positives per image (FPI) for RCC detection. The integration of spatial and class consistency modules into the architecture led to 0.2% increase in recall rate at 0.025 FPI, along with improvements of 0.1% in both accuracy and AUC scores for RCC classification. These enhancements allowed detection of cancer in an additional 21 slices and reduced false positives in 126 slices. Conclusion This study demonstrates high performance for RCC detection and classification using DL algorithm leveraging 2D neural networks and spatial and class consistency modules, to offer a novel, computationally simpler, and accurate DL approach to RCC characterization.

Φ-OTDR Signal Identification Method Based on Multimodal Fusion.

Distributed fiber optic sensing (DFS) systems are an effective method for long-distance pipeline safety inspections. Highly accurate vibration signal identification is crucial to DFS. In this paper, we propose an end-to-end high-accuracy fiber optic vibration signal detection and identification algorithm by extracting features from the time domain and frequency domain by a one-dimensional convolutional neural network and two-dimensional convolutional neural network, respectively, and introducing a self-attentive mechanism to fuse the features of multiple modes. First, the raw signal is segmented and normalized according to the statistical characteristics of the vibration signal combined with the distribution of noise. Then, the one-dimensional sequence of vibration signal and its two-dimensional image generated by short-time Fourier transform are input to the one-dimensional convolutional neural network and two-dimensional neural network, respectively, for automatic feature extraction, and the features are combined by long and short-time memory. Finally, the multimodal features generated from the time and frequency domains are fused by a multilayer TransformerEncoder structure with a multiheaded self-attentive mechanism and fed into a multilayer perceptron for classification. Experiments were conducted on an urban field database with complex noise and achieved 98.54% accuracy, which demonstrates the effectiveness of the proposed algorithm.

Augmentation dataset of a two-dimensional neural network model for use in the car parts segmentation and car classification of three dimensions

Augmentation dataset of a two-dimensional neural network model for use in the car parts segmentation and car classification of three dimensions

NNetEn2D: Two-Dimensional Neural Network Entropy in Remote Sensing Imagery and Geophysical Mapping

Measuring the predictability and complexity of 2D data (image) series using entropy is an essential tool for evaluation of systems’ irregularity and complexity in remote sensing and geophysical mapping. However, the existing methods have some drawbacks related to their strong dependence on method parameters and image rotation. To overcome these difficulties, this study proposes a new method for estimating two-dimensional neural network entropy (NNetEn2D) for evaluating the regularity or predictability of images using the LogNNet neural network model. The method is based on an algorithm for converting a 2D kernel into a 1D data series followed by NNetEn2D calculation. An artificial test image was created for the study. We demonstrate the advantage of using circular instead of square kernels through comparison of the invariance of the NNetEn2D distribution after image rotation. Highest robustness was observed for circular kernels with a radius of R = 5 and R = 6 pixels, with a NNetEn2D calculation error of no more than 10%, comparable to the distortion of the initial 2D data. The NNetEn2D entropy calculation method has two main geometric parameters (kernel radius and its displacement step), as well as two neural network hyperparameters (number of training epochs and one of six reservoir filling techniques). We evaluated our method on both remote sensing and geophysical mapping images. Remote sensing imagery (Sentinel-2) shows that brightness of the image does not affect results, which helps keep a rather consistent appearance of entropy maps over time without saturation effects being observed. Surfaces with little texture, such as water bodies, have low NNetEn2D values, while urban areas have consistently high values. Application to geophysical mapping of rocks to the northwest of southwest Australia is characterized by low to medium entropy and highlights aspects of the geology. These results indicate the success of NNetEn2D in providing meaningful entropy information for 2D in remote sensing and geophysical applications.

Intelligent fault diagnosis system design and implementation of diaphragm pump based on artificial intelligence

This article designs and implements a fault diagnosis system for diaphragm pump. The key components of the diaphragm pump are equipped with three-axis vibration intelligent sensor to collect vibration data, the edge side completes the data pre-processing, and then transmits it to the cloud platform through NBIoT. For the non-linear, non-smooth complex system, the time frequency representation is obtained by continuous wavelet transformation, after compressing the time frequency representation as the input of the two-dimensional neural network model, using the two-dimensional convolutional neural network model for training and monitoring, to complete the identification and positioning of diaphragm pump fault.

Precise and efficient heartbeat classification using a novel lightweight-modified method

Background and objectiveDevelopment of lightweight model with high accuracy is an important study in order to better transfer arrhythmia diagnostic models to mobile terminals or embedded devices. MethodsA novel lightweight-modified two-dimensional neural network is presented for accurately identifying whether certain types of heartbeats are occurred based on information of heartbeat images. The MIT-BIH arrhythmia database is taken as the dataset. Firstly, data preprocessing is performed that one-dimensional ECG signals are converted into two-dimensional heartbeat images. Then, the obtained heartbeat images are fed into the proposed two-dimensional network for training. Moreover, in order to reduce the network model parameters, we integrate the small-scale convolution kernels and inverted residual with linear bottleneck module into the proposed network. Finally, comparisons of several related classification networks are conducted to verify the effectiveness and superiority of the proposed method. ResultsResult shows the classification accuracy is 99.41 % in the test dataset originated in the aforementioned MIT-BIH database. The TPR of APC, LBBB, Normal, RBBB and VPC are 0.995, 1.0, 0.993, 1.0, 0.994, respectively. Compared with VGG16, the number of parameters is reduced by 8, 525, 312, and it achieves the optimal in network complexity and accuracy among the related networks of heartbeat classification. In terms of time consumption, the network proposed is better than some networks suitable for image classification. It takes around 0.0039478 s to complete the heartbeat classification of each ECG image using the proposed method. ConclusionsResult shows the effectiveness of the new method and the simplicity of the model parameters.

Visual Weather Property Prediction by Multi-Task Learning and Two-Dimensional RNNs

We attempted to employ convolutional neural networks to extract visual features and developed recurrent neural networks for weather property estimation using only image data. Four common weather properties are estimated, i.e., temperature, humidity, visibility, and wind speed. Based on the success of previous works on temperature prediction, we extended them in terms of two aspects. First, by considering the effectiveness of deep multi-task learning, we jointly estimated four weather properties on the basis of the same visual information. Second, we propose that weather property estimations considering temporal evolution can be conducted from two perspectives, i.e., day-wise or hour-wise. A two-dimensional recurrent neural network is thus proposed to unify the two perspectives. In the evaluation, we show that better prediction accuracy can be obtained compared to the state-of-the-art models. We believe that the proposed approach is the first visual weather property estimation model trained based on multi-task learning.

Identification of Pathway-Specific Protein Domain by Incorporating Hyperparameter Optimization Based on 2D Convolutional Neural Network

Pathway-specific protein domain (PSPD) are associated with specific pathways. Many protein domains are pervasive in various biological processes, whereas other domains are linked to specific pathways. Many human disease pathways, such as cancer pathways and signaling pathway-related diseases, have caused the loss of functional PSPD. Therefore, the creation of an accurate method to predict its roles is a critical step toward human disease and pathways. In this study, we proposed a deep learning model based on a two-dimensional neural network (2D-CNN-PSPD) with a pathway-specific protein domain association prediction. In terms of the purposes of a sub-pathway, its parent pathway and its super pathway are linked to the Uni-Pathway. We also proposed a dipeptide composition (DPC) model and a dipeptide deviation (DDE) model of feature extraction profiles as PSSM. Then, we predicted the proteins associated with the same sub-pathway or with the same organism. The DDE model and DPC model of the PSSM feature profile input was associated with our proposed 2D-CNN method. We deployed several parameters to optimize the model’s output performance and used the hyperparameter optimization approach to find the best model for our dataset based on the 10-fold cross-validation results. Ultimately, we assessed the predictive performance of the current model by using independent datasets and cross-validation datasets. Therefore, we enhanced the efficiency of deep learning methods. PSPD is involved in any known pathway and then follow the association in different stages of the pathway hierarchy with other proteins. Our proposed method could identify 2D-CNN-PSPD with 0.83% sensitivity, 0.92% specificity, 87.27% accuracy, and 0.75% accuracy. We provided an important method for the analysis of PSPD proteins in the proposed research, and our achievements might promote computational biological research. We concluded our proposed model architecture in the future, the use of the latest features, and the multi-one structure to predict different types of molecules, such as DNA, RNA, and disease-pathway specific proteins associations.

Gap junctions set the speed and nucleation rate of stage I retinal waves.

Spontaneous waves in the developing retina are essential in the formation of the retinotopic mapping in the visual system. From experiments in rabbits, it is known that the earliest type of retinal waves (stage I) is nucleated spontaneously, propagates at a speed of 451±91 μm/sec and relies on gap junction coupling between ganglion cells. Because gap junctions (electrical synapses) have short integration times, it has been argued that they cannot set the low speed of stage I retinal waves. Here, we present a theoretical study of a two-dimensional neural network of the ganglion cell layer with gap junction coupling and intrinsic noise. We demonstrate that this model can explain observed nucleation rates as well as the comparatively slow propagation speed of the waves. From the interaction between two coupled neurons, we estimate the wave speed in the model network. Furthermore, using simulations of small networks of neurons (N≤260), we estimate the nucleation rate in the form of an Arrhenius escape rate. These results allow for informed simulations of a realistically sized network, yielding values of the gap junction coupling and the intrinsic noise level that are in a physiologically plausible range.

An oscillatory network model with controllable synchronization and a neuromorphic dynamical method of information processing

A spatially two-dimensional oscillatory neural network model with inhomogeneous modifiable oscillatory coupling is designed and an adaptive dynamical method of brightness image segmentation (image reconstruction) based on self-organized cluster synchronization in the oscillatory network is developed. The method imitates the known phenomenon of dynamical binding via synchronization that is presumably used by a number of the brain neural structures in their work. The oscillatory-network approach demonstrates the following capabilities: (1) high-quality segmentation of real grey-level and color images; (2) selective image segmentation (exclusion of unnecessary information); (3) solution of the simplest problem of object selection in a visual scene—the problem of the successive selection of all spatially separated image fragments of almost equal brightness.

Learning Times Required to Identify the Stimulated Position and Shortening of Propagation Path by Hebb’s Rule in Neural Network

To deepen the understanding of the human brain, many researchers have created a new way of analyzing neural data. In many previous studies, researchers have examined neural networks from a macroscopic point of view, based on neuronal firing patterns. On the contrary, we have studied neural networks locally, in order to understand their communication strategies. To understand information processing in the brain, we simulated the firing activities of neural networks in a 9 × 9 two-dimensional neural network to analyze spike behavior. In this research study, we used two kinds of learning processes. As the main learning process, we implemented the learning process to identify the stimulated position. As the subsidiary one, we implemented Hebb’s learning rule which changes weight between neurons. Three channels with transmission and reception were preset, each of which has a different distance and direction. When all three channels succeeded in identifying the source stimulation in the receiving neuron group, it was regarded as an overall success and the learning was termed as successful. Furthermore, in order to see the effect of the second learning procedure, we elucidated the average of necessary learning times in each channel type and compared the firing propagation time of the first trial and an overall successful trial, in each channel. We found that the firing path after learning is shorter than the firing path before learning. Therefore, we deduced that Hebb’s rule contributes to shortening the firing path. Thus, Hebb’s rule contributes to speeding up communication in a neuronal network.

Modeling of the phase equilibria of aqueous two-phase systems using three-dimensional neural network

A three-dimensional neural network model has been designed for representing the phase equilibrium data related to aqueous two-phase systems. The polyvinyl pyrrolidone/phosphate/water system was selected as the model system to demonstrate the point of interest. The collected experimental data were categorized into two subsets, training and validation sets, not only to find the suitable network configuration but also to prevent the overfitting problem. Meanwhile, the weight comparison method was proposed to optimize the three-dimensional neural net. The results of accuracy comparison indicate that it outperforms the two-dimensional neural network on some details and can further enhance the calculation accuracy of the phase equilibrium data for these investigated aqueous two-phase systems. The development of the neural network in the three-dimensional space should be a research project of concern.

Performance Evaluation of Neuro ITI Canceller for Two-Dimensional Magnetic Recording by Shingled Magnetic Recording

A neuro intertrack interference canceller (neuro-ITIC) is proposed to diminish the influences of ITI and jitter-like medium noise for two-dimensional magnetic recording (TDMR) by shingled magnetic recording. The bit error rate (BER) performance of a low-density parity-check coding and iterative decoding system including a generalized partial response class-I channel with the neuro-ITIC is also obtained via computer simulation using a read/write channel model under TDMR specifications of 4 Tb/in , and it is compared to those for a two-dimensional neural network equalizer (2D-NNE) and an ITI canceller (ITIC) with a one-dimensional finite impulse response equalizer (1D-FIRE). It is clarified that the BER performance for the neuro-ITIC is far superior to those for the 2D-NNE and the ITIC with the 1D-FIRE.

Modeling of Writing Process for Two-Dimensional Magnetic Recording and Performance Evaluation of Two-Dimensional Neural Network Equalizer

Modeling of a simple writing process considering intergranular exchange fields and magnetostatic interaction fields between grains is studied for two-dimensional magnetic recording (TDMR). A new designing method of a two-dimensional neural network equalizer with a mis-equalization suppression function (2D-NNEMS) for TDMR is also proposed. The bit-error rate (BER) performance of a low-density parity-check coding and iterative decoding system with the designed 2D-NNEMS is obtained via computer simulation using a read/write channel model employing the proposed writing process under TDMR specifications of 4 Tb/in <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> , and it is compared with those for one- and two-dimensional finite impulse response equalizers (FIREs). It is clarified that the BER performance for the designed 2D-NNEMS is far superior to those for the FIREs.

A study on modeling of the writing process and two-dimensional neural network equalization for two-dimensional magnetic recording

A simple writing process considering magnetic clusters due to exchange coupling between grains is studied for two-dimensional magnetic recording. The bit error rate (BER) performance of a low-density parity-check coding and iterative decoding system with a two-dimensional neural network equalizer (2D-NNE) that can diminish the influences of jitter-like medium noise and inter-track interference is obtained using a read/write channel model based on the proposed writing process, and it is compared with those for one- and two-dimensional finite impulse response equalizers (FIREs). It is clarified that the BER performance for the 2D-NNE is far superior to those for the FIREs.

Read/Write Channel Modeling and Two-Dimensional Neural Network Equalization for Two-Dimensional Magnetic Recording

An accurate medium modeling method of discretized granular medium with non-magnetic grain boundaries using a discrete Voronoi diagram is proposed for two-dimensional magnetic recording. A simple closed-form representation of a double-shielded reader sensitivity function is also proposed for modeling the reading process. Moreover, a two-dimensional neural network equalizer (2D-NNE) is proposed to mitigate the influence of intertrack interference and jitter-like medium noise. The bit-error rate performance of partial response class-I maximum likelihood (PR1ML) system with the 2D-NNE is obtained by computer simulation based on the proposed read/write channel model. The performance is far superior to that of PR1ML system with a two-dimensional finite impulse response equalizer.

THE MOSAIC PATTERNS OF CNN WITH SYMMETRIC FEEDBACK TEMPLATE

In this paper, the mosaic patterns of the two-dimensional cellular neural network (CNN) with symmetric feedback template are investigated. For our CNN system, the parameter space is constructed by the output synaptic weights and the threshold, and it is partitioned into finitely many regions through geometric methods and variable substitution. Fixing the output synaptic weights and the threshold in some regions, we give the necessary and sufficient conditions to all mosaic patterns of the CNN systems.

Bifurcation Analysis for a Two-Dimensional Discrete-Time Hopfield Neural Network with Delays

A bifurcation analysis is undertaken for a discrete-time Hopfield neural network with four delays. Conditions ensuring the asymptotic stability of the null solution are obtained with respect to two parameters of the system. Using techniques developed by Kuznetsov to a discrete-time system, we study the Neimark-Sacker bifurcation (also called Hopf bifurcation for maps) of the system. The direction and the stability of the Neimark-Sacker bifurcation are investigated by applying the normal form theory and the center manifold theorem.

Permanence for a Generalized Discrete Neural Network System

We prove that the system of difference equationsxn+1(i)=λixn(i)+fi(αixn(i+1)−βixn−1(i+1)),i∈{1,2,…,k},n∈ℕ, (we regard thatxn(k+1)=xn(1)) is permanent, provided thatαi≥βi,λi+1∈[0,βi/αi),i∈{1,2,…,k},fi:ℝ→ℝ,i∈{1,2,…,k}, are nondecreasing functions bounded from below and such that there areδi∈(0,1)andM&gt;0such thatfi(αix)≤δix,i∈{1,2,…,k}, for allx≥M. This result considerably extends the results existing in the literature. The above system is an extension of a two-dimensional discrete neural network system.

SYNCHRONIZATION CONTROL OF NEURAL NETWORKS SUBJECT TO TIME-VARYING DELAYS AND INPUT NONLINEARITY

This paper investigates the synchronization problem for a particular class of neural networks subject to time-varying delays and input nonlinearity. Using the variable structure control technique, a memoryless decentralized control law is established which guarantees exponential synchronization even when input nonlinearity is present. The proposed controller is suitable for application in delayed cellular neural networks and Hopfield neural networks with no restriction on the derivative of the time-varying delays. A two-dimensional cellular neural network and a four-dimensional Hopfield neural network, both with time-varying delays, are presented as illustrative examples to demonstrate the effectiveness of the proposed synchronization scheme.