Parallel Neural Network Architecture Research Articles

A robust adapted Flexible Parallel Neural Network architecture for early prediction of lithium battery lifespan

Early prediction of End of Life (EPEOL) is crucial for improving lithium battery efficiency and lifespan. Traditional fixed-architecture neural networks often suffer from underfitting or overfitting due to diverse data distributions. To address this, we propose the Flexible Parallel Neural Network (FPNN). This model integrates modules like InceptionBlock, 3D CNN, 2D CNN, and dual-stream networks. By effectively extracting electrochemical features from video-format data through the 3D CNN and achieving multi-scale feature abstraction with InceptionBlock, FPNN ensures effective module coordination. The model can adaptively adjust the number of InceptionBlocks to handle tasks of varying complexity. Experimental results on the MIT dataset show that FPNN achieves MAPE values of 1.26%, 0.41%, 0.37%, 0.33%, 0.32%, 0.32%, 0.31%, 0.31%, 0.22%, and 0.34% using the first 10, 20, 30, 40, 50, 60, 70, 80, 90, and 100 cycle data, respectively. The interpretability of FPNN is reflected in its structural design and flexible unit choices, providing a basis for model interpretation. Comprising multiple modules, FPNN enables smooth feature extraction from electrochemical data through collaborative interaction. The diverse branching structure of the flexible units allows the model to capture features at different scales, learning richer information. Our approach offers a precise, adaptable, and easy-to-understand solution for EPEOL, opening new possibilities in battery health monitoring.

Multi-scale time series analysis using TT-ConvLSTM technique for bearing remaining useful life prediction

This study advocates the utilization of a parallel neural network (PNN) architecture for the estimation of remaining useful life (RUL) of rolling element bearings. Conventional machine learning methods have often fallen short in processing machinery sensor data efficiently and accurately, given its vast nature and the temporal dynamics of component health conditions. To address this limitation, a robust PNN architecture is proposed which incorporates multiple parallel processing pathways. These pathways are equipped with diverse input neurons, capturing data from mechanical component condition sensors. The output neuron then efficiently predicts the RUL, capturing the degradation stages throughout the component's lifespan. The PNN structure provides better accuracy and computation time by efficiently handling huge amounts of data simultaneously and integrating both spatial and temporal information present. Additionally, time-transformer and recurrent neural networks (RNNs) are used to handle complex time series data. Improvement methodologies, such as positional encoding combined with a multi-headed self-attention mechanism and the convolutional long short-term memory (ConvLSTM) neural network, are employed. These methodologies adeptly handle spatiotemporal dependencies inherent in multidimensional features extracted across the time domain, frequency domain, and time–frequency domain, thereby considerably boosting the model's efficacy. Two case studies are conducted on XJTU-SY rolling element bearing dataset and FEMTO bearing dataset to validate and display the generalization capabilities of the proposed methodology, where PNN performed exceptionally in terms of accuracy and efficiency. It is concluded that with a novel implementation of variational stride temporal window strategy and utilizing the full capabilities of a PNN, the RUL of bearings can be predicted very accurately. The algorithm strategy is very robust and can be applied to other machine components as well. Subsequently, the code for the algorithm will be open-sourced.

Modeling of Combustion Characteristics of Particles in Transient Gas–Solid Reacting Flow via a Machine Learning Approach

Particle group combustion presents a strong temporal and spatial inhomogeneity owing to the complicated interphase interactions. Based on the data set from the fictitious domain method, the recurrent fully connected and convolutional parallel neural network (R-FC&CNN) architecture and its two comparable simplified models, that is, the recurrent fully connected neural network (R-FCNN) and the recurrent convolutional neural network (R-CNN) architectures, were constructed for predicting the gas–solid momentum exchange coefficient, β (kg·s–1·m–3), average combustion rate per unit surface area of particles, rc¯/A (kg·s–1·m–2), and comprehensive NaCl release parameter, γ, selectively. A time sequence of average particle temperature, T̅ (K), and particle volume fraction, ε, which can be extended in the matrix form, were constructed as the features selectively according to their correlation with the target physical quantity. The average relative error, δ̅, and coefficient of determination, R2, were used as the evaluators. Through final testing, in the mild combustion domain, the R-CNN and R-FCNN models with simple structures showed good performance for β(δ̅ = 0.13, R2 = 0.8) and rc¯/A (δ̅ = 0.04, R2 = 0.84), respectively, while in the severe combustion domain, the R-FC&CNN model, with more complete features and functional structure, performed the best (for β, δ̅ = 0.12, R2 = 0.85; for rc¯/A, δ̅ = 0.05, R2 = 0.92; and for γ, δ¯=0.05,R2=0.96). A fine-tuning and interpolated prediction method was developed to investigate further the model’s expansibility. Both ensured acceptable performance on a new similar problem. In summary, the feasibility of physical meaning-oriented machine learning--based model(s) for predicting the combustion characteristics of nonuniformly distributed particles was confirmed.

Optimizing Implementations of Non-Profiled Deep Learning-Based Side-Channel Attacks

The differential deep learning analysis proposed by Timon is the first non-profiled side-channel attack technique that uses deep learning. The technique recovers the secret key using the phenomenon of deep learning metrics. However, the proposed technique made it difficult to observe the results from the intermediate process, while the neural network had to be retrained repeatedly, as the cost of learning increased with key size. In this paper, we propose three methods to solve the aforementioned problems and any challenges resulting from solving these problems. First, we propose a modified algorithm that allows the monitoring of the metrics in the intermediate process. Second, we propose a parallel neural network architecture and algorithm that works by training a single network, with no need to re-train the same model repeatedly. Attacks were performed faster with the proposed algorithm than with the previous algorithm. Finally, we propose a novel architecture that uses shared layers to solve memory problems in parallel architecture and also helps achieve better performance. We validated the performance of our methods by presenting the results of non-profiled attacks on the benchmark database ASCAD and for a custom dataset on power consumption collected from ChipWhisperer-Lite. On the ASCAD database, our shared layers method was up to 134 times more efficient than the previous method.

A hybrid fault diagnosis scheme for railway point machines by motor current signal analysis

Proper analysis of point machine current signal provides pervasive information of health status of their internal components. Point machines are subjected to several failure modes during their operation. “Gearbox,” “ball bearing,” “lead screw,” and “sliding chair” faults are among common mechanical failure modes. In this article, a two-stage prediction innovative process is proposed using Fault Detection based Decision Tree strategy (FDDT) where the healthy and faulty modes are first determined, followed by classifying the types of mechanical faults based on Parallel Neural Network Architecture and Fuzzy System (PNNFS). To differentiate between faulty and healthy point machines, some relevant features are extracted from the motors’ current signals which are used as input data for the proposed FDDT_PNNFS method. Feature selection has been performed using the ReliefF to select the dominant predictors in the point machine. Firstly, the Decision Tree (DT) algorithm is used to obtain a classifier model based on the offline training method for fault detection. The performance of DT is compared with the support vector machine algorithm. In the second stage, faulty data is fed to a bank of Neural Networks, designed in Parallel Neural Network Architecture (PNNA), which is used for identifying the type of failures. Each Neural Network Algorithm (NNA) is responsible for detecting only one type of failure and assessment of the NNA outputs shows the final failure of the point machine. If there is a discrepancy between the outputs of the NNAs, fuzzy logic plays the role of modifier and judges among outputs of NNAs and determines the more probable fault type.

Transfer Learning in ECG Classification from Human to Horse Using a Novel Parallel Neural Network Architecture

Automatic or semi-automatic analysis of the equine electrocardiogram (eECG) is currently not possible because human or small animal ECG analysis software is unreliable due to a different ECG morphology in horses resulting from a different cardiac innervation. Both filtering, beat detection to classification for eECGs are currently poorly or not described in the literature. There are also no public databases available for eECGs as is the case for human ECGs. In this paper we propose the use of wavelet transforms for both filtering and QRS detection in eECGs. In addition, we propose a novel robust deep neural network using a parallel convolutional neural network architecture for ECG beat classification. The network was trained and tested using both the MIT-BIH arrhythmia and an own made eECG dataset with 26.440 beats on 4 classes: normal, premature ventricular contraction, premature atrial contraction and noise. The network was optimized using a genetic algorithm and an accuracy of 97.7% and 92.6% was achieved for the MIT-BIH and eECG database respectively. Afterwards, transfer learning from the MIT-BIH dataset to the eECG database was applied after which the average accuracy, recall, positive predictive value and F1 score of the network increased with an accuracy of 97.1%.

Semi-Supervised Manifold Alignment Using Parallel Deep Autoencoders

The aim of manifold learning is to extract low-dimensional manifolds from high-dimensional data. Manifold alignment is a variant of manifold learning that uses two or more datasets that are assumed to represent different high-dimensional representations of the same underlying manifold. Manifold alignment can be successful in detecting latent manifolds in cases where one version of the data alone is not sufficient to extract and establish a stable low-dimensional representation. The present study proposes a parallel deep autoencoder neural network architecture for manifold alignment and conducts a series of experiments using a protein-folding benchmark dataset and a suite of new datasets generated by simulating double-pendulum dynamics with underlying manifolds of dimensions 2, 3 and 4. The dimensionality and topological complexity of these latent manifolds are above those occurring in most previous studies. Our experimental results demonstrate that the parallel deep autoencoder performs in most cases better than the tested traditional methods of semi-supervised manifold alignment. We also show that the parallel deep autoencoder can process datasets of different input domains by aligning the manifolds extracted from kinematics parameters with those obtained from corresponding image data.

Parallel neural networks for improved nonlinear principal component analysis

In this paper, a parallel neural network architecture is proposed to improve the performance of neural-network-based nonlinear principal component analysis. There exist two typical approaches for such analysis: simultaneous extraction of principal components using a single autoassociative neural network (also known as autoencoder), and sequential extraction using multiple neural networks in series. The proposed architecture can be obtained by systematically pruning the network connections of a fully connected autoassociative neural network, resulting in decoupled neural networks. As a result, more independent (i.e., less correlated) principal components can be obtained than the simultaneous extraction approach. The proposed architecture can be also viewed as a rearrangement of multiple neural networks for the sequential extraction in a parallel setting, and thus, the network training becomes more efficient. Simulation case studies are performed to illustrate the advantages of the proposed architecture, and it was shown that it is particularly beneficial for deep neural networks.

Pure color object extraction from a noisy state using quantum version parallel self organizing neural network

The classical techniques used for noisy object extraction recline that some a priori information concerning the noise characteristics is required during the extraction process. The proposed quantum version parallel self-organizing neural network (QVPSONN) architecture uses the quantum characteristics like superposition, coherence, decoherence, entanglement, etc. of quantum principle for its operation. The extraction of object achieved is better as well as the extraction time is reduced. QVPSONN architecture uses the phase shifting property of qubits to extract the objects from the noisy environment. At first the pure color input image is separated into three color components into the source layer. Then, the three color components are fed to the three parallel architectures of QMLSONNs for processing which are finally fused in the sink layer. Each of the processing layers of QVPSONN comprises qubit-based neurons. The weights between the network layers are demonstrated by single qubit rotation gates. Quantum measurement is applied for processing the information at the output layers, whereby the quantum states are destroyed using the decoherence property. When the system becomes stable, the sink layer fuses it and generates the output. Results of application on a synthetic image and a real-life reveal its efficacy.

Detection of multiple sclerosis with visual evoked potentials--an unsupervised computational intelligence system.

This paper describes the application of a novel unsupervised pattern recognition system to the classification of the Visual Evoked Potentials (VEP's) of normal and multiple sclerosis (MS) patients. The method combines a traditional statistical feature extractor with a fuzzy clustering method, all implemented in a parallel neural network architecture. The optimization routine, ALOPEX, is used to train the network while decreasing the likelihood of local solutions. The unsupervised system includes a feature extraction and clustering module, trained by the optimization routine ALOPEX. Through maximization of the output variance of each node, and an architecture which excludes redundancy, the feature extraction network retains the most significant Karhunen-Loeve expansion vectors. The clustering module uses a modification to the Fuzzy c-Means (FCM) clustering algorithms, where ALOPEX adjusts a set of cluster centers to minimize an objective error function. The result combines the power of the FCM algorithms with the advantage of a more global solution from ALOPEX. The new pattern recognition system is used to cluster the VEP's of 13 normal and 12 MS subjects. The classification with this technique can, without supervision, separate the patient population into two groups which largely correspond to the MS and control subject groups. A suitable threshold can be chosen so that the recognizer chooses no false negatives. The use of multiple stimulation patterns appears to improve the reliability of the decision. The reasoning of most neural networks in their decision making cannot easily be extracted upon the completion of training. However, due to the linearity of the network nodes, the cluster prototypes of this unsupervised system can be reconstructed to illustrate the reasoning of the system. In this application, this analysis hints at the usefulness of previously unused portions of the VEP in detecting MS. It also indicates a possible use of the system as a training aide.

A neural-network architecture for syntax analysis

Artificial neural networks (ANN's), due to their inherent parallelism, offer an attractive paradigm for implementation of symbol processing systems for applications in computer science and artificial intelligence. This paper explores systematic synthesis of modular neural-network architectures for syntax analysis using a prespecified grammar--a prototypical symbol processing task which finds applications in programming language interpretation, syntax analysis of symbolic expressions, and high-performance compilers. The proposed architecture is assembled from ANN components for lexical analysis, stack, parsing and parse tree construction. Each of these modules takes advantage of parallel content-based pattern matching using a neural associative memory. The proposed neural-network architecture for syntax analysis provides a relatively efficient and high performance alternative to current computer systems for applications that involve parsing of LR grammars which constitute a widely used subset of deterministic context-free grammars. Comparison of quantitatively estimated performance of such a system [implemented using current CMOS very large scale integration (VLSI) technology] with that of conventional computers demonstrates the benefits of massively parallel neural-network architectures for symbol processing applications.

Probabilistic neural networks and the polynomial Adaline as complementary techniques for classification

Two methods for classification based on the Bayes strategy and nonparametric estimators for probability density functions are reviewed. The two methods are named the probabilistic neural network (PNN) and the polynomial Adaline. Both methods involve one-pass learning algorithms that can be implemented directly in parallel neural network architectures. The performances of the two methods are compared with multipass backpropagation networks, and relative advantages and disadvantages are discussed. PNN and the polynomial Adaline are complementary techniques because they implement the same decision boundaries but have different advantages for applications. PNN is easy to use and is extremely fast for moderate-sized databases. For very large databases and for mature applications in which classification speed is more important than training speed, the polynomial equivalent can be found.

Masking fields: a massively parallel neural architecture for learning, recognizing, and predicting multiple groupings of patterned data

A massively parallel neural network architecture, called a masking field, is characterized through systematic computer simulations. A masking field is a multiple-scale self-similar automatically gain-controlled cooperative- competitive feedback network F(2). Network F(2) receives input patterns from an adaptive filter F(1) ? F(2) that is activated by a prior processing level F(1). Such a network F(2) behaves like a content-addressable memory. It activates compressed recognition codes that are predictive with respect to the activation patterns flickering across the feature detectors of F(1) and competitively inhibits, or masks, codes which are unpredictive with respect to the F(1) patterns. In particular, a masking field can simultaneously detect multiple groupings within its input patterns and assign activation weights to the codes for these groupings which are predictive with respect to the contextual information embedded within the patterns and the prior learning of the system. A masking field automatically rescales its sensitivity as the overall size of an input pattern changes, yet also remains sensitive to the microstructure within each input pattern. In this way, a masking field can more strongly activate a code for the whole F(1) pattern than for its salient parts, yet amplifies the code for a pattern part when it becomes a pattern whole in a new input context. A masking field can also be primed by inputs from F(1): it can activate codes which represent predictions of how the F(1) pattern may evolve in the subsequent time interval. Network F(2) can also exhibit an adaptive sharpening property: repetition of a familiar F(1) pattern can tune the adaptive filter to elicit a more focal spatial activation of its F(2) recognition code than does an unfamiliar input pattern. The F(2) recognition code also becomes less distributed when an input pattern contains more contextual information on which to base an unambiguous prediction of which the F(1) pattern is being processed. Thus a masking field suggests a solution of the credit assignment problem by embodying a real-time code for the predictive evidence contained within its input patterns. Such capabilities are useful in speech recognition, visual object recognition, and cognitive information processing. An absolutely stable design for a masking field is disclosed through an analysis of the computer simulations. This design suggests how associative mechanisms, cooperative-competitive interactions, and modulatory gating signals can be joined together to regulate the learning of compressed recognition codes. Data about the neural substrates of learning and memory are compared to these mechanisms.