Combination Of Neural Network Research Articles

Research on Acoustic Methods for Buried PE Pipeline Detection Based on LSTM Neural Networks

Abstract As an essential component of urban infrastructure construction, Polyethylene (PE) pipelines face the challenging task of underground detection due to the complex and dynamic nature of the subsurface environment, diverse installation paths, and the inherent insulating properties of PE materials. In order to address the non-excavation detection of buried PE pipelines, this paper proposes an acoustic method based on the Long Short-Term Memory (LSTM) neural network. The study begins by analyzing the propagation and reflection mechanisms of elastic waves in the pipe-soil coupling system, and a impact excitation source is designed to generate the excitation signal. After establishing the experimental environment and collecting experimental data, a comprehensive analysis is conducted, and the LSTM neural network is employed for data classification to determine the presence of buried PE pipelines. Through neural network training, accurate identification of the PE pipeline's existence and prediction of its burial depth are achieved, providing an efficient and reliable solution for buried PE pipeline detection. The practical results demonstrate the significant application prospects of the combined acoustic method and LSTM neural network in buried PE pipeline detection. This research contributes a novel solution to the field of non-destructive PE pipeline detection, with both theoretical and practical implications.

Kinetic models of HMX decomposition via chemical reaction neural network

1,3,5,7-Tetranitro-1,3,5,7-tetrazocane (HMX) is commonly used in solid propellants and explosives as energetic materials. The study of its kinetics and decomposition mechanism is of great significance to its application in the aerospace industries. This work investigates the thermal decomposition of HMX based on the combined thermogravimetric (TG) measurements and chemical reaction neural network (CRNN). Two compact kinetic models for HMX are introduced, with one consisting of four substances and a single global reaction (4–1 model) and the other consisting of four substances and four reactions (4–4 model). In the 4–1 model, the calculated activation energy is 328.44 kJ·mol−1, which agrees with the experimental value. As for the 4–4 model, the substances and reactions are assigned based on a skeleton mechanism involving reactions of N-N and C-N bond cleavage and HONO elimination. Moreover, the catalytic effects of TiO2 and Al2O3 on HMX are well simulated using the aforementioned kinetic models. The CRNN models can reproduce the peak temperature with a reduced activation energy, but the initial decomposition temperature is overestimated owing to the complex nature of catalytic impact. This work presents the application of the CRNN model to obtain a decomposition mechanism of HMX, highlighting its efficacy in accurately capturing the thermal decomposition behavior. The potential extension of CRNN to kinetic modeling of other energetic materials is anticipated in future studies.

Development of an Emulator for Radiation Physics Simulator Using Stochastic Variational Gaussian Process Model

This paper describes an emulator that uses a Stochastic Variational Gaussian process (SVGP) regression model to parameterize radiation in a numerical weather prediction (NWP) model that meteorologically models the Earth's weather system. The computation of radiative processes is very large, accounting for most of the total NWP model computation. Statistical emulators are surrogate models that represent simulators and can overcome the computational limitations of very complex simulators such as radiative processes. Recently, artificial neural network-based radiative transfer emulators have been developed, and in this study, a statistical model, GP, is used to develop a radiative transfer emulator. The GP model has the advantage of calculating the uncertainty of the prediction along with the prediction, so the uncertainty of the prediction can be utilized appropriately. However, the computational complexity of the conventional GP model is very high, making it difficult to apply to large data. To solve this problem, an approximate approach, the SVGP model, was utilized. To further reduce the dimensionality of the input variables, we used a combined neural network and SVGP model. As a result, the SVGP-based radiative physics emulator improved its accuracy by about 20% compared to the artificial neural network emulator. However, the computation speed was about 3 to 9 times slower than the neural network emulator, but it was faster than the computation speed of the NWP model. This suggests that statistical emulators can be used to replace NWP model simulators.

Hybrid forecasting model of building cooling load based on combined neural network

The inefficient operation of air-conditioning system, which leads to its high-energy consumption, has aroused far-ranging concern at present. And an accurate prediction of building cooling load can be conducive to matching the supply and demand of the air-conditioning system, thus improving its operational efficiency. Therefore, a hybrid forecasting model (BAS-GRNN&LSTM) combing generalized regression neural network (BAS-GRNN) and long short-term memory neural network (BAS-LSTM) optimized by beetle antennae search algorithm is proposed for building cooling load prediction. First, the factors affecting the cooling load are comprehensively analyzed, and the random forest combined with recursive feature elimination (RF-RFE) method is used for feature selection. Then, the BAS-GRNN&LSTM model is developed to forecast. Finally, a simulation experiment is carried out using the measured data of a large building in north of China. Compared with GRNN, LSTM, BAS-GRNN and BAS-LSTM, BAS-GRNN&LSTM performs better in five performance evaluation metrics, RMSE, MAPE, RRE, MBE and R2, demonstrating its higher prediction accuracy. Furthermore, the detailed algorithm performance analysis shows that the proposed hybrid model has significant advantages in robustness, generalization capability and computational complexity, and can be well applied to building cooling load prediction and beneficial to the optimal control of building air-conditioning system.

Merging Counter-Propagation and Back-Propagation Algorithms: Overcoming the Limitations of Counter-Propagation Neural Network Models.

Artificial neural networks (ANNs) are nowadays applied as the most efficient methods in the majority of machine learning approaches, including data-driven modeling for assessment of the toxicity of chemicals. We developed a combined neural network methodology that can be used in the scope of new approach methodologies (NAMs) assessing chemical or drug toxicity. Here, we present QSAR models for predicting the physical and biochemical properties of molecules of three different datasets: aqueous solubility, acute fish toxicity toward fat head minnow, and bio-concentration factors. A novel neural network modeling method is developed by combining two neural network algorithms, namely, the counter-propagation modeling strategy (CP-ANN) with the back-propagation-of-errors algorithm (BPE-ANN). The advantage is a short training time, robustness, and good interpretability through the initial CP-ANN part, while the extension with BPE-ANN improves the precision of predictions in the range between minimal and maximal property values of the training data, regardless of the number of neurons in both neural networks, either CP-ANN or BPE-ANN.

Boosting Clear Cell Renal Carcinoma-Specific Drug Discovery Using a Deep Learning Algorithm and Single-Cell Analysis.

Clear cell renal carcinoma (ccRCC), the most common subtype of renal cell carcinoma, has the high heterogeneity of a highly complex tumor microenvironment. Existing clinical intervention strategies, such as target therapy and immunotherapy, have failed to achieve good therapeutic effects. In this article, single-cell transcriptome sequencing (scRNA-seq) data from six patients downloaded from the GEO database were adopted to describe the tumor microenvironment (TME) of ccRCC, including its T cells, tumor-associated macrophages (TAMs), endothelial cells (ECs), and cancer-associated fibroblasts (CAFs). Based on the differential typing of the TME, we identified tumor cell-specific regulatory programs that are mediated by three key transcription factors (TFs), whilst the TF EPAS1/HIF-2α was identified via drug virtual screening through our analysis of ccRCC's protein structure. Then, a combined deep graph neural network and machine learning algorithm were used to select anti-ccRCC compounds from bioactive compound libraries, including the FDA-approved drug library, natural product library, and human endogenous metabolite compound library. Finally, five compounds were obtained, including two FDA-approved drugs (flufenamic acid and fludarabine), one endogenous metabolite, one immunology/inflammation-related compound, and one inhibitor of DNA methyltransferase (N4-methylcytidine, a cytosine nucleoside analogue that, like zebularine, has the mechanism of inhibiting DNA methyltransferase). Based on the tumor microenvironment characteristics of ccRCC, five ccRCC-specific compounds were identified, which would give direction of the clinical treatment for ccRCC patients.

A power load forecasting model based on a combined neural network

A power load forecasting model based on a combined neural network

Combined convolution recurrent neural network for the classification of dysarthria speech

Dysarthria is a neuromotor articulation condition that affects a person and weakens their tongue and lip muscles, and it additionally affects their capability to talk. Diffusion of factors, such as numbness, terrible coordination, and susceptible speech-generating muscle groups, can contribute to the condition. Cerebral palsy, Parkinson’s disorder, stroke, or some other kind of traumatic brain harm might also lead to dysarthria. People with dysarthria are not able to talk on a consistent basis because their speech is unclear and their phonemes are spoken irregularly because of trouble transferring the tongue, lips, and jaw as easily as in everyday speech. Dysarthric speech is stated to be 15 times slower than an ordinary speech.

Interpretable zero-inflated neural network models for predicting admission counts

Abstract In this paper, we construct interpretable zero-inflated neural network models for modeling hospital admission counts related to respiratory diseases among a health-insured population and their dependants in the United States. In particular, we exemplify our approach by considering the zero-inflated Poisson neural network (ZIPNN), and we follow the combined actuarial neural network (CANN) approach for developing zero-inflated combined actuarial neural network (ZIPCANN) models for modeling admission rates, which can accommodate the excess zero nature of admission counts data. Furthermore, we adopt the LocalGLMnet approach (Richman & Wüthrich (2023). Scandinavian Actuarial Journal, 2023(1), 71–95.) for interpreting the ZIPNN model results. This facilitates the analysis of the impact of a number of socio-demographic factors on the admission rates related to respiratory disease while benefiting from an improved predictive performance. The real-life utility of the methodologies developed as part of this work lies in the fact that they facilitate accurate rate setting, in addition to offering the potential to inform health interventions.

Periodic analysis of scenic spot passenger flow based on combination neural network prediction model

Abstract To prevent in a short time the rapid increase of tourists and corresponding traffic restriction measures’ lack in scenic areas, this study established a prediction model based on an improved convolutional neural network (CNN) and long- and short-term memory (LSTM) combined neural network. The study used this to predict the inflow and outflow of tourists in scenic areas. The model uses a residual unit, batch normalization, and principal component analysis to improve the CNN. The experimental results show that the model works best when batches’ quantity is 10, neurons’ quantity in the LSTM layer is 50, and the number of iterations is 50 on a workday; on non-working days, it is best to choose 10, 100, or 50. Using root mean square error (RMSE), normalized root mean square error (NRMSE), mean absolute percentage error (MAPE), and mean absolute error (MAE) as evaluation indicators, the inflow and outflow RMSEs of this study model are 82.51 and 89.80, MAEs are 26.92 and 30.91, NRMSEs are 3.99 and 3.94, and MAPEs are 1.55 and 1.53. Among the various models, this research model possesses the best prediction function. This provides a more accurate prediction method for the prediction of visitors’ flow rate in scenic spots. Meanwhile, the research model is also conducive to making corresponding flow-limiting measures to protect the ecology of the scenic area.

Automatic thoracic aorta calcium quantification using deep learning in non-contrast ECG-gated CT images

Thoracic aorta calcium (TAC) can be assessed from cardiac computed tomography (CT) studies to improve cardiovascular risk prediction. The aim of this study was to develop a fully automatic system to detect TAC and to evaluate its performance for classifying the patients into four TAC risk categories. The method started by segmenting the thoracic aorta, combining three UNets trained with axial, sagittal and coronal CT images. Afterwards, the surrounding lesion candidates were classified using three combined convolutional neural networks (CNNs) trained with orthogonal patches. Image datasets included 1190 non-enhanced ECG-gated cardiac CT studies from a cohort of cardiovascular patients (age 57 ± 9 years, 80% men, 65% TAC > 0). In the test set (N = 119), the combination of UNets was able to successfully segment the thoracic aorta with a mean volume difference of 0.3 ± 11.7 ml (<6%) and a median Dice coefficient of 0.947. The combined CNNs accurately classified the lesion candidates and 87% of the patients (N = 104) were accurately placed in their corresponding risk categories (Kappa = 0.826, ICC = 0.9915). TAC measurement can be estimated automatically from cardiac CT images using UNets to isolate the thoracic aorta and CNNs to classify calcified lesions.

Multi-spectral remote sensing and GIS-based analysis for decadal land use land cover changes and future prediction using random forest tree and artificial neural network

The integration of multi-spectral remote sensing and GIS-based analysis is significant for studying land cover changes, providing valuable insights for informed land management and sustainable development. The present study aims to examine land use land cover (LULC) changes of three decades from 1991 to 2021 and predict the future LULC change in Binh Duong province, Vietnam to explore a future research direction on land use change and associated challenges in the study region. Multi-spectral remote sensing data and random forest tree (RFT) were utilized to generate LULC maps. Areal statistics and annual change rate were considered to analyze the categorical land use change detection. Statistical measures such as user's accuracy, producer’s accuracy, kappa coefficient, and confusion matrix were employed to assess the accuracy of LULC classification. To predict future LULC and simulate the spatio-temporal change, we considered previous year LULC maps, independent spatial variables and a combined artificial neural network (ANN) multi-layer perceptron approach. The analysis revealed that there was a huge transition from agricultural lands to residential land with industry and commerce which resulting an expansion of impervious lands and a rapid decline of agricultural land as well as of scrub land and barren lands, and a changeability of forest and plantation, croplands, and waterbodies. Our study revealed that the impervious land has expanded 10 times within 30 years and will increase in the future.

An improved sparrow search algorithm and CNN-BiLSTM neural network for predicting sea level height

Accurate prediction of sea level height is critically important for the government in assessing sea level risk in coastal areas. However, due to the nonlinear, time-varying and highly uncertain characteristics of sea level change data, sea level prediction is challenging. To improve the accuracy of sea level prediction, this paper uses a new swarm intelligence algorithm named the sparrow search algorithm (SSA), which can imitate the foraging behavior and antipredation behavior of sparrows, to determine optimal solutions. To avoid the algorithm falling into a local optimal situation, this paper integrates the sine–cosine algorithm and the Cauchy variation strategy into the SSA to obtain an algorithm named the SCSSA. The SCSSA is used to optimize the parameter values of the CNN-BiLSTM (convolutional neural network combined with bidirectional long short-term memory neural network) model; finally, a combined neural network model (named SCSSA-CNN-BiLSTM) is proposed. In this paper, the time series data of seven tidal stations located in coastal China are used for experimental analysis. First, the SCSSA-CNN-BiLSTM model is compared with the CNN-BiLSTM model to predict the time series data of SHANWEI Station. With respect to the training and test sets of data, the SCSSA-CNN-BiLSTM model outperforms the other models on all the evaluation metrics. In addition, the remaining six tide station datasets and five neural network models, including the SCSSA-CNN-BiLSTM model, are used to further study the performance of the proposed prediction model. Four evaluation indices including the root mean squared error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE) and coefficient of determination (R2) are adopted. For six stations, the RMSE, MAE, MAPE and R2 of SCSSA-CNN-BiLSTM model are ranged from 20.9217 ~ 27.8427 mm, 9.4770 ~ 17.8603 mm, 0.1322% ~ 0.2482% and 0.9119 ~ 0.9759, respectively. The experimental analysis results show that the SCSSA-CNN-BiLSTM model makes effective predictions at all stations, and the prediction performance is better than that of the other models. Even though the combination of SCSSA algorithm may increase the complexity of the model, indeed the proposed model is a new prediction method with good accuracy and robustness for predicting sea level change.

CNN-LSTM and transfer learning models for malware classification based on opcodes and API calls

In this paper, we propose a novel model for a malware classification system based on Application Programming Interface (API) calls and opcodes, to improve classification accuracy. This system uses a novel design of combined Convolutional Neural Network and Long Short-Term Memory. We extract opcode sequences and API Calls from Windows malware samples for classification. We transform these features into N-grams (N = 2, 3, and 10)-gram sequences. Our experiments on a dataset of 9,749,57 samples produce high accuracy of 99.91% using the 8-gram sequences. Our method significantly improves the malware classification performance when using a wide range of recent deep learning architectures, leading to state-of-the-art performance. In particular, we experiment with ConvNeXt-T, ConvNeXt-S, RegNetY-4GF, RegNetY-8GF, RegNetY-12GF, EfficientNetV2, Sequencer2D-L, Swin-T, ViT-G/14, ViT-Ti, ViT-S, VIT-B, VIT-L, and MaxViT-B. Among these architectures, Swin-T and Sequencer2D-L architectures achieved high accuracies of 99.82% and 99.70%, respectively, comparable to our CNN-LSTM architecture although not surpassing it.

Next-level vegetation health index forecasting: A ConvLSTM study using MODIS Time Series

The Vegetation Health Index (VHI) is a metric used to assess the health and condition of vegetation, based on satellite-derived data. It offers a comprehensive indicator of stress or vigor, commonly used in agriculture, ecology, and environmental monitoring for forecasting changes in vegetation health. Despite its advantages, there are few studies on forecasting VHI as a future projection, particularly using up-to-date and effective machine learning methods. Hence, the primary objective of this study is to forecast VHI values by utilizing remotely sensed images. To achieve this objective, the study proposes employing a combined Convolutional Neural Network (CNN) and a specific type of Recurrent Neural Network (RNN) called Long Short-Term Memory (LSTM), known as ConvLSTM. The VHI time series images are calculated based on the Normalized Difference Vegetation Index (NDVI) and Land Surface Temperature (LST) data obtained from the Moderate Resolution Imaging Spectroradiometer (MODIS) aboard the Terra and Aqua satellites. In addition to the traditional image-based calculation, the study suggests using global minimum and global maximum values (global scale) of NDVI and LST time series for calculating the VHI. The results of the study showed that the ConvLSTM with a 1-layer structure generally provided better forecasts than 2-layer and 3-layer structures. The average Root Mean Square Error (RMSE) values for the 1-step, 2-step, and 3-step ahead VHI forecasts were 0.025, 0.026, and 0.026, respectively, with each step representing an 8-day forecast horizon. Moreover, the proposed global scale model using the applied ConvLSTM structures outperformed the traditional VHI calculation method.

Two-stage helmet security detection model combining CNN and MTCNN algorithm

Physical safety is of utmost importance for workers in an industrial and construction environment. Fatalities related to the violation of wearing a safety helmet account for a large proportion of occupational deaths. Hence, the requirement of a high-efficiency automatic system to monitor and promote wearing helmets is of vital importance, reducing hours from manual monitoring, facilitating efficient education and enforcement campaigns that increase industrial safety. In this paper, a combined Convolutional Neural Network (CNN) algorithm and Multi-Task CNN (MTCNN) to monitor violations of safety helmets is proposed. All images are sourced from industrial and construction locations, undergoing various image processing techniques before model detection. The proposed model includes the MTCNN locating individuals' heads from the picture data, followed by a CNN assessment to determine helmet wearing. Under the same evaluation criterion, the result, where an obvious detection accuracy improvement is shown, illustrates the effectiveness of the proposed model in the comparative study. Therefore, the research demonstrates the effectiveness of incorporating multi-scale feature extraction techniques for enhancing model performance. This establishes the groundwork for future research within this domain, facilitating the creation of more advanced helmet detection or other relevant items recognition systems that can be utilized in real-world applications.

A novel damage identification algorithm by combining the boundary element method and a series connection neural network

A novel damage identification approach based on a model-driven and a data-driven combined algorithm is developed. By using this approach with only boundary strains, the existence, location, classification as well as the extent of the damage in a plate can be predicted at once with high accuracy and efficiency. To accumulate the data, the boundary element method (BEM) is applied as a model-driven in modeling plates with one or multiple damages in the forms of circular or elliptical holes or cracks and for solving the boundary strains of the defective plates. The dimensionality reduction and semi-analytical characteristics of BEM not only can compress the feature data also can improve the accuracy of the database for the data-driven algorithm. The boundary strains are obtained directly from BEM models, which are also easily collected through the use of strain gauges mounted on the surfaces of structures being monitored in real applications. A series connection neural network algorithm is established to accomplish the novel damage identification assignments with deep learning. The number of the existing damages is firstly detected by a classification neural network model, then the extracted features are transmitted to the corresponding regression neural network model to prognosis the location, classification as well as the extent of each flaw. A high accuracy of about 99.86 % is achieved by the present combined neural network algorithm, which is promising in applications of actual structural health monitoring.

Experimental investigation of machine-learning-based soft-failure management using the optical spectrum

The demand for high-speed data is exponentially growing. To conquer this, optical networks have undergone significant changes, getting more complex and versatile. The increasing complexity necessitates that the fault management be more adaptive to enhance network assurance. In this paper, we experimentally compare the performance of soft-failure management of different machine learning algorithms. We further introduce a machine-learning-based soft-failure management framework. It utilizes a variational autoencoder-based generative adversarial network (VAE-GAN) running on optical spectral data obtained by optical spectrum analyzers. The framework is able to reliably run on a fraction of available training data as well as identify unknown failure types. The investigations show that the VAE-GAN outperforms the other machine learning algorithms when up to 10% of the total training data are available in identification tasks. Furthermore, the advanced training mechanism for the GAN shows a high F1-score for unknown spectrum identification. The failure localization comparison shows the advantage of a low complexity neural network in combination with a VAE over established machine learning algorithms.

Tree ring segmentation using UNEt TRansformer neural network on stained microsections for quantitative wood anatomy.

Forests are critical in the terrestrial carbon cycle, and the knowledge of their response to ongoing climate change will be crucial for determining future carbon fluxes and climate trajectories. In areas with contrasting seasons, trees form discrete annual rings that can be assigned to calendar years, allowing to extract valuable information about how trees respond to the environment. The anatomical structure of wood provides highly-resolved information about the reaction and adaptation of trees to climate. Quantitative wood anatomy helps to retrieve this information by measuring wood at the cellular level using high-resolution images of wood micro-sections. However, whereas large advances have been made in identifying cellular structures, obtaining meaningful cellular information is still hampered by the correct annual tree ring delimitation on the images. This is a time-consuming task that requires experienced operators to manually delimit ring boundaries. Classic methods of automatic segmentation based on pixel values are being replaced by new approaches using neural networks which are capable of distinguishing structures, even when demarcations require a high level of expertise. Although neural networks have been used for tree ring segmentation on macroscopic images of wood, the complexity of cell patterns in stained microsections of broadleaved species requires adaptive models to accurately accomplish this task. We present an automatic tree ring boundary delineation using neural networks on stained cross-sectional microsection images from beech cores. We trained a UNETR, a combined neural network of UNET and the attention mechanisms of Visual Transformers, to automatically segment annual ring boundaries. Its accuracy was evaluated considering discrepancies with manual segmentation and the consequences of disparity for the goals of quantitative wood anatomy analyses. In most cases (91.8%), automatic segmentation matched or improved manual segmentation, and the rate of vessels assignment to annual rings was similar between the two categories, even when manual segmentation was considered better. The application of convolutional neural networks-based models outperforms human operator segmentations when confronting ring boundary delimitation using specific parameters for quantitative wood anatomy analysis. Current advances on segmentation models may reduce the cost of massive and accurate data collection for quantitative wood anatomy.

Deep Learning-Based SC-FDMA Channel Equalization

It is very challenging to design an effective wireless communication system. That’s because of numerous factors affecting the performance of a typical wireless communication system, such as nonlinear channel distortions and impairments. single carrier frequency division multiple access (SC-FDMA) is a multiple access scheme that is an important part of the long-term evolution (LTE) standard for uplink transmission. An advanced mobile radio system’s multiple access schemes should indeed meet stringent requirements, such as a low bit error rate (BER). In this article, we investigate the equalization problem for nonlinear channel distortions and impairments using deep neural networks (NN). We introduce a novel combined deep neural network channel equalization and symbol detection scheme based on a deep learning (DL) recurrent feedback (RF) long short-term memory (LSTM) neural network to achieve blind equalization and decoding for SC-FDMA systems without knowing the channel state information (CSI). To train the model efficiently, the training data is gathered by simulation, with channel effects and noise treated as a complete black box. CSI and constellation demapping are learned by a deep neural network (DNN) model. Then, the frequency-domain sequences that have been corrupted are implicitly equalized to get the broadcasted signal back. Our specified SC-FDMA system, which uses a quadrature phase-shift keying (QPSK) modulation method and the suggested Deep Learning-based model channel equalizer, performs better than the existing equalizers by an average of 1 to 4 dB at moderate signal-to-noise (SNR) ratios, according to simulation data. A complexity comparison between the proposed and the conventional equalizers was conducted in terms of training time, execution time, and number of operations. On combined channel equalization and symbol detection, the suggested system delivers state-of-the-art performance.