Neural Network Model Research Articles

Rice Leaf Disease Recognition using CNN

Abstract: The occurrence of bacterial, viral, and fungal diseases on rice leaves significantly hampers rice production, posing a challenge to meet global demand for the staple crop. While the detection of rice leaf diseases is crucial, existing methods are constrained by limitations in image backgrounds and capture conditions. Convolutional Neural Network (CNN) models have emerged as a usefull avenue for disease recognition in rice leaves, yet current approaches suffer from decreased recognition rates when applied to independent datasets and are constrained by the need for large-scale network parameters. In this project, we propose an innovative CNN-based model aimed at mitigating these challenges by reducing network parameters. Through training multiple CNN- basedmodels to identify three common rice leaf diseases, our study aims to showcase the effectiveness and superiority of our approach compared to state-of-the-art CNN-based models for rice leaf disease recognition.

Prediction of Hydrogen Storage Vessel Explosion with Blast Wall using Machine Learning

We present a prediction of hydrogen storage vessel explosion using machine learning. A neural network model is trained to learn the waveform of a hydrogen explosion with a blast wall and used to predict the waveform with arbitrary blast wall position. The initial training revealed one of the features, blast wall distance, greatly affects the blast waveform. To emphasize the effect of that feature, feature multiplication is used instead of normalization and this enabled the training to learn the waveform correctly. The trained model can predict the blast waveform with an arbitrarily located blast wall. The result can be used in structural analysis and this will help to construct the blast wall in the hydrogen refuelling station. This research uses the CFD (Computational Fluid Dynamics) simulation by Pukyong University, South Korea, as training data.

Artificial neural network modelling of cyanide transport in a soil contaminated with cassava wastewater

Artificial neural network modelling of cyanide transport in a soil contaminated with cassava wastewater

Improved time series models for the prediction of lane-change intention

ABSTRACT To improve the accuracy of lane-change intention prediction and analyze the influence of driving styles on prediction outcomes, the T-Encoder-Sequence model is proposed in this paper. It integrates the Transformer’s encoder module with various recurrent neural network (RNN) models and introduces a multimodal fusion input structure. Building on this, a risk indicator model, capable of reflecting driver stress, is established to calculate the model’s input parameters. Consequently, the model can simultaneously capture global information and consider the impact of vehicle classes on drivers. Furthermore, the k-means++ algorithm is employed to categorize vehicle trajectories into conservative, conventional, and aggressive types for further analysis. The results demonstrate that training the model with risk indicator parameters markedly enhances prediction performance. Under identical input parameters, the T-Encoder-Sequence model exhibits notably superior prediction efficacy compared to the original model. The T-Encoder-Sequence model, trained with risk indicator parameters, demonstrates substantial advantages compared to other studies.

Exceptional performance with minimal data using a generative adversarial network for alzheimer's disease classification

The classification of Alzheimer's disease (AD) using deep learning models is hindered by the limited availability of data. Medical image datasets are scarce due to stringent regulations on patient privacy, preventing their widespread use in research. Moreover, although open-access databases such as the Open Access Series of Imaging Studies (OASIS) are available publicly for providing medical image data for research, they often suffer from imbalanced classes. Thus, to address the issue of insufficient data, this study proposes the integration of a generative adversarial network (GAN) that can achieve comparable accuracy with a reduced data requirement. GANs are unsupervised deep learning networks commonly used for data augmentation that generate high-quality synthetic data to overcome data scarcity. Experimental data from the OASIS database are used in this research to train the GAN model in generating synthetic MRI data before being included in a pretrained convolutional neural network (CNN) model for multistage AD classification. As a result, this study has demonstrated that a multistage AD classification accuracy above 80% can be achieved even with a reduced dataset. The exceptional performance of GANs positions them as a solution for overcoming the challenge of insufficient data in AD classification.

Study on the classification of sleep stages in EEG signals based on DoubleLinkSleepCLNet.

The classification of sleep stages based on Electroencephalogram (EEG) changes has significant implications for evaluating sleep quality and sleep status. Most polysomnography (PSG) systems have a limited number of channels and do not achieve optimal classification performance due to a paucity of raw data. To leverage the data characteristics and enhance the classification accuracy, we propose and evaluate a novel dual-link deep neural network model, 'DoubleLinkSleepCLNet'. The DoubleLinkSleepCLNet model performs feature extraction and efficient classification on both the raw EEG and the EEG processed with the Hilbert transform. It leverages the frequency domain and time domain feature modules, resulting in superior performance compared to other models. The DoubleLinkSleepCLNet model, using the 2 Raw/2 Hilbert data modes, achieved the highest classification performance with an accuracy of 88.47%. The average accuracy of the EEG was improved by approximately 4.08% after the application of the Hilbert transform. Additionally, Convolutional Neural Network (CNN) demonstrated superior performance in processing phase information, whereas Long Short-Term Memory (LSTM) excelled in handling time series data. The application of the Hilbert transform to EEG data, followed by processing it with a convolutional neural network, enhances the accuracy of the model. These findings introduce novel concepts for accelerating sleep stage prediction research, suggesting potential applications of these methods to other EEG analyses.

Machine learning-based predictive control of an electrically-heated steam methane reforming process

Hydrogen plays a crucial role in improving sustainability and offering a clean and efficient energy carrier that significantly reduces greenhouse gas emissions. However, the primary method of industrial hydrogen production, steam methane reforming (SMR), relies on the combustion of hydrocarbons as the heating source for the reforming reactions, resulting in significant carbon emissions. To address this issue, an experimental setup of an electrically-heated steam methane reformer (e-SMR) has been constructed at UCLA, and a lumped first-principle dynamic process model was built based on parameters estimated from the experimental data in a previous study. Subsequently, the first-principle dynamic process model was implemented into the computational model predictive control (MPC) scheme, successfully driving the hydrogen production rate to the desired setpoint. While these works are important and pave the way for developing MPC for large-scale e-SMR processes, the first-principle process model may not accurately reflect the actual process behavior, particularly as the process behavior changes with time. Therefore, the development and establishment of an adaptive data-driven approach for implementing model predictive control in the e-SMR process is necessary. To address this need, the present work investigates the construction of recurrent neural network (RNN) models for an e-SMR process in-depth, utilizing data from an experimentally-validated first-principle model. Specifically, a long short-term memory (LSTM) layer was utilized in the RNN model to effectively capture the complex correlations present in long-term sequential data. Subsequently, this LSTM-based RNN process model was employed to design an MPC, and its performance was evaluated through comparison with proportional-integral (PI) control. To address potential disturbances and variability in a typical e-SMR process, three distinct approaches were developed: MPC with an integrator, MPC with real-time online retraining (transfer learning), and offset-free MPC. These approaches effectively eliminated the offset caused by disturbances. Overall, this study underscores the effectiveness of utilizing RNN models to capture process dynamics in an experimental e-SMR process. It also outlines strategies for employing RNN-based control and multiple approaches to address disturbances in general processes with partially infrequent and delayed measurement feedback. This approach is particularly valuable in scenarios where developing first-principle models for a new process may be challenging.

Prediction of Surface Urban Heat Island based on predicted consequences of urban sprawl using deep learning: a way forward for a sustainable environment

The present work aimed at the spatiotemporal analysis of Land Surface Temperature (LST) and several land-use land-cover spectral indices, namely Normalized Difference Vegetation Index (NDVI), Normalized Difference Built-up Index (NDBI), Soil Adjusted Vegetation Index (SAVI), and Modified Normalized Difference Water Index (MNDWI) and to develop a Convolutional Neural Network (CNN) model to predict the LST and the indices for Surface Urban Heat Island (SUHI) evaluation based on the impervious surface area. The predictions were made only for LST and two indices (NDVI and NDBI) because of their strong correlation with LST. Landsat-8 satellite imageries acquired for 2013, 2015, 2017, 2019, and 2021 were utilized, and the Sialkot district, a swiftly urbanizing and advancing industrial city, was selected as a case study area. In the study, first, the indices (NDVI and NDBI) were predicted for 2019, 2021, and 2023 following a sequential procedure, and then based on the predicted indices, the LST for the mentioned years was predicted. Ultimately, to analyze the impact of impervious area on SUHI, impervious and SUHI were extracted for 2019, 2021, and 2023 using the predicted NDBI and LST, respectively, by employing Otsu’s Thresholding technique. The predicted LST and indices for 2019 and 2021 were compared with the obtained 2019 and 2021 LST and indices through several statistical measures, such as the kappa index, to evaluate the performance of the CNN model. The kappa index of different kappa statistics for 2019 and 2021 varied between 0.81 and 0.96, which resembles several previous studies, thus indicating sufficient future predictions. SUHI evaluation was conducted based on the observed (2016) and simulated (2021) impervious area and normalized LST images. The results showed that the distribution of the SUHI was highly related to the impervious area. Suppose the urbanization rate in the study area continues at its current pace. In that case, this expansion will cause a dramatic increase in SUHI distribution unless decision-makers consider any proper urban planning model for the study area.

Human gender estimation from CT images of skull using deep feature selection and feature fusion

This research endeavors to prognosticate gender by harnessing the potential of skull computed tomography (CT) images, given the seminal role of gender identification in the realm of identification. The study encompasses a corpus of CT images of cranial structures derived from 218 male and 203 female subjects, constituting a total cohort of 421 individuals within the age bracket of 25 to 65 years. Employing deep learning, a prominent subset of machine learning algorithms, the study deploys convolutional neural network (CNN) models to excavate profound attributes inherent in the skull CT images. In pursuit of the research objective, the focal methodology involves the exclusive application of deep learning algorithms to image datasets, culminating in an accuracy rate of 96.4%. The gender estimation process exhibits a precision of 96.1% for male individuals and 96.8% for female individuals. The precision performance varies across different selections of feature numbers, namely 100, 300, and 500, alongside 1000 features without feature selection. The respective precision rates for these selections are recorded as 95.0%, 95.5%, 96.2%, and 96.4%. It is notable that gender estimation via visual radiography mitigates the discrepancy in measurements between experts, concurrently yielding an expedited estimation rate. Predicated on the empirical findings of this investigation, it is inferred that the efficacy of the CNN model, the configurational intricacies of the classifier, and the judicious selection of features collectively constitute pivotal determinants in shaping the performance attributes of the proposed methodology.

Spiking Transfer Learning from RGB Image to Neuromorphic Event Stream.

Recent advances in bio-inspired vision with event cameras and associated spiking neural networks (SNNs) have provided promising solutions for low-power consumption neuromorphic tasks. However, as the research of event cameras is still in its infancy, the amount of labeled event stream data is much less than that of the RGB database. The traditional method of converting static images into event streams by simulation to increase the sample size cannot simulate the characteristics of event cameras such as high temporal resolution. To take advantage of both the rich knowledge in labeled RGB images and the features of the event camera, we propose a transfer learning method from the RGB to the event domain in this paper. Specifically, we first introduce a transfer learning framework named R2ETL (RGB to Event Transfer Learning), including a novel encoding alignment module and a feature alignment module. Then, we introduce the temporal centered kernel alignment (TCKA) loss function to improve the efficiency of transfer learning. It aligns the distribution of temporal neuron states by adding a temporal learning constraint. Finally, we theoretically analyze the amount of data required by the deep neuromorphic model to prove the necessity of our method. Numerous experiments demonstrate that our proposed framework outperforms the state-of-the-art SNN and artificial neural network (ANN) models trained on event streams, including N-MNIST, CIFAR10-DVS and N-Caltech101. This indicates that the R2ETL framework is able to leverage the knowledge of labeled RGB images to help the training of SNN on event streams.

Machine learning for the adsorptive removal of ciprofloxacin using sugarcane bagasse as a low-cost biosorbent: comparison of analytic, mechanistic, and neural network modeling.

Contamination with traces of pharmaceutical compounds, such as ciprofloxacin, has prompted interest in their removal via low-cost, efficient biomass-based adsorption. In this study, classical models, a mechanistic model, and a neural network model were evaluated for predicting ciprofloxacin breakthrough curves in both laboratory- and pilot scales. For the laboratory-scale (d = 2.2cm, Co = 5mg/L, Q = 7mL/min, T = 18°C) and pilot-scale (D = 4.4cm, Co = 5mg/L, Q = 28mL/min, T = 18°C) setups, the experimental adsorption capacities were 2.19 and 2.53mg/g, respectively. The mechanistic model reproduced the breakthrough data with high accuracy on both scales (R2 > 0.4 and X2 < 0.15), and its fit was higher than conventional analytical models, namely the Clark, Modified Dose-Response, and Bohart-Adams models. The neural network model showed the highest level of agreement between predicted and experimental data with values of R2 = 0.993, X2 = 0.0032 (pilot-scale) and R2 = 0.986, X2 = 0.0022 (laboratory-scale). This study demonstrates that machine learning algorithms exhibit great potential for predicting the liquid adsorption of emerging pollutants in fixed bed.

The changes prediction on terrestrial water storage in typical regions of China based on neural networks and satellite gravity data

Accurate prediction of regional terrestrial water storage change (TWSA) is of great significance for water resources planning and management, and early warning of extreme climate disasters. Aiming at the problem that the conventional methods on prediction of TWSA time series are difficult to be accurate, the six typical regions are selected in China as examples, including the upper reaches of the Yangtze River (UYR), the southwest region (SWR), the Liaohe River Basin (LRB), the North China Plain (NCP), the Qinghai-Tibet Plateau (QTP), and the Pearl River Basin (PRB). The mascon product from GRACE/GRACE-FO provided by CSR is used to extract TWSA time series in six typical areas. The improved Back Propagation (BP) neural network, Long Short-Term Memory (LSTM) neural network and the latest Bidirectional LSTM (BiLSTM-attention) neural network model based on attention mechanism are proposed to predict and analyze the regional TWSA. In the experiment, the selection of the optimal model parameters such as the number of hidden layer nodes and the number of hidden units of the neural network model is tested and analyzed in detail. Meanwhile, the model prediction results are compared with the traditional least squares method and random forest (RF) prediction method. The root mean square error (RMSE), determination coefficient (R2), Nash–Sutcliffe efficiency coefficient (NSE) and mean absolute percentage error (MAPE) were used to evaluate the accuracy of the predicted results. The results show that the improved BP, LSTM and Bi-LSTM-attention neural network models all achieve higher prediction accuracy in UYR and SWR areas. RMSE is less than 2.641 cm, R2 is as high as 0.8 or more, NSE is above 0.6, and MAPE is within 0.1. Compared with the least square method, the RMSE of the predicted results from three neural network decreased by 0.998 cm, 0.700 cm and 0.7563 on average, and the R2 increased by 81.75%, 69.89% and 72% on average. Compared with RFML method, the RMSE from three neural network is reduced by 0.601 cm, 0.316 cm and 0.360, and R2 is increased by 38.20%, 24.60% and 27.06% on average. NSE and RMSE are improved to varying degrees in the above regions. It shows that the improved BP, LSTM and BiLSTM-attention model used can effectively predict TWSA. The research methods and results in this paper can provide important reference for the rational utilization of regional water resources and disaster risk assessment.

Hybrid machine learning models to predict the shear strength of discontinuities with different joint wall compressive strength

ABSTRACT The shear strength of discontinuities with different joint wall compressive strength (DDJCS) is an important mechanical property. In this paper, the prediction of shear strength of DDJCS is performed using a hybrid machine learning model of artificial neural network (ANN) model with particle swarm optimisation (PSO). To develop the proposed model, a total of 168 cases are collected. Two classical models are introduced, and four performance indexes (i.e. coefficient of determination R2, mean absolute error MAE, root mean square error RMSE, and variance account for VAF) are utilised to evaluate the comprehensive predictive performances of these models. Finally, a graphical user interface (GUI) for predicting the shear strength of DDJCS is developed. The comparison results demonstrate that compared to the remaining models, this hybrid model has a better predictive performance and generalisation capability, with R2, RMSE, VAF, and MAE values of 0.999, 0.02, 99.9%, and 0.01, respectively. GUI signifies that this novel model provides an intuitive, efficient, and visual production experience for researchers. These findings can provide valuable insights for developing accurate predictive models.

Classification and Severity Level Assessment of Fusarium Wilt Disease in Chickpeas using Convolutional Neural Network

Background: The fusarium wilt disease of chickpea leaves is a common illness that leads to economic problems for farmers due decreased crop yield. Early disease detection and the implementation of suitable precautions can help to increase the yield of chickpeas. This study offers an improved method for Fusarium wilt disease prediction based on severity level using a convolutional neural learning algorithm. Methods: The Convolutional Neural Network (CNN) model is utilized in this work to identify leaf disease due to wilting. The dataset contains 4,339 images of chickpea leaves that were obtained from Kaggle. After preprocessing, the data is sent into the network model for training. The model shows acceptable classification and accuracy metrics. Result: Deep learning methods are very useful tools for tracking leaf diseases at their early stages and can help farmers with the use of controlling methods. The proposed work looks for changes in the shape and color of chickpea leaves in order to predict severe fusarium disease. Training and validation accuracies show a balanced trade-off by giving satisfactory outcomes. The model shows an overall accuracy of 74.79%. The confusion matrix and classification parameters increase the model’s performance.

Genome composition-based deep learning predicts oncogenic potential of HPVs

Human papillomaviruses (HPVs) account for more than 30% of cancer cases, with definite identification of the oncogenic role of viral E6 and E7 genes. However, the identification of high-risk HPV genotypes has largely relied on lagged biological exploration and clinical observation, with types unclassified and oncogenicity unknown for many HPVs. In the present study, we retrieved and cleaned HPV sequence records with high quality and analyzed their genomic compositional traits of dinucleotide (DNT) and DNT representation (DCR) to overview the distribution difference among various types of HPVs. Then, a deep learning model was built to predict the oncogenic potential of all HPVs based on E6 and E7 genes. Our results showed that the main three groups of Alpha, Beta, and Gamma HPVs were clearly separated between/among types in the DCR trait for either E6 or E7 coding sequence (CDS) and were clustered within the same group. Moreover, the DCR data of either E6 or E7 were learnable with a convolutional neural network (CNN) model. Either CNN classifier predicted accurately the oncogenicity label of high and low oncogenic HPVs. In summary, the compositional traits of HPV oncogenicity-related genes E6 and E7 were much different between the high and low oncogenic HPVs, and the compositional trait of the DCR-based deep learning classifier predicted the oncogenic phenotype accurately of HPVs. The trained predictor in this study will facilitate the identification of HPV oncogenicity, particularly for those HPVs without clear genotype or phenotype.

Evaluation of ethanol-gasoline blends in SI engines using experimental and ANN techniques

Fuel combustion has become a major global concern, with much research focusing on the various emissions resulting from different types of fuels. Due to the harmful pollutant emissions from fossil fuels, the world has turned to renewable and alternative fuels to limit toxic emissions and greenhouse effects. Ethanol is a biofuel that, when used in spark ignition engines with gasoline can improve the octane number, combustion efficiency, and produce less emissions. The current research studies the effect of different ethanol blends E0, E5, E10, and E15 with gasoline 92 on engine performance parameters and emissions of a GX35 four-stroke engine at different engine speeds. The results along the speed range reveal that increasing ethanol amount leads to an average increase of 2.7%, 1%, and 1.1% in brake power (BP), brake thermal efficiency (BTE), and CO2 emissions, respectively. Meanwhile, it causes an average decrease of 28 °C, 3%, 15 ppm, and 0.18% in exhaust gas temperature (EGT), brake-specific fuel consumption (BSFC), HC, and CO emissions respectively. Moreover, the current study develops an Artificial Neural Networks (ANN) model for predicting the performance and emissions of spark ignition (SI) engines. Python programming language is used for ANN coding to train and validate the ANN model with E15. Regression plots were generated to visualize the correlation between the target and predicted data, indicating outstanding performance. The results confirmed the model’s reliability for BP, EGT, CO, CO2, and HC parameters with R2 values more than 0.99 and with acceptable performance for BSFC and BTE with R2 of 0.9339, and 0.9708, respectively. To ensure that the is no overfitting during the ANN study, we used different statistical methods, such as validation set, cross-validation, and learning curves.

A Globally Accurate Neural Network Potential Energy Surface and Quantum Dynamics Studies on Be+(2S) + H2/D2 → BeH+/BeD+ + H/D Reactions

Chemical reactions between Be+ ions and H2 molecules have significance in the fields of ultracold chemistry and astrophysics, but the corresponding dynamics studies on the ground-state reaction have not been reported because of the lack of a global potential energy surface (PES). Herein, a globally accurate ground-state BeH2+ PES is constructed using the neural network model based on 18,657 ab initio points calculated by the multi-reference configuration interaction method with the aug-cc-PVQZ basis set. On the newly constructed PES, the state-to-state quantum dynamics calculations of the Be+(2S) + H2(v0 = 0; j0 = 0) and Be+(2S) + D2(v0 = 0; j0 = 0) reactions are performed using the time-dependent wave packet method. The calculated results suggest that the two reactions are dominated by the complex-forming mechanism and the direct abstraction process at relatively low and high collision energies, respectively, and the isotope substitution has little effect on the reaction dynamics characteristics. The new PES can be used to further study the reaction dynamics of the BeH2+ system, such as the effects of rovibrational excitations and alignment of reactant molecules, and the present dynamics data could provide an important reference for further experimental studies at a finer level.

Measurement of air-water counter-current flow rates in vertical annulus using multiple differential pressure signals and machine learning

Abstract Gas–liquid counter-current flow in vertical annulus is involved in several industrial fields such as petroleum engineering. For example, in coalbed methane wells with liquid pump drainage, obtaining the real-time flow rate of gas–liquid two-phase in the annulus is crucial for the development management of coalbed methane wells. However, due to complex flow conditions, this demand is difficult to achieve through traditional flow metering means. Therefore, this paper proposes a flow prediction method based on multi-group differential pressure signals and machine learning technology. Air-water two-phase flow experiments were carried out on a vertical annulus pipeline with inner and outer diameters of 75 mm/125 mm and adjustable eccentricity. The probability density function and power spectrum density of 3 groups of differential pressure signals collected at different height intervals in the annulus were used as model inputs. A gas–liquid two-phase flow prediction model was constructed based on the artificial neural network model, and the model hyper-parameters were optimized using Bayesian optimization. The results show that on the 440-group test dataset under the combined conditions of air superficial velocity of 0.06–5.04 m s−1, water superficial velocity of 0.03–0.25 m s−1, and pipeline eccentricity of 0, 0.25, 0.5, 0.75, 1, etc. This model can achieve average prediction errors of 9.12% and 29.34% for gas and water flow rates respectively. This method can be applied to the non-throttling, non-invasive measurement of phase flow rate under the condition of annulus air-liquid counter-current flow.

Predicting the Deformation of a Slope Using a Random Coefficient Panel Data Model

Engineering constructions in coastal areas not only affect existing landslides, but also induce new landslides. Variation of the water level makes the coastal area a geological hazard-prone. Prediction of the slope displacement based on monitoring data plays an important role in early warning of potential landslide and slope failure, and supports the risk management of hazards. Given the complex characteristic of the slope deformation, we proposed a prediction model using random coefficient model under the frame of panel data analysis, so as to take the correlation among monitoring points into consideration. In addition, we classified the monitoring data using Gaussian mixture model, to take the temporal-spatial characteristics into consideration. Monitoring data of Guobu slope was used to validate the model. Results indicated that the proposed model have a better performance in prediction accuracy. We also compared the proposed model with the BP neural network model and temporal – temperature model, and found that the prediction accuracy of the proposed model is better than those of the two control models.

DRL-Tomo: a deep reinforcement learning-based approach to augmented data generation for network tomography

Abstract Accurate and current comprehension of network status is crucial for efficient network management. Nevertheless, direct network measurement strategies entail substantial traffic overhead and demand intricate coordination among network entities, making them impractical. Network tomography, an indirect measurement approach, utilizes insights garnered from measured parts to deduce characteristics of the entire network. Past studies frequently depend on acquiring challenging-to-access information, such as the complete network topology or support from specialized protocols. Unfortunately, these constraints pose challenges in non-cooperative scenarios where obtaining such information is difficult. Recent endeavors pursue emancipating tomography from dependence on copious information, striving to predict unmeasured path performance using limited data. Nevertheless, the disparity between the measured data and actual performance has hindered the accuracy. In response, we introduce an innovative tomography framework named DRL-Tomo, designed to alleviate potential biases. DRL-Tomo initiates by generating augmented data through deep reinforcement learning, gradually approximating the genuine performance of unmeasured paths. Subsequently, a neural network model is trained using this augmented data, enabling precise inferences. Our experiments, encompassing both real-world and synthetic datasets, vividly demonstrate DRL-Tomo’s remarkable enhancement. Specifically, it achieves a substantial 10%–67% improvement in path delay prediction and an impressive 30%–98% enhancement in path loss rate prediction.