Neural Network Model Research Articles

Accelerometer-Based Pavement Classification for Vehicle Dynamics Analysis Using Neural Networks

This research examines the influence of various pavement types on vehicle dynamics, specifically concentrating on vertical acceleration and its implications for unsprung mass, including the wheels and suspension system. The objective of this project was to categorize pavement types with accelerometer data, enabling a deeper comprehension of the impact of road surface conditions on vehicle stability, comfort, and mechanical stress. Two categorization methods were utilized: a neural network and a multinomial logistic regression model. Accelerometer data were gathered while a car navigated diverse terrain types, such as grates, potholes, and cobblestones. The neural network model exhibited exceptional performance, with 100% accuracy in categorizing all surface types, while the multinomial logistic regression model reached 97.14% accuracy. The neural network demonstrated exceptional efficacy in differentiating intricate surface types such as potholes and grates, surpassing the logistic regression model which had difficulties with these surfaces. These results underscore the neural network’s effectiveness in the real-time categorization of road surfaces, enhancing the comprehension of vehicle dynamics influenced by pavement conditions. Future studies must tackle the difficulty of identifying analogous surfaces by enhancing methodologies or integrating more data attributes for greater precision.

Evaluation of green development of energy consumption in the Pearl River Delta urban agglomeration based on radial basis function neural network

AbstractThe study utilizes the RBF neural network model to evaluate the green development of the Pearl River Delta urban agglomeration, revealing a consistent increase in development levels from 2005 to 2020 despite notable inter‐city variations. The model adeptly handles complex interactions within urban green development, offering insights into comprehensive impacts and aiding in policy evaluation and optimization. It provides urban planners and decision‐makers with data‐driven support, enhancing the scientific basis and stability of policy‐making.

Predicting removal of arsenic from groundwater by iron based filters using deep neural network models

Arsenic (As) contamination in drinking water has been highlighted for its environmental significance and potential health implications. Iron-based filters are cost-effective and sustainable solutions for As removal from contaminated water. Applying Machine Learning (ML) models to investigate and optimize As removal using iron-based filters is limited. The present study developed Deep Learning Neural Network (DLNN) models for predicting the removal of As and other contaminants by iron-based filters from groundwater. A small Original Dataset (ODS) consisting of 20 data points and 13 groundwater parameters was obtained from the field performances of 20 individual iron-amended ceramic filters. Cubic-spline interpolation (CSI) expanded the ODS, generating 1600 interpolated data points (IDPs) without duplication. The Bayesian optimization algorithm tuned the model hyper-parameters and IDPs in a Stratified fivefold Cross-Validation (CV) setup trained all the models. The models demonstrated reliable performances with the coefficient of determination (R2) 0.990–0.999 for As, 0.774–0.976 for Iron (Fe), 0.934–0.954 for Phosphorus (P), and 0.878–0.998 for predicting manganese (Mn) in the effluent. Sobol sensitivity analysis revealed that As (total order index (ST) = 0.563), P (ST = 0.441), Eh (ST = 0.712), and Temp (ST = 0.371) are the most sensitive parameters for the removal of As, Fe, P, and Mn. The comprehensive approach, from data expansion through DLNN model development, provides a valuable tool for estimating optimal As removal conditions from groundwater.

Time Prediction in Ship Block Manufacturing Based on Transfer Learning

Accurate time prediction is critical to the success of ship block manufacturing. However, the emergence of new ship types with limited historical data poses challenges to existing prediction methods. In response, this paper proposes a novel framework for ship block manufacturing time prediction, integrating clustering and the transfer learning algorithm. Firstly, the concept of distributed centroids was innovatively adopted to achieve the clustering of categorical attribute features. Secondly, abundant historical data from other types of blocks (source domain) were incorporated into the neural network model to explore the effects of block features on manufacturing time, and the model was further transferred to blocks with limited data (target domain). Leveraging the similarities and differences between source and target domain blocks, actions involving freezing and fine-tuning parameters were adopted for the predictive model development. Despite a small sample size of only 80, our proposed block time prediction method achieves an impressive mean absolute percentage error (MAPE) of 8.62%. In contrast, the MAPE for the predictive model without a transfer learning algorithm is notably higher at 14.97%. Experimental validation demonstrates the superior performance of our approach compared to alternative methods in scenarios with small sample datasets. This research addresses a critical gap in ship block manufacturing time prediction.

Wheat growth stage identification method based on multimodal data

Accurate identification of crop growth stages is a crucial basis for implementing effective cultivation management. With the development of deep learning techniques in image understanding, research on intelligent real-time recognition of crop growth stages based on RGB images has garnered significant attention. However, the small differences and high similarity in crop morphological characteristics during the transition between adjacent growth stages pose challenges for accurate identification. To address this issue, this study proposes a multi-scale convolutional neural network model, termed MultiScalNet-Wheat (MSN-W), which enhances the algorithm's ability to learn complex features by utilizing multi-scale convolution and attention mechanisms. This model extracts key information from redundant data to identify winter wheat growth stages in complex field environments. Experimental results show that the MSN-W model achieves a recognition accuracy of 97.6 %, outperforming typical convolutional neural network models such as VGG19, ResNet50, MobileNetV3, and DenseNet. To further address the difficulty in recognizing growth stages during transition periods, where canopy morphological features are highly similar and show small differences, this paper introduces an innovative approach by incorporating sequential environmental data related to wheat growth stages. By extracting these features and performing multi-modal collaborative inference, a multi-modal feature-based wheat growth stage recognition model, termed MultiModalNet-Wheat (MMN-W), is constructed on the basis of the MSN-W model. Experimental results indicate that the MMN-W model achieves a recognition accuracy of 98.5 %, improving by 0.9 % over the MSN-W model. Both the MSN-W and MMN-W models provide accurate methods for observing wheat growth stages, thereby supporting the scientific management of winter wheat at different growth stages.

Optimization of green extraction technologies for recovering bioactive compounds from Ixora coccinea waste flower biomass: A comparative response surface methodology and artificial neural network modeling

Red pigment from Ixora coccinea waste flower biomass was extracted using water as a cost-effective solvent for a sustainable source of organic color. The significant effect of selected variables on the method of extraction was studied for the green-extraction techniques, i.e., microwave-assisted extraction (MAE) and ultrasound-assisted extraction (UAE), and conventional technique, i.e., heat-assisted extraction (HAE). Optimization of the pigment extraction was examined using response surface methodology (RSM) for each extraction technique. The parameters considered for MAE were power (W), time (s), pH, and solvent-to-sample ratio (mL/g) while temperature (°C), time (min), pH, and solvent-to-sample ratio (mL/g) were studied for both UAE and HAE. A training set for an artificial neural network (ANN) was created using the same design obtained from RSM. The modeling techniques (ANN and RSM) were then statistically differentiated by the coefficient of determination (R2), root mean square error (RMSE), and mean square error (MSE) which revealed the superiority of the RSM regression design over the ANN model. It was found that the energy consumption of pigment extraction was minimal for MAE. The obtained pigment was further characterized using Fourier transform infrared (FTIR) spectroscopy to reveal functional groups and a gas chromatography-mass spectrometry (GC-MS) to define its bioactive constituents.

Model Parameter Prediction Method for Accelerating Distributed DNN Training

As the size of deep neural network (DNN) models and datasets increases, distributed training becomes popular to reduce the training time. However, a severe communication bottleneck in distributed training limits its scalability. Many methods aim to solve this communication bottleneck by reducing communication traffic, such as gradient sparsification and quantification. However, these methods either are at the expense of losing model accuracy or introducing lots of computing overhead. We observe that the data distribution between layers of neural network models is similar. Thus, we propose a model parameter prediction method (MP2) to accelerate distributed DNN training under parameter server (PS) framework, where workers push only a subset of model parameters to PS, and residual model parameters are locally predicted by an already-trained deep neural network model on PS. Some key challenges are addressed. First, we build a hierarchical parameters dataset by randomly sampling a subset of model from normal distributed trainings. Second, we build a neural network model with the structure of “convolution + channel attention + Max pooling” for predicting model parameters by using a prediction result-based evaluation method. For VGGNet, ResNet, and AlexNet models on CIFAR10 and CIFAR100 datasets, compared with Baseline, Top-k, deep gradient compression (DGC), and weight nowcaster network (WNN), MP2 can reduce traffic by up to 88.98%; and accelerates the training by up to 47.32% while not losing the model accuracy. MP2 has shown good generalization.

Hybrid real-time wave forecasting model combining Gaussian process regression and neural networks

To safely and effectively conduct offshore operations, a real-time wave forecast model that can accurately and rapidly forecast waves hours or days in advance is highly valuable. Recent studies have investigated models with neural networks (NNs) and have demonstrated their advantages in terms of forecasting speed and accuracy. However, NNs are limited by their requirement for a large training dataset, which can hinder the construction of the model if sufficient training data are unavailable. Therefore, this study constructed a hybrid real-time wave forecast model by combining an NN model and Gaussian process regression (GPR) model. The objective was to obtain reasonably accurate forecasts that were more accurate than those from individual NN models even when sufficient training data for the NN model was unavailable. To derive an optimal prediction, the values predicted by the NN and GPR models were combined by utilizing the standard deviation of the latter and considering the accuracy of each model. Consequently, predictions that were more accurate compared to those of the individual models for an entire year were obtained. This improvement was particularly significant for predicting relatively large wave heights, as indicated by the daily maximum wave heights. Additionally, various accuracy evaluation metrics demonstrated that despite using a small training dataset, the optimal prediction of the hybrid model was still superior to those of the individual models for predicting the daily maximum wave heights.

An approach for built-up area extraction using different indices and deep neural network (DNN) model

Mapping urbanization and built-up areas from remotely sensed data poses a formidable challenge, particularly when the spectral reflectance of built-up regions intersects with other land types. To address this, numerous spectral indices have been developed. This paper utilizes multiple indices: Normalized Difference Built-up Index (NDBI), Built-up Area Extraction Index (BAEI), Normalized Built-up Area Index (NBAI), New Built-up Index (NBI), Modified Built-up Index (MBI), Band Ratio for Built-up Area (BRBA), and Normalized Difference Vegetation Index (NDVI) to delineate built-up regions. An intersection approach between BAEI, NBAI, and NDVI refines the methodology, resulting in a final built-up map with 92.5 % accuracy and a 0.848 Kappa coefficient. Subsequently, a Deep Neural Network (DNN) model trained on this map achieves over 95 % accuracy in predicting built-up areas from Landsat 5 imagery, and the resultant built-up map achieved an overall accuracy of 92 % and a Kappa coefficient of 0.85. The proposed methodology demonstrates efficiency for time-series analysis and addresses misclassification in built-up areas. Moreover, the optimization of the DNN model proves effective when meticulous training and validation processes incorporate more precise sample datasets.

A novel automated neural network architecture search method of air target intent recognition

Modern air battlefield operations are characterized by flexibility and change, and the battlefield evolves rapidly and intricately. However, traditional air target intent recognition methods, which mainly rely on manually designed neural network models, find it difficult to maintain sustained and excellent performance in such a complex and changing environment. To address the problem of the adaptability of neural network models in complex environments, we propose a lightweight Transformer model (TransATIR) with a strong adaptive adjustment capability, based on the characteristics of air target intent recognition and the neural network architecture search technique. After conducting extensive experiments, it has been proved that TransATIR can efficiently extract the deep feature information from battlefield situation data by utilizing the neural architecture search algorithm, in order to quickly and accurately identify the real intention of the target. The experimental results indicate that TransATIR significantly improves recognition accuracy compared to the existing state-of-the-art methods, and also effectively reduces the computational complexity of the model.

Information-theoretic sensor placement for large sewer networks

Utility operators face a challenging task in managing wastewater networks to proactively enhance network monitoring. To address this issue, this paper develops a framework for optimized placing of sensors in sewer networks with the aim of maximizing the information obtained about the state of the network. To that end, mutual information is proposed as a measure of the evidence acquired about the state of the network by the placed sensors. The problem formulation leverages a stochastic description of the network states to analytically characterize the mutual information in the system and pose the sensor placement problem. To circumvent the combinatorial problem that arises in the placement configurations, we propose a new algorithm coined the one-step modified greedy algorithm, which employs the greedy heuristic for all possible initial sensor placements. This algorithm enables further exploration of solutions outside the initial greedy solution within a computationally tractable time. The algorithm is applied to two real sewer networks, the first is a sewer network in the South of England with 479 nodes and 567 links, and the second is the sewer network in Bellinge, a village in Denmark that contains 1020 nodes and 1015 links. Sensor placements from the modified greedy algorithm are validated by comparing their performance in estimating unmonitored locations against other heuristic placements using linear and neural network models. Results show the one-step modified greedy placements outperform others in most cases and tend to cluster sensors for efficiently monitoring parts of the network. The proposed framework and modified greedy algorithm provide wastewater utility operators with a sensor placement method that enables them to design the data acquisition and monitoring infrastructure for large networks.

An Efficient Method for Batch Derivation of Detached Eclipsing Binary Parameters: Analysis of 34,907 OGLE Systems

Detached eclipsing binary (EB) systems are crucial for measuring the physical properties of stars that evolve independently. Large-scale time-domain surveys have released a substantial number of light curves for detached EBs. Utilizing the Physics of Eclipsing Binaries package in conjunction with Markov Chain Monte Carlo (MCMC) methods for batch parameter derivation poses significant computational challenges, primarily due to the high computational cost and time demands. Therefore, this paper develops an efficient method based on the neural network model and the stochastic variational inference method (denoted NNSVI) for the rapid derivation of parameters for detached EBs. For studies involving more than three systems, the NNSVI method significantly outperforms techniques that combine MCMC methods in terms of parameter inference speed, making it highly suitable for the batch derivation of large numbers of light curves. We efficiently derived parameters for 34,907 detached EBs, selected from the Optical Gravitational Lensing Experiment catalog and located in the Galactic bulge, using the NNSVI method. A catalog detailing the parameters of these systems is provided. Additionally, we compared the parameters of two double-lined detached EBs with those from previous studies and found the estimated parameters to be essentially identical.

Integrating Data Imputation and Augmentation with Interpretable Machine Learning for Efficient Strength Prediction of Fly Ash-Based Alkali-Activated Concretes

Fly ash-based alkali-activated concrete (AAC) is renowned for its superior mechanical performance and sustainability, presenting an attractive alternative to traditional Portland cement concrete. Despite these advantages, the broad compositional range of AACs presents challenges in precisely tailoring material properties. In this context, machine learning (ML) offers promising prospects to streamline and fast-track the development of advanced materials design strategies by predicting mechanical properties from compositional variations. Effective ML model development, however, hinges on the availability of a comprehensive, high-quality dataset. Previous studies often relied on literature-derived datasets, which typically include outliers, noise, and missing values, potentially leading to biased predictions. Moreover, limited dataset sizes could undermine the robustness of the models. Traditional ML methods applied to AACs also tend to lack interpretability. To address these issues, this paper utilizes several data imputation methods and Generative Adversarial Networks (GANs) for data augmentation, effectively doubling the dataset size. Following this, ML algorithms such as Random Forest (RF), Extreme Gradient Boosting (XGBoost), and Neural Networks (NNs) are leveraged to predict compressive strength. The NN model, especially when enhanced by k-nearest neighbors (kNN) imputation (k = 5), demonstrated superior predictive accuracy compared to RF and XGBoost models. Further, SHAP (SHapley Additive exPlanations) analysis reveals key determinants of compressive strength, such as water content, SiO2, and curing conditions. Visualizations such as SHAP violin and river flow plots further elucidated feature contributions and property distributions. Overall, this study provides a robust framework for exploring composition-strength relationships in AACs, advancing the design of these environment-friendly materials.

A novel Taguchi-based approach for optimizing neural network architectures: application to elastic short fiber composites

This study presents an innovative application of the Taguchi design of experiment method to optimize the structure of an Artificial Neural Network (ANN) model for the prediction of elastic properties of short fiber reinforced composites. The main goal is to minimize the computational effort required for hyperparameter optimization while enhancing the prediction accuracy. By utilizing a robust experimental design framework, the structure of an ANN model is optimized. This approach involves identifying a combination of hyperparameters that provides optimal predictive accuracy with the fewest algorithmic runs, thereby significantly reducing the required computational effort. The results suggested that the Taguchi-based developed ANN model with three hidden layers, 20 neurons in each hidden layer, elu activation function, Adam optimizer, and a learning rate of 0.001 can predict the anisotropic elastic properties of short fiber reinforced composites with a prediction accuracy of 97.71%. Then, external validation of the proposed ANN model was done using experimental data, and differences of less than 10% were obtained, indicating an appropriate predictive performance of the proposed ANN algorithm. Our findings demonstrate that the Taguchi method not only streamlines the hyperparameter tuning process but also substantially improves the algorithm's performance. These results highlight the potential of the Taguchi method as a powerful tool for optimizing machine learning algorithms, especially in scenarios where computational resources are limited. The implications of this study are far-reaching, offering insights for future research in the optimization of different algorithms for improved accuracies and computational efficiencies.

Decoding core genes and intercellular communication in osteosarcoma: bioinformatic investigation and immune cell profiling for diagnostic and therapeutic insights

The study focusing on developing an artificial neural network (ANN) model in accordance with genetic characteristics of osteosarcoma (OS) to accurately speculate OS cases. In the present study, we identified 467 DEGs through differentially acting gene investigation and that 345 exist suppressed and 122 exist stimulated. The resultant of GO enrichment analysis displayed the functions mainly included T cell activation, secretory granule lumen, antioxidant property etc. The pathways identified in the differentially acting genes (DAGs) were greatly interacted with Phagosome, Staphylococcus aureus infection, Human T − cell leukemia virus 1 infection, etc. Next, we found out top ten hub DEGs (HDEGs) by PPI network analysis. In addition, through the validation of ANN itself and Test set samples, it was proved that the prediction performance of our constructed ANN model is accurate and reliable. Finally, the penetration of immune cells and its interaction with target CDEGs were examined, and variations in penetration of 22 types of immune cells amongst different classes were found, additionally correlation amongst immune cells and between immune cells and target CDEGs. Furthermore, we analyzed the expression of the top two CDEGs (YES1 and MFNG) in OS tissues and normal tissues, also the interrelationship among the activity of YES1 and MFNG in OS tissues and clinicopathological properties of OS cases. Furthermore, the correlation analysis between the top two CDEGs and immune infiltrating cells was performed in OS tissues. Our research results revealed that CDEGs-based ANN model is effective at predicting OS patients, which facilitates early diagnosis and treatment of OS.

Micromechanics and microstructure based machine learning approach: Unveiling the role of porosity and hydrated phases on the tensile behaviour of cement pastes

The fracture process and the mechanical performance of the cement paste rely significantly on its tensile behaviour. Machine learning methods have been recently developed to predict this behaviour based on numerous experimental measurements. Due to multi-phase heterogeneous nature of cement paste microstructure, evaluation of the role of each chemical phase on the cracking of cement paste is necessary to develop optimized binder systems. Numerous experimental tests would be necessary to determine the best cement pastes in a considered environment. Therefore, in this study, a hybrid approach based on an ensemble machine learning model and a microstructure informed micromechanical model is proposed for predicting the tensile behaviour of hydrated cement pastes. The model can be used as a quick prediction tool for tensile strength of a given microstructure or for deep analysis on the optimization of binder system. A Deep Neural Networks (DNN) model was developed by training and validating a large dataset detailing the chemical phase composition, phase volume fractions and the tensile strength of the cement pastes. This dataset covers an extensive range of Portland cements, hydration ages and water-to-cement ratios. It was developed by simulating cement thermodynamics and tensile behaviour using a damage micromechanical model; which is otherwise almost impossible to obtain through experimental means. Additionally, two generative algorithms, tabular generative adversial networks (TGAN) and conditional generative adversial networks (CTGAN), were employed for dataset augmentation as a sampling method for the sensitivity analysis. This machine learning study provides an in-depth investigation to identify the role of volume fraction of solid phases (C-S-H, ettringite, portlandite and hydrogarnet) and micro porosity (dry and filled) in affecting the tensile behaviour of hydrated cement paste. Optimization assessment of the microstructure was then performed using micromechanical analysis and damage propagation in hydrated cement paste phases and porous network.

Deep Learning Based Song Recommendation System Using Facial Expression And Heart Rate

In today’s music streaming landscape, personalized song recommendations are crucial for enhancing user experiences. This research study combines heart rate monitoring and facial expression detection to present a novel deep learning method for song suggestion. With a 92% accuracy rate, a Convolutional Neural Network (CNN) model is used to identify emotional states from facial expressions, and heart rate information adds more context to the user's emotional intensity. The large music library is matched to these emotional cues, guaranteeing that suggested songs correspond with the user's present emotional state. This approach provides a really interesting, emotionally stirring, and contextually appropriate music finding experience.

Comprehensive investigation of gas hold-up in a double coaxial mixer with shear-thinning fluids exhibiting yield stress: Experimental, numerical, and artificial neural network approaches

This study addresses the challenge of uneven gas dispersion in yield-stress, non-Newtonian fluids, commonly encountered in industries such as biopharmaceuticals, cosmetics, and food processing. While previous research demonstrated the advantages of dual coaxial mixers for pseudoplastic fluids, limited attention has been given to aerating yield-pseudoplastic fluids with higher aspect ratios. This study bridges that gap by investigating both local and global gas hold-up under various conditions, utilizing electrical resistance tomography and computational fluid dynamics. Key findings showed that increasing the anchor speed from stationary to 30 rpm significantly enhanced aeration efficiency (gas hold-up per specific power consumption), with improvements of 78 % in UP-CO mode and 25 % in UP-COUNTER mode at Nc = 350 rpm and Qg = 20 L/min. These results underscore enhanced gas dispersion under specific operating conditions, driving overall process intensification. To ensure accurate prediction of gas hold-up, both dimensional and dimensionless empirical correlations, along with an artificial neural networks (ANNs) model, were developed. The ANNs model exhibited superior accuracy, achieving R² values of 0.99 for both rotation modes, outperforming empirical models, which achieved R² values of 0.90 and 0.89 for UP-CO and UP-COUNTER modes, respectively.

A fuzzy activation function based zeroing neural network for dynamic Arnold Map image cryptography

As an effective method for time-varying problems solving, zeroing neural network (ZNN) has been frequently applied in science and engineering. In order to improve its performances in practical applications, a fuzzy activation function (FAF) is designed by introducing the fuzzy logic technology, and a fuzzy activation function based zeroing neural network (FAF-ZNN) model for fast solving time-varying matrix inversion (TVMI) is proposed. Rigorous mathematical analysis and comparative simulation experiments with other models guarantee its superior convergence and robustness to noises. In addition, based on the proposed FAF-ZNN model, a new dynamic Arnold map image cryptography algorithm is designed. Specifically, in the new dynamic image encryption, a dynamic key matrix is introduced, and the FAF-ZNN model is applied to fast compute the inversion of the dynamic key matrix for the dynamic Arnold map image cryptography decryption process. The effectiveness of the dynamic image encryption algorithm is verified by experiment results, which enhances the security of existing image encryption algorithms.

Optimizing catalyst layer composition of PEM fuel cell via machine learning: Insights from in-house experimental data

The catalyst layer (CL) is a pivotal component of Proton Exchange Membrane (PEM) fuel cells, exerting a significant impact on both performance and durability. Its ink composition can be succinctly characterized by platinum (Pt) loading, Pt/carbon ratio, and ionomer/carbon ratio. The amount of each substance within the CL must be meticulously balanced to achieve optimal operation. In this work, we apply an Artificial Neural Network (ANN) model to forecast the performance and durability of a PEM fuel cell based on its cathode CL composition. The model is trained and validated based on experimental data measured at our laboratories, which consist of data from 49 fuel cells, detailing their cathode CL composition, operating conditions, accelerated stress test conditions, polarization curves and ECSA measurements throughout their lifespan. The presented ANN model demonstrates exceptional reliability in predicting PEM fuel cell behavior for both beginning and end of life. This allows for a deeper understanding of the influence of each input on performance and durability. Furthermore, the model can be effectively applied to optimize the CL composition. This paper demonstrates the immense potential of AI, combined with a high-quality database, to advance fuel cell research.