Neural Network-based Model Research Articles

Invariance embedded physics-infused deep neural network-based sub-grid scale models for turbulent flows

In this paper, we present two novel physics-infused neural network (NN) architectures that satisfy various invariance conditions for constructing efficient and robust Deep Learning (DL)-based sub-grid scale (SGS) turbulence models for use in Large Eddy Simulation (LES) procedures widely used in various fluid engineering applications. The first architecture, called tensor basis neural networks (TBNN), recently proposed in the context of Reynolds-averaged Navier–Stokes (RANS) modeling, and introduced herein for SGS modeling of wall-bounded turbulence, embeds the analytical expansion of the SGS turbulence stresses into integrity basis tensors composed of the symmetric and anti-symmetric parts of the resolved velocity gradient tensor, thus incorporating the Galilean, rotational and reflectional invariances. In our second approach, a relatively simple yet powerful architecture, called Galilean Invariance embedded Neural Network (GINN) incorporates the Galilean invariance, and takes in as inputs the independent components of the integrity basis tensors in addition to the invariant inputs in a single input layer. Explicit filtering of data from direct simulations of the canonical channel flow at two friction Reynolds numbers Reτ≈395 and 590 provided accurate data for training and testing these models. Both sets of models are used to predict the SGS stresses for feature datasets generated with different filter widths, and at different Reynolds numbers. The GINN model yields less prediction error on test datasets (mean squared error ∼0.4) compared to the TBNN model (mean squared error ∼0.5). Upon comparison, it is revealed that GINN has better feature extraction capacity owing to its ability to establish relations between and extract information from cross-components of the integrity basis tensors and the SGS stresses. Based on their predictive performance, both models proposed herein have shown great promise for use in a posteriori actual LESs. The present work illustrates the significance of physics infusion as well as invariance embedding into NN architecture for constructing efficient and robust DL-based SGS turbulent models in achieving superior predictive efficacy.

Artificial neural network-based surface reconstruction model of wire-arc additively manufactured surfaces using discrete cosine transform

Artificial neural network-based surface reconstruction model of wire-arc additively manufactured surfaces using discrete cosine transform

WT-CNN: A Hybrid Machine Learning Model for Heart Disease Prediction

Heart disease remains a predominant health challenge, being the leading cause of death worldwide. According to the World Health Organization (WHO), cardiovascular diseases (CVDs) take an estimated 17.9 million lives each year, accounting for 32% of all global deaths. Thus, there is a global health concern necessitating accurate prediction models for timely intervention. Several data mining techniques are used by researchers to help healthcare professionals to predict heart disease. However, the traditional machine learning models for predicting heart disease often struggle with handling imbalanced datasets. Moreover, when prediction is on the bases of complex data like ECG, feature extraction and selecting the most pertinent features that accurately represent the underlying pathophysiological conditions without succumbing to overfitting is also a challenge. In this paper, a continuous wavelet transformation and convolutional neural network-based hybrid model abbreviated as WT-CNN is proposed. The key phases of WT-CNN are ECG data collection, preprocessing, RUSBoost-based data balancing, CWT-based feature extraction, and CNN-based final prediction. Through extensive experimentation and evaluation, the proposed model achieves an exceptional accuracy of 97.2% in predicting heart disease. The experimental results show that the approach improves classification accuracy compared to other classification approaches and that the presented model can be successfully used by healthcare professionals for predicting heart disease. Furthermore, this work can have a potential impact on improving heart disease prediction and ultimately enhancing patient lifestyle.

Limitations and issues of conventional artificial neural network-based surrogate models for building energy retrofit

Artificial neural network (ANN) based surrogate models have been widely used in place of high-fidelity simulation tools. Error metrics such as mean absolute error, and root mean squared error (RMSE) have been widely used as de facto criteria. However, whether the ANN-based surrogate model can adequately reproduce the interwoven relationships and nonlinear causalities between design variables and simulated outputs are often overlooked. In this regard, the authors designed a case study regarding four ANN-based surrogate models. It was found that despite all of the models having low RMSEs, the models failed to adequately predict the causal relationships between input variables and energy use. In other words, the surrogate models were not always capable of providing accurate assessments of expected energy use reduction as a result of design changes. In this paper, we present a workflow for validating whether a surrogate model can reproduce the causal relationships between inputs and outputs.

Research on the Prediction Method of Clock Tester Calibration Data Based on Radial Basis Function Neural Network

A radial basis function (RBF) neural network-based calibration data prediction model for clock testers is proposed to address the issues of fixed calibration cycles, low efficiency, and waste of electrical energy. This provides a new method for clock tester traceability calibration. First, analyze the mechanism of clock tester calibration parameters and the influencing factors of prediction targets. Based on the learning rules of an RBF neural network, determine the data types of training and testing sets. Second, normalize the training and testing data to avoid the adverse effects of data characteristics and distribution differences on the prediction model. Finally, based on different prediction objectives, time-driven and data-driven calibration data prediction models are constructed using RBF neural networks. Through simulation analysis, it is shown that an RBF neural network is superior to a BP neural network in predicting clock tester calibration data, and time-driven prediction accuracy is superior to data-driven prediction accuracy. Moreover, the prediction error and mean square error of both prediction models are on the order of 10−9, meeting the prediction accuracy requirements.

A Neural Network-based Method for Predicting Dose to Organs at Risk in Intensity-modulated Radiotherapy for Nasopharyngeal Carcinoma

ObjectiveA neural network method was used to establish a dose prediction model for organs at risk (OARs) during intensity-modulated radiotherapy (IMRT) for nasopharyngeal carcinoma (NPC). Materials and methodsIn total, 103 patients with NPC were randomly selected for IMRT. Suborgans were automatically generated for OARs using ring structures based on distance to the target using a MATLAB program and the corresponding volume of each suborgan was determined. The correlation between the volume of each suborgan and the dose to each OAR was analysed and neural network prediction models of the OAR dose were established using the MATLAB Neural Net Fitting application. The R-value and mean square error in the regression analysis were used to evaluate the prediction model. ResultsThe OAR dose was related to the volume of the corresponding sub-OAR. The average R-values for the normalised mean dose (Dnmean) to parallel organs and serial organs and the normalised maximum dose (Dn0) to serial organs in the training set were 0.880, 0.927 and 0.905, respectively. The mean square error for each OAR in the prediction model was low (ranging from 1.72 × 10−4 to 7.06 × 10−3). ConclusionThe neural network-based model for predicting OAR dose during IMRT for NPC is simple, reliable and worth further investigation and application.

Quantitative and Qualitative Analysis of Agricultural Fields Based on Aerial Multispectral Images Using Neural Networks.

This article presents an integrated system that uses the capabilities of unmanned aerial vehicles (UAVs) to perform a comprehensive crop analysis, combining qualitative and quantitative evaluations for efficient agricultural management. A convolutional neural network-based model, Detectron2, serves as the foundation for detecting and segmenting objects of interest in acquired aerial images. This model was trained on a dataset prepared using the COCO format, which features a variety of annotated objects. The system architecture comprises a frontend and a backend component. The frontend facilitates user interaction and annotation of objects on multispectral images. The backend involves image loading, project management, polygon handling, and multispectral image processing. For qualitative analysis, users can delineate regions of interest using polygons, which are then subjected to analysis using the Normalized Difference Vegetation Index (NDVI) or Optimized Soil Adjusted Vegetation Index (OSAVI). For quantitative analysis, the system deploys a pre-trained model capable of object detection, allowing for the counting and localization of specific objects, with a focus on young lettuce crops. The prediction quality of the model has been calculated using the AP (Average Precision) metric. The trained neural network exhibited robust performance in detecting objects, even within small images.

Assessing the impact of borehole field data on AI-based deep learning models for heating and cooling prediction

Sensor readings play a critical role in the prediction performance of AI-based data-driven fault-detection and diagnostics for borehole heat exchangers in heating and cooling systems. The accuracy and algorithm selection of individual deep neural networks are well-investigated research topics but the impact of quality of sensors data on algorithm selection and its further impact on prediction performance has not been investigated and quantified for geothermal heating/cooling systems. The objective of the current work is to investigate the impact of data quality, available as measured time-series data from the borehole field, on the prediction accuracy of the deep neural networks-based models. In the current work four different data sets (D-1, D-2, D-3, D-4) are used for data-driven modeling using four different deep machine learning approaches, i.e., LSTM, BD-LSTM, GRU and CNN. The developed AI models have been trained, validated, and tested using the real-field data sets with same number of independent input parameters i.e., inlet temperature and mass flow rate to predict the fluid outlet temperature. The validated AI models are used to predict the performance of borehole heat exchanger to meet the heating and cooling loads. The prediction results show that the BD-LSTM outperforms other deep neural networks in terms of accuracy. With the data sets D-1, D-2, D-3 and D-4 it was observed that BD-LSTM had lowest MAPE of 2.99 %, 0.22 %, 6.39 % and 2.21 % respectively. Several experiments show that the BD-LSTM and LSTM models are better than the other used approaches in terms of prediction accuracy, but larger number of trainable parameters results in more computation time than CNN and GRU respectively. More data points with higher sampling frequency, results in more accurate prediction as depicted by the results from D-2 and D-4. Although, the results showed that the data collection interval is crucial, but different test runs indicate that avoiding the loss of time-series data during the cleaning process is equally important to accurately capture the Seasonality and Trend of the heating/cooling data. This is realized in case of D-1 where almost 58 % of the values were removed as outliers and these missing values were needed to be approximated which also affected the model prediction accuracy. The presented analysis highlights that the LSTM based DNN model has the capability to predict borehole heat exchanger performance accurately and the predictions are highly dependent on data frequency, the quality as well as data quantity from geothermal installations used to train the model.

Graph neural network-based similarity relationship construction model for geospatial services

ABSTRACT During the development of service-based software systems, Geospatial Service (GS) replacement is often performed, which requires the discovery of functionally similar services in service registries to replace failed services. Compared to real-time similarity computations, direct extraction of similar services from constructed similarity relationships can yield higher replacement efficiency. However, missing and inconsistent service-registry information impedes accurate similarity relationship construction. Here, we propose a Graph Neural Network (GNN)-based model for GS Similarity Relationship construction considering service descriptions and tags, which is named GSSR-GNN. As the sparsity of the service similarity relationship graph constructed based on labeled samples limits the information propagation ability, a graph augmentation method for similarity relationship construction among second-order neighbors is proposed. Considering the differences in the semantic-information feature distributions, such as the service descriptions and tags, a feed-forward neural network-based fusion method is designed to embed them into the same vector space. Pre-trained Bidirectional Encoder Representations from Transformers (BERT) and WordNet models are introduced to enhance the service-representation expressiveness. When an enhanced service representation is input to the GNN, the similarity is calculated and the service similarity relationship is obtained. Experimental results show that the proposed model constructs service similarity relationships with high precision, thus improving the service replacement efficiency and reducing the computational cost of service registry during service replacement.

Meaning-Sensitive Text Data Augmentation with Intelligent Masking

With the recent popularity of applying large-scale deep neural network-based models for natural language processing (NLP), attention to develop methods for text data augmentation is at its peak, since the limited size of training data tends to significantly affect the accuracy of these models. To this end, we propose a novel text data augmentation technique called Intelligent Masking with Optimal Substitutions Text Data Augmentation (IMOSA). IMOSA, developed for labelled sentences, can identify the most favourable sentences and locate the appropriate word combinations in a particular sentence to replace and generate synthetic sentences with a meaning closer to the original sentence, while also significantly increasing the diversity of the dataset. We demonstrate that the proposed technique notably improves the performance of classifiers based on attention-based transformer models through the extensive experiments for five different text classification tasks which are performed under the low data regime in a context-aware NLP setting. The analysis clearly shows that IMOSA effectively generates more sentences using favourable original examples and completely ignores undesirable examples. Furthermore, the experiments carried out confirm IMOSA’s ability to add diversity to the augmented dataset using multiple distinct masking patterns against the same original sentence, which remarkably adds variety to the training dataset. IMOSA consistently outperforms the two key masked language model-based text data augmentation techniques, and demonstrates a robust performance against the critical challenging NLP tasks.

A Multisource Data Fusion-based Heterogeneous Graph Attention Network for Competitor Prediction

Competitor identification is an essential component of corporate strategy. With the rapid development of artificial intelligence, various data-mining methodologies and frameworks have emerged to identify competitors. In general, the competitiveness among companies is determined by both market commonality and resource similarity. However, because resource information is more difficult to obtain than market information, existing studies primarily identify competitors via market commonality. To address this limitation, we introduce multisource company descriptions as well as heterogeneous business relationships, and we propose a novel method for simultaneously mining the market commonality and resource similarity. First, we use multisource company descriptions to represent companies and transform the heterogeneous business relationships into a heterogeneous business network. Then, we propose a novel multisource data fusion-based heterogeneous graph attention network (MHGAT) to learn the pairwise competitive relationships between companies. Specifically, a graph neural network-based model is proposed to learn the embeddings of companies by preserving their competition, and a multilevel attention framework is designed to integrate the embeddings from neighboring company level, heterogeneous relationship level, and multisource description level. Finally, experiments on a real-world dataset verify the effectiveness of our proposed MHGAT and demonstrate the usefulness of company descriptions and business relationships in competitor identification.

A computer vision approach to continuously monitor fatigue during resistance training

Monitoring fatigue during resistance training is essential to avoid injuries caused by overtraining. Fatigue can be comprehensively quantified by the external and internal load, where the external load is the work done by the athlete, and the internal load is the psychological and physiological response to the external load. This paper proposes a computer vision method to continuously monitor fatigue during resistance training by predicting external and internal parameters, namely the generated power and the rating of perceived exertion. We utilize the human pose estimation from two Microsoft Azure Kinect cameras to capture the movement of athletes while performing stationary exercises. Our method processes the obtained kinematic data, computes skeleton features to train traditional machine learning algorithms, and constructs feature maps to train convolutional neural network-based models to predict the load parameters. For evaluation, we recorded a dataset of 16 subjects who performed squat exercises on a Flywheel and rated their perceived exertion after each set. A measuring unit integrated into the Flywheel provided power readings for each repetition. The results show that our method achieves good results in predicting both parameters. Gradient Boosting Regression Trees best predicted perceived exertion with a mean absolute percentage error of 8.08% and a Spearman’s ρ=0.74. Multi-layer Perceptron performed best in predicting power with a mean absolute error of 23.13 Watts and ρ=0.79. Our findings show that our approach delivers promising external and internal load quantifications for fatigue, with great potential to provide external feedback to coaches or athletes.

Abstract 16324: A Deep Learning Electrocardiogram Model Identifies Risk of Incident Heart Failure in Patients With Subclinical Hypothyroidism

Introduction: Subclinical hypothyroidism (SH) is associated with an increased risk of heart failure (HF). However, there are no established measures to predict HF development within the SH population. Hypothesis: Risk of incident heart failure in SH can be predicted from a 12-lead electrocardiogram using features extracted by a convolutional neural network-based model. Methods: To test the hypothesis, we developed a convolutional neural network (CNN)-based model to detect ECG features linked with low thyroid function by training to discriminate the presence of hypothyroidism from 10,745 ECGs (2,693 from patients with low free T4 (FT4) and 8,052 from age, sex-matched patients without evidence of low FT4). The model was applied to baseline ECGs from an external cohort of 2,293 patients with SH (defined TSH >4.5mIU/L and FT4 within 0.9-1.9ng/dl) to estimate the cardiac phenotype induced by hypothyroidism (ECG-hypothyroidism score). Cumulative incidences of HF admission were compared between groups stratified by the tertile of the model output (low, intermediate, and high score groups). Adjusted hazard ratios (aHR) were calculated using Cox proportional hazard model. Results: HF admission was more frequent in the high score group (9.60/100 person-year), compared with the intermediate or low score groups (5.08 and 2.29 per 100 person-year in the, respectively; Fig ). Subgroup analyses by sex and age group yielded similar results. The association between the ECG-hypothyroidism score and HF admission remained significant after adjustment for patient characteristics, conventional ECG abnormalities, and baseline thyroid function (high vs low score group: aHR 3.56, [95%CI, 2.26-5.61]; high vs intermediate score group: aHR 1.78, [95%CI, 1.25-2.54]). Conclusions: The CNN-based model, trained to detect prevalent hypothyroidism, accurately identified patients at high risk for HF admission within the SH population using baseline ECG.

Developing A Neural Network-Based Model for Identifying Medicinal Plant Leaves Using Image Recognition Techniques

Herbal plants contribute an important role in people's health and the environment, as they can provide both medical benefits and oxygen. Many herbal plants contain valuable therapeutic elements that can be passed down to future generations. Traditional methods of identifying plant species, such as manual measurement and examination of characteristics, are labor-intensive and time-consuming. To address this, there has been a push to develop more efficient methods using technology, such as digital image processing and pattern recognition techniques. The exact recognition of plants uses methodologies like computer vision and neural networks, which have been proposed earlier. This approach involves neural network models such as CNN, ALexnet, and ResNet for identifying the medical plants based on their respective features. Classification metrics give the 96.82 average accuracies. These results have been promising, and further research will involve using a larger dataset and going more into deep-learning neural networks to improve the accuracy of medicinal plant identification. It is hoped that a web or mobile-based system for automatic plant identification can help increase knowledge about medicinal plants, improve techniques for species recognition, and participate in the preservation of species that are considered ad endangered.

Research and Evaluation of the Diagnostic Precision of an Artificial Intelligence Algorithm Using Ultrasound Images for Early Diagnosis of Arthropathy in People with Hemophilia

Background: Frequent joint bleeding in people with hemophilia can cause synovitis and progressive joint damage, leading to reduced quality of life over the long term. Despite widespread use of prophylactic treatment, frequent joint bleeding is still prevalent. Early diagnosis of symptomatic/asymptomatic hemarthrosis and synovitis is required through regular assessment to ensure appropriate therapeutic intervention. Ultrasonography is a convenient diagnostic tool, but accurate diagnosis of hemarthrosis and synovitis requires specialist skills and experience, and so its implementation in daily clinical practice is not currently widespread. By constructing an artificial intelligence (AI) algorithm to support diagnosis of hemarthrosis and synovitis using ultrasonography, increased uptake of joint ultrasonography is expected. This study researches and evaluates an AI algorithm for detecting the presence or absence of hemarthrosis and synovitis using ultrasound images obtained in people with hemophilia. Methods: This study evaluated the diagnostic precision of AI model-based diagnoses in people with congenital hemophilia A or B. Ankle, elbow, and knee joint ultrasound images obtained at study sites between January 1, 2010 and March 31, 2022 were used to train and test six AI models for each joint to estimate the presence or absence of hemarthrosis and synovitis. The ultrasound images were stratified by presence or absence of hemarthrosis and synovitis, as determined by an Executive Committee consisting of a physician and two orthopedists who are experts in hemophilia care. Images were then randomly divided, with 70% used to train the AI models and 30% used to test their diagnostic precision. The AI algorithm used various image pre-processing methods and the convolutional neural network-based AI model. The primary endpoint was the area under the receiver operating characteristic curve (AUC) for the diagnostic precision of the AI model to diagnose hemarthrosis and synovitis. The other endpoints were examined using parameters such as the rate of accuracy, precision, recall, and specificity. In addition, to evaluate whether the lesion area was detected correctly or not by visualization techniques, we displayed the lesion area with region of interest and expressed the discrimination basis using a heat map ( Figure 1). Results: A total of 5649 images were collected from five study sites. Following assessment by the Study Executive Committee,3435 images were used in this analysis. The AUC for hemarthrosis detection for the ankle, elbow, and knee joints was 0.87, 0.91, and 0.91, respectively, while for synovitis detection it was 0.90, 0.97, and 0.91, respectively ( Table 1). When evaluated in subgroups according to age, AUCs for hemarthrosis detection for at the ankle, elbow, and knee joints, respectively, were: 0.85, 0.94, and 0.98 at 10-19 years old; 0.82, 0.91, and 0.84 at 20-29 years old; 0.91, 0.90, and 0.89 at 30-39 years old; 0.90, 0.93, and 0.93 at 40-49 years old; and 0.91, 0.93, and 0.95 at 50-59 years old. AUCs for synovitis detection for the ankle, elbow, and knee joints, respectively, for different age groups were 0.95, 0.92, and 0.94 at 10-19 years old; 0.89, 0.97, and 0.95 at 20-29 years old; 0.94, 1.00, and 0.91 at 30-39 years old; 0.84, 0.96, and 0.91 at 40-49 years old; and 0.90, 0.98, and 0.91 at 50-59 years old. The analysis of the hemarthrosis and synovitis algorithms by joint site indicated high accuracy, precision, recall, and specificity ( Table 1). Conclusions: Although the AI model produced high AUC values for detecting hemarthrosis and synovitis, further refinement is needed before it can be used in diagnosis. Additional research should determine why the AI model and physician results differed. This may be due to image brightness or AI training data quality/quantity. As joint shape changes greatly with growth, and the AI model uses the bone surface as a landmark, pediatric (<10 years old) images were excluded; the model has low discrimination accuracy if a bone surface cannot be detected. In future, features associated with joint growth should be incorporated into the algorithm to allow more accurate pediatric diagnosis. The future ability of an AI model to diagnose hemarthrosis and synovitis in clinical practice would support appropriate therapeutic intervention. This would be expected to help with the achievement of a healthy and active life for people with hemophilia.

Identification of glaucoma in fundus images utilizing gray wolf optimization with deep convolutional neural network-based resnet50 model

Identification of glaucoma in fundus images utilizing gray wolf optimization with deep convolutional neural network-based resnet50 model

Firms' multi-sided platform construction efforts and ESG performance: An information processing theory perspective

This research explores the impact of business-to-business (B2B) firms' efforts to construct technology-enabled multi-sided platforms (MSP efforts) on their ESG performance through an information processing theory (IPT) lens. Integrating the literature on digital transformation and MSPs, we delineate firms' MSP efforts along with three dimensions (digital operation efforts, technology sharing efforts, and flat management efforts) and posit that each addresses specific challenges associated with the environmental, social, and governance components of ESG performance. Using a sample of 213 publicly listed Chinese firms, encompassing 1527 firm-year observations, we measure the variation in the three dimensions of MSP efforts through a deep neural network-based latent Dirichlet allocation topic model to analyze their annual reports. We find that digital operation efforts enhance environmental, social, and governance performance; technology sharing efforts primarily improve environmental and social performance; and flat management efforts boost only governance performance. In addition, these effects are more pronounced for large firms than for small firms. Two robustness checks, employing patent applications and granted patents as alternative indicators of firms' MSP efforts, further corroborate these findings. This research augments both the MSP and ESG literature, offering actionable insights for firms, governments, and non-governmental organizations.

Modeling and analysis of vehicle path dispersion at signalized intersections using explainable backpropagation neural networks

The dispersion of vehicular paths is a common phenomenon in the inner area of signalized intersections due to heterogeneous driver behavior and interactions. This study aims to develop an explainable neural network-based model to describe the vehicle path dispersion by exploring the relationship between the path dispersion and external factors. A backpropagation neural network model was established to analyze the effects of external factors on the dispersion of through and left-turn paths based on real trajectory data collected from 20 intersections in Shanghai, China. Twelve influencing factors in varying geometric, traffic, signalization, and traffic management conditions were considered. The predictive power and transferability of the model were verified by applying the trained model on the four new intersections. The contributions of the influencing factors on the path dispersion were explored based on the neural interpretation diagram, relative importance of influencing factors, and sensitivity analysis to offer explanatory insights for the proposed model. The results show that the mean absolute percentage errors of the path dispersion models for the through and left-turn movements are only 14.67% and 17.65%, respectively. The through path dispersion is primarily influenced by the number of exit lanes, the offset degree between the approach and exit lanes, and the traffic saturation degree on the through lane. In contrast, the path dispersion of the left turn is mainly affected by the number of exit lanes, the left-turn angle, and the setting of guide lines.

Multi-Time Scale Optimal Dispatch for the Wind Power Integrated System With Demand Response of Data Centers Based on Neural Network-Based Model Predictive Control

Multi-Time Scale Optimal Dispatch for the Wind Power Integrated System With Demand Response of Data Centers Based on Neural Network-Based Model Predictive Control

GNR: A universal and efficient node ranking model for various tasks based on graph neural networks

Node ranking is an essential problem in complex networks, which aims to identify influential or central nodes in a graph. Existing methods for node ranking are either computationally expensive or suboptimal for different networks and multiple tasks. In this paper, we propose GNR, a graph neural network-based node ranking model that can sort nodes quickly and efficiently for any type of networks. To achieve this, GNR takes into account three factors to calculate the importance of nodes and train the model: degree, infection score, and dismantling score, which capture the local and global structure of the network and the effect of nodes on network propagation and dismantling. We conduct extensive experiments on 16 real networks, and compare GNR with other centrality-based node ranking models on three tasks: network dismantling, virus propagation, and information spreading. The results show that GNR performs better in most cases and validates the effectiveness and superiority of our model.