Trained Neural Network Model Research Articles

Edge Intelligence for Autonomous Driving in 6G Wireless System: Design Challenges and Solutions

In a level-5 autonomous driving system, the autonomous driving vehicles (AVs) are expected to sense the surroundings via analyzing a large amount of data captured by a variety of onboard sensors in near-real-time. As a result, enormous computing costs will be introduced to the AVs for processing the tasks with the deployed machine learning (ML) model, while the inference accuracy may not be guaranteed. In this context, the advent of edge intelligence (EI) and sixth-generation (6G) wireless networking are expected to pave the way to more reliable and safer autonomous driving by providing multi-access edge computing (MEC) together with ML to AVs in close proximity. To realize this goal, we propose a two-tier EI-empowered autonomous driving framework. In the autonomous-vehicles tier, the autonomous vehicles are deployed with the shallow layers by splitting the trained deep neural network model. In the edge-intelligence tier, an edge server is implemented with the remaining layers (also deep layers) and an appropriately trained multi-task learning (MTL) model. In particular, obtaining the optimal offloading strategy (including the binary offloading decision and the computational resources allocation) can be formulated as a mixed-integer nonlinear programming (MINLP) problem, which is solved via MTL in near-real-time with high accuracy. On another note, an edge-vehicle joint inference is proposed through neural network segmentation to achieve efficient online inference with data privacy-preserving and less communication delay. Experiments demonstrate the effectiveness of the proposed framework, and open research topics are finally listed.

Analysis on SPECT myocardial perfusion imaging with a tool derived from dynamic programming to deep learning

Our study tries to propose a method that can rapidly process and calibrate data sets through our previous dynamic programming (DP) tool, manually adjust error-heavy labels, and then train neural network models to accurately segment gated SPECT MPI images. Lastly, LV contours can be accurately determined on the gated SPECT MPI images and corresponding LV volumes can also be evaluated. First, we used our previous DP-based tool to coarsely determine the myocardial profiles on SPECT MPI images. In order to obtain a more accurate LV volumes and LVEF, it was also necessary to manually adjust the profiles that do not meet the clinical requirements. Then the results of manual adjustment were used as the ground truths to train the end-to-end full convolution neural networks by deep learning (DL). The method proposed in this paper was evaluated with the Dice similarity coefficient and Hausdorff distance, and the results showed that the proposed method had a high similarity with the ground truth. The LV volumes and LVEFs estimated by different methods were calculated separately and compared each other, and the experimental results showed that the performance of the proposed DL-based method was superior to that of the DP-based method and close to that of QGS program. As a quantitative analysis tool, the proposed DL-based method is fully automatic and more accurate in the evaluation of LV volumes and LVEFs, so it is expected that the DL-based method may have a wide application prospect in clinical practices.

The Use of Graph Databases for Artificial Neural Networks

Storing and using trained artificial neural network (ANN) models face technical difficulties. These models are usually stored as files and cannot be run directly. An artificial neural network can be structurally expressed as a graph. Therefore, it would be much more useful to store ANN models in a database and use the graph database as this database system. In this study, training and testing stages of ANN models are provided with software that will allow multiple researchers to conduct joint research on ANN models. The developed software platform is aimed to increase the representation power of the currently used methods by transferring the models developed in the popular ANN frameworks used today. With the study conducted, even someone who has started learning artificial neural network models from scratch will see the process and can visually develop their own model. When models are stored in the graph database, it will be easier to making versions and observing how the model grows. In addition, data to be input and output to the model can be stored in this database, also. In order to feed ANN models with input data and produce outputs, the graph database's own query language was used. This eliminates the dependency on another software library.

Prediction of activity cliffs on the basis of images using convolutional neural networks

An activity cliff (AC) is formed by a pair of structurally similar compounds with a large difference in potency. Accordingly, ACs reveal structure–activity relationship (SAR) discontinuity and provide SAR information for compound optimization. Herein, we have investigated the question if ACs could be predicted from image data. Therefore, pairs of structural analogs were extracted from different compound activity classes that formed or did not form ACs. From these compound pairs, consistently formatted images were generated. Image sets were used to train and test convolutional neural network (CNN) models to systematically distinguish between ACs and non-ACs. The CNN models were found to predict ACs with overall high accuracy, as assessed using alternative performance measures, hence establishing proof-of-principle. Moreover, gradient weights from convolutional layers were mapped to test compounds and identified characteristic structural features that contributed to successful predictions. Weight-based feature visualization revealed the ability of CNN models to learn chemistry from images at a high level of resolution and aided in the interpretation of model decisions with intrinsic black box character.

Physics-Driven Machine-Learning-Based Borehole Sonic Interpretation in the Presence of Casing and Drillpipe

SummaryInterpretation of sonic data acquired by a logging-while-drilling (LWD) tool or wireline tool in cased holes is complicated by the presence of drillpipe or casing because those steel pipes can act as a strong waveguide. Traditional solutions, which rely on using a frequency bandpass filter or waveform arrival-time separation to filter out the unwanted pipe mode, often fail when formation and pipe signals coexist in the same frequency band or arrival-time range. We hence developed a physics-driven machine-learning-based method to overcome the challenge. In this method, two synthetic databases are generated from a general root-finding mode-search routine on the basis of two assumed models: One is defined as a cemented cased hole for a wireline scenario, and the other is defined as a steel pipe immersed in a fluid-filled borehole for the logging-while-drilling scenario. The synthetic databases are used to train neural network models, which are first used to perform global sensitivity analysis on all relevant model parameters so that the influence of each parameter on the dipole dispersion data can be well understood. A least-squares inversion scheme using the trained model was developed and tested on synthetic cases. The scheme showed good results, and a reasonable uncertainty estimate was made for each parameter. We then extended the application of the trained model to develop a method for automated labeling and extraction of the dipole flexural dispersion mode from other disturbances. The method combines the clustering technique with the neural-network-model-based inversion and an adaptive filter. Testing on field data demonstrates that the new method is superior to traditional methods because it introduces a mechanism from which unwanted pipe mode can be physically filtered out.This novel physics-driven machine-learning-based method improved the interpretation of sonic dipole dispersion data to cope with the challenge brought by the existence of steel pipes. Unlike data-driven machine learning methods, it can provide global service with just one-time offline training. Compared with traditional methods, the new method is more accurate and reliable because the processing is confined by physical laws. This method is less dependent on input parameters; hence, a fully automated solution could be achieved.

Modeling Fused Filament Fabrication using Artificial Neural Networks

With industries pushing towards digitalized production, adaption to expectations and increasing requirements for modern applications, has brought additive manufacturing (AM) to the forefront of Industry 4.0. In fact, AM is a main accelerator for digital production with its possibilities in structural design, such as topology optimization, production flexibility, customization, product development, to name a few. Fused Filament Fabrication (FFF) is a widespread and practical tool for rapid prototyping that also demonstrates the importance of AM technologies through its accessibility to the general public by creating cost effective desktop solutions. An increasing integration of systems in an intelligent production environment also enables the generation of large-scale data to be used for process monitoring and process control. Deep learning as a form of artificial intelligence (AI) and more specifically, a method of machine learning (ML) is ideal for handling big data. This study uses a trained artificial neural network (ANN) model as a digital shadow to predict the force within the nozzle of an FFF printer using filament speed and nozzle temperatures as input data. After the ANN model was tested using data from a theoretical model it was implemented to predict the behavior using real-time printer data. For this purpose, an FFF printer was equipped with sensors that collect real time printer data during the printing process. The ANN model reflected the kinematics of melting and flow predicted by models currently available for various speeds of printing. The model allows for a deeper understanding of the influencing process parameters which ultimately results in the determination of the optimum combination of process speed and print quality.

Location of abnormal energy consumption and optimization of energy efficiency of hydraulic press considering uncertainty

As a high energy consumption machine, there is plenty of abnormal energy consumption in the operation of a hydraulic press, which leads to energy loss and reduces energy efficiency. The time series of energy consumption data are characterized by intense uncertainty, nonlinearity, and non–Gaussianity. The traditional method is used to locate it, which has the problems of a large error and uncertain locating result. Therefore, an abnormal energy consumption locating technique considering uncertainty is developed. To quantitatively evaluate the correlation between abnormal machine components and energy consumption, an uncertainty evaluation method combining entropy weight and fuzzy theory is proposed, which aims to provide a decision basis for the location of abnormal energy consumption. To accurately locate abnormal energy consumption, a novel localization technique is proposed, which is called genetic algorithm–based wavelet neural network. Firstly, the energy consumption data is processed by wavelet packet decomposition and reconstruction. The energy value is calculated, and then the dimension reduction is performed by principal component analysis. Finally, energy value is input into the genetic algorithm–based backpropagation neural network model for training and anomaly location. We integrate the proposed method into the energy management system and perform energy efficiency optimization analysis. The results show that the proposed method can improve the energy efficiency of a hydraulic press, reduce energy cost, and reduce environmental pollution. This provides a solution for solving the cleaner production problem of high energy consumption manufacturing industry and realizing sustainable intelligent manufacturing.

Clinicopathological features of fibrosarcomatous dermatofibrosarcoma protuberans and the construction of a back-propagation neural network recognition model

BackgroundFibrosarcomatous dermatofibrosarcoma protuberans (FS-DFSP) is a form of tumor progression of dermatofibrosarcoma protuberans (DFSP) with an increased risk of metastasis and recurrence. Few studies have compared the clinicopathological features of FS-DFSP and conventional DFSP (C-DFSP).ObjectivesTo better understand the epidemiological and clinicopathological characteristics of FS-DFSP.MethodsWe conducted a cohort study of 221 patients diagnosed with DFSP and built a recognition model with a back-propagation (BP) neural network for FS-DFSP.ResultsTwenty-six patients with FS-DFSP and 195 patients with C-DFSP were included. There were no differences between FS-DFSP and C-DFSP regarding age at presentation, age at diagnosis, sex, size at diagnosis, size at presentation, and tumor growth. The negative ratio of CD34 in FS-DFSP (11.5%) was significantly lower than that in C-DFSP (5.1%) (P = 0.005). The average Ki-67 index of FS-DFSP (18.1%) cases was significantly higher than that of C-DFSP (8.1%) cases (P < 0.001). The classification accuracy of the BP neural network model training samples was 100%. The correct rates of classification and misdiagnosis were 84.1% and 15.9%.ConclusionsThe clinical manifestations of FS-DFSP and C-DFSP are similar but have large differences in immunohistochemistry. The classification accuracy and feasibility of the BP neural network model are high in FS-DFSP.

A data-driven artificial neural network model for predicting wind load of buildings using GSM-CFD solver

Wind load prediction is critical to the design of buildings, especially skyscrapers, because wind-induced dynamic loads and vibrations can often be a major concern that dominates the design. This paper presents a real-time wind load predictor for buildings by training an artificial neural network, with big data generated from a well-established software package for computational fluid dynamics (CFD), known as GSM-CFD. The cross-section of a building is idealized as a rectangle with various aspect ratios. The attack angles and velocity of the wind are also treated as variables in a wide range for building design considerations. We firstly establish a large number of computational models of major possible situations, and intensive computations using the GSM-CFD solver are conducted to generate detailed flow fields. Due to the high Reynold numbers of the flow fields, large eddy simulation (LES) model has been adopted in the calculation process. Both drag and lift coefficients are then computed based on the pressure distribution around the surface of the rectangular blocks. The GSM-CFD enables us to generate a mass of data considering different aspect ratios, wind velocities and attacking angles. Finally, an artificial neural network is trained to predict the drag coefficients for any given aspect ratios of the rectangular cross-sections of building subject to any given attack angles of wind with various velocities. The trained neural network is verified by comparing the results of the neural network predictions and that of the numerical simulations. The accuracy based on the test data set is found about 3.17%, which is sufficient for engineering design purposes. Our trained neural network model is applicable to other problems with a flow around rectangles with aspect ratios from 0.2 to 1, air-flow velocity of 34 m/s, and attack angles from 0° to 90°. The prediction is practically in real-time.

Deep Learning Denoising Applied to Regional Distance Seismic Data in Utah

ABSTRACTSeismic waveform data are generally contaminated by noise from various sources. Suppressing this noise effectively so that the remaining signal of interest can be successfully exploited remains a fundamental problem for the seismological community. To date, the most common noise suppression methods have been based on frequency filtering. These methods, however, are less effective when the signal of interest and noise share similar frequency bands. Inspired by source separation studies in the field of music information retrieval (Jansson et al., 2017) and a recent study in seismology (Zhu et al., 2019), we implemented a seismic denoising method that uses a trained deep convolutional neural network (CNN) model to decompose an input waveform into a signal of interest and noise. In our approach, the CNN provides a signal mask and a noise mask for an input signal. The short-time Fourier transform (STFT) of the estimated signal is obtained by multiplying the signal mask with the STFT of the input signal. To build and test the denoiser, we used carefully compiled signal and noise datasets of seismograms recorded by the University of Utah Seismograph Stations network. Results of test runs involving more than 9000 constructed waveforms suggest that on average the denoiser improves the signal-to-noise ratios (SNRs) by ∼5 dB, and that most of the recovered signal waveforms have high similarity with respect to the target waveforms (average correlation coefficient of ∼0.80) and suffer little distortion. Application to real data suggests that our denoiser achieves on average a factor of up to ∼2–5 improvement in SNR over band-pass filtering and can suppress many types of noise that band-pass filtering cannot. For individual waveforms, the improvement can be as high as ∼15 dB.

Crack Identification on the Fresh Chilli (Capsicum) Fruit Destemmed System

Destemming fresh chilli fruit (Capsicum) in large productivity is necessary, especially in the Mekong Delta region. Several studies have been done to solve this problem with high applicability, but a certain percentage of the output consisted of cracked fruits, thus reducing the quality of the system. The manual sorting results in high costs and low quality, so it is necessary that automatic grading is performed after destemming. This research focused on developing a method to identify and classify cracked chilli fruits caused by the destemming process. The convolution neural network (CNN) model was built and trained to identify cracks; then, appropriate control signals were sent to the actuator for classification. Image processing operations are supported by the OpenCV library, while the TensorFlow data structure is used as a database and the Keras application programming interface supports the construction and training of neural network models. Experiments were carried out in both the static and working conditions, which, respectively, achieved an accurate identification rate of 97 and 95.3%. In addition, a success rate of 93% was found even when the chilli body is wrinkled due to drying after storage time at 120 hours. Practical results demonstrate that the reliability of the model was useful and acceptable.

Research on Defect Recognition of Lithium Battery Pole Piece Based on Deep Learning

In the field of defect recognition, deep learning technology has the advantages of strong generalization and high accuracy compared with mainstream machine learning technology. This paper proposes a deep learning network model, which first processes the self-made 3, 600 data sets, and then sends them to the built convolutional neural network model for training. The final result can effectively identify the three defects of lithium battery pole pieces. The accuracy rate is 92%. Compared with the structure of the AlexNet model, the model proposed in this paper has higher accuracy.

Confirming the Buzz about Hornets

With the occurrence of many Vespa mandarinia sightings, Washington has established various channels to encourage people to report Vespa mandarinia. This paper uses the Time Series to predict the latitude and longitude, and verify the prediction through Neural Network and Linear Regression Fitting the accuracy of the result. The relative error of bringing the prediction data into the Neural Network training model is 8.73%. Using SVM model in this paper, unverified date training and analysis were carried out on the original data, and the analysis results showed that the accuracy was 89.95% and the recall rate was 93%, demonstrating the good matching degree of the model. Finally using a Decision Tree Classification Model calculate the total weight corresponding to each unverified data through PYTHON programming.

Detecting Pronunciation Errors in Spoken English Tests Based on Multifeature Fusion Algorithm

In this study, multidimensional feature extraction is performed on the U‐language recordings of the test takers, and these features are evaluated separately, with five categories of features: pronunciation, fluency, vocabulary, grammar, and semantics. A deep neural network model is constructed to model the feature values to obtain the final score. Based on the previous research, this study uses a deep neural network training model instead of linear regression to improve the correlation between model score and expert score. The method of using word frequency for semantic scoring is replaced by the LDA topic model for semantic analysis, which eliminates the need for experts to manually label keywords before scoring and truly automates the critique. Also, this paper introduces text cleaning after speech recognition and deep learning‐based speech noise reduction technology in the scoring model, which improves the accuracy of speech recognition and the overall accuracy of the scoring model. Also, innovative applications and improvements are made to key technologies, and the latest technical solutions are integrated and improved. A new open oral grading model is proposed and implemented, and innovations are made in the method of speech feature extraction to improve the dimensionality of open oral grading.

Evaluation of Germany’s Vulnerability Based on Economic Principles and Data Science

Vulnerability refers to the ability of a country or a region to resist internal and external natural factors such as ecological environment, economy, and society during its development. Germany is a country with low overall vulnerability and distinct regional differences, so research on its regional vulnerability can be a representative case for developing countries, as it provides a comprehensive assessment of regional vulnerability via scientific methodology and at the same time proposes rational solutions. The research collects quarterly data of 16 states of Germany from 2000 to 2015. This study describes a series of features of the data and establishes a comprehensive assessment of regional vulnerability including 33 indicators. Combined with the multi-criteria decision analysis method (MCDM), the analytic hierarchy process (AHP) method and the entropy method are applied to calculate the weights. A linear weighted sum method is applied to obtain the regional vulnerability index of Germany. Afterwards, by performing regression tests, this study empirically assess the influencing factors of the regional vulnerability of Germany. Moreover, this study adopts the neural network training model and forecasts the regional vulnerability of Germany of 2016 to 2020. This study identifies the main factors that influence the regional vulnerability of Germany, and proposes policy implications on the overall regulation to reduce the vulnerability of different regions in Germany accordingly.

Applying Artificial Intelligence Methods for the Estimation of Disease Incidence: The Utility of Language Models.

Background: AI-driven digital health tools often rely on estimates of disease incidence or prevalence, but obtaining these estimates is costly and time-consuming. We explored the use of machine learning models that leverage contextual information about diseases from unstructured text, to estimate disease incidence.Methods: We used a class of machine learning models, called language models, to extract contextual information relating to disease incidence. We evaluated three different language models: BioBERT, Global Vectors for Word Representation (GloVe), and the Universal Sentence Encoder (USE), as well as an approach which uses all jointly. The output of these models is a mathematical representation of the underlying data, known as “embeddings.” We used these to train neural network models to predict disease incidence. The neural networks were trained and validated using data from the Global Burden of Disease study, and tested using independent data sourced from the epidemiological literature.Findings: A variety of language models can be used to encode contextual information of diseases. We found that, on average, BioBERT embeddings were the best for disease names across multiple tasks. In particular, BioBERT was the best performing model when predicting specific disease-country pairs, whilst a fusion model combining BioBERT, GloVe, and USE performed best on average when predicting disease incidence in unseen countries. We also found that GloVe embeddings performed better than BioBERT embeddings when applied to country names. However, we also noticed that the models were limited in view of predicting previously unseen diseases. Further limitations were also observed with substantial variations across age groups and notably lower performance for diseases that are highly dependent on location and climate.Interpretation: We demonstrate that context-aware machine learning models can be used for estimating disease incidence. This method is quicker to implement than traditional epidemiological approaches. We therefore suggest it complements existing modeling efforts, where data is required more rapidly or at larger scale. This may particularly benefit AI-driven digital health products where the data will undergo further processing and a validated approximation of the disease incidence is adequate.

Design of HL-2A plasma position predictive model based on deep learning

In tokamak discharge experiments, the plasma position prediction model’s research is to understand the law of plasma motion and verify the correctness of the plasma position controller design. Although Maxwell equations can completely describe plasma movement, obtaining an accurate physical model for predicting plasma behavior is still challenging. This paper describes a deep neural network model that can accurately predict the HL-2A plasma position. That is a hybrid neural network model based on a long short-term memory network. We introduce the topology, training parameter setting, and prediction result analysis of this model in detail. The test results show that a trained deep neural network model has high prediction accuracy for plasma vertical and horizontal displacements.

A fusion of salient and convolutional features applying healthy templates for MRI brain tumor segmentation

This paper proposes an improved brain tumor segmentation method based on visual saliency features on MRI image volumes. The proposed method introduces a novel combination of multiple MRI modalities used as pseudo-color channels for highlighting the potential tumors. The novel pseudo-color model incorporates healthy templates generated from the MRI slices without tumors. The constructed healthy templates are also used during the training of neural network models. Based on a saliency map built using the pseudo-color templates, combination models are proposed, fusing the saliency map with convolutional neural networks’ prediction maps to improve predictions and to reduce the networks’ eventual overfitting which may result in weaker predictions for previously unseen cases. By introducing the combination technique for deep learning techniques and saliency-based, handcrafted feature models, the fusion approach shows good abstraction capabilities and it is able to handle diverse cases that the networks were less trained for. The proposed methods were tested on the BRATS2015 and BRATS2018 databases, and the quantitative results show that hybrid models (including both trained and handcrafted features) can be promising alternatives for reaching higher segmentation performance. Moreover, healthy templates can provide additional information for the training process, enhancing the prediction performance of neural network models.

SCrop: A Novel Device for Sustainable Automatic Disease Prediction, Crop Selection, and Irrigation in Internet-of-Agro-Things for Smart Agriculture

Agriculture Cyber-Physical System (A-CPS) is becoming increasingly important in enhancing crop quality and productivity by utilizing minimum cropland. This paper introduces the innovative idea of the Internet-of-Agro-Things (IoAT) with an explanation of the automatic detection of plant disease for the development of ACPS. Majority of the crops were infected by microbial diseases in conventional agriculture. Also, the constantly mutating pathogens cannot be known to the knowledge of the farmer, due to which, there arises a demand to develop a disease prediction system. To prevent this, we use a trained Convolutional Neural Network (CNN) model to perform an analysis of the crop image captured by a health maintenance system. The image capturing along with continuous sensing and intelligent automation is performed by the solar sensor node. The sensor node houses a developed soil moisture sensor which has a high longevity compared to its peers. A real time implementation of the proposed system is demonstrated using a solar sensor node with a camera module, a microcontroller and a smartphone application using which a farmer can monitor the field. The prototype was deployed for three months and has achieved a robust performance by remaining rust-free and sustaining the varied weather conditions. An accuracy of 99.24% is achieved by the proposed plant disease prediction framework.

Push the Limit of Acoustic Gesture Recognition

With the flourish of the smart devices and their applications, controlling devices using gestures has attracted increasing attention for ubiquitous sensing and interaction. Recent works use acoustic signals to track hand movement and recognize gestures. However, they suffer from low robustness due to frequency selective fading, interference and insufficient training data. In this work, we propose RobuCIR, a robust contact-free gesture recognition system that can work under different practical impact factors with high accuracy and robustness. RobuCIR adopts frequency-hopping mechanism to mitigate frequency selective fading and avoid signal interference. To further increase system robustness, we investigate a series of data augmentation techniques based on a small volume of collected data to emulate different practical impact factors. The augmented data is used to effectively train neural network models and cope with various influential factors (e.g., gesture speed, distance to transceiver, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">etc</i> .). Our experiment results show that RobuCIR can recognize 15 gestures and outperform state-of-the-art works in terms of accuracy and robustness.