Multiple Neural Network Models Research Articles

Lightweight Neural Networks-Based Safety Evaluation for Smart Construction Devices.

Based on the theory of lightweight neural networks, this paper presents a safety evaluation model for smart construction devices. The model index system includes the internal logical relationship between the input and output indexes, and the input indexes are specifically refined. According to the safety evaluation results, the article observes what type of accidents will occur at the construction site. According to the detailed and specific output index system, the six input factor layer indicators correspond to the indicators of several multiple network index layers, respectively. In the simulation process, MATLAB software was used to write the multiple neural network model program for the safety evaluation of the construction site, and the appropriate multiple network structure and related parameters were selected. The experimental results show that the multiple neural networks are trained by collecting 10 expert evaluation samples, and the trained multiple neural networks are applied to real construction cases. Comparing the two sets of data, it can be seen that the gap is relatively small, and the sample training is better. The multiple neural networks have relatively good evaluation performance. The method has a fast calculation speed and effectively improves the efficiency and practical value of safety evaluation.

Automatic Implementation Algorithm of Pressure Relief Drilling Depth Based on an Innovative Monitoring-While-Drilling Method.

An innovative monitoring-while-drilling method of pressure relief drilling was proposed in a previous study, and the periodic appearance of amplitude concentrated enlargement zone in vibration signals can represent the drilling depth. However, there is a lack of a high accuracy model to automatically identify the amplitude concentrated enlargement zone. So, in this study, a neural network model is put forward based on single-sensor and multi-sensor prediction results. The neural network model consists of one Deep Neural Network (DNN) and four Long Short-Term Memory (LSTM) networks. The accuracy is only 92.72% when only using single-sensor data for identification, while the proposed multiple neural network model could improve the accuracy to being greater than 97.00%. In addition, an optimization method was supplemented to eliminate some misjudgment due to data anomalies, which improved the final accuracy to the level of manual recognition. Finally, the research results solved the difficult problem of identifying the amplitude concentrated enlargement zone and provided the foundation for automatically identifying the drilling depth.

Which neural network makes more explainable decisions? An approach towards measuring explainability

Neural networks are getting increasingly popular thanks to their exceptional performance in solving many real-world problems. At the same time, they are shown to be vulnerable to attacks, difficult to debug and subject to fairness issues. To improve people’s trust in the technology, it is often necessary to provide some human-understandable explanation of neural networks’ decisions, e.g., why is that my loan application is rejected whereas hers is approved? That is, the stakeholder would be interested to minimize the chances of not being able to explain the decision consistently and would like to know how often and how easy it is to explain the decisions of a neural network before it is deployed. In this work, we provide two measurements on the decision explainability of neural networks. Afterwards, we develop algorithms for evaluating the measurements of user-provided neural networks automatically. We evaluate our approach on multiple neural network models trained on benchmark datasets. The results show that existing neural networks’ decisions often have low explainability according to our measurements. This is in line with the observation that adversarial samples can be easily generated through adversarial perturbation, which are often hard to explain. Our further experiments show that the decisions of the models trained with robust training are not necessarily easier to explain, whereas decisions of the models retrained with samples generated by our algorithms are easier to explain.

Evaluating Machine Learning and Remote Sensing in Monitoring NO2 Emission of Power Plants

Effective and precise monitoring is a prerequisite to control human emissions and slow disruptive climate change. To obtain the near-real-time status of power plant emissions, we built machine learning models and trained them on satellite observations (Sentinel 5), ground observed data (EPA eGRID), and meteorological observations (MERRA) to directly predict the NO2 emission rate of coal-fired power plants. A novel approach to preprocessing multiple data sources, coupled with multiple neural network models (RNN, LSTM), provided an automated way of predicting the number of emissions (NO2, SO2, CO, and others) produced by a single power plant. There are many challenges on overfitting and generalization to achieve a consistently accurate model simply depending on remote sensing data. This paper addresses the challenges using a combination of techniques, such as data washing, column shifting, feature sensitivity filtering, etc. It presents a groundbreaking case study on remotely monitoring global power plants from space in a cost-wise and timely manner to assist in tackling the worsening global climate.

Forecasting Day-Ahead Electricity Metrics with Artificial Neural Networks.

As artificial neural network architectures grow increasingly more efficient in time-series prediction tasks, their use for day-ahead electricity price and demand prediction, a task with very specific rules and highly volatile dataset values, grows more attractive. Without a standardized way to compare the efficiency of algorithms and methods for forecasting electricity metrics, it is hard to have a good sense of the strengths and weaknesses of each approach. In this paper, we create models in several neural network architectures for predicting the electricity price on the HUPX market and electricity load in Montenegro and compare them to multiple neural network models on the same basis (using the same dataset and metrics). The results show the promising efficiency of neural networks in general for the task of short-term prediction in the field, with methods combining fully connected layers and recurrent neural or temporal convolutional layers performing the best. The feature extraction power of convolutional layers shows very promising results and recommends the further exploration of temporal convolutional networks in the field.

Moisture-carryover performance optimization using physics-constrained machine learning

A data-driven model for predicting moisture carryover (MCO) in the General Electric Type-4 boiling water reactor (BWR) was constructed using a physics-constrained artificial intelligence technique. An accurate prediction of the MCO is of great value for commercial BWR operators as it can be used to modify the operational plan during a power cycle to mitigate high MCO, thereby avoiding elevated dose to on-site personnel and damage to turbine components. Using data from operational plants and preliminary features selected through physics and engineering analyses, a neural network based model for predicting MCO was built. A final feature set was then obtained through a hyperspace optimization performed using a genetic algorithm. Multiple neural network models possessing good generalization were obtained, the best of these having a mean-square error (MSE) of 9.69E-5 for prediction of an un-seen cycle, which is in agreement with the uncertainty in the measured MCO data. This predictive capability is of great value for the planning of a power generation cycle, and for scheduling of operations for cycles already underway.

Comparison of Multiple Linear Regression and Neural Network Models in Bank Performance Prediction in Botswana

Comparison of Multiple Linear Regression and Neural Network Models in Bank Performance Prediction in Botswana

GDP forecast of accommodation and catering industry in X city based on multi variable grey model —— Energy saving development of accommodation and catering industry

As the GDP of lodging and catering industry is generally quarterly or annual, the data volume is very small, so it is difficult to accurately predict the GDP of lodging and catering industry in this city for a city. However, a city can accurately predict the GDP development of the lodging and catering industry through electricity consumption. And the local government can formulate relevant policies to promote the stable development of lodging and catering industry and economy, and find the relationship between electricity consumption and GDP development of this industry, so as to achieve the purpose of low energy consumption and fast economic growth.t. This paper first conducted desensitization and normalization processing on the data, and then used Matlab to forecast the GDP of the lodging and catering industry in X City respectively by using GM (1, N) grey model, multiple linear regression model and BP neural network model. The experimental results show that the GM (1, N) grey model is more accurate and reliable than the other two methods in the case of less data. This paper solves the problem that the GDP of accommodation and catering industry in a single city is inaccurate.

Deep learning based network traffic matrix prediction

Network traffic matrix prediction is a methodology of predicting network traffic behavior ahead of time in order to improve network management and planning. Different neural network models ranging from simple recurrent neural network (RNN) to long short-term memory neural network (LSTM) and gated recurrent unit (GRU) are being used to predict traffic matrix. In this paper, for the first time the bidirectional LSTM (Bi-LSTM) and the bidirectional GRU (Bi-GRU) are applied to predict the network traffic matrix due to their high effectiveness and efficiency. The proposed models were designed as hybrid models that support multiple neural network models in a chained manner to support higher feature learning and subsequently higher accuracies in traffic matrix prediction. The hybrid models combined convolutional neural network (CNN) with either Bi-LSTM or Bi-GRU along with the unidirectional versions. With this approach, it gives the ability to eliminate unneeded information in order to obtain good data prediction. The comparisons of the proposed methods were applied on real traffic data from the GÉANT network. The results showed that the proposed models have a considerable improvement in prediction accuracy when compared to other existing models found in literature.

Accurate Recognition and Simulation of 3D Visual Image of Aerobics Movement

The structure of the deep artificial neural network is similar to the structure of the biological neural network, which can be well applied to the 3D visual image recognition of aerobics movements. A lot of results have been achieved by applying deep neural networks to the 3D visual image recognition of aerobics movements, but there are still many problems to be overcome. After analyzing the expression characteristics of the convolutional neural network model for the three-dimensional visual image characteristics of aerobics, this paper builds a convolutional neural network model. The model is improved on the basis of the traditional model and unifies the process of aerobics 3D visual image segmentation, target feature extraction, and target recognition. The convolutional neural network and the deep neural network based on autoencoder are designed and applied to aerobics action 3D visual image test set for recognition and comparison. We improve the accuracy of network recognition by adjusting the configuration parameters in the network model. The experimental results show that compared with other simple models, the model based on the improved AdaBoost algorithm can improve the final result significantly when the accuracy of each model is average. Therefore, the method can improve the recognition accuracy when multiple neural network models with general accuracy are obtained, thereby avoiding the complicated parameter adjustment process to obtain a single optimal network model.

Human activity recognition-based path planning for autonomous vehicles

Human activity recognition (HAR) is a wide research topic in a field of computer science. Improving HAR can lead to massive breakthrough in humanoid robotics, robots used in medicine and in the field of autonomous vehicles. The system that is able to recognise human and its activity without any errors and anomalies would lead to safer and more empathetic autonomous systems. During this research work, multiple neural networks models, with different complexity, are being investigated. Each model is re-trained on the proposed unique data set, gathered on automated guided vehicle (AGV) with the latest and the modest sensors used commonly on autonomous vehicles. The best model is picked out based on the final accuracy for action recognition. Best models pipeline is fused with YOLOv3, to enhance the human detection. In addition to pipeline improvement, multiple action direction estimation methods are proposed.

A neural network enhanced system for learning nonlinear constitutive law and failure initiation criterion of composites using indirectly measurable data

A neural network enhanced system containing a subsystem with one or multiple neural networks is proposed. Instead of defining the loss function as the direct output of a neural network model, the proposed method uses the system output, which can be measured from experiments, to define the loss function. The loss function is contributed by the outputs from one or multiple neural network models through a subsystem. As a result, the direct output of the ANN model is not required to be measurable from experiments. A set of new back-propagation equations have been derived for this system. Two examples are given using the proposed system: learning the nonlinear in-plane shear constitutive law and learning the failure initiation criterion of fiber-reinforced composites (FRC). The neural network models in both examples are trained at the lamina level using the measurable experimental responses of laminates. The results obtained from the learned neural network models agree well with the corresponding analytical solutions. The proposed method can be used to train neural network models in a subsystem when only the input and output of the system is measurable.

DARB: A Density-Adaptive Regular-Block Pruning for Deep Neural Networks

The rapidly growing parameter volume of deep neural networks (DNNs) hinders the artificial intelligence applications on resource constrained devices, such as mobile and wearable devices. Neural network pruning, as one of the mainstream model compression techniques, is under extensive study to reduce the model size and thus the amount of computation. And thereby, the state-of-the-art DNNs are able to be deployed on those devices with high runtime energy efficiency. In contrast to irregular pruning that incurs high index storage and decoding overhead, structured pruning techniques have been proposed as the promising solutions. However, prior studies on structured pruning tackle the problem mainly from the perspective of facilitating hardware implementation, without diving into the deep to analyze the characteristics of sparse neural networks. The neglect on the study of sparse neural networks causes inefficient trade-off between regularity and pruning ratio. Consequently, the potential of structurally pruning neural networks is not sufficiently mined.In this work, we examine the structural characteristics of the irregularly pruned weight matrices, such as the diverse redundancy of different rows, the sensitivity of different rows to pruning, and the position characteristics of retained weights. By leveraging the gained insights as a guidance, we first propose the novel block-max weight masking (BMWM) method, which can effectively retain the salient weights while imposing high regularity to the weight matrix. As a further optimization, we propose a density-adaptive regular-block (DARB) pruning that can effectively take advantage of the intrinsic characteristics of neural networks, and thereby outperform prior structured pruning work with high pruning ratio and decoding efficiency. Our experimental results show that DARB can achieve 13× to 25× pruning ratio, which are 2.8× to 4.3× improvements than the state-of-the-art counterparts on multiple neural network models and tasks. Moreover, DARB can achieve 14.3× decoding efficiency than block pruning with higher pruning ratio.

Research on Chinese medical named entity recognition based on collaborative cooperation of multiple neural network models.

Medical named entity recognition (NER) in Chinese electronic medical records (CEMRs) has drawn much research attention, and plays a vital prerequisite role for extracting high-value medical information. In 2018, China Health Information Processing Conference (CHIP2018) organized a medical NER academic competition aiming to extract three types of malignant tumor entity from CEMRs. Since the three types of entity are highly domain-specific and interdependency, extraction of them cannot be achieved with a single neural network model. Based on comprehensive study of the three types of entity and the entity interdependencies, we propose a collaborative cooperation of multiple neural network models based approach, which consists of two BiLSTM-CRF models and a CNN model. In order to tackle the problem that target scene dataset is small and entity distributions are sparse, we introduce non-target scene datasets and propose sentence-level neural network model transfer learning. Based on 30,000 real-world CEMRs, we pre-train medical domain-specific Chinese character embeddings with word2vec, GloVe and ELMo, and apply them to our approach respectively to validate effects of pre-trained language models in Chinese medical NER. Also, as control experiments, we apply Gated Recurrent Unit to our approach. Finally, our approach achieves an overall F1-score of 87.60%, which is the state-of-the-art performance to the best of our knowledge. In addition, our approach has won the champion of the medical NER academic competition organized by 2019 China Conference on Knowledge Graph and Semantic Computing, which proves the outstanding generalization ability of our approach.

Extracting Representative Images of Tourist Attractions from Flickr by Combining an Improved Cluster Method and Multiple Deep Learning Models

Extracting representative images of tourist attractions from geotagged photos is beneficial to many fields in tourist management, such as applications in touristic information systems. This task usually begins with clustering to extract tourist attractions from raw coordinates in geotagged photos. However, most existing cluster methods are limited in the accuracy and granularity of the places of interest, as well as in detecting distinct tags, due to its primary consideration of spatial relationships. After clustering, the challenge still exists for the task of extracting representative images within the geotagged base image data, because of the existence of noisy photos occupied by a large area proportion of humans and unrelated objects. In this paper, we propose a framework containing an improved cluster method and multiple neural network models to extract representative images of tourist attractions. We first propose a novel time- and user-constrained density-joinable cluster method (TU-DJ-Cluster), specific to photos with similar geotags to detect place-relevant tags. Then we merge and extend the clusters according to the similarity between pairs of tag embeddings, as trained from Word2Vec. Based on the clustering result, we filter noise images with Multilayer Perceptron and a single-shot multibox detector model, and further select representative images with the deep ranking model. We select Beijing as the study area. The quantitative and qualitative analysis, as well as the questionnaire results obtained from real-life tourists, demonstrate the effectiveness of this framework.

High-Quality Object Reconstruction From One-Dimensional Compressed Encrypted Signal Based on Multi-Network Mixed Learning

Conventional optical image encryption methods based on phase-shifting interferometry need at least two interferograms, and the storage or transmission of interferograms needs to occupy a lot of resources. At the same time, the low quality of reconstructed complex natural images has always been a main limiting factor in the application of optical image security. In this paper, a high-quality object reconstruction method from one-dimensional compressed encrypted signal based on multi-network mixed learning is proposed. First, an encrypted interferogram can be obtained using the double random phase encoding (DRPE) method. Then, we can obtain the one-dimensional compressed sampling signal of the encrypted hologram on the photodiode using single-pixel compressive holographic imaging method. Finally, the mapping of 1D signal to 2D object image can be learned utilizing multiple neural network models. Numerical simulation results show that the complex natural images can be reconstructed using the proposed method with high quality at lower sampling ratio.

Analysis of pressure distribution during movement for the top part of female socks

Female sports socks were studied to achieve the correlation between the ankle surface curvature and pressure distribution of the top part of socks. The transverse tension performance of the socks’ top part was obtained using an Instron universal strength tester, and the leg size was measured with a [TC]2 contactless 3D body scanner. The pressure was monitored by a Pliance-X-32 pressure test system. Gray correlation, variance, and regression analysis were applied to study the correlation between movement velocity, fabric performance, leg circumference, and ankle pressure distribution. The dynamic pressure prediction models of multiple regression and back propagation (BP) neural network on the top part of socks were also established. The results show that the transverse tension performance and sock density have a significant effect on the ankle static pressure. Movement velocity, sock density, and leg circumference are positively correlated with dynamic pressure, while the elastic recovery rate of the fabric is negatively correlated with the pressure. Both of the multiple regression and BP neural network models can predict the dynamic pressure, and the BP neural network model is better than multiple regression at prediction error, which was kept to less than 0.5%. Therefore, the BP neural network model can be effectively used in female ribbed sock top design.

Strength characteristics and stability analysis of GGBS stabilised fly ash-overburden dump

ABSTRACT Stability of OB dump slope has been a major concern and require a proper design for safe mining operation. Utilisation of fly ash with OB material for construction of mine haul road and external dumping, may provide an alternative mode of fly ash disposal. In this study, fly ash was mixed with OB material at different percentages to evaluate the compaction characteristics, California Bearing Ratio (CBR), Unconfined Compressive Strength (UCS) and shear strength parameter. Tests were also performed with the addition of a different percentage of Ground Granulated Blast Furnace Slag (GGBS) with fly ash-OB mix. Further, the effects of GGBS at various curing period on CBR and UCS characteristics were highlighted. Numerical modelling was carried out for dump containing OB material, fly ash and GGBS at different proportions to evaluate the Factor of Safety (FoS) and to suggest the optimum dump geometry. Experimental result infers that maximum CBR and UCS values were obtained for 78% OB material, 10% fly ash and 12% GGBS at 28 days curing period. Based on the observed FoS from numerical modelling, an optimal multiple regression model and artificial neural network model have been suggested for the direct prediction of FoS by various input parameters.

DeepLN: A framework for automatic lung nodule detection using multi-resolution CT screening images

Computed tomography (CT) is an important and valuable tool for detecting and diagnosing lung cancer at an early stage. Commonly, CT screenings with lower dose and resolution are used for preliminary screening. In particular, many hospitals in smaller towns only provide CT screenings at low resolution. However,when patients are diagnosed with suspected cancer, they are transferred or recommended to larger hospitals for more sophisticated examinations with high-resolution CT scans. Therefore, multi-resolution CT images deserve attention and are critical in clinical practice. Currently, the available open source datasets only contain high-resolution CT screening images. To address this problem, a multi-resolution CT screening image dataset called the DeepLNDataset is constructed. A three-level labeling criterion and a semi-automatic annotation system are presented to guarantee the correctness and efficiency of lung nodule annotation. Moreover, a novel framework called DeepLN is proposed to detect lung nodules in both low-resolution and high-resolution CT screening images. The multi-level features are extracted by a neural-network based detector to locate the lung nodules. Hard negative mining and a modified focal loss function are employed to solve the common category imbalance problem. A novel non-maximum suppression based ensemble strategy is proposed to synthesize the results from multiple neural network models trained on CT image datasets of different resolutions. To the best of our knowledge, this is the first work that considers the influence of multiple resolutions on lung nodule detection. The experimental results demonstrate that the proposed method can address this issue well.

Neural network-based construction of inverse kinematics model for serial redundant manipulators

Solving the inverse kinematics of redundant manipulators is difficult, because knowledge of the manipulators and their evaluation functions is required. To solve this problem, we propose a novel method of enabling a neural network model to learn the inverse kinematics. The method achieves learning independent of the structure of the evaluation function, by combining multiple neural network models. The method can obtain the neural network models of the inverse kinematics via an automatic calculation process using only training data, which consist of the postures, end-points, and evaluation values. In this paper, the algorithm used by the method and its background is explained, and the effectiveness of the method is validated by a numerical simulation.