Kind Of Deep Neural Network Research Articles

Prediction of thermo-mechanical performance for effusion cooling by machine learning method

Fast and accurate evaluation of performance of cooling structures is very critical to the design of hot-section components in gas turbines. In the present study, machine learning method was developed to predict thermo-mechanical performance for effusion cooling on flat plate. The geometric and thermodynamic parameters including blowing ratio, density ratio, hole inclination angle, hole pitch and spacing were studied. Latin hypercube sampling was used for the design of numerical experiments, and the data sets for training model were generated by computational fluid dynamics (CFD) solutions. A kind of deep neural networks (DNNs) was used to predict the distributions of solid temperature and stress. Based on the outputs of neural network, the effects of the geometric and thermodynamic parameters on thermo-mechanical performance were investigated in detail. The results show that thermal stress decreases with the increases of density ratio and blowing ratio, while the increases of hole spacing and pitch cause the increase of thermal stress. Compared with the CFD results, the proposed DNN method has mean absolute percent error of 0.5% and 0.08% and root mean square error of 6.95 K and 5.1 MPa as applying for predicting solid temperature and stress. On the whole, machine learning is a promising method for modelling effusion cooling with high accuracy.

CNN based deep learning methods for precise analysis of cardiac arrhythmias

In contemporary day, Deep Learning (DL) is a developing discipline in the science of Machine Learning (ML) (ML). The research in this field is evolving extremely fast and its consequence leads to breakthrough in advance technology. Deep learning approaches are meant to gradually learn characteristics from several layers by adopting a general purpose learning mechanism, without relying on the human built features. This enables the system to learn the complicated functions and translate the input to the output straight from the data. This study effort primarily focuses on emphasising the Convolutional Neural Networks (CNNs), a kind of Deep Neural Networks (DNNs) and develop an 11 layered CNN for effective ECG arrhythmia classification. In this study the relevance of CNNs, the major building blocks and layers are explored, the design of the suggested CNN model is described.

Adjusted EfficientNet for the diagnostic of orbital angular momentum spectrum.

Orbital angular momentum (OAM) is one of multiple dimensions of beams. A beam can carry multiple OAM components, and their intensity weights form the OAM spectrum. The OAM spectrum determines complex amplitude distributions of a beam and features unique characteristics. Thus, measuring the OAM spectrum is of great significance, especially for OAM-based applications. Here we employ a deep neural network combined with a phase-only diffraction optical element to measure the OAM spectrum. The diffraction optical element is designed to diffract incident beams into distinct patterns corresponding to OAM distributions. Then, the EfficientNet, a kind of deep neural network, is adjusted to adapt and analyze the diffraction pattern to calculate the OAM spectrum. The favorable experimental results show that our proposal can reconstruct the OAM spectra with high precision and speed, works well for different numbers of OAM channels, and is also robust to Gaussian noise and random zooming. This work opens a new, to the best of our knowledge, ability for OAM spectrum recognition and will find applications in a number of advanced domains including large capacity optical communications, quantum key distribution, optical trapping, rotation detection, and so on.

Probing Higgs boson exotic decays at the LHC with machine learning

We study the tagging of Higgs exotic decay signals using different types of deep neural networks (DNNs), focusing on the $W^\pm h$ associated production channel followed by Higgs decaying into $n$ $b$-quarks with $n=4$, 6 and 8. All the Higgs decay products are collected into a fat-jet, to which we apply further selection using the DNNs. Three kinds of DNNs are considered, namely convolutional neural network (CNN), recursive neural network (RecNN) and particle flow network (PFN). The PFN can achieve the best performance because its structure allows enfolding more information in addition to the four-momentums of the jet constituents, such as particle ID and tracks parameters. Using the PFN as an example, we verify that it can serve as an efficient tagger even though it is trained on a different event topology with different $b$-multiplicity from the actual signal. The projected sensitivity to the branching ratio of Higgs decaying into $n$ $b$-quarks at the HL-LHC are 10\%, 3\% and 1\%, for $n=4$, 6 and 8, respectively.

Deep Learning of the Eddington Tensor in Core-collapse Supernova Simulation

Abstract We trained deep neural networks (DNNs) as a function of the neutrino energy density, flux, and the fluid velocity to reproduce the Eddington tensor for neutrinos obtained in our first-principles core-collapse supernova simulation. Although the moment method, which is one of the most popular approximations for neutrino transport, requires a closure relation, none of the analytical closure relations commonly employed in the literature capture all aspects of the neutrino angular distribution in momentum space. In this paper, we develop a closure relation by using DNNs that take the neutrino energy density, flux, and the fluid velocity as the inputs and the Eddington tensor as the output. We consider two kinds of DNNs: a conventional DNN, named a component-wise neural network (CWNN), and a tensor-basis neural network (TBNN). We find that the diagonal component of the Eddington tensor is better reproduced by the DNNs than the M1 closure relation, especially for low to intermediate energies. For the off-diagonal component, the DNNs agree better with the Boltzmann solver than the M1 closure relation at large radii. In the comparison between the two DNNs, the TBNN displays slightly better performance than the CWNN. With these new closure relations at hand, based on DNNs that well reproduce the Eddington tensor at much lower costs, we have opened up a new possibility for the moment method.

Research on the Extraction Method of Book Number Region Based on Bayesian Optimization and Deep Learning

The continuous development of artificial intelligence technology has promoted the construction of smart libraries and their intelligent services. In the process of intelligent access to books, the extraction of the requested book number region has become an important part of the process. The requested book number is generally affixed to the bottom of the spine of the book, which is small in size, and the height of the book is not always the same, so it’s difficult to identify. By the way, due to the images’ resolution, shooting angle and other practical problems, the difficulty of the extraction work will be increased. To improve the identification accuracy, in this paper, Bayesian Optimization (BO) and one kind of deep neural networks ‘Faster R-CNN’ are combined for the extraction work mentioned above. The data preparation, network training, optimization variable selection, establishment of BO objective function, optimization training, and network parameter evaluation have been introduced in detail. The performance of the designed algorithm has been tested with actual images of book spines taken in the academy library and compared with several other conventional recognition algorithms. The experimental results show that the requested book number region extraction method based on Bayesian optimization and deep neural network is effective and reliable, and its recognition rate can reach 91.82%, which has advantages in both recognition rate and extraction time compared with other algorithms.

Analysis of Deep Transfer Learning Using DeepConvLSTM for Human Activity Recognition from Wearable Sensors

Human Activity Recognition (HAR) from wearable sensors has gained significant attention in the last few decades, largely because of the potential healthcare benefits. For many years, HAR was done using classical machine learning approaches that require extraction of features. With the resurgence of deep learning, a major shift happened and at the moment, HAR researchers are mainly investigating different kinds of deep neural networks. However, deep learning comes with the challenge of having access to large amounts of labeled examples, which in the field of HAR is considered an expensive task, both in terms of time and effort. Another challenge is the fact that the training and testing data in HAR can be different due to the personal preferences of different people when performing the same activity. In order to try and mitigate these problems, in this paper we explore transfer learning, a paradigm for transferring knowledge from a source domain, to another related target domain. More specifically, we explore the effects of transferring knowledge between two open-source datasets, the Opportunity and JSI-FOS datasets, using weight-transfer for the DeepConvLSTM architecture. We also explore the performance of this transfer at different amounts of labeled data from the target domain. The experiments showed that it is beneficial to transfer the weights of fewer layers, and that deep transfer learning can perform better than a domain-specific deep end-to-end model in specific circumstances. Finally, we show that deep transfer learning is a viable alternative to classical machine learning approaches as it produces comparable results and does not require feature extraction.

WITHDRAWN: Role of convolutional neural networks for any real time image classification, recognition and analysis

Abstract Convolutional neural networks in short form often called as CNN is a kind of Deep neural networks, it is the part of Deep Learning, based on their features Convolutional neural networks are also called as space invariant artificial neural networks. The Word Convolution is the name given to it as it creates complexity over anything. Applications of Convolutional neural networks will be treated as Significant due to their wide use in various diverse areas such as medical image analysis, recognition of images and moving objects, Medical image classification, Language processing’s etc. There exit regularized Convolutional neural networks for MPI-Multilayer perceptron’s due to their significance Convolutional neural networks take a variety of approach to deal with the regularization, they follow hierarchical patterns to achieve it which makes things simpler. The main motivation of convolutional neural networks is recognizing objects in a video. The reason behind the popularity of CNN are the neurons of CNNs are resembles the neurons of Living species which is the inspiration of Convolutional neural networks to accomplish desired tasks. There exists many popular object detection and object classifications with convolutional neural networks for a diversity of data sets. In this paper we are going to do a comparative study on convolutional neural networks for any real time image classification and object recognition. Convolutional neural networks have that much of capability to create optimized image classifications and object recognitions.

A Double-Channel Hybrid Deep Neural Network Based on CNN and BiLSTM for Remaining Useful Life Prediction

In recent years, prognostic and health management (PHM) has played an important role in industrial engineering. Efficient remaining useful life (RUL) prediction can ensure the development of maintenance strategies and reduce industrial losses. Recently, data-driven based deep learning RUL prediction methods have attracted more attention. The convolution neural network (CNN) is a kind of deep neural network widely used in RUL prediction. It shows great potential for application in RUL prediction. A CNN is used to extract the features of time-series data according to the spatial feature method. This way of processing features without considering the time dimension will affect the prediction accuracy of the model. On the contrary, the commonly used long short-term memory (LSTM) network considers the timing of the data. However, compared with CNN, it lacks spatial data extraction capabilities. This paper proposes a double-channel hybrid prediction model based on the CNN and a bidirectional LSTM network to avoid those drawbacks. The sliding time window is used for data preprocessing, and an improved piece-wise linear function is used for model validating. The prediction model is evaluated using the C-MAPSS dataset provided by NASA. The predicted results show the proposed prediction model to have a better prediction performance compared with other state-of-the-art models.

The heat source layout optimization using deep learning surrogate modeling

In practical engineering, the layout optimization technique driven by the thermal performance is faced with a severe computational burden when directly integrating the numerical analysis tool of temperature simulation into the optimization loop. To alleviate this difficulty, this paper presents a novel deep learning surrogate-assisted heat source layout optimization method. First, two sampling strategies, namely the random sampling strategy and the evolving sampling strategy, are proposed to produce diversified training data. Then, regarding mapping between the layout and the corresponding temperature field as an image-to-image regression task, the feature pyramid network (FPN), a kind of deep neural network, is trained to learn the inherent laws, which plays as a surrogate model to evaluate the thermal performance of the domain with respect to different input layouts accurately and efficiently. Finally, the neighborhood search-based layout optimization (NSLO) algorithm is proposed and combined with the FPN surrogate to solve discrete heat source layout optimization problems. A typical two-dimensional heat conduction optimization problem is investigated to demonstrate the feasibility and effectiveness of the proposed deep learning surrogate-assisted layout optimization framework.

Seismic horizon tracking using a deep convolutional neural network

Seismic horizons are essential for structural analysis, inversion, time-to-depth conversion, and seismic attribution analysis. However, seismic horizons are obtained commonly by manual tracking or a combination of manual tracking and traditional auto-tracking techniques, which is a time consuming and error-prone process. Although many conventional auto-tracking techniques have been proposed to improve the efficiency of horizon tracking, there are still many challenges to track seismic horizons with complex seismic reflection signatures. A deep convolutional neural network (CNN) method, a typical kind of deep neural networks in the field of deep learning, was proposed in this paper to solve the problems. Seismic horizon tracking is formulated as an image classification task. CNN can take full advantage of the high flexibility in network architecture and hierarchical feature extraction ability to track horizons from seismic volumes in which discontinuities, lateral variations and faults are apparent. We designed a CNN structure for horizon tracking and trained CNN models by using only 1% and 1.5% of the seismic volume to perform a single horizon and multiple horizon tracking. The root mean square error (RMSE) and the coefficient of determination (R2) between the horizon tracked by CNN model and the manual-tracking are 1.78 and 0.998 in single horizon tracking, respectively. The average RMSE and R2 are 2.68 and 0.992 in multiple horizon tracking, respectively. The research we have done suggests that the proposed method can track seismic horizons from 3-D seismic volumes much more accurately and efficiently than conventional methods.

Sleep apnea detection from a single-lead ECG signal with automatic feature-extraction through a modified LeNet-5 convolutional neural network.

Sleep apnea (SA) is the most common respiratory sleep disorder, leading to some serious neurological and cardiovascular diseases if left untreated. The diagnosis of SA is traditionally made using Polysomnography (PSG). However, this method requires many electrodes and wires, as well as an expert to monitor the test. Several researchers have proposed instead using a single channel signal for SA diagnosis. Among these options, the ECG signal is one of the most physiologically relevant signals of SA occurrence, and one that can be easily recorded using a wearable device. However, existing ECG signal-based methods mainly use features (i.e. frequency domain, time domain, and other nonlinear features) acquired from ECG and its derived signals in order to construct the model. This requires researchers to have rich experience in ECG, which is not common. A convolutional neural network (CNN) is a kind of deep neural network that can automatically learn effective feature representation from training data and has been successfully applied in many fields. Meanwhile, most studies have not considered the impact of adjacent segments on SA detection. Therefore, in this study, we propose a modified LeNet-5 convolutional neural network with adjacent segments for SA detection. Our experimental results show that our proposed method is useful for SA detection, and achieves better or comparable results when compared with traditional machine learning methods.

Selected technical issues of deep neural networks for image classification purposes

In recent years, deep learning and especially Deep Neural Networks (DNN) have obtained amazing performance on a variety of problems, in particular in classification or pattern recognition. Among many kinds of DNNs, the Convolutional Neural Networks (CNN) are most commonly used. However, due to their complexity, there are many problems related but not limited to optimizing network parameters, avoiding overfitting and ensuring good generalization abilities. Therefore, a number of methods have been proposed by the researchers to deal with these problems. In this paper, we present the results of applying different, recently developed methods to improve deep neural network training and operating. We decided to focus on the most popular CNN structures, namely on VGG based neural networks: VGG16, VGG11 and proposed by us VGG8. The tests were conducted on a real and very important problem of skin cancer detection. A publicly available dataset of skin lesions was used as a benchmark. We analyzed the influence of applying: dropout, batch normalization, model ensembling, and transfer learning. Moreover, the influence of the type of activation function was checked. In order to increase the objectivity of the results, each of the tested models was trained 6 times and their results were averaged. In addition, in order to mitigate the impact of the selection of learning, test and validation sets, k-fold validation was applied.

Fast unsupervised deep fusion network for change detection of multitemporal SAR images

In this paper, a fast unsupervised deep fusion framework for change detection of multitemporal synthetic aperture radar (SAR) images is presented. It mainly aim at generating a difference image (DI) in the feature learning procedure by stacked auto-encoders (SAEs). Stacked auto-encoders, as one kind of deep neural network, can learn feature maps that retain the structural information but suppress the noise in the SAR images, which will be beneficial for DI generation. Compared with shallow network, the proposed framework can extract more available features, and be favorable for getting better change results. Different with other common deep neural networks, our proposed method does not need labeled data to train the network. In addition, we find a subset of the entire samples that appropriately represent the whole dataset to speed up the training of the deep neural network without under-fitting. Moreover, we design a fusion network structure that can combine ratio operator based method to ensure that the representations of higher layers are better than that of the lower ones. To summarize, the main contribution of our work lies in using of deep fusion network for generation of DI in a fast and unsupervised way. Experiments on four real SAR images confirm that our network performs better than traditional ratio methods and convolutional neural network.

Deep Neural Networks for Linear Sum Assignment Problems

Many resource allocation issues in wireless communications can be modeled as assignment problems and can be solved online with global information. However, traditional methods for assignment problems take a lot of time to find the optimal solutions. In this letter, we solve the assignment problem using machine learning approach. Specifically, the linear sum assignment problems (LSAPs) are solved by the deep neural networks (DNNs). Since LSAP is a combinatorial optimization problem, it is first decomposed into several sub-assignment problems. Each of them is a classification problem and can be solved effectively with DNNs. Two kinds of DNNs, feed-forward neural network and convolutional neural network, are implemented to deal with the sub-assignment problems, respectively. Based on computer simulation, DNNs can effectively solve LSAPs with great time efficiency and only slight loss of accuracy.

Multi-view Opinion Mining with Deep Learning

With the explosive growth of social media on the Internet, people are expressing an increasing number of opinions. As for objectives like business decision making and public opinion analysis, how to make the best of these precious opinionated words is a new challenge in the field of NLP. The field of opinion mining, or sentiment analysis, has become active in recent years. Since different kinds of deep neural networks differ in their structures, they are probably extracting different features. We investigated whether features generated by heterogeneous deep neural networks can be combined by multi-view learning to improve the overall performance. With document level opinion mining being the objective, we implemented multi-view learning based on heterogeneous deep neural networks. Experiments show that multi-view learning utilizing these heterogeneous features outperforms single-view deep neural networks. Our framework makes better use of single-view data.

Prediction for Tourism Flow based on LSTM Neural Network

Accurate tourism flow prediction is one of the most difficult problems in the Intelligent Tourism System (ITS), especially in the short-term forecast. Existing models such as ARMA, ARIMA which are mainly linear models and cannot describe the stochastic and non-linear nature of tourism flow. In recent years, deep-learning-based methods have been applied as novel alternatives for tourism flow prediction. However, which kind of deep neural networks is the most appropriate model for tourism flow prediction remains unsolved. In this paper, we use Long Short Term Memory Neural Network (LSTM) methods to predict tourism flow, and experiments demonstrate that LSTM methods perform better than Auto Regressive Integrated Moving Average (ARIMA) model and Back Propagation Neural Network(BPNN). To the best of our knowledge, this is the first time that LSTM NN is applied to tourism flow prediction.