Proposed Deep Neural Network Architecture Research Articles

Machine learning-based prediction of hysteretic behaviors of two-side clamped steel shear walls

This study proposes a methodology that can construct the hysteretic curves of two-side clamped steel shear walls (TCSSWs) using a machine learning technology to address the inherent deficiencies of multivariable regression analysis which has been typically used for developing the hysteretic model of a common structural element. A deep neural network (DNN) is adopted for this study and a new architecture is proposed by modifying numbers of neurons and hidden layers to well capture the cyclic response of TCSSWs. Using a training datasets DNN architectures with several combinations of input parameters and configurations relating to numbers of neurons and hidden layers had been trained. This study considers a total of 70 DNN architectures and selects the best-fitting DNN architecture which produces the minimum mean-square-error value. Estimates from the proposed DNN architecture are evaluated with a test dataset and their accuracy is acceptable. Finally, the selected DNN architecture is further verified with a validation dataset consisting of TCSSWs with different material properties of steel. The modeling parameters for the hysteretic curves of TCSSWs can be properly captured by the proposed DNN architecture.

Deep Learning Based Auction-Driven Beamforming for Wireless Information and Power Transfer

In this paper, we design a deep learning based resource allocation framework, in the form of an auction, for simultaneous information and power transfer from a hybrid access point (AP) to information devices and energy harvesting devices, respectively. Using Myerson’s lemma and the concept of virtual welfare maximization, we develop an optimal dominant-strategy incentive-compatible mechanism for the AP to maximize its expected revenue, based on the devices’ bid profiles, valuation distributions, demand profiles, and channel state information. In so doing, we formulate the revenue maximization problem, which is a mixed-integer non-linear program, and propose an efficient Branch-and-Bound (BnB) algorithm to solve the problem using semidefinite relaxation technique in each branch. Since the problem has exponential time complexity, using BnB algorithms can be impractical for real-time applications. To circumvent this, a deep neural network (DNN) is proposed, and trained to predict the optimal mechanism for beamforming the data and the energy towards the information and energy devices, respectively. We use the BnB algorithm to solve the problem offline and populate the training dataset. The proposed DNN architecture is indeed a multi-layer perceptron, which is trained well to map the heterogeneous input to the desired output with high accuracy. Furthermore, we propose a heuristic iterative solution whose accuracy performance is comparable to that of the DNN-based solution. The heuristic solution has polynomial time complexity whereas the DNN-based solution has linear time complexity.

Brain Age Prediction With Morphological Features Using Deep Neural Networks: Results From Predictive Analytic Competition 2019.

Morphological changes in the brain over the lifespan have been successfully described by using structural magnetic resonance imaging (MRI) in conjunction with machine learning (ML) algorithms. International challenges and scientific initiatives to share open access imaging datasets also contributed significantly to the advance in brain structure characterization and brain age prediction methods. In this work, we present the results of the predictive model based on deep neural networks (DNN) proposed during the Predictive Analytic Competition 2019 for brain age prediction of 2638 healthy individuals. We used FreeSurfer software to extract some morphological descriptors from the raw MRI scans of the subjects collected from 17 sites. We compared the proposed DNN architecture with other ML algorithms commonly used in the literature (RF, SVR, Lasso). Our results highlight that the DNN models achieved the best performance with MAE = 4.6 on the hold-out test, outperforming the other ML strategies. We also propose a complete ML framework to perform a robust statistical evaluation of feature importance for the clinical interpretability of the results.

Reconstructing Spatial Aspects of Motion by Image-to-Path Deep Neural Networks

The choice of an appropriate representation is important when reconstructing motion from images. In this letter we propose a new type of deep neural network (DNN) that maps images to spatial paths represented by a recently introduced motion representation called arc-length dynamic movement primitive (AL-DMP). This representation separates the spatial from temporal aspects of motion and is therefore suitable for processing data that do not contain temporal information. We propose a physically meaningful loss function for training of AL-DMPs, which improves the performance of the trained DNN, and derive its gradients, which are needed to apply the backpropagation algorithm. Moreover, the proposed DNN architecture supports the processing of variable-size input images and images with cluttered background. The developed approach was applied to the reproduction of handwritten digits from single images that do not contain temporal aspects of motion. Our experiments also show that the proposed network can be applied to input images of sizes that are different from the size of training images. Finally, the proposed approach was successfully applied for reproducing real handwritten digits with a humanoid robot.

Mining Mobile Intelligence for Wireless Systems: A Deep Neural Network Approach

Wireless big data contain valuable information on users' behaviors and preferences, which can drive the design and optimization for wireless systems. The fundamental issue is how to mine mobile intelligence and further incorporate them into wireless systems. To this end, this article discusses two challenges on big data based wireless system design and optimization, and proposes a unified framework to tackle them with the help of Deep Neural Networks (DNNs) and online learning techniques. In particular, we propose a DNN architecture by incorporating an embedding layer to project different types of raw data to a latent space and utilize a regression or classification function to predict the mobile access pattern. It outperforms the best traditional machine learning algorithm (76% vs. 63%) significantly. Moreover, combining the proposed DNN architecture with online learning techniques, we show two cases on how to apply the mobile intelligence for wireless video applications, including video adaption and video pre-fetching. In the former case, we utilize the proposed DNN method to predict the dynamics of user count within the coverage of base stations, and adaptively adjust the bitrate for video streaming to improve the video watching experience. In the latter one, we utilize the proposed method to predict the user trajectory, i.e., the associated base stations, and conduct content prefetching to reduce the access latency. Evaluating the performance with a real wireless dataset, we show that the perceived video QoE and cache hit ratio are greatly improved (0.7db and 25% respectively).

A deep learning framework for football match prediction

An efficient framework is developed by deep neural networks (DNNs) and artificial neural network (ANNs) for predicting the outcomes of football matches. A dataset is used with the rankings, team performances, all previous international football match results and so on. ANN and DNN are used to explore and process the sporting data to generate prediction value. Datasets are divided into sections for training, validating and testing. By using the proposed DNN architecture, corresponding model performed excellently on predicting the FIFA world cup 2018 matches. This model had predicted 63.3% matches accurately. However, this accuracy can be increased with proper datasets and more accurate information of the teams. The outcome of this hypothesis can be derived that deep learning may be used for successfully predicting the outcomes of football matches or any other sporting events. For more accurate performance of the prediction, prior and more information about each team, player and match is desirable.

A Novel Action Recognition Framework Based on Deep-Learning and Genetic Algorithms

Recognition of human actions in partially cluttered environments is an important research field of computer vision and human-computer interaction. This field has recently garnered attention from a large number of academic researchers in various fields of application. This study proposes a novel deep-learning-based architecture for the recognition and prediction of human actions based on a hybrid model. The main contribution of this study is to propose a new hybrid architecture, integrating four wide-ranging pre-trained network models in an optimized manner, using a metaheuristic algorithm. This architecture consists of four main stages: namely, the creation of the data set, the design of deep neural network (DNN) architecture, training and optimization of the proposed DNN architecture, and evaluation of the trained DNN. By adapting the aforementioned architecture, reliable features are obtained for the training procedure. In order to validate the superiority of the proposed architecture over other state-of-the-art studies, a performance evaluation between these architectures is presented using benchmark datasets. The results reveal that the proposed architecture outperforms previously developed architectures in terms of predicting human actions.

Anomaly Detection of CAN Bus Messages Using a Deep Neural Network for Autonomous Vehicles

The in-vehicle controller area network (CAN) bus is one of the essential components for autonomous vehicles, and its safety will be one of the greatest challenges in the field of intelligent vehicles in the future. In this paper, we propose a novel system that uses a deep neural network (DNN) to detect anomalous CAN bus messages. We treat anomaly detection as a cross-domain modelling problem, in which three CAN bus data packets as a group are directly imported into the DNN architecture for parallel training with shared weights. After that, three data packets are represented as three independent feature vectors, which corresponds to three different types of data sequences, namely anchor, positive and negative. The proposed DNN architecture is an embedded triplet loss network that optimizes the distance between the anchor example and the positive example, makes it smaller than the distance between the anchor example and the negative example, and realizes the similarity calculation of samples, which were originally used in face detection. Compared to traditional anomaly detection methods, the proposed method to learn the parameters with shared-weight could improve detection efficiency and detection accuracy. The whole detection system is composed of the front-end and the back-end, which correspond to deep network and triplet loss network, respectively, and are trainable in an end-to-end fashion. Experimental results demonstrate that the proposed technology can make real-time responses to anomalies and attacks to the CAN bus, and significantly improve the detection ratio. To the best of our knowledge, the proposed method is the first used for anomaly detection in the in-vehicle CAN bus.

Deep Neural Network Supervised Bandwidth Allocation Decisions for Low-Latency Heterogeneous E-Health Networks

This paper proposes the first use of a deep neural network (DNN) to characterize the dependence of end-to-end latency on key bandwidth allocation decision parameters and diverse network features over heterogeneous body area network (BAN), wireless local area network (WLAN), and passive optical access network (PON) toward ubiquitous e-health. Using the association between latency and bandwidth allocation decisions, the DNN supervises bandwidth values that significantly reduce end-to-end latency. In this paper, we present the proposed DNN architecture and the selection of input features and output targets based on latency tradeoff analysis. We show that with supervised training, the DNN can accurately predict end-to-end latency corresponding to varying bandwidth allocation decisions, thereby critically analyzing the impact that multiple bandwidth decision parameters and network features have on latency. Using the trained DNN, bandwidth allocation decisions, i.e., bandwidth values that reduce the latency in WLAN and PON can be obtained. Our simulation results show that by using such bandwidth values supervised by the DNN in allocating bandwidth in WLAN and PON, the latency in each network segment over a heterogeneous BAN + WLAN + PON can be effectively reduced, regardless of variations in network features during network operation. Overall, an end-to-end latency μ s with a significant 40%–70% improvement is achieved as compared to conventional bandwidth allocation algorithms.

Estimating Vehicle Movement Direction from Smartphone Accelerometers Using Deep Neural Networks.

Characterization of driving maneuvers or driving styles through motion sensors has become a field of great interest. Before now, this characterization used to be carried out with signals coming from extra equipment installed inside the vehicle, such as On-Board Diagnostic (OBD) devices or sensors in pedals. Nowadays, with the evolution and scope of smartphones, these have become the devices for recording mobile signals in many driving characterization applications. Normally multiple available sensors are used, such as accelerometers, gyroscopes, magnetometers or the Global Positioning System (GPS). However, using sensors such as GPS increase significantly battery consumption and, additionally, many current phones do not include gyroscopes. Therefore, we propose the characterization of driving style through only the use of smartphone accelerometers. We propose a deep neural network (DNN) architecture that combines convolutional and recurrent networks to estimate the vehicle movement direction (VMD), which is the forward movement directional vector captured in a phone’s coordinates. Once VMD is obtained, multiple applications such as characterizing driving styles or detecting dangerous events can be developed. In the development of the proposed DNN architecture, two different methods are compared. The first one is based on the detection and classification of significant acceleration driving forces, while the second one relies on longitudinal and transversal signals derived from the raw accelerometers. The final success rate of VMD estimation for the best method is of 90.07%.

A Gender Mixture Detection Approach to Unsupervised Single-Channel Speech Separation Based on Deep Neural Networks

We propose an unsupervised speech separation framework for mixtures of two unseen speakers in a single-channel setting based on deep neural networks (DNNs). We rely on a key assumption that two speakers could be well segregated if they are not too similar to each other. A dissimilarity measure between two speakers is first proposed to characterize the separation ability between competing speakers. We then show that speakers with the same or different genders can often be separated if two speaker clusters, with large enough distances between them, for each gender group could be established, resulting in four speaker clusters. Next, a DNN-based gender mixture detection algorithm is proposed to determine whether the two speakers in the mixture are females, males, or from different genders. This detector is based on a newly proposed DNN architecture with four outputs, two of them representing the female speaker clusters and the other two characterizing the male groups. Finally, we propose to construct three independent speech separation DNN systems, one for each of the female–female, male–male, and female–male mixture situations. Each DNN gives dual outputs, one representing the target speaker group and the other characterizing the interfering speaker cluster. Trained and tested on the speech separation challenge corpus, our experimental results indicate that the proposed DNN-based approach achieves large performance gains over the state-of-the-art unsupervised techniques without using any specific knowledge about the mixed target and interfering speakers being segregated.