A Survey of Clustering With Deep Learning: From the Perspective of Network Architecture

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Generalized linear models with nonlinear feature transformations are widely used for large-scale regression and classification problems with sparse inputs. Memorization of feature interactions through a wide set of cross-product feature transformations are effective and interpretable, while generalization requires more feature engineering effort. With less feature engineering, deep neural networks can generalize better to unseen feature combinations through low-dimensional dense embeddings learned for the sparse features. However, deep neural networks with embeddings can over-generalize and recommend less relevant items when the user-item interactions are sparse and high-rank. In this paper, we present Wide & Deep learning---jointly trained wide linear models and deep neural networks---to combine the benefits of memorization and generalization for recommender systems. We productionized and evaluated the system on Google Play, a commercial mobile app store with over one billion active users and over one million apps. Online experiment results show that Wide & Deep significantly increased app acquisitions compared with wide-only and deep-only models. We have also open-sourced our implementation in TensorFlow.

Feature extraction is the key element when aiming at robust speech recognition. Both linear and nonlinear data-driven feature transformations are applied to the logarithmic mel-spectral context feature vectors in the TIMIT phone recognition task. Transformations are based on principal component analysis (PCA), independent component analysis (ICA), linear discriminant analysis (LDA) and multilayer perceptron network based nonlinear discriminant analysis (NLDA). All four methods outperform the baseline system which consists of the standard feature representation based on MFCCs (mel-frequency cepstral coefficients) with the first-order deltas, using a mixture-of-Gaussians HMM recognizer. Further improvement is gained by forming the feature vector as a concatenation of the outputs of all four feature transformations.

Comprehensive Study for Breast Cancer Using Deep Learning and Traditional Machine Learning

As the scale of software and the complexity of programs continue to grow, it is hard to meet development of modern computer technology only by manually extracting program features. In recent years, deep learning has achieved rapid development in different fields. Program representation learning based on deep learning has been widely used in many works, such as software vulnerability analysis, program analysis and malware detection. And it has gradually become a hot research direction in information security. After the deep analysis of existing research work on automatic program security detection, we catalog deep program representation learning for program security based on data representation and provide a comprehensive overview of deep program representation learning for program security under different application scenes. Then, we propose a deep program representation learning framework for program security. Finally, we conduct comparative analysis and summarize the challenges in deep program representation learning for program security.

A novel feature adaptive extraction method based on deep learning for bearing fault diagnosis

In the recent decades, the application of “machine learning”, “deep learning” and “artificial intelligence” has become widespread. In pattern recognition, deep artificial neural networks and machine learning have a wide range of applications. All the phrases are used often, sometimes interchangeably, with differing significance, in science and in media. Artificial intelligence is the technology that comprises science and engineering in making intelligent computer programs that make intelligent machines. Here, strong artificial intelligence makes use of machine learning and deep learning. Without programming explicitly, the ability of a system that improves by learning automatically and from its own experience is named as machine learning. Deep learning is an increasing field of research in machine learning (ML). And it is a subfield of machine learning that uses algorithms inspired by human brain with the help of large data sets using artificial neural networks. It consists of several occult layers of artificial neural networks. The approach of profound study employs high-level models and nonlinear transformations in huge databases. Recent improvements in artificial intelligence using deep learning architectures in several sectors have already contributed significantly to this. This study seeks to elucidate the link between the above words and to specify, in particular, the contribution to artificial intelligence by machine education and profound learning. We examine the relevant literature and provide a conceptual framework that explains the role of machine learning and profound learning in the development of intelligent (artificial) beings. In addition, the higher and more advantageous approach and the hierarchy of deep learning are given in layers and nonlinear operations in common applications and contrasted with the more standard techniques. We therefore want to give additional terminology and a basis for (interdisciplinary) talks.

Clustering is a fundamental challenge in many data-driven application fields and machine learning techniques. The data distribution determines the quality of the outcomes, which has a significant impact on clustering performance. As a result, deep neural networks can be used to learn more accurate data representations for clustering. Many recent studies have focused on employing deep neural networks to develop a clustering-friendly representation, which has resulted in a significant improvement in clustering performance. We present a systematic survey of clustering with deep learning in this study. Then, a taxonomy of deep clustering is proposed, as well as some sample algorithms for our overview. Finally, we discuss some exciting future possibilities for clustering using deep learning and offer some remarks.

Deep dual-side learning ensemble model for Parkinson speech recognition

The research progress in multimodal learning has grown rapidly over the last decade in several areas, especially in computer vision. The growing potential of multimodal data streams and deep learning algorithms has contributed to the increasing universality of deep multimodal learning. This involves the development of models capable of processing and analyzing the multimodal information uniformly. Unstructured real-world data can inherently take many forms, also known as modalities, often including visual and textual content. Extracting relevant patterns from this kind of data is still a motivating goal for researchers in deep learning. In this paper, we seek to improve the understanding of key concepts and algorithms of deep multimodal learning for the computer vision community by exploring how to generate deep models that consider the integration and combination of heterogeneous visual cues across sensory modalities. In particular, we summarize six perspectives from the current literature on deep multimodal learning, namely: multimodal data representation, multimodal fusion (i.e., both traditional and deep learning-based schemes), multitask learning, multimodal alignment, multimodal transfer learning, and zero-shot learning. We also survey current multimodal applications and present a collection of benchmark datasets for solving problems in various vision domains. Finally, we highlight the limitations and challenges of deep multimodal learning and provide insights and directions for future research.

There has been an enormous growth of the Internet, mobile phone, medical facilities, and many more in the 21st century, which can also be known as the beginning of the knowledge era. Knowledge is defined not for what it is, but for what it can do. In this fast-moving technological era, as a result, a huge amount of data is generated in different regions of the world and it is growing day by day, this growing data is known as “Big Data”. To extract useful information (analyze) from large unstructured data (like Web, sales, customer contact center, social media, mobile data, and so on) is a complex task, as data being generated is a combination of structured, semi-structured and unstructured data. Traditional systems are not capable to handle semi-structured or unstructured data generated whose volume could range in petabytes or exabytes, as the major challenges are limited memory usage, computational hurdles and slower response time, data redundancy, etc. This problem can be overcome with big data analytics having technologies like Apache Hadoop, Apache Spark, Hive, Pig, etc. which can extract useful information from these large data. Authors are going to explore more on them in these chapters.Alongside authors will explore “Deep Learning” also known as “Deep Neural Learning” or “Deep Neural Network”, which is a class of Machine Learning that progressively extract higher-level features from raw data automatically. It performs 'end-to- end learning' and uses layers of algorithms to process data, understand human speech, and visually recognize objects, which is an important part of it. Feature extraction, self-driving cars, fraud detection, healthcare, neural language processing, etc. are some of the areas where it is applied in daily life. Algorithms like RNN, CNN, FNN, Backpropagation, etc, are some of the algorithms used in deep learning. The authors will explore how Machine learning is different from deep learning.Deep learning (DL) is also associated with data science in many ways as the DL algorithms work better than older learning algorithms for prediction or feature extraction etc. Which has brought it, more closer towards one of its main objectives i.e., artificial intelligence (AI)? Hence it is immensely advantageous to the data scientists who aim for making predictions and draw useful information to analyze and interpret it for helping the organization in its growth. The processing of Big Data and the evolution of Artificial Intelligence are both dependent on Deep Learning. Deep learning technology came up along with big data analytics. The concept of deep learning is supportive in the big data analytics due to its efficient use for processing huge and enormous data.This chapter explains about deep learning and big data analytics use in healthcare and alongside authors will study about algorithms used in deep learning and technologies used in big data analytics with its architecture. After reading this chapter, authors must be able to connect deep learning with big data analytics for building new products and contribute to society in a much better way.

Predicting Atrial Fibrillation from Automated Measurements of Left Atrial Volume Using Routine Chest CT Examination: Overlooked and Underrecognized Risk Factors.

Deep learning (DL)-based PV Power Forecasting (PVPF) emerged nowadays as a promising research direction to intelligentize energy systems. With the massive smart meter integration, DL takes advantage of the large-scale and multi-source data representations to achieve a spectacular performance and high PV forecastability potential compared to classical models. This review article taxonomically dives into the nitty-gritty of the mainstream DL-based PVPF methods while showcasing their strengths and weaknesses. Firstly, we draw connections between PVPF and DL approaches and show how this relation might cross-fertilize or extend both directions. Then, fruitful discussions are conducted based on three classes: discriminative learning, generative learning, and deep reinforcement learning. In addition, this review analyzes recent automatic architecture optimization algorithms for DL-based PVPF. Next, the notable DL technologies are thoroughly described. These technologies include federated learning, deep transfer learning, incremental learning, and big data DL. After that, DL methods are taxonomized into deterministic and probabilistic PVPF. Finally, this review concludes with some research gaps and hints about future challenges and research directions in driving the further success of DL techniques to PVPF applications. By compiling this study, we expect to help aspiring stakeholders widen their knowledge of the staggering potential of DL for PVPF.

Artificial intelligence in interdisciplinary life science and drug discovery research.

Deep learning applied to electromyography (EMG) signals enables accurate hand gesture recognition, revolutionizing diverse applications such as human-machine interaction, neural interfaces, and rehabilitative robotics. A well-designed deep learning architecture is crucial for accurately and robustly modeling and decoding the multidimensional information embedded in the EMG data. This survey presents a comprehensive review of state-of-the-art deep learning models and, for the first time, offers a categorization of advanced architectures from the perspective of data representations. EMG, as a distinctive biosignal modality, can be characterized through multiple representational forms, including temporal waveforms, spatial images, spectral domains, and graph-based structures comprising interconnected nodes. Consequently, the optimal model architecture is closely tied to the specific data representation employed. In addition, the limited availability of EMG datasets, particularly those with high-quality labels, remains a critical bottleneck and continues to impede the translation of research advances into widespread real-world applications. We therefore examine emerging semi-supervised and self-supervised learning frameworks, which serve as complementary approaches to fully supervised paradigms. Finally, we outline promising future directions for the development of generalizable and robust deep learning for practical EMG decoding.