Deep learning with Gaussian continuation

Purpose: Although deep learning (DL) models have shown increasing ability to accurately classify diagnostic images in oncology, significantly large amounts of well-curated data are often needed to match human level performance. Given the relative paucity of imaging datasets for less prevalent cancer types, there is an increasing need for methods which can improve the performance of deep learning models trained using limited diagnostic images. Deep metric learning (DML) is a potential method which can improve accuracy in deep learning models trained on limited datasets. Leveraging a triplet-loss function, DML exponentially increases training data compared to a traditional DL model. In this study, we investigated the utility of DML to improve the accuracy of DL models trained to classify cancerous lesions found on screening mammograms. Methods: Using a dataset of 2620 lesions found on routine screening mammogram, we trained both a traditional DL and DML models to classify suspicious lesions as cancerous or benign. The VGG16 architecture was used as the basis for the DL and DML models. Model performance was compared by calculating model accuracy, sensitivity, and specificity on a blinded test set of 378 lesions. In addition to individual model performance, we also measured agreement accuracy when both the DL and DML models were combined. Sub-analyses were conducted to identify phenotypes which were best suited for each model type. Both models underwent hyperparameters optimization to identify ideal batch size, learning rate, and regularization to prevent overfitting. Results: We found that the combination of the traditional DL model with DML model resulted in the highest overall accuracy (78.7%) representing a 7.1% improvement compared to the traditional DL model (p<.001). Alone, the traditional DL model had an improved accuracy compared to the DML model (71.4% vs 66.4%). The traditional DL model had a higher sensitivity (94.8% vs 73.6 %) , but lower specificity (34.7% vs 55.1%) compared the DML model. Sub-analyses suggested the traditional DL model was more accurate on higher density breasts, whereas the DML model was more accurate on lower density breasts. Additionally, the traditional DL model had the highest accuracy on oval shaped lesions, compared to the DML model which was most accurate on irregularly shaped breast lesions. Conclusion: Our study suggests that addition of DML models with traditional DL models can improve diagnostic image classification performance in cancer. Our results suggest DML models may provide increased specificity and help with classification of unique populations often misclassified by traditional DL models. Further studied investigating the utility of DML on other cancer imaging tasks are necessary to successfully build more robust DL models in cancer imaging. Citation Format: Justin Du, Sachin Umrao, Enoch Chang, Marina Joel, Aidan Gilson, Guneet Janda, Rachel Choi, Yongfeng Hui, Sanjay Aneja. The utility of deep metric learning for breast cancer identification on mammographic images [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr 184.

Deep dual-side learning ensemble model for Parkinson speech recognition

Background: In problem-based learning (PBL) curricula implemented around the world, it is assumed that students adopt a deep learning approach to studying and aim to gain a profound understanding of the subjects being studied. However, it is not clear which PBL components initiate or deter deep learning and to what extent this happens and why. Aim: This study explored to which extent students used a deep or surface learning approach in PBL and whether this differs across years. We also investigated which PBL components students perceived to be hindrances to deep or surface learning. Methods: The study took place at Sulaiman Al Rajhi Medical College, Qassim, Kingdom of Saudi Arabia. A mixed-methods approach was applied. A validated questionnaire and semi-structured focus group interviews were conducted sequentially. Results: First-, second-, and third-year students reported, in scale 1–5, for deep learning scores, respectively, with mean (M) = 3.55, M = 3.41, and M = 3.55. First-, second-, and third-year students reported, in scale 1–5, for surface learning scores, respectively, with M = 2.88, M = 2.78, and M = 2.89. The differences for both deep and surface learning across the years were statistically nonsignificant. According to students, they study deeply on main learning objectives and superficially on minor objectives as indicated by tutors, they are stimulated toward deep learning through interesting topics during self-study, and examinations drive them toward deep or surface learning depending on the question format and necessity to pass. Conclusions: The results of this study confirm that students' perceptions of PBL components affect their approaches to deep and surface learning. These effects are not entirely negative or positive. Students seem to frequently employ a deep learning approach in PBL throughout the 3 years. These conclusions will allow program administrators/educationalists to constructively design curricula around the perceptions of learners of PBL tutors, topics, and examinations.

Comprehensive Study for Breast Cancer Using Deep Learning and Traditional Machine Learning

The pedestrian re-identification problem (i.e., re-id) is essential and pre-requisite in multi-camera video surveillance studies, provided the fact that pedestrian targets need to be accurately re-identified across a network of multiple cameras with non-overlapping fields of views before other post-hoc high-level utilizations (i.e., tracking, behaviors analyses, activities monitoring, etc.) can be carried out. Driven by recent developments in deep learning techniques, the important re-id problem is often tackled via either deep discriminant learning or deep generative learning techniques. However, most contemporary deep learning-based models with tremendously deep structures are not easy to be trained because of the notorious vanishings gradient problem. In this study, a novel full-scaled deep discriminant learning model is proposed. The novelty of the full-scale model is significant, as three crucial concepts in designing a deep learning model, including depth, width, and cardinality, are all taken into consideration, simultaneously. Therefore, the new model needs not to be tremendously deep but is more convenient to be trained. Moreover, based on the new model, a novel deep metric learning method is proposed to further solve the important re-id problem. Technically, two algorithms either based on the conventional SGD (stochastic gradient descent) or an alternative more efficient PGD (proximal gradient descent) are both derived. For experimental analyses, the newly introduced full-scaled deep metric learning method has been comprehensively compared with dozens of popular re-id methods proposed from either deep learning or shallow learning perspectives. Several well-known public re-id datasets have been incorporated and rigorous statistical analyses have been carried out to compare all methods regarding their re-id performance. The superiority of the novel full-scaled deep metric learning method has been substantiated, from the statistical point of view.

There has been an enormous growth of the Internet, mobile phone, medical facilities, and many more in the 21st century, which can also be known as the beginning of the knowledge era. Knowledge is defined not for what it is, but for what it can do. In this fast-moving technological era, as a result, a huge amount of data is generated in different regions of the world and it is growing day by day, this growing data is known as “Big Data”. To extract useful information (analyze) from large unstructured data (like Web, sales, customer contact center, social media, mobile data, and so on) is a complex task, as data being generated is a combination of structured, semi-structured and unstructured data. Traditional systems are not capable to handle semi-structured or unstructured data generated whose volume could range in petabytes or exabytes, as the major challenges are limited memory usage, computational hurdles and slower response time, data redundancy, etc. This problem can be overcome with big data analytics having technologies like Apache Hadoop, Apache Spark, Hive, Pig, etc. which can extract useful information from these large data. Authors are going to explore more on them in these chapters. Alongside authors will explore “Deep Learning” also known as “Deep Neural Learning” or “Deep Neural Network”, which is a class of Machine Learning that progressively extract higher-level features from raw data automatically. It performs 'end-to- end learning' and uses layers of algorithms to process data, understand human speech, and visually recognize objects, which is an important part of it. Feature extraction, self-driving cars, fraud detection, healthcare, neural language processing, etc. are some of the areas where it is applied in daily life. Algorithms like RNN, CNN, FNN, Backpropagation, etc, are some of the algorithms used in deep learning. The authors will explore how Machine learning is different from deep learning. Deep learning (DL) is also associated with data science in many ways as the DL algorithms work better than older learning algorithms for prediction or feature extraction etc. Which has brought it, more closer towards one of its main objectives i.e., artificial intelligence (AI)? Hence it is immensely advantageous to the data scientists who aim for making predictions and draw useful information to analyze and interpret it for helping the organization in its growth. The processing of Big Data and the evolution of Artificial Intelligence are both dependent on Deep Learning. Deep learning technology came up along with big data analytics. The concept of deep learning is supportive in the big data analytics due to its efficient use for processing huge and enormous data. This chapter explains about deep learning and big data analytics use in healthcare and alongside authors will study about algorithms used in deep learning and technologies used in big data analytics with its architecture. After reading this chapter, authors must be able to connect deep learning with big data analytics for building new products and contribute to society in a much better way

Predicting Atrial Fibrillation from Automated Measurements of Left Atrial Volume Using Routine Chest CT Examination: Overlooked and Underrecognized Risk Factors.

Deep learning (DL)-based PV Power Forecasting (PVPF) emerged nowadays as a promising research direction to intelligentize energy systems. With the massive smart meter integration, DL takes advantage of the large-scale and multi-source data representations to achieve a spectacular performance and high PV forecastability potential compared to classical models. This review article taxonomically dives into the nitty-gritty of the mainstream DL-based PVPF methods while showcasing their strengths and weaknesses. Firstly, we draw connections between PVPF and DL approaches and show how this relation might cross-fertilize or extend both directions. Then, fruitful discussions are conducted based on three classes: discriminative learning, generative learning, and deep reinforcement learning. In addition, this review analyzes recent automatic architecture optimization algorithms for DL-based PVPF. Next, the notable DL technologies are thoroughly described. These technologies include federated learning, deep transfer learning, incremental learning, and big data DL. After that, DL methods are taxonomized into deterministic and probabilistic PVPF. Finally, this review concludes with some research gaps and hints about future challenges and research directions in driving the further success of DL techniques to PVPF applications. By compiling this study, we expect to help aspiring stakeholders widen their knowledge of the staggering potential of DL for PVPF.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Advancements in Deep Learning Algorithms is a comprehensive exploration of the cutting-edge developments in deep learning, a subset of artificial intelligence that has revolutionized the way machines learn from data. This book starts with the basics, introducing the reader to the fundamental concepts and terminologies of deep learning, before delving into the core algorithms that form the backbone of this field, including neural networks, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and generative adversarial networks (GANs). It further explores advanced architectures and techniques such as attention mechanisms, deep reinforcement learning, federated learning, and autoencoders, providing a deep dive into the mechanisms that enable machines to mimic human-like learning processes. The book also addresses critical aspects of data handling and preprocessing, optimization and regularization techniques, and the practical applications of deep learning in various industries, highlighting real-world case studies. Additionally, it discusses the challenges, ethical considerations, and future implications of deploying deep learning technologies. With an eye towards recent trends and the future directions of deep learning, this book aims to equip researchers, practitioners, and enthusiasts with the knowledge to understand and leverage the potential of deep learning in solving complex problems. Keywords: Deep Learning, Neural Networks, Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), Attention Mechanisms, Deep Reinforcement Learning, Federated Learning, Autoencoders, Data Preprocessing, Optimization Techniques, Artificial Intelligence, Industry Applications, Ethical Considerations, Future Directions.

As the scale of software and the complexity of programs continue to grow, it is hard to meet development of modern computer technology only by manually extracting program features. In recent years, deep learning has achieved rapid development in different fields. Program representation learning based on deep learning has been widely used in many works, such as software vulnerability analysis, program analysis and malware detection. And it has gradually become a hot research direction in information security. After the deep analysis of existing research work on automatic program security detection, we catalog deep program representation learning for program security based on data representation and provide a comprehensive overview of deep program representation learning for program security under different application scenes. Then, we propose a deep program representation learning framework for program security. Finally, we conduct comparative analysis and summarize the challenges in deep program representation learning for program security.

A systematic review of deep transfer learning for machinery fault diagnosis

Intelligent classification of maize straw types from UAV remote sensing images using DenseNet201 deep transfer learning algorithm

Objective:To evaluate the diagnostic efficacy of traditional radiomics, deep learning, and deep learning radiomics in differentiating normal and inner ear malformations on temporal bone computed tomography（CT）. Methods:A total of 572 temporal bone CT data were retrospectively collected, including 201 cases of inner ear malformation and 371 cases of normal inner ear, and randomly divided into a training cohort（n=458） and a test cohort（n=114） in a ratio of 4∶1. Deep transfer learning features and radiomics features were extracted from the CT images and feature fusion was performed to establish the least absolute shrinkage and selection operator. The CT results interpretated by two chief otologists from the National Clinical Research Center for Otorhinolaryngological Diseases served as the gold standard for diagnosis. The model performance was evaluated using receiver operating characteristic（ROC）, and the accuracy, sensitivity, specificity, and other indicators of the models were calculated. The predictive power of each model was compared using the Delong test. Results:1 179 radiomics features were obtained from traditional radiomics, 2 048 deep learning features were obtained from deep learning, and 137 features fusion were obtained after feature screening and fusion of the two. The area under the curve（AUC） of the deep learning radiomics model on the test cohort was 0.964 0（95%CI 0.931 4-0.996 8）, with an accuracy of 0.922, sensitivity of 0.881, and specificity of 0.945. The AUC of the radiomics features alone on the test cohort was 0.929 0（95%CI 0.882 2-0.974 9）, with an accuracy of 0.878, sensitivity of 0.881, and specificity of 0.877. The AUC of the deep learning features alone on the test cohort was 0.947 0（95%CI 0.898 2-0.994 8）, with an accuracy of 0.913, sensitivity of 0.810, and specificity of 0.973. The results indicated that the prediction accuracy and AUC of the deep learning radiomics model are the highest. The Delong test showed that the differences between any two models did not reach statistical significance. Conclusion:The feature fusion model can be used for the differential diagnosis of normal and inner ear malformations, and its diagnostic performance is superior to radiomics or deep learning models alone.

This study focuses on the impact of learning experience on college students' deep learning of English and the chain-mediated effects of motivation and strategy. In the context of globalization, English is crucial for university students, but traditional teaching models often neglect the role of learning experience in deep learning. Deep learning emphasizes critical understanding, creative application and long-term memory construction, which is particularly important for English learning. Learning experience covers affective, cognitive and behavioral responses, and influences learning attitudes and effects, but there are fewer studies on its impact on college students' deep learning of English and the related mechanisms. In this study, college students of different genders, ages, educational backgrounds and academic achievement levels were selected as samples, and learning experience, motivation, learning strategies and deep learning were comprehensively assessed by well-designed scales and statistically analyzed with the help of SPSS and AMOS software. The results of the study show that learning experience has a significant positive effect on English deep learning, and motivation and learning strategies play an important chain mediating role. Specifically, learning experience enhances motivation, which in turn promotes the use of learning strategies and ultimately improves English deep learning. This study validates for the first time the chain mediation model of "learning experience→learning motivation→learning strategies→deep learning"in the field of English language learning, which provides a new perspective for understanding the intrinsic mechanism of college students' English language learning and enriches related research. In practice, it provides specific guidance for English teaching, and teachers can enhance students' English deep learning by optimizing learning experience, stimulating learning motivation and guiding the use of learning strategies. However, there are some limitations in this study, such as the limited sample scope and the use of a cross-sectional design, etc. Future studies can expand the sample scope, adopt a longitudinal research design, and further explore other potential mediating variables.