Deep Learning Library Research Articles

AnalogVNN: A fully modular framework for modeling and optimizing photonic neural networks

In this paper, we present AnalogVNN, a simulation framework built on PyTorch that can simulate the effects of optoelectronic noise, limited precision, and signal normalization present in photonic neural network accelerators. We use this framework to train and optimize linear and convolutional neural networks with up to nine layers and ∼1.7 × 106 parameters, while gaining insights into how normalization, activation function, reduced precision, and noise influence accuracy in analog photonic neural networks. By following the same layer structure design present in PyTorch, the AnalogVNN framework allows users to convert most digital neural network models to their analog counterparts with just a few lines of code, taking full advantage of the open-source optimization, deep learning, and GPU acceleration libraries available through PyTorch.

Hybrid data augmentation and deep attention-based dilated convolutional-recurrent neural networks for speech emotion recognition

Recently, speech emotion recognition (SER) has become an active research area in speech processing, particularly with the advent of deep learning (DL). Numerous DL-based methods have been proposed for SER. However, most of the existing DL-based models are complex and require a large amounts of data to achieve a good performance. In this study, a new framework of deep attention-based dilated convolutional-recurrent neural networks coupled with a hybrid data augmentation method was proposed for addressing SER tasks. The hybrid data augmentation method constitutes an upsampling technique for generating more speech data samples based on the traditional and generative adversarial network approaches. By leveraging both convolutional and recurrent neural networks in a dilated form along with an attention mechanism, the proposed DL framework can extract high-level representations from three-dimensional log Mel spectrogram features. Dilated convolutional neural networks acquire larger receptive fields, whereas dilated recurrent neural networks overcome complex dependencies as well as the vanishing and exploding gradient issues. Furthermore, the loss functions are reconfigured by combining the SoftMax loss and the center-based losses to classify various emotional states. The proposed framework was implemented using the Python programming language and the TensorFlow deep learning library. To validate the proposed framework, the EmoDB and ERC benchmark datasets, which are imbalanced and/or small datasets, were employed. The experimental results indicate that the proposed framework outperforms other related state-of-the-art methods, yielding the highest unweighted recall rates of 88.03 ± 1.39 (%) and 66.56 ± 0.67 (%) for the EmoDB and ERC datasets, respectively.

PyComplexHeatmap: a Python package to visualize multimodal genomics data.

Python has emerged as a robust programming language increasingly employed in genomics data analysis, largely due to its comprehensive deep learning libraries and proficiency in handling large-scale data, such as single-cell multi-omics datasets. Although Python has become a prominent data science ecosystem for bioinformatics, there remains a growing demand for advanced heatmap visualization and assembly tools, which are not sufficiently addressed by existing Python-based data visualization libraries. We present PyComplexHeatmap, an all-inclusive Python library for heatmap visualization, inspired by the ComplexHeatmap package currently available in R. PyComplexHeatmap is built upon the matplotlib library and features a versatile, modular interface that seamlessly integrates with other Python-based data science tools, such as Pandas, NumPy, and genomics tools, such as Scanpy, in a standard-compliant manner. This library caters to the requirements of exquisite rendering of multimodal matrix data, incorporating both textual and graphical annotations, thereby enabling efficient integrative analysis of multimodal data and associated metadata.

Genomic benchmarks: a collection of datasets for genomic sequence classification

BackgroundRecently, deep neural networks have been successfully applied in many biological fields. In 2020, a deep learning model AlphaFold won the protein folding competition with predicted structures within the error tolerance of experimental methods. However, this solution to the most prominent bioinformatic challenge of the past 50 years has been possible only thanks to a carefully curated benchmark of experimentally predicted protein structures. In Genomics, we have similar challenges (annotation of genomes and identification of functional elements) but currently, we lack benchmarks similar to protein folding competition.ResultsHere we present a collection of curated and easily accessible sequence classification datasets in the field of genomics. The proposed collection is based on a combination of novel datasets constructed from the mining of publicly available databases and existing datasets obtained from published articles. The collection currently contains nine datasets that focus on regulatory elements (promoters, enhancers, open chromatin region) from three model organisms: human, mouse, and roundworm. A simple convolution neural network is also included in a repository and can be used as a baseline model. Benchmarks and the baseline model are distributed as the Python package ‘genomic-benchmarks’, and the code is available at https://github.com/ML-Bioinfo-CEITEC/genomic_benchmarks.ConclusionsDeep learning techniques revolutionized many biological fields but mainly thanks to the carefully curated benchmarks. For the field of Genomics, we propose a collection of benchmark datasets for the classification of genomic sequences with an interface for the most commonly used deep learning libraries, implementation of the simple neural network and a training framework that can be used as a starting point for future research. The main aim of this effort is to create a repository for shared datasets that will make machine learning for genomics more comparable and reproducible while reducing the overhead of researchers who want to enter the field, leading to healthy competition and new discoveries.

Analysis of Urine Sediment Images for Detection and Classification of Cells

Urine sediment tests are important in the diagnosis of abnormal diseases related to the urinary tract. The formation of cells such as red blood cells and white blood cells in the urine of patients is important for the diagnosis of the disease. Therefore, cells need to be fully identified in clinical urinalysis. Urinalysis with human eyes; Since it is subjective, time consuming and causing errors, methods have been developed to automate microscopic analysis with the help of computer and software systems. In this study, the YOLO-v7 algorithm, which gives successful results in image processing technology, was used as a method and model. The dataset used in the study was created by using microscopic images of urine sediment taken from the Biochemistry Laboratory of the Faculty of Medicine, Selcuk University. Seven different cell segmentation and classification studies have been carried out, including WBC, RBC, WBCC, Epithelial, Flat Epithelial, Mucs and Bubbles, which have clinical value for the diagnosis of the disease. Experimental studies were carried out with the YOLO-v7 algorithm and the results were presented. The contributions of this study can be summarized as follows. (1) In this study, which is proposed for segmentation of cells on the urine cell images in the Urine Sediment dataset, for the experimental studies carried out with the YOLO model, whose performance was evaluated; Precision, Recall, mAP(0.5) and F1-Score(%) segmentation performance metrics were calculated as 0.384, 0.759, 0.432 and 0.510, respectively. (2) A computer-aided support system to assist physicians in segmenting urine cells is presented as a secondary tool. Classification accuracy for WBC, RBC, WBCC, Epithelial, Flat Epithelial, Mucs and Bubbles cells was calculated as 0.78, 0.94, 0.90, 0.57, 0.92, 0.68 and 0.97, respectively. A mean classification success of 0.822 was achieved for all classes. Thus, it has been seen that the Yolov7 model can be used by experts as a tool for recognizing cells in the urine sediment. As a result, it has been shown that suitable deep learning models can be used to recognize the biometric properties of urinary sediment cells. With the model created using deep learning libraries, urine sediment cells can be easily classified, and it is possible to define many different cells if there is a dataset with sufficient number of images.

Deep learning-enabled segmentation of ambiguous bioimages with deepflash2

Bioimages frequently exhibit low signal-to-noise ratios due to experimental conditions, specimen characteristics, and imaging trade-offs. Reliable segmentation of such ambiguous images is difficult and laborious. Here we introduce deepflash2, a deep learning-enabled segmentation tool for bioimage analysis. The tool addresses typical challenges that may arise during the training, evaluation, and application of deep learning models on ambiguous data. The tool’s training and evaluation pipeline uses multiple expert annotations and deep model ensembles to achieve accurate results. The application pipeline supports various use-cases for expert annotations and includes a quality assurance mechanism in the form of uncertainty measures. Benchmarked against other tools, deepflash2 offers both high predictive accuracy and efficient computational resource usage. The tool is built upon established deep learning libraries and enables sharing of trained model ensembles with the research community. deepflash2 aims to simplify the integration of deep learning into bioimage analysis projects while improving accuracy and reliability.

A deep learning approach for solving diffusion-induced stress in large-deformed thin film electrodes

Understanding the effects of large deformation on electrochemical performance of electrodes, such as silicon and tin, in lithium-ion battery has stimulated great interest in analyzing stress evolution during electrochemical cycling and in developing mechanochemical models for the stress analysis. As the complexity of mechanochemical models continues to increase, there is an urgent need to develop computational methods to reduce computational costs and increase computational efficiency. In this work, we propose a physics-inspired neural network (PINN) based on the DeepXDE deep learning library in analyzing diffusion-induced stresses (DIS) in a thin film electrode with large deformation, in which a loss function, which is associated with the loss functions of partial differential equations (PDEs) in the domain and the initial/boundary conditions for the mechanochemical problem, is introduced. Using the proposed physics-inspired neural network, the mechanical equations, including geometric, constitutive and equilibrium equations, in the framework of finite deformation theory as well as the mass transport equation are solved to obtain the stress evolution in the large-deformed thin-film electrode. The numerical results are in accord with the results obtained from finite element method. This work provides a unique approach for deep learning to solve the coupling mechanochemical problems in energy storage.

Automatic Detection of Oranges Peel Based on the YOLOv5 Model

In recent years, the deep learning algorithms have been widely used in target detection with the development of artificial intelligence algorithms. This experiment tends to detect whether the oranges are totally peeled or not by using the YOLOv5 automatic detection system. By using this automatic detection, the factory can classify the peeled and unpeeled orange easily and successfully, hence improving the efficiency and quality of the production of orange products. To achieve this goal, this experiment uses the YOLOv5s.pt model for training the detection; for collecting the images, considering the real situation, this experiment collect the whole orange with peel, partly peeled, the whole orange without peel, and some cloves of peeled oranges; overall 1100 images were collected, 310 came from the real-life photos and other found online; the ratio of the number of training images and the validation images is 8:2; set up two classes which are “orange with peel” and “peeled orange”; Using the online tool to label the images for training; Using the PyTorch (1.12.1) as the deep learning library. The experimental result shows that the proposed method achieved 88.4% for the Mean Average Precision (mAP) value and 88.7% for the average recall rate. Besides, in the testing part, this result can successfully detect the “orange with peel” and the “peeled orange”.

PyMIC: A deep learning toolkit for annotation-efficient medical image segmentation

Background and Objective: Open-source deep learning toolkits are one of the driving forces for developing medical image segmentation models that are essential for computer-assisted diagnosis and treatment procedures. Existing toolkits mainly focus on fully supervised segmentation that assumes full and accurate pixel-level annotations are available. Such annotations are time-consuming and difficult to acquire for segmentation tasks, which makes learning from imperfect labels highly desired for reducing the annotation cost. We aim to develop a new deep learning toolkit to support annotation-efficient learning for medical image segmentation, which can accelerate and simplify the development of deep learning models with limited annotation budget, e.g., learning from partial, sparse or noisy annotations.Methods: Our proposed toolkit named PyMIC is a modular deep learning library for medical image segmentation tasks. In addition to basic components that support development of high-performance models for fully supervised segmentation, it contains several advanced components that are tailored for learning from imperfect annotations, such as loading annotated and unannounced images, loss functions for unannotated, partially or inaccurately annotated images, and training procedures for co-learning between multiple networks, etc. PyMIC is built on the PyTorch framework and supports development of semi-supervised, weakly supervised and noise-robust learning methods for medical image segmentation.Results: We present several illustrative medical image segmentation tasks based on PyMIC: (1) Achieving competitive performance on fully supervised learning; (2) Semi-supervised cardiac structure segmentation with only 10% training images annotated; (3) Weakly supervised segmentation using scribble annotations; and (4) Learning from noisy labels for chest radiograph segmentation.Conclusions: The PyMIC toolkit is easy to use and facilitates efficient development of medical image segmentation models with imperfect annotations. It is modular and flexible, which enables researchers to develop high-performance models with low annotation cost. The source code is available at:https://github.com/HiLab-git/PyMIC.

MiraBest: a data set of morphologically classified radio galaxies for machine learning

Abstract The volume of data from current and future observatories has motivated the increased development and application of automated machine learning methodologies for astronomy. However, less attention has been given to the production of standardized data sets for assessing the performance of different machine learning algorithms within astronomy and astrophysics. Here we describe in detail the MiraBest data set, a publicly available batched data set of 1256 radio-loud AGN from NVSS and FIRST, filtered to 0.03 < z < 0.1, manually labelled by Miraghaei and Best according to the Fanaroff–Riley morphological classification, created for machine learning applications and compatible for use with standard deep learning libraries. We outline the principles underlying the construction of the data set, the sample selection and pre-processing methodology, data set structure and composition, as well as a comparison of MiraBest to other data sets used in the literature. Existing applications that utilize the MiraBest data set are reviewed, and an extended data set of 2100 sources is created by cross-matching MiraBest with other catalogues of radio-loud AGN that have been used more widely in the literature for machine learning applications.

Knowledge Guided Attention and Graph Convolutional Networks for Chemical-Disease Relation Extraction.

The automatic extraction of the chemical-disease relation (CDR) from the text becomes critical because it takes a lot of time and effort to extract valuable CDR manually. Studies have shown that prior knowledge from the biomedical knowledge base is important for relation extraction. The method of combining deep learning models with prior knowledge is worthy of our study. In this paper, we propose a new model called Knowledge Guided Attention and Graph Convolutional Networks (KGAGN) for CDR extraction. First, to make full advantage of domain knowledge, we train entity embedding as a feature representation of input sequence, and relation embedding to capture weighted contextual information further through the attention mechanism. Then, to make full advantage of syntactic dependency information in cross-sentence CDR extraction, we construct document-level syntactic dependency graphs and encode them using a graph convolution network (GCN). Finally, the chemical-induced disease (CID) relation is extracted by using weighted context features and long-range dependency features both of which contain additional knowledge information We evaluated our model on the CDR dataset published by the BioCreative-V community and achieves an F1-score of 73.3%, surpassing other state-of-the-art methods. the code implemented by PyTorch 1.7.0 deep learning library can be downloaded from Github: https://github.com/sunyi123/cdr.

Prediction of Alzheimer's disease from magnetic resonance imaging using a convolutional neural network

ObjectivesThe primary goal of this study is to examine if a convolutional neural network (CNN) can be applied as a diagnostic tool for predicting Alzheimer's Disease (AD) from magnetic resonance imaging (MRI) using the MIRIAD-dataset (Minimal Interval Resonance Imaging in Alzheimer's Disease) from one single central slice of the brain. MethodsThe MIRIAD dataset contains patients' health records represented by a set of MRI scans of the brain and further diagnostic data. Hyperparameters and configurations of CNNs were optimized to determine the best-performing model. The CNN was implemented in Python with the deep learning library ‘Keras’ using Linux/Ubuntu as the operating system. ResultsThis study obtained the following best performance metrics for predicting Alzheimer's Disease from MRI with Matthew's Correlation Coefficient (MCC) of 0.77; accuracy of 0.89; F1-score of 0.89; AUC of 0.92. The computational time for the training of a CNN takes less than 30 sec. s with a GPU (graphics processing unit). The prediction takes less than 1 sec. on a standard PC. ConclusionsThe study suggests that an axial MRI scan can be used to diagnose if a patient has Alzheimer's Disease with an AUC score of 0.92.

Simultaneous Segmentation and Classification of Pressure Injury Image Data Using Mask-R-CNN.

Pressure injuries (PIs) impose a substantial burden on patients, caregivers, and healthcare systems, affecting an estimated 3 million Americans and costing nearly $18 billion annually. Accurate pressure injury staging remains clinically challenging. Over the last decade, object detection and semantic segmentation have evolved quickly with new methods invented and new application areas emerging. Simultaneous object detection and segmentation paved the way to segment and classify anatomical structures. In this study, we utilize the Mask-R-CNN algorithm for segmentation and classification of stage 1-4 pressure injuries. Images from the eKare Inc. pressure injury wound data repository were segmented and classified manually by two study authors with medical training. The Mask-R-CNN model was implemented using the Keras deep learning and TensorFlow libraries with Python. We split 969 pressure injury images into training (87.5%) and validation (12.5%) subsets for Mask-R-CNN training. We included 121 random pressure injury images in our test set. The Mask-R-CNN model showed overall classification accuracy of 92.6%, and the segmentation demonstrated 93.0% accuracy. Our F1 scores for stages 1-4 were 0.842, 0.947, 0.907, and 0.944, respectively. Our Dice coefficients for stages 1-4 were 0.92, 0.85, 0.93, and 0.91, respectively. Our Mask-R-CNN model provides levels of accuracy considerably greater than the average healthcare professional who works with pressure injury patients. This tool can be easily incorporated into the clinician's workflow to aid in the hospital setting.

A modular framework for stabilizing deep reinforcement learning control

We propose a framework for the design of feedback controllers that combines the optimization-driven and model-free advantages of deep reinforcement learning with the stability guarantees provided by using the Youla-Kučera parameterization to define the search domain. Recent advances in behavioral systems allow us to construct a data-driven internal model; this enables an alternative realization of the Youla-Kučera parameterization based entirely on input-output exploration data. Using a neural network to express a parameterized set of nonlinear stable operators enables seamless integration with standard deep learning libraries. We demonstrate the approach on a realistic simulation of a two-tank system.

A Generic Cryptographic Deep-Learning Inference Platform for Remote Sensing Scenes

Deep learning plays an essential role in multidisciplinary research of Remote sensing. We will encounter security problems during the data acquisition, processing, and result generation stages. Therefore, secure deep-learning inference services are one of the most important links. Some theoretical progress has been made in cryptographic deep-learning inference, but it lacks a general platform that can be realized in reality. Constantly modifying the corresponding models to approximate the plaintext results reveal the model information to a certain extent. This paper proposes a generic post-quantum platform named the PyHENet, which perfectly combines cryptography with plaintext deep learning libraries. Secondly, we optimize the convolution, activation, and pooling functions and complete the ciphertext operation under floating point numbers for the first time. Moreover, the computation process is accelerated by SIMD and GPU parallel computing. The experimental results show that the PyHENet is closer to the plaintext inference platform than any other cryptographic model and has satisfactory robustness. The optimized PyHENet obtained a better accuracy of 95.05% in the high-resolution NaSC-TG2 database, which the Tiangong-2 space station received.

Deepregression: A Flexible Neural Network Framework for Semi-Structured Deep Distributional Regression

In this paper we describe the implementation of semi-structured deep distributional regression, a flexible framework to learn conditional distributions based on the combination of additive regression models and deep networks. Our implementation encompasses (1) a modular neural network building system based on the deep learning library TensorFlow for the fusion of various statistical and deep learning approaches, (2) an orthogonalization cell to allow for an interpretable combination of different subnetworks, as well as (3) pre-processing steps necessary to set up such models. The software package allows to define models in a user-friendly manner via a formula interface that is inspired by classical statistical model frameworks such as mgcv. The package's modular design and functionality provides a unique resource for both scalable estimation of complex statistical models and the combination of approaches from deep learning and statistics. This allows for state-of-the-art predictive performance while simultaneously retaining the indispensable interpretability of classical statistical models.

Inside A Deep Learning Library, the Symptoms, Causes, and Fixes for Bugs

Inside A Deep Learning Library, the Symptoms, Causes, and Fixes for Bugs

Capturing Provenance from Deep Learning Applications Using Keras-Prov and Colab: a Practical Approach

Due to the exploratory nature of DNNs, DL specialists often need to modify the input dataset, change a filter when preprocessing input data, or fine-tune the models’ hyperparameters, while analyzing the evolution of the training. However, the specialist may lose track of what hyperparameter configurations have been used and tuned if these data are not properly registered. Thus, these configurations must be tracked and made available for the user’s analysis. One way of doing this is to use provenance data derivation traces to help the hyperparameter’s fine-tuning by providing a global data picture with clear dependencies. Current provenance solutions present provenance data disconnected from W3C PROV recommendation, which is difficult to reproduce and compare to other provenance data. To help with these challenges, we present Keras-Prov, an extension to the Keras deep learning library to collect provenance data compliant with PROV. To show the flexibility of Keras-Prov, we extend a previous Keras-Prov demonstration paper with larger experiments using GPUs with the help of Google Colab. Despite the challenges of running a DBMS with virtual environments, DL analysis with provenance has added trust and persistence in databases and PROV serializations. Experiments show Keras-Prov data analysis, during training execution, to support hyperparameter fine-tuning decisions, favoring the comparison, and reproducibility of such DL experiments. Keras-Prov is open source and can be downloaded from https://github.com/dbpina/keras-prov.

A Real-Time Visage Expression Detection Using Convolutional Neural Network (RTVED)

In this work, the purpose is to implement parts of seven Real-Time Visage Expression Detection using Convolutional Neural network (RTVED), Data Augmentation to get high accuracy, and focus on the Methodology of Real-Time Visage Expression Detection using CNN. In this paper, considered a convention Convolutional Neural Network model and utilized it to prepare and test diverse look pictures with the Keras, TensorFlow, and deep learning library. Real-Time human visage Expressions Detection has two sections, recognizer Validation, and data preparation, and a training model for data preparation and training. The recognizer contains a visage Expression detector and a visage Expression recognizer. The visage expression detector extricates facial pictures from the camera and the visage Expression recognizer recognizes the extracted pictures. The Data Training model utilizes the Convolutional Neural Network to prepare data and the recognizer likewise utilizes Convolutional Neural Network to identify the emotional condition through their visage Expressions. The framework perceives the six widespread emotions as anger, disgust, happiness, sadness, surprise neutral, and contempt.

Compressing convolutional neural networks with hierarchical Tucker-2 decomposition

Convolutional neural networks (CNNs) play a crucial role and achieve top results in computer vision tasks but at the cost of high computational cost and storage complexity. One way to solve this problem is the approximation of the convolution kernel using tensor decomposition methods. In this way, the original kernel is replaced with a sequence of kernels in a lower-dimensional space. This study proposes a novel CNN compression technique based on the hierarchical Tucker-2 (HT-2) tensor decomposition and makes an important contribution to the field of neural network compression based on low-rank approximations. We demonstrate the effectiveness of our approach on many CNN architectures on CIFAR-10 and ImageNet datasets. The obtained results show a significant reduction in parameters and FLOPS with a minor drop in classification accuracy. Compared to different state-of-the-art compression methods, including pruning and matrix/tensor decomposition, the HT-2, as a new alternative, outperforms most of the cited methods. The implementation of the proposed approach is very straightforward and can be easily coded in every deep learning library.