Deep Learning Tools Research Articles

Classification and recognition of milk somatic cell images based on PolyLoss and PCAM-Reset50.

Somatic cell count (SCC) is a fundamental approach for determining the quality of cattle and bovine milk. So far, different classification and recognition methods have been proposed, all with certain limitations. In this study, we introduced a new deep learning tool, i.e., an improved ResNet50 model constructed based on the residual network and fused with the position attention module and channel attention module to extract the feature information more effectively. In this paper, macrophages, lymphocytes, epithelial cells, and neutrophils were assessed. An image dataset for milk somatic cells was constructed by preprocessing to increase the diversity of samples. PolyLoss was selected as the loss function to solve the unbalanced category samples and difficult sample mining. The Adam optimization algorithm was used to update the gradient, while Warm-up was used to warm up the learning rate to alleviate the overfitting caused by small sample data sets and improve the model's generalization ability. The experimental results showed that the classification accuracy, precision rate, recall rate, and comprehensive evaluation index F value of the proposed model reached 97%, 94.5%, 90.75%, and 92.25%, respectively, indicating that the proposed model could effectively classify the milk somatic cell images, showing a better classification performance than five previous models (i.e., ResNet50, ResNet18, ResNet34, AlexNet andMobileNetv2). The accuracies of the ResNet18, ResNet34, ResNet50, AlexNet, MobileNetv2, and the new model were 95%, 93%, 93%, 56%, 37%, and 97%, respectively. In addition, the comprehensive evaluation index F1 showed the best effect, fully verifying the effectiveness of the proposed method in this paper. The proposed method overcame the limitations of image preprocessing and manual feature extraction by traditional machine learning methods and the limitations of manual feature selection, improving the classification accuracy and showing a strong generalization ability.

Development of a real-time face tracking system based on the MOSSE correlation filter

The functioning acceleration of the face tracking system can be achieved by using simultaneously a highprecision neural network face detector for initial face detection and a tracking system based on modern high-speed correlation filters. A face tracking system using a MOSSE correlation filter (filter with minimum output sum of squared errors) is proposed. The proposed system allows tracking a human face at high speed in real time, as well as in various face positions. The proposed approach was tested using image processing and deep learning tools from the MATLAB application. Keywords convolutional neural network, YOLO v3 object detection algorithm, deep learning, MOSSE filter, face tracking

Spotted Hyena Optimizer with Deep Learning Driven Cybersecurity for Social Networks

Recent developments on Internet and social networking have led to the growth of aggressive language and hate speech. Online provocation, abuses, and attacks are widely termed cyberbullying (CB). The massive quantity of user generated content makes it difficult to recognize CB. Current advancements in machine learning (ML), deep learning (DL), and natural language processing (NLP) tools enable to detect and classify CB in social networks. In this view, this study introduces a spotted hyena optimizer with deep learning driven cybersecurity (SHODLCS) model for OSN. The presented SHODLCS model intends to accomplish cybersecurity from the identification of CB in the OSN. For achieving this, the SHODLCS model involves data pre-processing and TF-IDF based feature extraction. In addition, the cascaded recurrent neural network (CRNN) model is applied for the identification and classification of CB. Finally, the SHO algorithm is exploited to optimally tune the hyperparameters involved in the CRNN model and thereby results in enhanced classifier performance. The experimental validation of the SHODLCS model on the benchmark dataset portrayed the better outcomes of the SHODLCS model over the recent approaches.

Application of deep learning tools in rugoscopy: Exploring digital imaging study

Application of deep learning tools in rugoscopy: Exploring digital imaging study

Deep learning-based 2D keypoint detection in alpine ski racing – A performance analysis of state-of-the-art algorithms applied to regular skiing and injury situations

ObjectivesIn this study, we examined the practicability of deep learning-based 2D keypoint detection applied to regular skiing and injury situations (i.e., out-of-balance situations and fall situations) on an alpine ski racing track. MethodsWe therefore created a regular skiing- and injury situation-specific dataset (hereinafter called "Injury Ski Dataset"), on which the state-of-the-art keypoint detection algorithms OpenPose, Mask-R-CNN, AlphaPose and DCPose were compared. The performance of each keypoint detector was evaluated by calculating the mean per joint position error (MPJPE) and the percentage of correct keypoints (PCK). Failure cases and common error patterns were further investigated by a visual analysis. ResultsWe observed the best results for regular skiing, with 81%–92% of all keypoints detected correctly at an MPJPE of 9 (2) to 14 (3) pixels. In injury situations, self-occlusions and rare poses became more likely, similar to occlusions due to snow spray and motion blur. As a result, the performance in out-of-balance situations decreased to 68%–80% (PCK), while in fall situations, only 35%–54% of all keypoints were detected correctly, with mean errors of 26–36 pixels. Among all algorithms, AlphaPose was the most robust and achieved the best results. ConclusionsPCK and MPJPE for regular skiing were in the range of manual annotation errors and can be considered low enough for further biomechanical analysis. For fall situations, keypoint detection should be further improved. Regarding the development of a deep learning tool for injury analysis in alpine skiing in the future, we propose to fine-tune a well-performing keypoint detector, such as AlphaPose, on a ski- and injury-specific dataset, such as ours.

Real-Time Echocardiography Guidance for Optimized Apical Standard Views.

Measurements of cardiac function such as left ventricular ejection fraction and myocardial strain are typically based on 2-D ultrasound imaging. The reliability of these measurements depends on the correct pose of the transducer such that the 2-D imaging plane properly aligns with the heart for standard measurement views and is thus dependent on the operator's skills. We propose a deep learning tool that suggests transducer movements to help users navigate toward the required standard views while scanning. The tool can simplify echocardiography for less experienced users and improve image standardization for more experienced users. Training data were generated by slicing 3-D ultrasound volumes, which permits simulation of the movements of a 2-D transducer. Neural networks were further trained to calculate the transducer position in a regression fashion. The method was validated and tested on 2-D images from several data sets representative of a prospective clinical setting. The method proposed the adequate transducer movement 75% of the time when averaging over all degrees of freedom and 95% of the time when considering transducer rotation solely. Real-time application examples illustrate the direct relation between the transducer movements, the ultrasound image and the provided feedback.

Towards interactive deep-learning for tumour segmentation in head and neck cancer radiotherapy.

With deep-learning, gross tumour volume (GTV) auto-segmentation has substantially been improved, but still substantial manual corrections are needed. With interactive deep-learning (iDL), manual corrections can be used to update a deep-learning tool while delineating, minimising the input to achieve acceptable segmentations. We present an iDL tool for GTV segmentation that took annotated slices as input and simulated its performance on a head and neck cancer (HNC) dataset. Multimodal image data of 204 HNC patients with clinical tumour and lymph node GTV delineations were used. A baseline convolutional neural network (CNN) was trained (n=107 training, n=22 validation) and tested (n=24). Subsequently, user input was simulated on initial test set by replacing one or more of predicted slices with ground truth delineation, followed by re-training the CNN. The objective was to optimise re-training parameters and simulate slice selection scenarios while limiting annotations to maximally-five slices. The remaining 51 patients were used as an independent test set, where Dice similarity coefficient (DSC), mean surface distance (MSD), and 95% Hausdorff distance (HD95%) were assessed at baseline and after every update. Median segmentation accuracy at baseline was DSC=0.65, MSD=4.3mm, HD95% = 17.5mm. Updating CNN using three slices equally sampled from the craniocaudal axis of the GTV in the first round, followed by two rounds of annotating one extra slice, gave the best results. The accuracy improved to DSC=0.82, MSD=1.6mm, HD95% = 4.8mm. Every CNN update took 30s. The presented iDL tool achieved substantial segmentation improvement with only five annotated slices.

Hybrid offline-online optimization, monitoring and control of milling processes

A hybrid offline-online optimization, monitoring and control (HOMC) system was developed for milling processes. Safe cutting regions (SCRs) are defined based on offline analysis considering process dynamic stability, tool deflection and machined part geometric accuracy and surface quality. Near-optimum cutting conditions are defined offline using the cutter-workpiece contact information along the toolpath. A deep-learning tool condition monitoring approach is developed to detect the wear state in real-time using minimal learning efforts. The HOMC monitors the machining power signals and adaptively controls the feedrate within SCR using online predictions. Validation tests proved better process productivity, part quality and extended tool life.

Deciphering Voice of the Customer using Text Analytics and Sentiment Analysis: An Interpretable Review Rating Prediction using RoBERTa

In this era of cut-throat competition among traditional and newer digital organizations, capturing, listening, and understanding customer voices are critical for success in the marketplace. The challenge to decipher the voice of the customer (VOC) has multiplied many times today, as now the number of customer reviews are present in multiple platforms and the data to be analyzed is huge. Sentiment analysis, and text analytics using machine learning, deep learning tools and transformer-based tools can be applied to gather meaningful insights from these data. This paper applies the traditional machine learning tools of the Naive Bayes classifier, Random Forest and AdaBoost to predict the customer review ratings. These results are compared with deep learning methods of CNN, RNN and Bi-LSTM and transformer-based approaches of BERT, DistilBERT and RoBERTa. The results show that RoBERTa has the highest accuracy among these methods. Paper also uses the explainable AI tool of LIME to understand the customer sentiments deeper in terms of why customers are giving a particular rating to the product. Business organizations will continue to use more and more AI tools to understand the customer feedback and the attempt in this paper is to learn how we can make predictions faster and more accurately.

Validation of a Deep Learning–Based Model to Predict Lung Cancer Risk Using Chest Radiographs and Electronic Medical Record Data

Lung cancer screening with chest computed tomography (CT) prevents lung cancer death; however, fewer than 5% of eligible Americans are screened. CXR-LC, an open-source deep learning tool that estimates lung cancer risk from existing chest radiograph images and commonly available electronic medical record (EMR) data, may enable automated identification of high-risk patients as a step toward improving lung cancer screening participation. To validate CXR-LC using EMR data to identify individuals at high-risk for lung cancer to complement 2022 US Centers for Medicare & Medicaid Services (CMS) lung cancer screening eligibility guidelines. This prognostic study compared CXR-LC estimates with CMS screening guidelines using patient data from a large US hospital system. Included participants were persons who currently or formerly smoked cigarettes with an outpatient posterior-anterior chest radiograph between January 1, 2013, and December 31, 2014, with no history of lung cancer or screening CT. Data analysis was performed between May 2021 and June 2022. CXR-LC lung cancer screening eligibility (previously defined as having a 3.297% or greater 12-year risk) based on inputs (chest radiograph image, age, sex, and whether currently smoking) extracted from the EMR. 6-year incident lung cancer. A total of 14 737 persons were included in the study population (mean [SD] age, 62.6 [6.8] years; 7154 [48.5%] male; 204 [1.4%] Asian, 1051 [7.3%] Black, 432 [2.9%] Hispanic, 12 330 [85.2%] White) with a 2.4% rate of incident lung cancer over 6 years (361 patients with cancer). CMS eligibility could be determined in 6277 patients (42.6%) using smoking pack-year and quit-date from the EMR. Patients eligible by both CXR-LC and 2022 CMS criteria had a high rate of lung cancer (83 of 974 patients [8.5%]), higher than those eligible by 2022 CMS criteria alone (5 of 177 patients [2.8%]; P < .001). Patients eligible by CXR-LC but not 2022 CMS criteria also had a high 6-year incidence of lung cancer (121 of 3703 [3.3%]). In the 8460 cases (57.4%) where CMS eligibility was unknown, CXR-LC eligible patients had a 5-fold higher rate of lung cancer than ineligible (127 of 5177 [2.5%] vs 18 of 2283 [0.5%]; P < .001). Similar results were found in subgroups, including female patients and Black persons. Using routine chest radiographs and other data automatically extracted from the EMR, CXR-LC identified high-risk individuals who may benefit from lung cancer screening CT.

SpheroScan: a user-friendly deep learning tool for spheroid image analysis.

In recent years, 3-dimensional (3D) spheroid models have become increasingly popular in scientific research as they provide a more physiologically relevant microenvironment that mimics in vivo conditions. The use of 3D spheroid assays has proven to be advantageous as it offers a better understanding of the cellular behavior, drug efficacy, and toxicity as compared to traditional 2-dimensional cell culture methods. However, the use of 3D spheroid assays is impeded by the absence of automated and user-friendly tools for spheroid image analysis, which adversely affects the reproducibility and throughput of these assays. To address these issues, we have developed a fully automated, web-based tool called SpheroScan, which uses the deep learning framework called Mask Regions with Convolutional Neural Networks (R-CNN) for image detection and segmentation. To develop a deep learning model that could be applied to spheroid images from a range of experimental conditions, we trained the model using spheroid images captured using IncuCyte Live-Cell Analysis System and a conventional microscope. Performance evaluation of the trained model using validation and test datasets shows promising results. SpheroScan allows for easy analysis of large numbers of images and provides interactive visualization features for a more in-depth understanding of the data. Our tool represents a significant advancement in the analysis of spheroid images and will facilitate the widespread adoption of 3D spheroid models in scientific research. The source code and a detailed tutorial for SpheroScan are available at https://github.com/FunctionalUrology/SpheroScan.

Hydrology and small pelagic fish drive the spatio–temporal dynamics of springtime zooplankton assemblages over the Bay of Biscay continental shelf

As mesozooplankton is the preferential prey of small pelagic fish (SPF), environmentally–driven mesozooplankton dynamics can have critical effects on SPF population dynamics. Despite previous studies on SPF habitats’ dynamics, hydrological landscapes and mesozooplankton dynamics in the Bay of Biscay (BoB), knowledge gaps persist at the BoB regional–scale pelagic ecology and in particular about the mesozooplankton assemblages and their long–term space–time patterns. Here, we present 16 years of spring mesozooplankton assemblage interannual spatial dynamics over the BoB continental shelf and we describe the correlations between the mesozooplankton space–time patterns and those in hydrology, primary producers and SPF. We gathered data originating from the PELGAS surveys (2004–2019) and remote sensing products. Mesozooplankton samples were collected with a 200–µm mesh size WP2 net vertically towed from 100 m depth (or 5 m above the sea floor) to the surface. They were analysed with imaging and deep-learning tools and the biomass in 24 coarse taxonomic groups was calculated. Automated procedures for spatial gridding and missing data imputation enable the generation of yearly maps time series with the same spatial resolution across the pelagic ecosystem components and years. These comprehensive multivariate datasets were analysed with a multi–table method known as Multiple Factor Analyses to depict time–consistent spatial patterns in each ecosystem component and the temporal variability around them. Finally, the main time–consistent spatial patterns in the hydrology, primary producers and SPF ecosystem components were used as predictors in generalized linear models, to explain those in the mesozooplankton. Mesoscale coastal-offshore and north–south gradients were the main patterns observed in each of the pelagic ecosystem components studied. The spatial patterns in the mesozooplankton assemblage were stable, without any significant changes detected in the taxonomic composition nor its spatial structure over the studied period. Small copepods, gelatinous and meroplanktonic organisms characterised coastal areas. Euchaetidae and meroplanktonic crustaceans’ larvae displayed higher biomass in the northern part of the BoB while Metridinidae, Cladocera, Appendicularia and Echinodermata had higher biomass in the southern part. Surface and bottom water temperature, salinity–related parameters, water column stratification and SPF biomasses were the variables that best explained the observed space–time patterns in the mesozooplankton communities.

Measuring Comfort Behaviours in Laying Hens Using Deep-Learning Tools.

Image analysis using machine learning (ML) algorithms could provide a measure of animal welfare by measuring comfort behaviours and undesired behaviours. Using a PLF technique based on images, the present study aimed to test a machine learning tool for measuring the number of hens on the ground and identifying the number of dust-bathing hens in an experimental aviary. In addition, two YOLO (You Only Look Once) models were compared. YOLOv4-tiny needed about 4.26 h to train for 6000 epochs, compared to about 23.2 h for the full models of YOLOv4. In validation, the performance of the two models in terms of precision, recall, harmonic mean of precision and recall, and mean average precision (mAP) did not differ, while the value of frame per second was lower in YOLOv4 compared to the tiny version (31.35 vs. 208.5). The mAP stands at about 94% for the classification of hens on the floor, while the classification of dust-bathing hens was poor (28.2% in the YOLOv4-tiny compared to 31.6% in YOLOv4). In conclusion, ML successfully identified laying hens on the floor, whereas other PLF tools must be tested for the classification of dust-bathing hens.

HiGNN: A Hierarchical Informative Graph Neural Network for Molecular Property Prediction Equipped with Feature-Wise Attention.

Elucidating and accurately predicting the druggability and bioactivities of molecules plays a pivotal role in drug design and discovery and remains an open challenge. Recently, graph neural networks (GNNs) have made remarkable advancements in graph-based molecular property prediction. However, current graph-based deep learning methods neglect the hierarchical information of molecules and the relationships between feature channels. In this study, we propose a well-designed hierarchical informative graph neural network (termed HiGNN) framework for predicting molecular property by utilizing a corepresentation learning of molecular graphs and chemically synthesizable breaking of retrosynthetically interesting chemical substructure (BRICS) fragments. Furthermore, a plug-and-play feature-wise attention block is first designed in HiGNN architecture to adaptively recalibrate atomic features after the message passing phase. Extensive experiments demonstrate that HiGNN achieves state-of-the-art predictive performance on many challenging drug discovery-associated benchmark data sets. In addition, we devise a molecule-fragment similarity mechanism to comprehensively investigate the interpretability of the HiGNN model at the subgraph level, indicating that HiGNN as a powerful deep learning tool can help chemists and pharmacists identify the key components of molecules for designing better molecules with desired properties or functions. The source code is publicly available at https://github.com/idruglab/hignn.

Deep neural network reducing numerical dispersion for post-processing of seismic modeling results

The article describes a new approach to seismic modeling that combines calculations using traditional finite difference methods with the deep learning tools. Seismograms for the training data set are calculated using a finite difference scheme with high-quality spatial and temporal discretization. A numerical dispersion mitigation neural network is trained on the training dataset and applied to inaccurate seismograms calculated on a raw grid with a large spatial spacing. The paper presents a demonstration of this approach for 2D model; it is showing a tenfold acceleration of seismic modeling.

Investigation of a Fused in Sarcoma Splicing Mutation in a Chinese Amyotrophic Lateral Sclerosis Patient.

Genetic mutations of fused in sarcoma (FUS) causing amyotrophic lateral sclerosis (ALS) may disrupt mRNA splicing events. For example, the FUS c.1394-2delA variant was reported in two western ALS patients, but its molecular mechanism is unclear. In this study, we aim to investigate FUS splice site mutations in Chinese ALS patients. Sanger sequencing was used to identify FUS splicing mutations in Chinese ALS patients. We combined a deep learning tool (SpliceAI), RNA sequencing, and RT-PCR/RT-qPCR to analyze the effect of FUS c.1394-2delA mutation on RNA splicing and expression. AlphaFold was used to predict the protein structure of mutant FUS. In transfected cell lines, we used immunofluorescence to assess cytoplasmic mislocalization of mutant FUS protein. We identified a de novo FUS splice acceptor site mutation (c.1394-2delA, p. Gly466Valfs*14) in one Chinese sporadic ALS patient, which is linked to exon 14 skipping, and upregulated total FUS mRNA expression. The FUS splice site mutation was predicted to be translated into a truncated protein product at C-terminal. In vitro studies revealed that the FUS mutation increased cytoplasmic mislocalization in both HEK293T and SH-SY5Y cells. We identified a de novo FUS splicing mutation (c.1394-2delA, p. Gly466Valfs*14) in 1 out of 233 Chinese ALS patients. It caused abnormal RNA splicing, upregulated gene expression, truncated FUS translation, and cytosolic mislocalization. Our findings suggested that FUS splice site mutation is rare in Chinese ALS patients and extended our knowledge of molecular mechanisms of the FUS c.1394-2delA mutation.

ReMODE: a deep learning-based web server for target-specific drug design

Deep learning (DL) and machine learning contribute significantly to basic biology research and drug discovery in the past few decades. Recent advances in DL-based generative models have led to superior developments in de novo drug design. However, data availability, deep data processing, and the lack of user-friendly DL tools and interfaces make it difficult to apply these DL techniques to drug design. We hereby present ReMODE (Receptor-based MOlecular DEsign), a new web server based on DL algorithm for target-specific ligand design, which integrates different functional modules to enable users to develop customizable drug design tasks. As designed, the ReMODE sever can construct the target-specific tasks toward the protein targets selected by users. Meanwhile, the server also provides some extensions: users can optimize the drug-likeness or synthetic accessibility of the generated molecules, and control other physicochemical properties; users can also choose a sub-structure/scaffold as a starting point for fragment-based drug design. The ReMODE server also enables users to optimize the pharmacophore matching and docking conformations of the generated molecules. We believe that the ReMODE server will benefit researchers for drug discovery. ReMODE is publicly available at http://cadd.zju.edu.cn/relation/remode/.Graphical

Impact of Asp/Glu-ADP-ribosylation on protein-protein interaction and protein function.

PARylation plays critical role in regulating multiple cellular processes such as DNA damage response and repair, transcription, RNA processing, and stress response. More than 300 human proteins have been found to be modified by PARylation on acidic residues, that is, Asp (D) and Glu (E). We used the deep-learning tool AlphaFold to predict protein-protein interactions (PPIs) and their interfaces for these proteins based on coevolution signals from joint multiple sequence alignments (MSAs). AlphaFold predicted 260 confident PPIs involving PARylated proteins, and about one quarter of these PPIs have D/E-PARylation sites in their predicted PPI interfaces. AlphaFold predictions offer novel insights into the mechanisms of PARylation regulations by providing structural details of the PPI interfaces. D/E-PARylation sites have a preference to occur in coil regions and disordered regions, and PPI interfaces containing D/E-PARylation sites tend to occur between short linear sequence motifs in disordered regions and globular domains. The hub protein PCNA is predicted to interact with more than 20 proteins via the common PIP box motif and the structurally variable flanking regions. D/E-PARylation sites were found in the interfaces of key components of the RNA transcription and export complex, the SF3a spliceosome complex, and H/ACA and C/D small nucleolar ribonucleoprotein complexes, suggesting that systematic PARylation have a profound effect in regulating multiple RNA-related processes such as RNA nuclear export, splicing, and modification. Finally, PARylation of SUMO2 could modulate its interaction with CHAF1A, thereby representing a potential mechanism for the cross-talk between PARylation and SUMOylation in regulation of chromatin remodeling.

AtomAI framework for deep learning analysis of image and spectroscopy data in electron and scanning probe microscopy

AtomAI is an open-source software package bridging instrument-specific Python libraries, deep learning, and simulation tools into a single ecosystem. AtomAI allows direct applications of the deep convolutional neural networks for atomic and mesoscopic image segmentation converting image and spectroscopy data into class-based local descriptors for downstream tasks such as statistical and graph analysis. For atomically-resolved imaging data, the output is types and positions of atomic species, with an option for subsequent refinement. AtomAI further allows the implementation of a broad range of image and spectrum analysis functions, including invariant variational autoencoders (VAEs). The latter consists of VAEs with rotational and (optionally) translational invariance for unsupervised and class-conditioned disentanglement of categorical and continuous data representations. In addition, AtomAI provides utilities for mapping structure-property relationships via im2spec and spec2im type of encoder-decoder models. Finally, AtomAI allows seamless connection to the first principles modeling with a Python interface, including molecular dynamics and density functional theory calculations on the inferred atomic position. While the majority of applications to date were based on atomically resolved electron microscopy, the flexibility of AtomAI allows straightforward extension towards the analysis of mesoscopic imaging data once the labels and feature identification workflows are established/available. The source code and example notebooks are available at https://github.com/pycroscopy/atomai.

DEEP BUILDING FOOTPRINT EXTRACTION FOR URBAN RISK ASSESSMENT – REMOTE SENSING AND DEEP LEARNING BASED APPROACH

Abstract. Mapping building footprints can play a crucial role in urban dynamics moni-toring, risk assessment and disaster management. Available free building footprints, like OpenStreetMap, provide manually annotated building foot-print information for some urban areas; however, frequently it does not en-tirely cover urban areas in many parts of the world and is not always availa-ble. The huge potential for meaningful ground information extraction from high-resolution Remote Sensing imagery can be considered as an alternative and a reliable source of data for building footprint generation. Therefore, the aim of the study is to explore the use of satellite imagery data and some of the state-of-the art deep learning tools to fully automate building footprint extraction. To better understand the usability and generalization ability of those approaches, this study proposes a comparative analysis of the perfor-mances and characteristics of two of the most recent deep learning models such as Unet and Attention-Unet for building footprint generation.