Deep Neural Network-based Model Research Articles

Combining handcrafted and learned features using deep learning to improve protein-protein interaction prediction performance

ABSTRACT Understanding protein-protein interactions (PPIs) is essential because knowledge regarding PPIs helps in determining the biochemical functions of organisms. Advances made in machine learning techniques, for example, DeepFE-PPI, GcForest-PPI, and DeepPPI, have been applied to enhance PPI prediction performance. However, most methods use either handcrafted or learned features. Improving protein representation quality can significantly impact PPI model performance. This study aims to propose a deep neural network-based model, in which both handcrafted and learned features are fused to improve PPI prediction. In our method, the handcrafted features are computed by sequence descriptors, including Amino Acid Composition (AAC), Local Descriptors, Pseudo-AAC, Amphiphilic Pseudo-AAC, and Quasi-Sequence-Order. The learned features are generated by Word2vec after being trained on a large database of protein sequences. Standard evaluation methods and benchmark datasets are used to assess the proposed model. The results show that our model achieves 95.8% and 99.3% accuracy on the Saccharomyces cerevisiae and Human datasets, respectively. Additionally, the generalizability of our model is evaluated through independent tests, and our findings indicate that the proposed feature combination method correctly classifies experimentally validated interactions in eight datasets, including cross-species and PPI networks. The source code and data for this work are included at https://github.com/npxquynhdhsp/ComFeatPPI.git.

Improving Clinical Decision Making With a Two-Stage Recommender System.

Clinical decision-making is complex and time-intensive. To help in this effort, clinical recommender systems (RS) have been designed to facilitate healthcare practitioners with personalized advice. However, designing an effective clinical RS poses challenges due to the multifaceted nature of clinical data and the demand for tailored recommendations. In this article, we introduce a 2-Stage Recommendation framework for clinical decision-making, which leverages a publicly accessible dataset of electronic health records. In the first stage, a deep neural network-based model is employed to extract a set of candidate items, such as diagnoses, medications, and prescriptions, from a patient's electronic health records. Subsequently, the second stage utilizes a deep learning model to rank and pinpoint the most relevant items for healthcare providers. Both retriever and ranker are based on pre-trained transformer models that are stacked together as a pipeline. To validate our model, we compared its performance against several baseline models using different evaluation metrics. The results reveal that our proposed model attains a performance gain of approximately 12.3% macro-average F1 compared to the second best performing baseline. Qualitative analysis across various dimensions also confirms the model's high performance. Furthermore, we discuss challenges like data availability, privacy concerns, and shed light on future exploration in this domain.

Deep Neural Network-Based Model for Breast Cancer Lesion Diagnosis in Mammography Images

Deep Neural Network-Based Model for Breast Cancer Lesion Diagnosis in Mammography Images

A comparative analysis between deep neural network-based 1D-CNN and LSTM models to harness the self-sensing property of the shape memory alloy wire actuator for position estimation

The article presents a performance-based comparative analysis of popular deep neural network (DNN) models such as 1-dimensional convolutional neural network (1D-CNN) and long short-term memory (LSTM) for position estimation of shape memory alloy (SMA)-based wire actuator. These DNN models utilize the self-sensing property (SSP) for position estimation of the SMA actuator. The phase-dependent electrical resistivity of SMA wire acts as SSP, where the electrical resistivity in the form of SMA wire resistance acts as inputs to the proposed models for precise estimation of the current position of the SMA actuator. For effective position control of the SMA actuator, accurate position sensor feedback is required, utilizing SSP results in the elimination of this external sensor. This will improve the overall system in terms of compactness and reduced interface complexity. Coming to DNN models, 1D-CNN has been meagerly explored in the current literature landscape for self-sensing estimation of SMA actuators. These 1D-CNN models are becoming quite popular for time series prediction for various applications and are emerging as an alternative to widely used LSTM models. In this paper, a novel implementation of a 1D-CNN model for SMA actuator position estimation has been done. A comparative analysis between 1D-CNN and LSTM has been done for prediction capability and inference speed based on performance measures such as Mean Square Error (MSE), Mean Absolute Error (MAE), symmetric Mean Absolute Percentage Error (sMAPE), data distribution, and average inference speed. The proposed comparative results show that 1D-CNN has a matching performance with the LSTM model with respect to prediction capability, however, 1D-CNN offers faster inference speed. The analysis of the proposed work can be useful for choosing a suitable DNN model for deployment on low computing platforms such as microcontrollers for SMA actuator-based real-time applications where time latency is a critical parameter.

A predictive model for classifying college students' academic performance based on visual-spatial skills.

As the application of visual-spatial skills in academic disciplines, vocational fields and daily life is becoming more and more prominent, it is of great theoretical and practical significance how to make use of big data and artificial intelligence technology to conduct research on the relationship between visual-spatial skills and students' grades. This paper explores and analyses from the perspective of artificial intelligence, combining students' visual-spatial skills and students' specific attribute characteristics to construct an expert system, which defines the prediction of academic performance as a classification problem corresponding to the five categories of excellent, good, moderate, passing, and weak, respectively, and based on which a deep neural network-based classification prediction model for students' performance is designed. The experimental results show that visual-spatial skills plays an important role in the professional learning of science and engineering students, while the classification model designed in this paper has high accuracy in the grade prediction process. This paper not only helps to fill the gaps in the current research field, but is also expected to provide scientific basis for educational practice and promote the development of the education field in a more intelligent and personalized direction.

AI-based segmentation of renal enhanced CT images for quantitative evaluate of chronic kidney disease

To quantitatively evaluate chronic kidney disease (CKD), a deep convolutional neural network-based segmentation model was applied to renal enhanced computed tomography (CT) images. A retrospective analysis was conducted on a cohort of 100 individuals diagnosed with CKD and 90 individuals with healthy kidneys, who underwent contrast-enhanced CT scans of the kidneys or abdomen. Demographic and clinical data were collected from all participants. The study consisted of two distinct stages: firstly, the development and validation of a three-dimensional (3D) nnU-Net model for segmenting the arterial phase of renal enhanced CT scans; secondly, the utilization of the 3D nnU-Net model for quantitative evaluation of CKD. The 3D nnU-Net model achieved a mean Dice Similarity Coefficient (DSC) of 93.53% for renal parenchyma and 81.48% for renal cortex. Statistically significant differences were observed among different stages of renal function for renal parenchyma volume (VRP), renal cortex volume (VRC), renal medulla volume (VRM), the CT values of renal parenchyma (HuRP), the CT values of renal cortex (HuRC), and the CT values of renal medulla (HuRM) (F = 93.476, 144.918, 9.637, 170.533, 216.616, and 94.283; p < 0.001). Pearson correlation analysis revealed significant positive associations between glomerular filtration rate (eGFR) and VRP, VRC, VRM, HuRP, HuRC, and HuRM (r = 0.749, 0.818, 0.321, 0.819, 0.820, and 0.747, respectively, all p < 0.001). Similarly, a negative correlation was observed between serum creatinine (Scr) levels and VRP, VRC, VRM, HuRP, HuRC, and HuRM (r = − 0.759, − 0.777, − 0.420, − 0.762, − 0.771, and − 0.726, respectively, all p < 0.001). For predicting CKD in males, VRP had an area under the curve (AUC) of 0.726, p < 0.001; VRC, AUC 0.765, p < 0.001; VRM, AUC 0.578, p = 0.018; HuRP, AUC 0.912, p < 0.001; HuRC, AUC 0.952, p < 0.001; and HuRM, AUC 0.772, p < 0.001 in males. In females, VRP had an AUC of 0.813, p < 0.001; VRC, AUC 0.851, p < 0.001; VRM, AUC 0.623, p = 0.060; HuRP, AUC 0.904, p < 0.001; HuRC, AUC 0.934, p < 0.001; and HuRM, AUC 0.840, p < 0.001. The optimal cutoff values for predicting CKD in HuRP are 99.9 Hu for males and 98.4 Hu for females, while in HuRC are 120.1 Hu for males and 111.8 Hu for females. The kidney was effectively segmented by our AI-based 3D nnU-Net model for enhanced renal CT images. In terms of mild kidney injury, the CT values exhibited higher sensitivity compared to kidney volume. The correlation analysis revealed a stronger association between VRC, HuRP, and HuRC with renal function, while the association between VRP and HuRM was weaker, and the association between VRM was the weakest. Particularly, HuRP and HuRC demonstrated significant potential in predicting renal function. For diagnosing CKD, it is recommended to set the threshold values as follows: HuRP < 99.9 Hu and HuRC < 120.1 Hu in males, and HuRP < 98.4 Hu and HuRC < 111.8 Hu in females.

Development and validation of a deep learning-based model for diabetes diagnosis

Diabetes is a systemic disease caused by hyperglycemia, and the number of people with diabetes worldwide may reach 1.31 billion by 2050. The traditional diagnosis of early diabetes is difficult and inaccurate. Computer-aided automatic method has been widely applied in diabetes diagnosis at early stage. In order to realize automatic diagnosis of diabetes, this study proposed a deep neural network-based model for diabetes diagnosis, and its performance was compared with those of other classical machine leaning models. After data cleaning, Synthetic Minority Over-sampling Technique (SMOTE) sampling and feature selecting of the diabetes dataset, multiple models were applied to the features such as Body Mass Index (BMI), Age and other features for prediction. The results showed that Deep Neural Network (DNN) is the best diagnostic solution for diabetes with excellent performance with an accuracy of 99.5%. More specific features will be considered to ensure the accuracy and credibility of clinical diagnosis of diabetes.

A position-enhanced sequential feature encoding model for lung infections and lymphoma classification on CT images.

Differentiating pulmonary lymphoma from lung infections using CT images is challenging. Existing deep neural network-based lung CT classification models rely on 2D slices, lacking comprehensive information and requiring manual selection. 3D models that involve chunking compromise image information and struggle with parameter reduction, limiting performance. These limitations must be addressed to improve accuracy and practicality. We propose a transformer sequential feature encoding structure to integrate multi-level information from complete CT images, inspired by the clinical practice of using a sequence of cross-sectional slices for diagnosis. We incorporate position encoding and cross-level long-range information fusion modules into the feature extraction CNN network for cross-sectional slices, ensuring high-precision feature extraction. We conducted comprehensive experiments on a dataset of 124 patients, with respective sizes of 64, 20 and 40 for training, validation and testing. The results of ablation experiments and comparative experiments demonstrated the effectiveness of our approach. Our method outperforms existing state-of-the-art methods in the 3D CT image classification problem of distinguishing between lung infections and pulmonary lymphoma, achieving an accuracy of 0.875, AUC of 0.953 and F1 score of 0.889. The experiments verified that our proposed position-enhanced transformer-based sequential feature encoding model is capable of effectively performing high-precision feature extraction and contextual feature fusion in the lungs. It enhances the ability of a standalone CNN network or transformer to extract features, thereby improving the classification performance. The source code is accessible at https://github.com/imchuyu/PTSFE .

Evolutionary Approach to Optimal Oil Skimmer Assignment for Oil Spill Response: A Case Study.

We propose a genetic algorithm for optimizing oil skimmer assignments, introducing a tailored repair operation for constrained assignments. Methods essentially involve simulation-based evaluation to ensure adherence to South Korea's regulations. Results show that the optimized assignments, compared to current ones, reduced work time on average and led to a significant reduction in total skimmer capacity. Additionally, we present a deep neural network-based surrogate model, greatly enhancing efficiency compared to simulation-based optimization. Addressing inefficiencies in mobilizing locations that store oil skimmers, further optimization aimed to minimize mobilized locations and was validated through scenario-based simulations resembling actual situations. Based on major oil spills in South Korea, this strategy significantly reduced work time and required locations. These findings demonstrate the effectiveness of the proposed genetic algorithm and mobilized location minimization strategy in enhancing oil spill response operations.

Unraveling Financial Markets: Deep Neural Network-Based Models for Stock Price Prediction

This paper delves into the potential and challenges of leveraging deep neural networks (DNNs) in stock price forecasting. Traditional econometric models often grapple with the complexities of financial time series data, leading to the exploration of DNNs, especially architectures like Long Short-Term Memory (LSTM) networks, to capture intricate patterns in such data. While these networks present promising results, challenges such as model interpretability, non-stationarity of data, overfitting, and computational demands remain. The financial sector's increasing digitization and influx of alternative data offer a unique opportunity for DNNs, emphasizing their capability for automatic feature extraction. However, the integration of DNNs necessitates a multi-disciplinary approach, involving financial experts, data scientists, and computational specialists. The paper concludes with an optimistic outlook for the synergy between deep learning and traditional financial theories, aiming for a harmonious blend of modern computational techniques with foundational financial principles.

A CNN Model for Physical Activity Recognition and Energy Expenditure Estimation from an Eyeglass-Mounted Wearable Sensor.

Metabolic syndrome poses a significant health challenge worldwide, prompting the need for comprehensive strategies integrating physical activity monitoring and energy expenditure. Wearable sensor devices have been used both for energy intake and energy expenditure (EE) estimation. Traditionally, sensors are attached to the hip or wrist. The primary aim of this research is to investigate the use of an eyeglass-mounted wearable energy intake sensor (Automatic Ingestion Monitor v2, AIM-2) for simultaneous recognition of physical activity (PAR) and estimation of steady-state EE as compared to a traditional hip-worn device. Study data were collected from six participants performing six structured activities, with the reference EE measured using indirect calorimetry (COSMED K5) and reported as metabolic equivalents of tasks (METs). Next, a novel deep convolutional neural network-based multitasking model (Multitasking-CNN) was developed for PAR and EE estimation. The Multitasking-CNN was trained with a two-step progressive training approach for higher accuracy, where in the first step the model for PAR was trained, and in the second step the model was fine-tuned for EE estimation. Finally, the performance of Multitasking-CNN on AIM-2 attached to eyeglasses was compared to the ActiGraph GT9X (AG) attached to the right hip. On the AIM-2 data, Multitasking-CNN achieved a maximum of 95% testing accuracy of PAR, a minimum of 0.59 METs mean square error (MSE), and 11% mean absolute percentage error (MAPE) in EE estimation. Conversely, on AG data, the Multitasking-CNN model achieved a maximum of 82% testing accuracy in PAR, a minimum of 0.73 METs MSE, and 13% MAPE in EE estimation. These results suggest the feasibility of using an eyeglass-mounted sensor for both PAR and EE estimation.

The effect of data transformation on low-dimensional integration of single-cell RNA-seq

BackgroundRecent developments in single-cell RNA sequencing have opened up a multitude of possibilities to study tissues at the level of cellular populations. However, the heterogeneity in single-cell sequencing data necessitates appropriate procedures to adjust for technological limitations and various sources of noise when integrating datasets from different studies. While many analysis procedures employ various preprocessing steps, they often overlook the importance of selecting and optimizing the employed data transformation methods.ResultsThis work investigates data transformation approaches used in single-cell clustering analysis tools and their effects on batch integration analysis. In particular, we compare 16 transformations and their impact on the low-dimensional representations, aiming to reduce the batch effect and integrate multiple single-cell sequencing data. Our results show that data transformations strongly influence the results of single-cell clustering on low-dimensional data space, such as those generated by UMAP or PCA. Moreover, these changes in low-dimensional space significantly affect trajectory analysis using multiple datasets, as well. However, the performance of the data transformations greatly varies across datasets, and the optimal method was different for each dataset. Additionally, we explored how data transformation impacts the analysis of deep feature encodings using deep neural network-based models, including autoencoder-based models and proto-typical networks. Data transformation also strongly affects the outcome of deep neural network models.ConclusionsOur findings suggest that the batch effect and noise in integrative analysis are highly influenced by data transformation. Low-dimensional features can integrate different batches well when proper data transformation is applied. Furthermore, we found that the batch mixing score on low-dimensional space can guide the selection of the optimal data transformation. In conclusion, data preprocessing is one of the most crucial analysis steps and needs to be cautiously considered in the integrative analysis of multiple scRNA-seq datasets.

WindSeer: real-time volumetric wind prediction over complex terrain aboard a small uncrewed aerial vehicle

Real-time high-resolution wind predictions are beneficial for various applications including safe crewed and uncrewed aviation. Current weather models require too much compute and lack the necessary predictive capabilities as they are valid only at the scale of multiple kilometers and hours – much lower spatial and temporal resolutions than these applications require. Our work demonstrates the ability to predict low-altitude time-averaged wind fields in real time on limited-compute devices, from only sparse measurement data. We train a deep neural network-based model, WindSeer, using only synthetic data from computational fluid dynamics simulations and show that it can successfully predict real wind fields over terrain with known topography from just a few noisy and spatially clustered wind measurements. WindSeer can generate accurate predictions at different resolutions and domain sizes on previously unseen topography without retraining. We demonstrate that the model successfully predicts historical wind data collected by weather stations and wind measured by drones during flight.

A deep neural network for vehicle detection in aerial images

This research paper highlights the significance of vehicle detection in aerial images for surveillance systems, focusing on deep learning methods that outperform traditional approaches. However, the challenge of high computation complexity due to diverse vehicle appearances persists. The motivation behind this study is to highlight the crucial role of vehicle detection in aerial images for surveillance systems, emphasizing the superior performance of deep learning methods compared to traditional approaches. To address this, a lightweight deep neural network-based model is developed, striking a balance between accuracy and efficiency enabling real-time operation. The model is trained and evaluated on a standardized dataset, with extensive experiments demonstrating its ability to achieve accurate vehicle detection with significantly reduced computation costs, offering a practical solution for real-world aerial surveillance scenarios.

Human motion activity recognition and pattern analysis using compressed deep neural networks

ABSTRACT This work presents an on-device machine learning model with the ability to identify different mobility gestures called human activity recognition (HAR), which includes running, walking, squatting, jumping, and others. The data is collected through an Arduino Nano 33 BLE Sense board with a sampling rate of 119 Hz, which is embedded with an Inertial Measurement Unit (IMU) sensor. The same board is used as a microcontroller to identify human gestures by developing an end-to-end edge computing application. A deep neural network model is trained and then compressed for deployment on the board to create a self-contained, embedded device capable of identifying the type of gesture performed. Three deep learning models, namely Multi-Layer Perceptron (MLP), Convolutional Neural Network – Long Short Term Memory (CNN-LSTM) & CNN-Gated Recurrent Unit (CNN-GRU), are evaluated in the identification of the mobility gestures. The observed accuracy of the models is 96%, 97.1% and 97.8%, respectively, MLP, CNN-LSTM & CNN-GRU across the different gesture categories. The study shows the utility of embedded devices with deep neural network-based models, which can provide low cost, minimal power usage, and meet data privacy requirements in HAR.

Grammatical versus Spelling Error Correction: An Investigation into the Responsiveness of Transformer-Based Language Models Using BART and MarianMT

Text continues to remain a relevant form of representation for information. Text documents are created either in digital native platforms or through the conversion of other media files such as images and speech. While the digital native text is invariably obtained through physical or virtual keyboards, technologies such as OCR and speech recognition are utilised to transform the images and speech signals into text content. All these variety of mechanisms of text generation also introduce errors into the captured text. This project aims at analysing different kinds of errors that occur in text documents. The work employs two of the advanced deep neural network-based language models, namely, BART and MarianMT, to rectify the anomalies present in the text. Transfer learning of these models with available dataset is performed to finetune their capacity for error correction. A comparative study is conducted to investigate the effectiveness of these models in handling each of the defined error categories. It is observed that while both models can bring down the erroneous sentences by 20+%, BART can handle spelling errors far better (24.6%) than grammatical errors (8.8%).

A cost-sensitive deep neural network-based prediction model for the mortality in acute myocardial infarction patients with hypertension on imbalanced data.

Hypertension is one of the most serious risk factors and the leading cause of mortality in patients with cardiovascular diseases (CVDs). It is necessary to accurately predict the mortality of patients suffering from CVDs with hypertension. Therefore, this paper proposes a novel cost-sensitive deep neural network (CSDNN)-based mortality prediction model for out-of-hospital acute myocardial infarction (AMI) patients with hypertension on imbalanced data. The synopsis of our research is as follows. First, the experimental data is extracted from the Korea Acute Myocardial Infarction Registry-National Institutes of Health (KAMIR-NIH) and preprocessed with several approaches. Then the imbalanced experimental dataset is divided into training data (80%) and test data (20%). After that, we design the proposed CSDNN-based mortality prediction model, which can solve the skewed class distribution between the majority and minority classes in the training data. The threshold moving technique is also employed to enhance the performance of the proposed model. Finally, we evaluate the performance of the proposed model using the test data and compare it with other commonly used machine learning (ML) and data sampling-based ensemble models. Moreover, the hyperparameters of all models are optimized through random search strategies with a 5-fold cross-validation approach. In the result, the proposed CSDNN model with the threshold moving technique yielded the best results on imbalanced data. Additionally, our proposed model outperformed the best ML model and the classic data sampling-based ensemble model with an AUC of 2.58% and 2.55% improvement, respectively. It aids in decision-making and offers a precise mortality prediction for AMI patients with hypertension.

Skin cancer classification using Inception Network

Since skin disease is a universally recognized condition among humans, there has been a growing interest in utilizing intelligent systems to classify various skin ailments. This line of research in deep learning holds immense significance for dermatologists. However, accurately determining the presence of a disease is a formidable task due to the intricate nature of skin texture and the visual similarities between different diseases. To address this challenge, skin images undergo filtration to eliminate unwanted noise and undergo further processing to enhance the overall quality of the image. The primary purpose of this study is to construct a deep neural network-based model that is capable of automatically classifying several types of skin cancer as either melanoma or non-melanoma with a prominent level of accuracy. We propose an optimized Inception architecture, in which the InceptionNet model is enhanced with data augmentation and basic layers. The strategy that has been proposed enhances the model's capacity to deal with incomplete and inconsistent data. A dataset of 2637 skin images are used to demonstrate the benefits of the technique that has been proposed. We analyze the performance of the suggested method by looking at its precision, sensitivity, specificity, F1-score, and area under the ROC curve. Proposed InceptionNet provides an accuracy of 84.39% and 85.94%, respectively for Adam and Nadam optimizer. The training process in each subsequent layer exhibits a notable enhancement in effectiveness. An examination of this inquiry can assist experts in making early diagnoses, thereby providing them with insight into the nature of the infection and enabling them to initiate the necessary treatment, if deemed necessary.

WFSS: weighted fusion of spectral transformer and spatial self-attention for robust hyperspectral image classification against adversarial attacks

The emergence of adversarial examples poses a significant challenge to hyperspectral image (HSI) classification, as they can attack deep neural network-based models. Recent adversarial defense research tends to establish global connections of spatial pixels to resist adversarial attacks. However, it cannot yield satisfactory results when only spatial pixel information is used. Starting from the premise that the spectral band is equally important for HSI classification, this paper explores the impact of spectral information on model robustness. We aim to discover potential relationships between different spectral bands and establish global connections to resist adversarial attacks. We design a spectral transformer based on the transformer structure to model long-distance dependency relationships among spectral bands. Additionally, we use a self-attention mechanism in the spatial domain to develop global relationships among spatial pixels. Based on the above framework, we further explore the influence of both spectral and spatial domains on the robustness of the model against adversarial attacks. Specifically, a weighted fusion of spectral transformer and spatial self-attention (WFSS) is designed to achieve the multi-scale fusion of spectral and spatial connections, which further improves the model’s robustness. Comprehensive experiments on three benchmarks show that the WFSS framework has superior defensive capabilities compared to state-of-the-art HSI classification methods.

Microscopic identification of foodborne bacterial pathogens based on deep learning method

Accurate and rapid detection of foodborne bacterial pathogens is critical for food quality control. Nowadays, tracking morphological bacterial properties using microscope is still a priority at the grass-roots food supervision department due to its simplicity and low cost. However, the method requires highly professional personnel and there are certain misjudgments in the process of analysis. Automatically recognizing foodborne pathogen using deep learning algorithm to replace manual microscopy will not only reduce expert cost, artificial misjudgment, and operation time in detection, but also provide more objective and accurate identification. Here, we firstly constructed a high-quality and large-scale dataset of foodborne pathogenic bacteria, allowing the deep learning-based model to be efficiently trained and achieve accurate identification. The deep convolutional neural network-based model is capable of identifying six common foodborne pathogens, including Escherichia coli (O157:H7), Vibrio parahaemolyticus, Staphylococcus aureus, Bacillus cereus, Salmonella typhi, and Streptococcus hemolyticus, with accuracy rates of 90%–100%. This method can assist or replace the manual microscopic inspection step in traditional detection methods, and is promising to break through the traditional approach that heavily relies on manual judgment, greatly reduce the cost of experts and human errors, and provide rapid, accurate, and powerful discriminatory results in large quantities for detection.