Deep Neural Network Classifier Research Articles

Developing picking route policies with genetic algorithms and order batching with deep neural networks in picker to part warehouses

ABSTRACT This paper proposes a hybrid approach to warehouse management, combining Genetic Algorithm (GA) optimization for picking routes with Deep Neural Network (DNN) classification for order batching. The GA-DNN method significantly reduces travel distances while lowering computational time. Experimental results demonstrate up to 69.01% reduction in travel distance in large warehouses, outperforming traditional methods. This study underscores the effectiveness of GA-DNN solutions in addressing modern logistics challenges.

CATIL: Customized Adversarial Training based on Instance Loss

Adversarial training is one of the most effective adversarial defense methods currently recognized. It enhances the robustness of deep neural network (DNN) classifiers by generating adversarial samples. However, current adversarial training methods cannot effectively trade off the robust accuracy and natural accuracy when training DNN classifiers, and are prone to overfit. To solve these problems, we propose Customized Adversarial Training based on Instance Loss (CATIL), aiming to improve robust accuracy and natural accuracy while alleviating overfitting. We first comprehensively consider the influencing factors of adversarial training and propose the comprehensive customization strategy (CCS), which crafts unique attack strategies for each sample, fine-tunes the classifier's decision boundary, and boosts the robustness of the DNN classifier without compromising its natural accuracy. Second, the loss adjustment strategy (LAS) is designed to update the attack strategy according to the loss value. This increases the fitting difficulty of the DNN classifier and alleviates the overfitting problem. Finally, numerous experiments show that CATIL can effectively enhance robust accuracy and alleviate the overfitting problem without significantly compromising natural accuracy. When evaluating CIFAR-10 on Wide ResNet, CATIL achieves the best performance in both natural and robust accuracy compared to all benchmarks.

A Theory-Based Explainable Deep Learning Architecture for Music Emotion

This paper develops a theory-based, explainable deep learning convolutional neural network classifier to predict the time-varying emotional response to music.

Grasshopper-Inspired Deep Neural Network for Enhanced Breast Cancer Classification

Early-stage disease diagnosis is critical for effective treatment, and software-aided design can analyze disease architecture for timely detection. Many fail to identify disease severity before it becomes chronic, contributing to global mortality rates. Breast cancer, a prime reason of death among women, can be treated if detected early. Computer-aided diagnosis aids practitioners in accurately assessing disease criticality. This paper introduces an automated diagnosis system utilizing an enhanced Grasshopper Optimization technique and a Deep Neural Network (DNN) classifier. The Grasshopper Algorithm optimally selects features from segmented images, extracted through SIFT and BRISK hybrid techniques. The DNN classifies breast cancer using a partitioned dataset for training and testing. Performance metrics, including accuracy, precision, F-measure, and recall, demonstrate that the proposed system significantly outperforms existing methods, with an F-measure improvement of 5.1% and an accuracy increase of 11.19%.

Fixed frequency range empirical wavelet transform based acoustic and entropy features for speech emotion recognition

The primary goal of automated speech emotion recognition (SER) is to accurately and effectively identify each specific emotion conveyed in a speech signal utilizing machines such as computers and mobile devices. The widespread recognition of the popularity of SER among academics for three decades is mainly attributed to its broad application in practical scenarios. The utilization of SER has proven to be beneficial in various fields, such as medical intervention, bolstering safety strategies, conducting vigil functions, enhancing online search engines, enhancing road safety, managing customer relationships, strengthening the connection between machines and humans, and numerous other domains. Many researchers have used diverse methodologies, such as the integration of different attributes, the use of different feature selection techniques, and designed a hybrid or complex model using more than one classifier, to augment the effectiveness of emotion classification. In our study, we used a novel technique called the fixed frequency range empirical wavelet transform (FFREWT) filter bank decomposition method to extract the features, and then used those features to accurately identify each and every emotion in the speech signal. The FFREWT filter bank method segments the speech signal frame (SSF) into many sub-signals or modes. We used each FFREWT-based decomposed mode to get features like the mel frequency cepstral coefficient (MFCC), approximate entropy (ApEn), permutation entropy (PrEn), and increment entropy (IrEn). We then used the different combinations of the proposed FFREWT-based feature sets and the deep neural network (DNN) classifier to classify the speech emotion. Our proposed method helps to achieve an emotion classification accuracy of 89.35%, 84.69%, and 100% using the combinations of the proposed FFREWT-based feature (MFCC + ApEn + PrEn + IrEn) for the EMO-DB, EMOVO, and TESS datasets, respectively. Our experimental results were compared with the other methods, and we found that the proposed FFREWT-based feature combinations with a DNN classifier performed better than the state-of-the-art methods in SER.

Convolutional Neural Networks: A Comprehensive Evaluation and Benchmarking of Pooling Layer Variants

Convolutional Neural Networks (CNNs) are a class of deep neural networks that have proven highly effective in areas such as image and video recognition. CNNs typically include several types of layers, such as convolutional layers, activation layers, pooling layers, and fully connected layers, all of which contribute to the network’s ability to recognize patterns and features. The pooling layer, which often follows the convolutional layer, is crucial for reducing computational complexity by performing down-sampling while maintaining essential features. This layer’s role in balancing the symmetry of information across the network is vital for optimal performance. However, the choice of pooling method is often based on intuition, which can lead to less accurate or efficient results. This research compares various standard pooling methods (MAX and AVERAGE pooling) on standard datasets (MNIST, CIFAR-10, and CIFAR-100) to determine the most effective approach in preserving detail, performance, and overall computational efficiency while maintaining the symmetry necessary for robust CNN performance.

Performance Comparison between Deep Neural Network and Machine Learning based Classifiers for Huntington Disease Prediction from Human DNA Sequence.

Huntington Disease (HD) is a type of neurodegenerative disorder which causes problems like psychiatric disturbances, movement problem, weight loss and problem in sleep. It needs to be addressed in earlier stage of human life. Nowadays Deep Learning (DL) based system could help physicians provide second opinion in treating patient's disease. In this work, human Deoxyribo Nucleic Acid (DNA) sequence is analyzed using Deep Neural Network (DNN) algorithm to predict the HD disease. The main objective of this work is to identify whether the human DNA is affected by HD or not. Human DNA sequences are collected from National Center for Biotechnology Information (NCBI) and synthetic human DNA data are also constructed for process. Then numerical conversion of human DNA sequence data is done by Chaos Game Representation (CGR) method. After that, numerical values of DNA data are used for feature extraction. Mean, median, standard deviation, entropy, contrast, correlation, energy and homogeneity are extracted. Additionally, the following features such as counts of adenine, thymine, guanine and cytosine are extracted from the DNA sequence data itself. The extracted features are used as input to the DNN classifier and other machine learning based classifiers such as NN (Neural Network), Support Vector Machine (SVM), Random Forest (RF) and Classification Tree with Forward Pruning (CTWFP). Six performance measures are used such as Accuracy, Sensitivity, Specificity, Precision, F1 score and Mathew Correlation Co-efficient (MCC). The study concludes DNN, NN, SVM, RF achieve 100% accuracy and CTWFP achieves accuracy of 87%.

Enhancing Real-Time Cursor Control with Motor Imagery and Deep Neural Networks for Brain–Computer Interfaces

This paper advances real-time cursor control for individuals with motor impairments through a novel brain–computer interface (BCI) system based solely on motor imagery. We introduce an enhanced deep neural network (DNN) classifier integrated with a Four-Class Iterative Filtering (FCIF) technique for efficient preprocessing of neural signals. The underlying approach is the Four-Class Filter Bank Common Spatial Pattern (FCFBCSP) and it utilizes a customized filter bank for robust feature extraction, thereby significantly improving signal quality and cursor control responsiveness. Extensive testing under varied conditions demonstrates that our system achieves an average classification accuracy of 89.1% and response times of 663 milliseconds, illustrating high precision in feature discrimination. Evaluations using metrics such as Recall, Precision, and F1-Score confirm the system’s effectiveness and accuracy in practical applications, making it a valuable tool for enhancing accessibility for individuals with motor disabilities.

HepNet: Deep Neural Network for Classification of Early-Stage Hepatic Steatosis Using Microwave Signals.

Hepatic steatosis, a key factor in chronic liver diseases, is difficult to diagnose early. This study introduces a classifier for hepatic steatosis using microwave technology, validated through clinical trials. Our method uses microwave signals and deep learning to improve detection to reliable results. It includes a pipeline with simulation data, a new deep-learning model called HepNet, and transfer learning. The simulation data, created with 3D electromagnetic tools, is used for training and evaluating the model. HepNet uses skip connections in convolutional layers and two fully connected layers for better feature extraction and generalization. Calibration and uncertainty assessments ensure the model's robustness. Our simulation achieved an F1-score of 0.91 and a confidence level of 0.97 for classifications with entropy ≤0.1, outperforming traditional models like LeNet (0.81) and ResNet (0.87). We also use transfer learning to adapt HepNet to clinical data with limited patient samples. Using 1H-MRS as the standard for two microwave liver scanners, HepNet achieved high F1-scores of 0.95 and 0.88 for 94 and 158 patient samples, respectively, showing its clinical potential.

AppPoet: Large language model based android malware detection via multi-view prompt engineering

Due to the vast array of Android applications, their multifarious functions and intricate behavioral semantics, attackers can adopt various tactics to conceal their genuine attack intentions within legitimate functions. However, numerous learning-based methods suffer from a limitation in mining behavioral semantic information, thus impeding the accuracy and efficiency of Android malware detection. Besides, the majority of existing learning-based methods are weakly interpretive and fail to furnish researchers with effective and readable detection reports. Inspired by the success of the Large Language Models (LLMs) in natural language understanding, we propose AppPoet, a LLM-assisted multi-view system for Android malware detection. Firstly, AppPoet employs a static method to comprehensively collect application features and formulate various observation views. Then, using our carefully crafted multi-view prompt templates, it guides the LLM to generate function descriptions and behavioral summaries for each view, enabling deep semantic analysis of the views. Finally, we collaboratively fuse the multi-view information to efficiently and accurately detect malware through a deep neural network (DNN) classifier and then generate the human-readable diagnostic reports. Experimental results demonstrate that our method achieves a detection accuracy of 97.15% and an F1 score of 97.21%, which is superior to the baseline methods. Furthermore, the case study evaluates the effectiveness of our generated diagnostic reports.

Deep learning as Ricci flow.

Deep neural networks (DNNs) are powerful tools for approximating the distribution of complex data. It is known that data passing through a trained DNN classifier undergoes a series of geometric and topological simplifications. While some progress has been made toward understanding these transformations in neural networks with smooth activation functions, an understanding in the more general setting of non-smooth activation functions, such as the rectified linear unit (ReLU), which tend to perform better, is required. Here we propose that the geometric transformations performed by DNNs during classification tasks have parallels to those expected under Hamilton's Ricci flow-a tool from differential geometry that evolves a manifold by smoothing its curvature, in order to identify its topology. To illustrate this idea, we present a computational framework to quantify the geometric changes that occur as data passes through successive layers of a DNN, and use this framework to motivate a notion of 'global Ricci network flow' that can be used to assess a DNN's ability to disentangle complex data geometries to solve classification problems. By training more than 1500 DNN classifiers of different widths and depths on synthetic and real-world data, we show that the strength of global Ricci network flow-like behaviour correlates with accuracy for well-trained DNNs, independently of depth, width and data set. Our findings motivate the use of tools from differential and discrete geometry to the problem of explainability in deep learning.

Oxygen Content Control in the Electroslag Remelting Process: An Incremental Learning Strategy Based on Optimized Wasserstein Generative Adversarial Network with Gradient Penalty Data Augmentation

Electroslag remelting (ESR) is essential for producing high‐end special steel, but its complex process and numerous influencing factors make quality control challenging. This study addresses oxygen content control during ESR using a big data machine learning approach. An incremental learning strategy is proposed based on an optimized Wasserstein generative adversarial network with gradient penalty (WGAN‐GP) for data enhancement, focusing on G20Cr2Ni4A bearing steel. The WGAN‐GP model enhances time‐series data and metadata, utilizing long short‐term memory networks, fully connected networks, and attention mechanisms. The effectiveness of data enhancement is verified using a deep neural network classifier and statistical methods. Data is divided into historical and data streams, with an incremental learning strategy based on histogram gradient boosting regression trees to prevent catastrophic forgetting and improve efficiency through knowledge distillation and real‐time hyperparameter adjustment. Results show that the data augmentation method significantly improves model generalization and accuracy in small sample metallurgy. The incremental learning strategy enhances prediction accuracy for oxygen content, contributing to better cleanliness quality of electroslag steel. This study offers a novel approach for addressing small sample challenges in metallurgical processes.

Comparison of three artificial intelligence methods for predicting 90% quantile interval of future insulin sensitivity of intensive care patients

Insulin dosing of hyperglycemic patients in the intensive care unit (ICU) is a complex and nonlinear clinical control problem. Recent model-based glycemic control protocols predict a patient-specific and time-specific future insulin sensitivity distribution, which defines the future patient state in response to insulin and nutrition inputs. The prediction methods provide a 90% confidence interval for a future insulin sensitivity distribution for a given time horizon, making the prediction problem more specific compared to common prediction problems where the aim is to predict the expected value of the given stochastic parameter. This study proposes three alternative artificial intelligence-based insulin sensitivity prediction methods to improve the prediction accuracy and make prediction parameters better fit the clinical requirements. The proposed prediction methods use different neural network models: a classification deep neural network model, a Mixture Density Network model, and a Quantile Regression-based model. A large patient data set was used to create the neural network models, including 2357 patients and 92646 blood glucose measurements from three clinical sites (Christchurch, New Zealand, Gyula, Hungary, and Liege, Belgium). Prediction accuracy was assessed by statistical metrics expressing clinical requirements, as well as via validated in-silico virtual patient simulations comparing the clinical performance of a proven glycaemic control protocol using the alternative prediction methods to assess impact on glycemic control performance and thus the need for these alternative models.

Classification of Lung Cancer using Pre-Trained Deep Learning Models

Many Computer Assisted Diagnosis (CAD) systems have been designed and used in recent past for diagnosingdifferenttypes of cancer. Identification of carcinoma at an earlier stage is more important, and it is made possible due to the use modern image processing and deep learning methods.The occurrence of Lung cancer is seen to be increased and Computed Tomography (CT) scan images were utilized in investigation to locate and classify lung cancer, also for determining the severity of those cancer. This study is aimed at employing pre-trained deep neural networks for classification of lung cancer images. A gaussian-based approach is used to segment CT scan images. In this research exploits a transfer learning-based classification method for the chest CT images acquired from Cancer Image Archive and available in the Kaggle platform. Pre- trained models VGG and RESNET were trained using segmented chest CT images, and their performance was evaluated using different optimization algorithms were analyzed. Keywords: Computer Aided Diagnosis, lung cancer, deep learning, CT image, gaussian, transfer learning, pre-trained models, optimization algorithms.

Machine-learning-based impairment-aware dynamic RMSCA in multi-core elastic optical networks

This paper presents a routing, modulation, spectrum, and core assignment (RMSCA) algorithm for space-division-multiplexing-based elastic optical networks (SDM-EONs) comprising multi-core links. A network state-dependent route and core selection method is proposed using a deep neural network (DNN) classifier. The DNN is trained using a metaheuristic optimization algorithm to predict lightpath suitability, considering the quality of transmission and resource availability. Physical layer impairments, including inter-core crosstalk, amplified spontaneous emission, and Kerr fiber nonlinearities, are considered, and a random forest (RF)-based link noise estimator is proposed. A feature importance selection analysis is provided for all the features considered for the DNN classifier and the RF link noise estimator. The proposed machine-learning-enabled RMSCA approach is evaluated on three network topologies, USNET, NSFNET, and COST-239 with 7-core and 12-core fiber links. It is shown to be superior in terms of blocking probability, bandwidth blocking probability, and acceptable computational speed compared to the standard and published benchmarks at different traffic loads.

Fault diagnosis method of rotating machinery based on MSResNet feature fusion and CAM

To solve the problem of noise interference, it is difficult to extract multi-scale information from complex vibration signals in fault diagnosis with the single-scale convolution kernel of classical deep learning model convolutional neural network (CNN). Therefore, a fault diagnosis method of rotating machinery based on MSResNet feature fusion and CAM is proposed. The residual network (ResNet) and multi-scale convolutional neural network (MSCNN) are combined to extract multi-scale feature information according to convolution kernels with different sizes, so as to avoid the loss of single-scale feature extraction. Make full use of the advantages of the residual network to skip the connection and prevent the feature information extracted by the multi-scale convolution kernel from being lost when the convolution layer propagates forward. In addition, in order to avoid the interference of invalid features after multi-scale information feature fusion, a channel attention mechanism module (CAM) is introduced to screen important features adaptively. The effectiveness of MSResNet-CAM is verified by the bearing data set of Western Reserve University (CWRU) and the data set of QPZZ-II gearbox, and the anti-noise ability is verified by adding noise to the two data sets. The experimental results show that MSResNet-CAM has the characteristics of high fault classification accuracy, good robustness and strong anti-noise ability.

Neural Operator Approximations for Boundary Stabilization of Cascaded Parabolic PDEs

ABSTRACTThis article proposes a novel method to accelerate the boundary feedback control design of cascaded parabolic difference equations (PDEs) through DeepONet. The backstepping method has been widely used in boundary control problems of PDE systems, but solving the backstepping kernel function can be time‐consuming. To address this, a neural operator (NO) learning scheme is leveraged for accelerating the control design of cascaded parabolic PDEs. DeepONet, a class of deep neural networks designed for approximating nonlinear operators, has shown potential for approximating PDE backstepping designs in recent studies. Specifically, we focus on approximating gain kernel PDEs for two cascaded parabolic PDEs. We utilize neural operators to map only two kernel functions, while the other two are computed using the analytical solution, thus simplifying the training process. We establish the continuity and boundedness of the kernels, and demonstrate the existence of arbitrarily close DeepONet approximations to the kernel PDEs. Furthermore, we demonstrate that the DeepONet approximation gain kernels ensure stability when replacing the exact backstepping gain kernels. Notably, DeepONet operator exhibits computation speeds two orders of magnitude faster than PDE solvers for such gain functions, and their theoretically proven stabilizing capability is validated through simulations.

STNMDA: A Novel Model for Predicting Potential Microbe-Drug Associations with Structure-Aware Transformer

Introduction: Microbes are intimately involved in the physiological and pathological processes of numerous diseases. There is a critical need for new drugs to combat microbe-induced diseases in clinical settings. Predicting potential microbe-drug associations is, therefore, essential for both disease treatment and novel drug discovery. However, it is costly and time-consuming to verify these relationships through traditional wet lab approaches. Methods: We proposed an efficient computational model, STNMDA, that integrated a StructureAware Transformer (SAT) with a Deep Neural Network (DNN) classifier to infer latent microbedrug associations. The STNMDA began with a “random walk with a restart” approach to construct a heterogeneous network using Gaussian kernel similarity and functional similarity measures for microorganisms and drugs. This heterogeneous network was then fed into the SAT to extract attribute features and graph structures for each drug and microbe node. Finally, the DNN classifier calculated the probability of associations between microbes and drugs. Results: Extensive experimental results showed that STNMDA surpassed existing state-of-the-art models in performance on the MDAD and aBiofilm databases. In addition, the feasibility of STNMDA in confirming associations between microbes and drugs was demonstrated through case validations. Conclusion: Hence, STNMDA showed promise as a valuable tool for future prediction of microbedrug associations.

Detection of Encroachment in Civil Structure Using Dynamic Learning Techniques

Identifying cracks through visual inspection and automated surveys are both effective methods. While both approaches yield satisfactory distress analysis results, automated crack recognition technology stands out for its speed and cost-effectiveness compared to traditional human visual detection methods. This study introduces an innovative approach, namely "crack encroachment in concrete structures classified using DCNN" which employs a Deep Convolutional Neural Network. To enhance the precision of crack detection, the input image undergoes denoising through ADNLMF (Anisotropic Diffusion Non-Local Mean Filtering), preserving edges, textures, and features. Crack discrimination between crack and non-crack images was achieved using Yolo v3, and a deep convolutional neural network classifier was employed to identify specific crack types based on their widths, utilizing crack width transform. This method not only enhances accuracy but also reduces network complexity and processing time. A comprehensive performance comparison with existing crack identification techniques indicates that our proposed methodology for concrete structure crack recognition produces superior results.

Performance of Deep Feed-Forward Neural Network Algorithm Based on Content-Based Filtering Approach

Background: Selecting a restaurant in a diverse city like Bandung can be challenging. This study leverages Twitter data and local restaurant information to develop an advanced recommendation system to improve decision-making. Objective: The system integrates content-based filtering (CBF) with deep feedforward neural network (DFF) classification to enhance the accuracy and relevance of restaurant recommendations. Methods: Data was sourced from Twitter and PergiKuliner, with restaurant-related tweets converted into rating values. The CBF combined Bag of Words (BoW) and cosine similarity, followed by DFF classification. SMOTE was applied during training to address data imbalance. Results: The initial evaluation of CBF showed a Mean Absolute Error (MAE) of 0.0614 and a Root Mean Square Error (RMSE) of 0.0934. The optimal DFF configuration in the first phase used two layers with 32/16 nodes, a dropout rate of 0.3, and a 20% test size. This setup achieved an accuracy of 81.08%, precision of 82.89%, recall of 76.93%, and f1-scores of 79.23%. In the second phase, the RMSprop optimizer improved accuracy to 81.30%, and tuning the learning rate to 0.0596 further increased accuracy to 89%, marking a 9.77% improvement. Conclusion: The research successfully developed a robust recommendation system, significantly improving restaurant recommendation accuracy in Bandung. The 9.77% accuracy increase highlights the importance of hyperparameter tuning. SMOTE also proved crucial in balancing the dataset, contributing to a well-rounded learning model. Future studies could explore additional contextual factors and experiment with recurrent or convolutional neural networks to enhance performance further.