Neural Network-based Classification Research Articles

CTCTime: A New Model for Unidimensional Time Series Classification

One-dimensional time series classification has always been an important research direction, playing an irreplaceable role in various fields. With the rapid development of deep learning, most traditional time series classification methods have gradually been replaced by neural network-based classification methods. At present, in the field of time series classification, models that perform well are mostly based on Convolutional Neural Networks (CNNs), while models with the Transformer architecture, which have shown outstanding performance in natural language processing and computer vision, do not stand out. In order to change this situation, We have investigated the feasibility of training models based on a single Transformer architecture directly on small datasets without additional data processing. Furthermore, we propose a new model called CTCTime, which combines the Transformer architecture with CNNs to address one-dimensional time series classification problems. We compared CTCTime with 13 traditional algorithms on 44 datasets from the UCR archive and with 7 advanced methods on 85 datasets. The UCR datasets are classic datasets for one-dimensional time series classification tasks. The experimental results demonstrate its feasibility, accuracy, and scalability.

Pattern recognition of drying lysozyme-glucose droplets using machine learning classifiers

Out-of-equilibrium processes, such as sessile droplet drying, often result in distinctive macroscopic residual patterns in systems containing molecules, proteins, and colloids. Protein-glucose mixtures are particularly effective models for studying the behavior of complex fluids containing biomolecules. This study investigates the drying patterns of lysozyme droplets with varying initial glucose concentrations. Without glucose, the crack patterns are chaotic and dispersed throughout the droplet. Interestingly, cracks predominantly form around the droplet edges at intermediate glucose concentrations, while the deposits become uniform and crack-free at high glucose concentrations. To understand and classify the unique patterns related to the initial compositional changes, we developed an automated pattern recognition pipeline. We used two methods for analyzing images captured throughout the drying process. The first method involved extracting statistical textural parameters from the images as quantitative features for machine learning classifiers. The second method utilized a neural network-based classifier to directly classify the images, achieving an accuracy of 97%. The results demonstrate the effectiveness of using images from the entire drying process, not just the final images, for pattern classification. This approach may be useful in gaining a fundamental understanding of unique crack pattern that emerge when glucose is added to a protein solution.

Low-power adaptive sampling electronic nose system with a Radon transform-based convolutional neural network for optimized gas recognition

The electronic nose (E-nose) systems provide great promise in capturing information about the identity of gaseous chemical analytes as well as their temporal variation. However, the regular and dense sampling events of the E-nose often lead to a substantial amount of redundancy in the temporal structure, which in turn limits their application in efficiency-sensitive scenarios such as advanced robotics and the Internet of Things. Herein, we introduce a novel operating mechanism for E-noses, which dynamically adjusts the time interval of operation to reduce power consumption and information content through an adaptive sampling mechanism. Three strategic schemes with different operating principles are presented to implement this mechanism: the accurate scheme targets the optimized discrimination accuracy, the vague scheme is oriented toward power-sensitive scenarios, and the balanced scheme aims to balance cost and benefit. In order to ensure that the adaptively sampled inhomogeneous data can be compatible with each other, we further propose an algorithm based on the Radon transform that can convert non-uniform time series with timestamp information into the same size tensor. Finally, we designed a convolutional neural network-based classification model for gaseous chemical analytes, providing guidance on scheme and parameter selection with an accuracy of up to 99% and power savings of up to 96%. Overall, this work provides a novel solution to optimize temporal redundancy in E-nose systems and a generalized approach to the corresponding deep-learning data processing.

Classifier surrogates: sharing AI-based searches with the world

In recent years, neural network-based classification has been used to improve data analysis at collider experiments. While this strategy proves to be hugely successful, the underlying models are not commonly shared with the public and rely on experiment-internal data as well as full detector simulations. We show a concrete implementation of a newly proposed strategy, so-called Classifier Surrogates, to be trained inside the experiments, that only utilise publicly accessible features and truth information. These surrogates approximate the original classifier distribution, and can be shared with the public. Subsequently, such a model can be evaluated by sampling the classification output from high-level information without requiring a sophisticated detector simulation. Technically, we show that continuous normalizing flows are a suitable generative architecture that can be efficiently trained to sample classification results using conditional flow matching. We further demonstrate that these models can be easily extended by Bayesian uncertainties to indicate their degree of validity when confronted with unknown inputs by the user. For a concrete example of tagging jets from hadronically decaying top quarks, we demonstrate the application of flows in combination with uncertainty estimation through either inference of a mean-field Gaussian weight posterior, or Monte Carlo sampling network weights.

Machine Learning-Based Gesture Recognition Glove: Design and Implementation.

In the evolving field of human-computer interaction (HCI), gesture recognition has emerged as a critical focus, with smart gloves equipped with sensors playing one of the most important roles. Despite the significance of dynamic gesture recognition, most research on data gloves has concentrated on static gestures, with only a small percentage addressing dynamic gestures or both. This study explores the development of a low-cost smart glove prototype designed to capture and classify dynamic hand gestures for game control and presents a prototype of data gloves equipped with five flex sensors, five force sensors, and one inertial measurement unit (IMU) sensor. To classify dynamic gestures, we developed a neural network-based classifier, utilizing a convolutional neural network (CNN) with three two-dimensional convolutional layers and rectified linear unit (ReLU) activation where its accuracy was 90%. The developed glove effectively captures dynamic gestures for game control, achieving high classification accuracy, precision, and recall, as evidenced by the confusion matrix and training metrics. Despite limitations in the number of gestures and participants, the solution offers a cost-effective and accurate approach to gesture recognition, with potential applications in VR/AR environments.

Convolutional Neural Network-Based Drone Detection and Classification Using Overlaid Frequency-Modulated Continuous-Wave (FMCW) Range-Doppler Images.

This paper proposes a novel drone detection method based on a convolutional neural network (CNN) utilizing range-Doppler map images from a frequency-modulated continuous-wave (FMCW) radar. The existing drone detection and identification techniques, which rely on the micro-Doppler signature (MDS), face challenges when a drone is small or located far away, leading to performance degradation due to signal attenuation and faint (MDS). In order to address these issues, this paper suggests a method where multiple time-series range-Doppler images from an FMCW radar are overlaid onto a single image and fed to a CNN. The experimental results, using actual data for three different drone sizes, show significant performance improvements in drone detection accuracy compared to conventional methods.

Review on spiking neural network-based ECG classification methods for low-power environments.

This paper reviews arrhythmia classification studies using electrocardiogram (ECG) signals. Research on automatically diagnosing arrhythmia in daily life has been actively underway for early detection and treatment of heart disease. Development of automatic arrhythmia classification using ECG signal began based on handcrafted morphological feature extraction and machine learning-based classification methods. As deep neural networks (DNN) show excellent performance in the signal processing field, studies using various types of DNN are also being conducted in ECG classification. However, these DNN-based studies have extremely high computational complexity, making it challenging to perform real-time classification, and are unsuitable for low-power environments such as wearable devices due to high power consumption. Currently, research based on spiking neural network (SNN), which mimics the low-power operation of the human nervous system, is attracting attention as a method that can dramatically reduce complexity and power consumption. The classification accuracy of the SNN-based ECG classification studies is close to that of the DNN-based studies. When combined with neuromorphic hardware, it shows ultra-low-power performance, suggesting the possibility of use in lightweight devices. In this paper, the SNN-based ECG classification studies for low-power environments are mainly reviewed, and prior to this, conventional and DNN-based ECG classification studies are also reviewed. We hope that this review will be helpful to researchers and engineers interested in the field of ECG classification.

Explainable AI (XAI) Techniques for Convolutional Neural Network-Based Classification of Drilled Holes in Melamine Faced Chipboard

The furniture manufacturing sector faces significant challenges in machining composite materials, where quality issues such as delamination can lead to substandard products. This study aims to improve the classification of drilled holes in melamine-faced chipboard using Explainable AI (XAI) techniques to better understand and interpret Convolutional Neural Network (CNN) models’ decisions. We evaluated three CNN architectures (VGG16, VGG19, and ResNet101) pretrained on the ImageNet dataset and fine-tuned on our dataset of drilled holes. The data consisted of 8526 images, divided into three categories (Green, Yellow, Red) based on the drill’s condition. We used 5-fold cross-validation for model evaluation and applied LIME and Grad-CAM as XAI techniques to interpret the model decisions. The VGG19 model achieved the highest accuracy of 67.03% and the lowest critical error rate among the evaluated models. LIME and Grad-CAM provided complementary insights into the decision-making process of the model, emphasizing the significance of certain features and regions in the images that influenced the classifications. The integration of XAI techniques with CNN models significantly enhances the interpretability and reliability of automated systems for tool condition monitoring in the wood industry. The VGG19 model, combined with LIME and Grad-CAM, offers a robust solution for classifying drilled holes, ensuring better quality control in manufacturing processes.

CNN-based Network Intrusion Detection and Classification Model for Cyber-Attacks

A Convolution Neural Network (CNN)-based Network Intrusion Detection Model for Cyber-attacks is of great value in identifying and classifying attacks on any network. The Knowledge Discovery in Database Cup '99 dataset containing approximately 4,900,000 single connection vectors was divided into two phases; 75% of the total dataset was used during the learning process of the machine learning technique, while 25% was used on a fully trained model to validate and evaluate its performance. The model's performance indicated that it can detect and classify different classes of attacks with an accuracy of 98% with 20 epochs at a 0.001 learning rate using machine learning. The model loss for the training and validation was 7.48% and 7.98%, respectively, over 20 epochs, which implies that the model performed better on the training dataset. This study demonstrated that the convolutional Neural network-based Network Intrusion Detection and classification model shows high detection and low false negative rates. The CNN model offers a high detection rate and fidelity to unknown attacks, i.e., it can differentiate between already-seen attacks and new zero-day attacks. At the end of the experiment, the proposed approach is suitable in modeling the network IDS for detecting intrusion attacks on computer networks thereby enabling a secured environment for the proper functioning of the system

An improved deep convolutional neural network-based YouTube video classification using textual features

Video content on the web platform has increased explosively during the past decade, thanks to the open access to Facebook, YouTube, etc. YouTube is the second-largest social media platform nowadays containing more than 37 million YouTube channels. YouTube revealed at a recent press event that 30,000 new content videos per hour and 720,000 per day are posted. There is a need for an advanced deep learning-based approach to categorize the huge database of YouTube videos. This study aims to develop an artificial intelligence-based approach to categorize YouTube videos. This study analyzes the textual information related to videos like titles, descriptions, user tags, etc. using YouTube exploratory data analysis (YEDA) and shows that such information can be potentially used to categorize videos. A deep convolutional neural network (DCNN) is designed to categorize YouTube videos with efficiency and high accuracy. In addition, recurrent neural network (RNN), and gated recurrent unit (GRU) are also employed for performance comparison. Moreover, logistic regression, support vector machines, decision trees, and random forest models are also used. A large dataset with 9 classes is used for experiments. Experimental findings indicate that the proposed DCNN achieves the highest receiver operating characteristics (ROC) area under the curve (AUC) score of 99% in the context of YouTube video categorization and 96% accuracy which is better than existing approaches. The proposed approach can be used to help YouTube users suggest relevant videos and sort them by video category.

A predictive model for classifying college students' academic performance based on visual-spatial skills.

As the application of visual-spatial skills in academic disciplines, vocational fields and daily life is becoming more and more prominent, it is of great theoretical and practical significance how to make use of big data and artificial intelligence technology to conduct research on the relationship between visual-spatial skills and students' grades. This paper explores and analyses from the perspective of artificial intelligence, combining students' visual-spatial skills and students' specific attribute characteristics to construct an expert system, which defines the prediction of academic performance as a classification problem corresponding to the five categories of excellent, good, moderate, passing, and weak, respectively, and based on which a deep neural network-based classification prediction model for students' performance is designed. The experimental results show that visual-spatial skills plays an important role in the professional learning of science and engineering students, while the classification model designed in this paper has high accuracy in the grade prediction process. This paper not only helps to fill the gaps in the current research field, but is also expected to provide scientific basis for educational practice and promote the development of the education field in a more intelligent and personalized direction.

A position-enhanced sequential feature encoding model for lung infections and lymphoma classification on CT images.

Differentiating pulmonary lymphoma from lung infections using CT images is challenging. Existing deep neural network-based lung CT classification models rely on 2D slices, lacking comprehensive information and requiring manual selection. 3D models that involve chunking compromise image information and struggle with parameter reduction, limiting performance. These limitations must be addressed to improve accuracy and practicality. We propose a transformer sequential feature encoding structure to integrate multi-level information from complete CT images, inspired by the clinical practice of using a sequence of cross-sectional slices for diagnosis. We incorporate position encoding and cross-level long-range information fusion modules into the feature extraction CNN network for cross-sectional slices, ensuring high-precision feature extraction. We conducted comprehensive experiments on a dataset of 124 patients, with respective sizes of 64, 20 and 40 for training, validation and testing. The results of ablation experiments and comparative experiments demonstrated the effectiveness of our approach. Our method outperforms existing state-of-the-art methods in the 3D CT image classification problem of distinguishing between lung infections and pulmonary lymphoma, achieving an accuracy of 0.875, AUC of 0.953 and F1 score of 0.889. The experiments verified that our proposed position-enhanced transformer-based sequential feature encoding model is capable of effectively performing high-precision feature extraction and contextual feature fusion in the lungs. It enhances the ability of a standalone CNN network or transformer to extract features, thereby improving the classification performance. The source code is accessible at https://github.com/imchuyu/PTSFE .

Unbiased analysis of spatial learning strategies in a modified Barnes maze using convolutional neural networks

Assessment of spatial learning abilities is central to behavioral neuroscience and a useful tool for animal model validation and drug development. However, biases introduced by the apparatus, environment, or experimentalist represent a critical challenge to the test validity. We have recently developed the Modified Barnes Maze (MBM) task, a spatial learning paradigm that overcomes inherent behavioral biases of animals in the classical Barnes maze. The specific combination of spatial strategies employed by mice is often considered representative of the level of cognitive resources used. Herein, we have developed a convolutional neural network-based classifier of exploration strategies in the MBM that can effectively provide researchers with enhanced insights into cognitive traits in mice. Following validation, we compared the learning performance of female and male C57BL/6J mice, as well as that of Ts65Dn mice, a model of Down syndrome, and 5xFAD mice, a model of Alzheimer’s disease. Male mice exhibited more effective navigation abilities than female mice, reflected in higher utilization of effective spatial search strategies. Compared to wildtype controls, Ts65Dn mice exhibited delayed usage of spatial strategies despite similar success rates in completing this spatial task. 5xFAD mice showed increased usage of non-spatial strategies such as Circling that corresponded to higher latency to reach the target and lower success rate. These data exemplify the need for deeper strategy classification tools in dissecting complex cognitive traits. In sum, we provide a machine-learning-based strategy classifier that extends our understanding of mice’s spatial learning capabilities while enabling a more accurate cognitive assessment.

Quantum state reconstruction in a noisy environment via deep learning

Quantum noise is currently limiting efficient quantum information processing and computation, impacting on the fidelity and reliability of quantum states. In this work, we consider the tasks of reconstructing and classifying quantum states corrupted by the action of an unknown noisy channel using classical feed-forward neural networks. By framing reconstruction as a regression problem, we show how such an approach can be used to recover with fidelities exceeding 99% the noiseless density matrices of quantum states of up to three qubits undergoing noisy evolution, and we test its performance with both single-qubit (bit-flip, phase-flip, depolarizing, and amplitude damping) and two-qubit quantum channels (correlated amplitude damping). Furthermore, a critical aspect of our investigation involves also a comprehensive comparison between mean squared error and infidelity as loss functions. Our findings reveal that these two metrics yield comparable results in the context of state reconstruction. Moreover, we also consider the task of distinguishing between different quantum noisy channels, and show how a neural network-based classifier is able to solve such a classification problem with perfect accuracy.

A deep learning-based algorithm for pulmonary tuberculosis detection in chest radiography

In tuberculosis (TB), chest radiography (CXR) patterns are highly variable, mimicking pneumonia and many other diseases. This study aims to evaluate the efficacy of Google teachable machine, a deep neural network-based image classification tool, to develop algorithm for predicting TB probability of CXRs. The training dataset included 348 TB CXRs and 3806 normal CXRs for training TB detection. We also collected 1150 abnormal CXRs and 627 normal CXRs for training abnormality detection. For external validation, we collected 250 CXRs from our hospital. We also compared the accuracy of the algorithm to five pulmonologists and radiological reports. In external validation, the AI algorithm showed areas under the curve (AUC) of 0.951 and 0.975 in validation dataset 1 and 2. The accuracy of the pulmonologists on validation dataset 2 showed AUC range of 0.936–0.995. When abnormal CXRs other than TB were added, AUC decreased in both human readers (0.843–0.888) and AI algorithm (0.828). When combine human readers with AI algorithm, the AUC further increased to 0.862–0.885. The TB CXR AI algorithm developed by using Google teachable machine in this study is effective, with the accuracy close to experienced clinical physicians, and may be helpful for detecting tuberculosis by CXR.

A neural network-based classifier for identifying and locating neutral wire breaks in low voltage distribution networks

The breakage of the neutral conductor in low voltage distribution networks is a major concern for distribution companies. This breakage causes significant voltage deviations that can damage the connected equipment as well as jeopardizing people. The detection and localization of the breakage is a major challenge as it does not always manifest in the same way. This work presents a methodology based on artificial intelligence for the detection and localization of neutral conductor breaks in distribution networks. Two neural networks are trained in attempt to solve each of the challenges. For this purpose, measurements commonly taken by smartmeters such as power and nodal voltages are used. The methodology is evaluated in simulation exhibiting a good performance.

Graph structure estimation neural network-based service classification

PurposeWith the introduction of graph structure learning into service classification, more accurate graph structures can significantly improve the precision of service classification. However, existing graph structure learning methods tend to rely on a single information source when attempting to eliminate noise in the original graph structure and lack consideration for the graph generation mechanism. To address this problem, this paper aims to propose a graph structure estimation neural network-based service classification (GSESC) model.Design/methodology/approachFirst, this method uses the local smoothing properties of graph convolutional networks (GCN) and combines them with the stochastic block model to serve as the graph generation mechanism. Next, it constructs a series of observation sets reflecting the intrinsic structure of the service from different perspectives to minimize biases introduced by a single information source. Subsequently, it integrates the observation model with the structural model to calculate the posterior distribution of the graph structure. Finally, it jointly optimizes GCN and the graph estimation process to obtain the optimal graph.FindingsThe authors conducted a series of experiments on the API data set and compared it with six baseline methods. The experimental results demonstrate the effectiveness of the GSESC model in service classification.Originality/valueThis paper argues that the data set used for service classification exhibits a strong community structure. In response to this, the paper innovatively applies a graph-based learning model that considers the underlying generation mechanism of the graph to the field of service classification and achieves good results.

Performance Comparison of Convolutional Neural Network-Based Hearing Loss Classification Model Using Auditory Brainstem Response Data.

This study evaluates the efficacy of several Convolutional Neural Network (CNN) models for the classification of hearing loss in patients using preprocessed auditory brainstem response (ABR) image data. Specifically, we employed six CNN architectures-VGG16, VGG19, DenseNet121, DenseNet-201, AlexNet, and InceptionV3-to differentiate between patients with hearing loss and those with normal hearing. A dataset comprising 7990 preprocessed ABR images was utilized to assess the performance and accuracy of these models. Each model was systematically tested to determine its capability to accurately classify hearing loss. A comparative analysis of the models focused on metrics of accuracy and computational efficiency. The results indicated that the AlexNet model exhibited superior performance, achieving an accuracy of 95.93%. The findings from this research suggest that deep learning models, particularly AlexNet in this instance, hold significant potential for automating the diagnosis of hearing loss using ABR graph data. Future work will aim to refine these models to enhance their diagnostic accuracy and efficiency, fostering their practical application in clinical settings.

Imaging Flow Cytometry and Convolutional Neural Network-Based Classification Enable Discrimination of Hematopoietic and Leukemic Stem Cells in Acute Myeloid Leukemia.

Acute myeloid leukemia (AML) is a heterogenous blood cancer with a dismal prognosis. It emanates from leukemic stem cells (LSCs) arising from the genetic transformation of hematopoietic stem cells (HSCs). LSCs hold prognostic value, but their molecular and immunophenotypic heterogeneity poses challenges: there is no single marker for identifying all LSCs across AML samples. We hypothesized that imaging flow cytometry (IFC) paired with artificial intelligence-driven image analysis could visually distinguish LSCs from HSCs based solely on morphology. Initially, a seven-color IFC panel was employed to immunophenotypically identify LSCs and HSCs in bone marrow samples from five AML patients and ten healthy donors, respectively. Next, we developed convolutional neural network (CNN) models for HSC-LSC discrimination using brightfield (BF), side scatter (SSC), and DNA images. Classification using only BF images achieved 86.96% accuracy, indicating significant morphological differences. Accuracy increased to 93.42% when combining BF with DNA images, highlighting differences in nuclear morphology, although DNA images alone were inadequate for accurate HSC-LSC discrimination. Model development using SSC images revealed minor granularity differences. Performance metrics varied substantially between AML patients, indicating considerable morphologic variations among LSCs. Overall, we demonstrate proof-of-concept results for accurate CNN-based HSC-LSC differentiation, instigating the development of a novel technique within AML monitoring.

Design and Implementation of Model Parameter Independent Robust Current Control Scheme of Three-Phase Inverter - A Neural Network-Based Classification Approach

Design and Implementation of Model Parameter Independent Robust Current Control Scheme of Three-Phase Inverter - A Neural Network-Based Classification Approach