Classification Problem Research Articles

Improving the Generalization Abilities of Constructed Neural Networks with the Addition of Local Optimization Techniques

Constructed neural networks with the assistance of grammatical evolution have been widely used in a series of classification and data-fitting problems recently. Application areas of this innovative machine learning technique include solving differential equations, autism screening, and measuring motor function in Parkinson’s disease. Although this technique has given excellent results, in many cases, it is trapped in local minimum and cannot perform satisfactorily in many problems. For this purpose, it is considered necessary to find techniques to avoid local minima, and one technique is the periodic application of local minimization techniques that will adjust the parameters of the constructed artificial neural network while maintaining the already existing architecture created by grammatical evolution. The periodic application of local minimization techniques has shown a significant reduction in both classification and data-fitting problems found in the relevant literature.

Enhancing adaptivity of active sonar classifier considering loss landscape under dataset shift

We study the loss landscape of deep neural networks and use it to solve dataset shift problems in active sonar classification. The dataset shift degrades the generalization performance of supervised learning-based classifiers. When test data samples are available, fine-tuning methods can be used to mitigate the performance decrease. However, they can induce catastrophic forgetting and negative transfer because fine-tuned weights could be overfitted to the test data distribution. These problems are more severe in active sonar datasets with small amounts and less diversity. To mitigate the two problems in conventional fine-tuning, we explore the loss landscape of supervised learning-based active sonar classifiers and apply it to derive adaptative weights from pre-trained weights. Our results showed promising performance.

Adjustable robust optimization approach for SVM under uncertainty

The support vector machine (SVM) is one of the successful approaches to the classification problem. Since the values of features are typically affected by uncertainty, it is important to incorporate uncertainty into the SVM formulation. This paper focuses on developing a robust optimization (RO) model for SVM. A key distinction from existing literature lies in the timing of optimizing decision variables. To the best of our knowledge, in all existing RO models developed for SVM, a common assumption is that all decision variables are decided before the uncertainty realization, which leads to an overly conservative decision boundary. However, this paper adopts a different strategy by determining the variables that assess the misclassification error of data points or their fall within the margin post-realization, resulting in a less conservative model. The RO models where decisions are made in two stages (some before and the rest after the uncertainty resolution), are called adjustable RO models. This adjustment results in a three-level optimization model for which two decomposition-based algorithms are proposed. In these algorithms, after providing a bi-level reformulation, the model is divided into a master-problem (MP) and a sub-problem the interaction of which yields the optimal solution. Acceleration of algorithms via incorporating valid inequalities into MP is another novelty of this paper. Computational results over simulated and real-world datasets confirm the efficiency of the proposed model and algorithms.

CTCTime: A New Model for Unidimensional Time Series Classification

One-dimensional time series classification has always been an important research direction, playing an irreplaceable role in various fields. With the rapid development of deep learning, most traditional time series classification methods have gradually been replaced by neural network-based classification methods. At present, in the field of time series classification, models that perform well are mostly based on Convolutional Neural Networks (CNNs), while models with the Transformer architecture, which have shown outstanding performance in natural language processing and computer vision, do not stand out. In order to change this situation, We have investigated the feasibility of training models based on a single Transformer architecture directly on small datasets without additional data processing. Furthermore, we propose a new model called CTCTime, which combines the Transformer architecture with CNNs to address one-dimensional time series classification problems. We compared CTCTime with 13 traditional algorithms on 44 datasets from the UCR archive and with 7 advanced methods on 85 datasets. The UCR datasets are classic datasets for one-dimensional time series classification tasks. The experimental results demonstrate its feasibility, accuracy, and scalability.

Development of brain tumor radiogenomic classification using GAN-based augmentation of MRI slices in the newly released gazi brains dataset

Significant progress has been made recently with the contribution of technological advances in studies on brain cancer. Regarding this, identifying and correctly classifying tumors is a crucial task in the field of medical imaging. The disease-related tumor classification problem, on which deep learning technologies have also become a focus, is very important in the diagnosis and treatment of the disease. The use of deep learning models has shown promising results in recent years. However, the sparsity of ground truth data in medical imaging or inconsistent data sources poses a significant challenge for training these models. The utilization of StyleGANv2-ADA is proposed in this paper for augmenting brain MRI slices to enhance the performance of deep learning models. Specifically, augmentation is applied solely to the training data to prevent any potential leakage. The StyleGanv2-ADA model is trained with the Gazi Brains 2020, BRaTS 2021, and Br35h datasets using the researchers’ default settings. The effectiveness of the proposed method is demonstrated on datasets for brain tumor classification, resulting in a notable improvement in the overall accuracy of the model for brain tumor classification on all the Gazi Brains 2020, BraTS 2021, and Br35h datasets. Importantly, the utilization of StyleGANv2-ADA on the Gazi Brains 2020 Dataset represents a novel experiment in the literature. The results show that the augmentation with StyleGAN can help overcome the challenges of working with medical data and the sparsity of ground truth data. Data augmentation employing the StyleGANv2-ADA GAN model yielded the highest overall accuracy for brain tumor classification on the BraTS 2021 and Gazi Brains 2020 datasets, together with the BR35H dataset, achieving 75.18%, 99.36%, and 98.99% on the EfficientNetV2S models, respectively. This study emphasizes the potency of GANs for augmenting medical imaging datasets, particularly in brain tumor classification, showcasing a notable increase in overall accuracy through the integration of synthetic GAN data on the used datasets.

SKH-NLP at LLMs4OL 2024 Task B: Taxonomy Discovery in Ontologies Using BERT and LLaMA 3

Taxonomy discovery in ontologies refers to extracting the parent class from the child class. By modeling this task as a classification problem, we addressed it using two different approaches. The first approach involved fine-tuning the “BERT-Large” model with various prompts and using it in a classification system. In the second approach, we utilized the “LLaMA 3 70B” model, experimenting with different prompts and modifying them to achieve the best results. Additionally, we evaluated the correctness of the answers using substring and Levenshtein distance functions. The results indicate that, with appropriate fine-tuning, the BERT model can achieve performance levels comparable to those of more recent and significantly larger language models, such as LLaMA 3 70B. However, with appropriate prompts, LLaMA 3 70B performs slightly better than BERT, highlighting the importance of prompt quality. Ultimately, further experiments on different settings for fine-tuning BERT, few-shot learning, and using knowledge graphs for validating the model's answers for LLaMA are recommended to improve the results. Additionally, testing other models and examining the results of various encoder-based and decoder-based models can be employed.

Learning to detect slip through tactile estimation of the contact force field and its entropy properties

Slip detection during object grasping and manipulation plays a vital role in object handling. Visual feedback can help devise a strategy for grasping. However, for robotic systems to attain a proficiency comparable to humans, integrating artificial tactile sensing is increasingly essential, especially in consistently handling unfamiliar objects. We introduce a novel physics-informed, data-driven approach to detect slip continuously for control-oriented tasks. Our work leverages the inhomogeneity of tactile sensor readings during slip events to develop distinct features and formulates slip detection as a classification problem. We test multiple data-driven models on 10 common objects under different loading conditions, textures, and materials to evaluate our approach. The resulting best classification algorithm achieves a high average accuracy of 95.61%. Practical application in dynamic robotic manipulation demonstrates the effectiveness of the proposed real-time slip detection and prevention.

TC-DTA: Predicting Drug-Target Binding Affinity With Transformer and Convolutional Neural Networks.

Bioinformatics is a rapidly evolving field that applies computational methods to analyze and interpret biological data. A key task in bioinformatics is identifying novel drug-target interactions (DTIs), which plays a crucial role in drug discovery. Most computational approaches treat DTI prediction as a binary classification problem, determining whether drug-target pairs interact. However, with the growing availability of drug-target binding affinity data, this binary task can be reframed as a regression problem focused on drug-target affinity (DTA). DTA quantifies the strength of drug-target binding, offering more detailed insights than DTI and serving as a valuable tool for virtual screening in drug discovery. Accurately predicting compound interactions with targets can accelerate the drug development process. In this study, we introduce a deep learning model named TC-DTA for DTA prediction, leveraging convolutional neural networks (CNN) and the encoder module of the transformer architecture. We begin by extracting raw drug SMILES strings and protein amino acid sequences from the dataset, which are then represented using various encoding methods. Subsequently, we employ CNN and the transformer's encoder module to extract features from the drug SMILES strings and protein sequences, respectively. Finally, the feature information is concatenated and input into a multi-layer perceptron to predict binding affinity scores. We evaluated our model on two benchmark DTA datasets, Davis and KIBA, comparing it with methods such as KronRLS, SimBoost, and DeepDTA. Our model, TC-DTA, outperformed these baseline methods based on evaluation metrics like Mean Squared Error (MSE), Concordance Index (CI), and Regression towards the Mean Index ( rm2 ). These results highlight the effectiveness of the Transformer's encoder and CNN in extracting meaningful representations from sequences, thereby enhancing DTA prediction accuracy. This deep learning model can accelerate drug discovery by identifying drug candidates with high binding affinity to specific targets. Compared to traditional methods, machine learning technology offers a more effective and efficient approach to drug discovery.

Hawk: An industrial-strength multi-label document classifier

There are a plethora of methods for solving the classical multi-label document classification problem. However, when it comes to deployment and usage in an industry setting, most if not all the contemporary approaches fail to address some of the vital aspects or requirements of an ideal solution: i) ability to operate on variable-length texts or rambling documents, ii) catastrophic forgetting problem, and iii) ability to visualize the model’s predictions. The paper describes the significance of these problems in detail and adopts the hydranet architecture to address these problems. The proposed architecture views documents as a sequence of sentences and leverages sentence-level embeddings for input representation, turning the problem into a sequence classification task. Furthermore, two specific architectures are explored as the architectures for the heads, Bi-LSTM and transformer heads. The proposed architecture is benchmarked on some of the popular benchmarking datasets such as Web of Science - 5763, Web of Science - 11967, BBC Sports, and BBC News datasets. The experimental results reveal that the proposed model performs at least as best as previous SOTA architectures and even outperforms prior SOTA in a few cases, along with the added advantages of the practicality issues discussed. The ablation study includes comparisons of the impact of the attention mechanism and the application of weighted loss functions to train the task-specific heads in the hydranet. The claims regarding catastrophic forgetfulness are further corroborated by empirical evaluations under incremental learning scenarios. The results reveal the robustness of the proposed architecture compared to other benchmarks.

A [formula omitted]-means triangular synthesis large margin classifier with unified pinball loss for imbalanced data

Large Margin Distribution Machine (LDM) improves the Support Vector Machine (SVM) by integrating the marginal distribution of samples into the objective function, which exhibits excellent classification and generalization performance. However, most of the existing LDM-based models exhibit inherent flaws: (1) The assumption of a category uniform distribution, resulting in an inability to effectively address the issue of class imbalance. (2) The inheritance of the hinge loss in SVM, leading to inferior anti-noise performance. Given the shortcomings above, this paper constructs a novel K-means Triangular Synthesis (KTS) method for imbalanced data classification and introduces Unified Pinball (UP) loss to ensure robust performance, demonstrated as a novel KTS large margin classifier with UP Loss (KTS-UPLMC) model. Specifically, the KTS method utilizes the triangular synthesis and generates samples based on the cluster density, effectively mitigating the problems of introducing noise samples and strip-shaped sample distribution in the traditional synthetic minority oversampling technique, further alleviating the LDM’s classification line offset. In addition, the UP Loss considers quantile distance, improving the anti-noise performance of the model. Comparative experiments on artificial datasets and benchmark datasets validate the effectiveness and noise resistance of the proposed KTS-UPLMC in imbalanced data classification problems. Furthermore, the stability of the KTS-UPLMC is substantiated by the parameter sensitivity analysis experiment. In conclusion, the KTS method effectively addresses category imbalance, while the integration of the UP Loss enhances noise resistance.

The ROC Curve Examination on Traveling Ionospheric Disturbances in FORMOSAT‐7/COSMIC‐2 IVM Ion Density Triggered by the 15 January 2022 Tonga Volcanic Eruption

AbstractThe receiver operating characteristic (ROC) and the area under the curve (AUC), initially developed for signal processing and psychology, are a test for assessing the performance of a binary classification problem at varying threshold values. The ion density (Ni) observed by FORMOSAT‐7/COSMIC‐2 is used to study traveling ionospheric disturbances (TIDs) triggered by the 15 January 2022 Tonga volcanic eruption. We examine parameters of Ni, differential Ni, and standard deviation Ni (STD_Ni) in January 2022, simulate TID wavefronts traveling with various speeds from 10 to 1,000 m/s, and apply the ROC curve to globally identify the significance of TIDs in STD_Ni triggered by the volcanic eruption. ROC and AUC results show that in addition to TIDs related to tsunami/tropospheric Lamb waves and a series of fast‐moving waves with propagation speeds of 180–350 and 450–600 m/s, respectively, those long‐lasting low‐speeds less than 70 m/s and high‐speeds about 690, 860, and 990 m/s meet 95% statistical significance, which confirms TIDs being detected. These show that the FORMOSAT‐7/COSMIC‐2 ion density can be used to globally detect various TIDs triggered by the Tonga volcanic eruption. The ROC test results also show a potential use case for detecting other geophysical signals in future applications.

An evolutionary algorithm‐based classification method for high‐dimensional imbalanced mixed data with missing information

AbstractThe data scale keeps growing by leaps and the majority of it is high‐dimensional imbalanced data, which is hard to classify. Data missing often happens in software which further aggravates the difficulty of classifying the data. In order to resolve high‐dimensional imbalanced mixed‐variables missing data classification problem, a novel method based on particle swarm optimization is developed. It has three original components including multiple feature selection, mixed attribute imputation, and quantum oversampling. Multiple feature selection uses a two‐stage strategy to obtain stable relevant features. Mixed attribute imputation separates features into continuous and discrete features and fills missing values with different models. Quantum oversampling chooses instances to balance data based on the quantum operator. Furthermore, particle swarm optimization is employed to optimize the parameters of the components to obtain preferable classification results. Six representative classification datasets, six typical algorithms, and four measures are taken to conduct exhaust experiments, and results indicate that the proposed method is superior to the comparison algorithms.

An Industrial Micro Parts Recognition Technology Based on Improved Yolov8

Abstract Aiming at the recognition and classification problem of micro assembly parts in industrial production, the Yolov8 object detection algorithm was used as the base model to improve the C2f class in the backbone. And apply the improved object detection algorithm to the recognition field of detecting assembly parts. By collecting images of gears, nails, and screws from various angles on Roboflow, a training set and a validation set containing three types of gears, nails, and screws are formed. At the same time, in order to improve the robustness of the Yolov8 object detection algorithm, data processing methods such as adding noise and image rotation were adopted for the samples in the data. Due to the use of Yolov8 as the base model, a TXT text file containing label prediction box position information was formed after labelling the dataset on Roboflow. Based on Python (3.10.12) and cloud platform, improved Yolov8 object detection algorithm was trained and network validation was carried out. Use TXT files as input for the model to obtain and learn features of gears, nails, and screws. After 200 epochs, the mAP for detecting gears, nails, and screws increased by about 3% compared to the base model.

CT-Net: an interpretable CNN-Transformer fusion network for fNIRS classification.

Functional near-infrared spectroscopy (fNIRS), an optical neuroimaging technique, has been widely used in the field of brain activity recognition and brain-computer interface. Existing works have proposed deep learning-based algorithms for the fNIRS classification problem. In this paper, a novel approach based on convolutional neural network and Transformer, named CT-Net, is established to guide the deep modeling for the classification of mental arithmetic (MA) tasks. We explore the effect of data representations, and design a temporal-level combination of two raw chromophore signals to improve the data utilization and enrich the feature learning of the model. We evaluate our model on two open-access datasets and achieve the classification accuracy of 98.05% and 77.61%, respectively. Moreover, we explain our model by the gradient-weighted class activation mapping, which presents a high consistent between the contributing value of features learned by the model and the mapping of brain activity in the MA task. The results suggest the feasibility and interpretability of CT-Net for decoding MA tasks.

A multi-label legal charge prediction method based on summarization generation

Legal Charge Prediction (LCP) is a task of predicting charge labels for which a criminal should be charged based on the factual case descriptions. In recent years, the advancement of deep learning networks has led to the emergence of effective approaches for LCP through the Fine-tuning of pre-trained language models. While existing methods primarily treat it as multiple binary classifications, excelling in the single charge label given factual case descriptions, they still exhibit substantial room for improvement in multi-label LCP. This is because single-label classification methods do not effecitively deal with the relationship between multiple labels, mainly determining the likelihood of a factual description’s association to each individual charge label. Recognizing the need to enhance the relationship among multi-label charges in Fine-tuning-based discriminative charge prediction, this paper introduces a multi-label LCP method based on summarization generation. This method reconstructs the multi-label classification problem as a summarization generation task, capturing the relationship among multiple labels within the summarization content. Empirical findings from the Chinese legal domain’s CAIL2018 dataset demonstrate the method’s capacity to discern the relationship between multiple legal charge labels with Micro-F1 and Macro-F1 enhancements, compared to the fine-tuning-based approaches.

RFMiD: Retinal Image Analysis for multi-Disease Detection challenge

In the last decades, many publicly available large fundus image datasets have been collected for diabetic retinopathy, glaucoma, and age-related macular degeneration, and a few other frequent pathologies. These publicly available datasets were used to develop a computer-aided disease diagnosis system by training deep learning models to detect these frequent pathologies. One challenge limiting the adoption of a such system by the ophthalmologist is, computer-aided disease diagnosis system ignores sight-threatening rare pathologies such as central retinal artery occlusion or anterior ischemic optic neuropathy and others that ophthalmologists currently detect. Aiming to advance the state-of-the-art in automatic ocular disease classification of frequent diseases along with the rare pathologies, a grand challenge on “Retinal Image Analysis for multi-Disease Detection” was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI - 2021). This paper, reports the challenge organization, dataset, top-performing participants solutions, evaluation measures, and results based on a new “Retinal Fundus Multi-disease Image Dataset” (RFMiD). There were two principal sub-challenges: disease screening (i.e. presence versus absence of pathology — a binary classification problem) and disease/pathology classification (a 28-class multi-label classification problem). It received a positive response from the scientific community with 74 submissions by individuals/teams that effectively entered in this challenge. The top-performing methodologies utilized a blend of data-preprocessing, data augmentation, pre-trained model, and model ensembling. This multi-disease (frequent and rare pathologies) detection will enable the development of generalizable models for screening the retina, unlike the previous efforts that focused on the detection of specific diseases.

Improvement of deformable DETR model for insulator defect classification detection method

Abstract In response to challenges faced by traditional detection methods such as image blurring, scarcity of insulator defect datasets, and insulators as small targets, we propose an improved Deformable DETR network based on the DETR defect detection algorithm. This network model accurately identifies the position information of insulators and segments the insulators on the insulator string. To address the classification problem after defect detection, we introduce a fused insulator defect classifier with a self-attention mechanism built behind the Deformable DETR model. The detected insulators are classified into three categories. Due to the limited dataset of damaged insulators, corresponding loss functions are set to address the issue of sample imbalance, thereby increasing the model’s focus on damaged insulators. Experimental results demonstrate an accuracy of 97.5% on the test set, highlighting the network’s strong generalization ability.

Integrating machine learning models to learn potentially non-monotonic preferences for multi-criteria sorting from large-scale assignment examples

Learning preferences from assignment examples has attracted considerable attention in the field of multi-criteria sorting (MCS). However, traditional MCS methods, designed to infer decision makers’ preferences from small-scale assignment examples, encounter limitations when confronted with large-scale data sets. Additionally, the presence of decision makers’ non-monotonic preferences for certain criteria in MCS problems necessitates accounting for potential non-monotonicity when devising preference learning methods. To address this, this paper proposes some new models to learn potentially non-monotonic preferences for MCS problems from large-scale assignment examples by leveraging machine learning models. Specifically, we first introduce the Piecewise-Linear Neural Network (PLNN) model, which leverages the threshold-based value-driven sorting procedure as the underlying sorting model and integrates a perceptron-based model to establish piecewise-linear marginal value functions to approximate real ones. On this basis, we address MCS problems with criteria interactions and extend the PLNN model to develop the Piecewise-Linear Factorization Machine-based Neural Network (PLFMNN) model by incorporating the factorization machine to factorize interaction coefficients. Training these models allows us to learn potentially non-monotonic preferences of decision makers. To illustrate the proposed models, we apply them to a red wine quality classification problem. Furthermore, we assess the performance of the proposed models through computational experiments on both artificial and real-world data sets. Additionally, we conduct statistical tests to ascertain the significance of the performance differences. Experimental results reveal that the proposed models are comparable to the multilayer perceptron model and outperform other baseline models on most data sets, thus affirming their efficacy. Finally, we conduct some sensitivity analysis to assess the impact of certain parameters on the performance of the proposed models and compare them with existing studies from a theoretical perspective, further demonstrating their effectiveness.

Unveiling image source: Instance-level camera device linking via context-aware deep Siamese network

Unveiling the source of an image is one of the most effective ways to validate the originality, authenticity, and reliability in the field of digital forensics. Source camera device identification can identify the specific camera device used to take a photo under investigation. While great progress has been made by the photo-response non-uniformity (PRNU)-based methods over the past decade, the challenge of instance-level source camera device linking, which verifies whether two images in question were captured by the same camera device, remains significant. This challenge is mainly due to the absence of auxiliary images to construct a clean camera fingerprint for each camera, particularly dealing with small image sizes. To overcome this limitation, in this paper, we formulate the task of source device linking as a binary classification problem and propose a simple yet effective framework based on a context-aware deep Siamese network. We take advantage of a Siamese architecture to extract the intrinsic camera device-related noise patterns from a pair of image patches in parallel for comparisons without any auxiliary images. Moreover, a recurrent criss-cross group is utilized to aggregate contextual information in the noise residual maps to alleviate the problem that PRNU noise maps are easily contaminated by the additive noises from image contents. For reliable device linking, we employ a patch-selection strategy on a pair of test images to adaptively choose suitable image patch pairs according to image contents. The final decision of a pair of test images is obtained from the average similarity score of the selected image patch pairs. Compared with existing state-of-the-art methods, our proposed framework can achieve better performance on both the tasks of source camera identification and source device linking without any prior knowledge, i.e., reliable camera fingerprints, regardless of whether the camera devices are “seen” or “unseen” in the training stage. The experimental results on two standard image forensic datasets demonstrate that the proposed method not only shows robustness with respect to different image patch sizes and image quality degenerations, but also has a generalization ability across digital camera and smartphone devices.

Discriminative Deep Canonical Correlation Analysis for Multi-View Data.

Over the past few years, multimodal data analysis has emerged as an inevitable method for identifying sample categories. In the multi-view data classification problem, it is expected that the joint representation should include the supervised information of sample categories so that the similarity in the latent space implies the similarity in the corresponding concepts. Since each view has different statistical properties, the joint representation should be able to encapsulate the underlying nonlinear data distribution of the given observations. Another important aspect is the coherent knowledge of the multiple views. It is required that the learning objective of the multi-view model efficiently captures the nonlinear correlated structures across different modalities. In this context, this article introduces a novel architecture, termed discriminative deep canonical correlation analysis (D2CCA), for classifying given observations into multiple categories. The learning objective of the proposed architecture includes the merits of generative models to identify the underlying probability distribution of the given observations. In order to improve the discriminative ability of the proposed architecture, the supervised information is incorporated into the learning objective of the proposed model. It also enables the architecture to serve as both a feature extractor as well as a classifier. The theory of CCA is integrated with the objective function so that the joint representation of the multi-view data is learned from maximally correlated subspaces. The proposed framework is consolidated with corresponding convergence analysis. The efficacy of the proposed architecture is studied on different domains of applications, namely, object recognition, document classification, multilingual categorization, face recognition, and cancer subtype identification with reference to several state-of-the-art methods.