Competitive Classification Research Articles

Learning high-dependence Bayesian network classifier with robust topology

The increase in data variability and quantity makes it urgent for learning complex multivariate probability distributions. The state-of-the-art Tree Augmented Naive Bayes (TAN) classifier uses maximum weighted spanning tree (MWST) to graphically model data with excellent time and space complexity. In this paper, we theoretically prove the feasibility of scaling up one-dependence MWST to model high-dependence relationships, and then propose to apply a heuristic search strategy to improve the fitness of extended topology to data. The newly added edges to each attribute node may provide a local optimal solution. Then ensemble learning is introduced to improve the generalization performance and reduce the sensitivity to variation in training data. The experimental results on 32 benchmark datasets reveal that this highly scalable algorithm inherits the expressive power of TAN and achieves an excellent bias–variance tradeoff, and it also demonstrates competitive classification performance when compared to a range of high-dependence or ensemble learning algorithms.

Competitive Decomposition-Based Multiobjective Architecture Search for the Dendritic Neural Model.

The dendritic neural model (DNM) is computationally faster than other machine-learning techniques, because its architecture can be implemented by using logic circuits and its calculations can be performed entirely in binary form. To further improve the computational speed, a straightforward approach is to generate a more concise architecture for the DNM. Actually, the architecture search is a large-scale multiobjective optimization problem (LSMOP), where a large number of parameters need to be set with the aim of optimizing accuracy and structural complexity simultaneously. However, the issues of irregular Pareto front, objective discontinuity, and population degeneration strongly limit the performances of conventional multiobjective evolutionary algorithms (MOEAs) on the specific problem. Therefore, a novel competitive decomposition-based MOEA is proposed in this study, which decomposes the original problem into several constrained subproblems, with neighboring subproblems sharing overlapping regions in the objective space. The solutions in the overlapping regions participate in environmental selection for the neighboring subproblems and then propagate the selection pressure throughout the entire population. Experimental results demonstrate that the proposed algorithm can possess a more powerful optimization ability than the state-of-the-art MOEAs. Furthermore, both the DNM itself and its hardware implementation can achieve very competitive classification performances when trained by the proposed algorithm, compared with numerous widely used machine-learning approaches.

All-analog photoelectronic chip for high-speed vision tasks

Photonic computing enables faster and more energy-efficient processing of vision data1–5. However, experimental superiority of deployable systems remains a challenge because of complicated optical nonlinearities, considerable power consumption of analog-to-digital converters (ADCs) for downstream digital processing and vulnerability to noises and system errors1,6–8. Here we propose an all-analog chip combining electronic and light computing (ACCEL). It has a systemic energy efficiency of 74.8 peta-operations per second per watt and a computing speed of 4.6 peta-operations per second (more than 99% implemented by optics), corresponding to more than three and one order of magnitude higher than state-of-the-art computing processors, respectively. After applying diffractive optical computing as an optical encoder for feature extraction, the light-induced photocurrents are directly used for further calculation in an integrated analog computing chip without the requirement of analog-to-digital converters, leading to a low computing latency of 72 ns for each frame. With joint optimizations of optoelectronic computing and adaptive training, ACCEL achieves competitive classification accuracies of 85.5%, 82.0% and 92.6%, respectively, for Fashion-MNIST, 3-class ImageNet classification and time-lapse video recognition task experimentally, while showing superior system robustness in low-light conditions (0.14 fJ μm−2 each frame). ACCEL can be used across a broad range of applications such as wearable devices, autonomous driving and industrial inspections.

Double similarities weighted multi-instance learning kernel and its application

Multi-instance learning (MIL), as a special version of classification, focuses on labeled sets (bags) consisting of unlabeled instances and has drawn accumulative attention due to its significant importance in practical applications. However, most existing MIL methods just utilize partial information (bags or instances) of MIL data to construct the kernel function, resulting in deteriorated classification performance of MIL. In this paper, we propose a Double Similarities weighted Multi-Instance Learning (DSMIL) kernel framework, which utilizes the similarities of Bag-to-Bag (B2B) and Instance-to-Bag (I2B). In the proposed kernel framework, the similarities of B2B and I2B could be derived from the prototypes distance of inter-bag and similarity matrix of intra-bag, respectively, based on the affinity propagation (AP) clustering of the bag. Meanwhile, we give theoretical proof of the validity of the designed kernel function. Experimental results on benchmark and semi-synthetic datasets show that our proposed method obtains competitive classification performance and achieves robustness to parameters and noise.

Abnormality Detection of Blast Furnace Tuyere Based on Knowledge Distillation and a Vision Transformer

The blast furnace tuyere is a key position in hot metal production and is primarily observed to assess the internal state of the furnace. However, detecting abnormal tuyere conditions has relied heavily on manual judgment, leading to certain limitations. We proposed a tuyere abnormality detection model based on knowledge distillation and a vision transformer (ViT) to address this issue. In this approach, ResNet50 is employed as the Teacher model to distill knowledge into the Student model, ViT. Firstly, we introduced spatial attention modules to enhance the model’s perception and feature-extraction capabilities for different image regions. Furthermore, we simplified the depth of the ViT and improved its self-attention mechanism to alleviate training losses. In addition, we employed the knowledge distillation strategy to achieve model lightweighting and enhance the model’s generalization capability. Finally, we evaluate the model’s performance in tuyere abnormality detection and compare it with other classification methods such as VGG-19, ResNet-101, and ResNet-50. Experimental results showed that our model achieved a classification accuracy of 97.86% on a tuyere image dataset from a company, surpassing the original ViT model by 1.12% and the improved ViT model without knowledge distillation by 0.34%. Meanwhile, the model achieved a competitive classification accuracy of 90.31% and 77.65% on the classical fine-grained image datasets, Stanford Dogs and CUB-200-2011, respectively, comparable to other classification models.

Continuous Cartesian Genetic Programming based representation for multi-objective neural architecture search

We propose a novel neural architecture search (NAS) approach for the challenge of designing convolutional neural networks (CNNs) that achieve a good tradeoff between complexity and accuracy. We rely on Cartesian genetic programming (CGP) and integrated real-based and block-chained CNN representation, for optimization using multi-objective evolutionary algorithms (MOEAs) in the continuous domain. We introduce two variants, CGP-NASV1 and CGP-NASV2, which differ in the granularity of their respective search spaces. To evaluate the proposed algorithms, we utilized the non-dominated sorting genetic algorithm II (NSGA-II) on the CIFAR-10, CIFAR-100,and SVHN datasets. Additionally, we extended the empirical analysis while maintaining the same solution representation to assess other searching techniques such as differential evolution (DE), the multi-objective evolutionary algorithm based on decomposition (MOEA/D), and the S metric selection evolutionary multi-objective algorithm (SMS-EMOA). The experimental results demonstrate that our approach exhibits competitive classification performance and model complexity compared to state-of-the-art methods.

Defect detection and classification on semiconductor wafers using two-stage geometric transformation-based data augmentation and SqueezeNet lightweight convolutional neural network

The manufacturing industry is evolving in line with the principles of Industry 4.0, with the aim of achieving higher levels of automation and digitization. In particular, deep learning algorithms such as convolutional neural networks (CNNs) are key enabling technologies to achieve this goal. The semiconductor industry is a particular case where CNNs are used to assist inspection systems and human operators in defect classification of wafers. However, deep CNN models are time consuming and resource intensive. It is therefore necessary to look for alternatives. One of these alternatives is the use of lightweight models, which provide competitive classification performance with low time and resource consumption. Therefore, the motivation of this work is to apply these lightweight models to semiconductor defect classification and compare their performance with that obtained by deep CNN models in similar work. In this line, this paper introduces an efficient two-step approach combining traditional computer vision techniques and a lightweight SqueezeNet CNN for defect detection and classification. The lightweight SqueezeNet model is tuned using a grid search algorithm. After obtaining the optimal model, its metrics are presented and compared with results from related work. Using a semiconductor surface defect dataset from a multinational semiconductor company, our lightweight model can achieve really competitive classification results (99.356% versus 99.443% obtained by ResNet50) while consuming significantly less time than other heavyweight models (80.146% less time than ResNet50).

Deep multi-feature fusion residual network for oral squamous cell carcinoma classification and its intelligent system using Raman spectroscopy

Using fiber optic Raman spectroscopy and deep neural networks, we develop an intelligent system which will be used to assist surgeons accurately and efficiently to identify oral squamous cell carcinomas (OSCC). This system is able to classify 6 types of oral tissues. To achieve this goal, a novel classification framework called deep multi-feature fusion residual network (DMFF-ResNet) is proposed. This model is based on 16,200 Raman spectral data, obtained from the normal oral tissues and the OSCC of 90 patients through the surgical resection. Firstly, the 1-dimensional RestNet50 is taken as its backbone network. Then, the output spectral features of last three blocks are extracted from backbone network for feature fusion, which is expected to learn more spatial representations and have more discriminative power. Lastly, the derived spectral features are sent into a fully-connected neural network for performing the multiclassification task. Experimental results show that the proposed model achieves a competitive classification performance compared with state-of-the-art classifiers, and its accuracy, precision, and sensitivity reach 93.28%, 93.53%, and 93.13%, respectively. Further, the proposed framework is deployed on an edge computing device to form a prototype intelligent system for OSCC detection. To validate this system, we perform an offline test experiment in another 20 patients which demonstrates the developed intelligent system can successfully discriminate OSCC and normal oral tissues, with accuracy, precision, and recall of 92.78%, 92.33%, and 92.57%, respectively. The code was available at https://github.com/ISCLab-Bistu/retinanet-OSCC.

Real-Time 3-D Human Action Recognition Based on Hyperpoint Sequence

Real-time 3D human action recognition has broad industrial applications, such as surveillance, human-computer interaction, and healthcare monitoring. By relying on complex spatio-temporal local encoding, most existing point cloud sequence networks capture spatio-temporal local structures to recognize 3D human actions. To simplify the point cloud sequence modeling task, we propose a lightweight and effective point cloud sequence network referred to as SequentialPointNet for real-time 3D action recognition. Instead of capturing spatio-temporal local structures, SequentialPointNet encodes the temporal evolution of static appearances to recognize human actions. Firstly, we define a novel type of point data, Hyperpoint, to better describe the temporally changing human appearances. A theoretical foundation is provided to clarify the information equivalence property for converting point cloud sequences into Hyperpoint sequences. Secondly, the point cloud sequence modeling task is decomposed into a Hyperpoint embedding task and a Hyperpoint sequence modeling task. Specifically, for Hyperpoint embedding, the static point cloud technology is employed to convert point cloud sequences into Hyperpoint sequences, which introduces inherent frame-level parallelism; for Hyperpoint sequence modeling, a Hyperpoint-Mixer module is designed as the basic building block to learning the spatio-temporal features of human actions. Extensive experiments on three widely-used 3D action recognition datasets demonstrate that the proposed SequentialPointNet achieves competitive classification performance with up to 10X faster than existing approaches.

Region Purity-Based Local Feature Selection: A Multiobjective Perspective

In contrast from the traditional feature selection, local feature selection (LFS) partitions the whole sample space and obtains the feature subset for each local region. However, most existing LFS algorithms lack a problem-specific objective function and instead simply apply the distance-like objective function, which limits their classification performance. In addition, obtaining a good LFS model is essentially a multi-objective optimization problem. Therefore, in this paper we propose a region purity-based LFS (RP-LFS) where, besides the proportion of the selected features and region-based distance metric, we design a novel objective function, region purity, from the perspective of combining local features with classifiers. To solve the RP-LFS, an improved non-dominated sorting genetic algorithm III is proposed. Specifically, a network-inspired crossover operator and a quick bit mutation are applied, which can improve the ability to search for better solutions. A regional feature sharing strategy between different local models is developed, which can preserve more effective features. Experimental studies on 11 UCI datasets and nine high-dimensional datasets validate the effectiveness of our proposed RP. In comparison with various state-of-the-art feature selection and LFS algorithms, RP-LFS can achieve very competitive classification accuracy while obtaining a reduced feature subset size.

Faster and Better: A Lightweight Transformer Network for Remote Sensing Scene Classification

Remote sensing (RS) scene classification has received considerable attention due to its wide applications in the RS community. Many methods based on convolutional neural networks (CNNs) have been proposed to classify complex RS scenes, but they cannot fully capture the context in RS images because of the lack of long-range dependencies (the dependency relationship between two distant elements). Recently, some researchers fine-tuned the large pretrained vision transformer (ViT) on small RS datasets to extract long-range dependencies effectively in RS scenes. However, it usually takes more time to fine-tune the ViT on account of high computational complexity. The lack of good local feature representation in the ViT limits classification performance improvement. To this end, we propose a lightweight transformer network (LTNet) for RS scene classification. First, a multi-level group convolution (MLGC) module is presented. It enriches the diversity of local features and requires a lower computational cost by co-representing multi-level and multi-group features in a single module. Then, based on the MLGC module, a lightweight transformer block, LightFormer, was designed to capture global dependencies with fewer computing resources. Finally, the LTNet was built using the MLGC and LightFormer. The experiments of fine-tuning the LTNet on four RS scene classification datasets demonstrate that the proposed network achieves a competitive classification performance under less training time.

Exploiting the implicit independence assumption for learning directed graphical models

Bayesian network classifiers (BNCs) provide a sound formalism for representing probabilistic knowledge and reasoning with uncertainty. Explicit independence assumptions can effectively and efficiently reduce the size of the search space for solving the NP-complete problem of structure learning. Strong conditional dependencies, when added to the network topology of BNC, can relax the independence assumptions, whereas the weak ones may result in biased estimates of conditional probability and degradation in generalization performance. In this paper, we propose an extension to the k-dependence Bayesian classifier (KDB) that achieves the bias/variance trade-off by verifying the rationality of implicit independence assumptions implicated. The informational and probabilistic dependency relationships represented in the learned robust topologies will be more appropriate for fitting labeled and unlabeled data, respectively. The comprehensive experimental results on 40 UCI datasets show that our proposed algorithm achieves competitive classification performance when compared to state-of-the-art BNC learners and their efficient variants in terms of zero-one loss, root mean square error (RMSE), bias and variance.

ASMEvoNAS: Adaptive segmented multi-objective evolutionary network architecture search

Network architecture search (NAS) has attracted much attention as an automatic design technique of network architecture. In particular, multi-objective evolutionary algorithms (MOEAs) have been a popular kind of optimizer in NAS due to their global optimization capability. However, as a population-based iterative search method, MOEAs are subject to the unbearable computational cost of individual evaluation on multiple objectives at each generation, which affects their generalization ability and transferability of MOEA-based NAS. Therefore, an adaptive segmented multi-objective evolutionary network architecture search (ASMEvoNAS) method is proposed in this paper. Firstly, an adaptive segmented evaluation strategy is designed to adaptively select different but more suitable objectives to efficiently assess the candidate architectures at different evolutionary stages, instead of evaluating them by all the considered objectives simultaneously. Thus, the computational cost and complexity of the search process can be controlled and reduced to some extent. Secondly, a preference-based pre-selection strategy is designed to filter out the initialized architectures with high parameter quantities to reduce the total parameter scale of the whole population and memory consumption. Last, a novel desirable gene reservation-based crossover and a directed connection-based mutation are proposed to produce offspring. Experimental results show that ASMEvoNAS shows promising performance on CIFAR-10, CIFAR-100, and ImageNet with error rates of 2.21%, 15.57%, and 24.43% top-1, respectively. The proposed method reduces the search cost to 0.36 GPU-Days on CIFAR-10 while maintaining competitive classification performance compared to state-of-the-art networks. In addition, ASMEvoNAS presents superior performance when dealing with the considered transfer tasks, as well as the benchmark dataset of NAS.

HAR-DeepConvLG: Hybrid deep learning-based model for human activity recognition in IoT applications

Smartphones and wearable devices have built-in sensors that can collect multivariant time-series data that can be used to recognize human activities. Research on human activity recognition (HAR) has gained significant attention in recent years due to its growing demand in various application domains. As wearable sensor-aided devices and the Internet of Things (IoT) became more common, great attention has been paid to the HAR ubiquitous computing and mobile computing. To infer human activity data from a massive amount of multivariant data generated by different wearable devices, in this study an innovative deep learning–based model named HAR-DeepConvLG is proposed. It includes three convolution layers and a squeezing and excitation (SE) block, which are employed to precisely learn and extract the spatial representation data from the collected raw sensor data. The extracted features are used as input of three parallel paths, each of which includes a long short-term memory (LSTM) layer connected in sequence with a gated recurrent unit (GRU) layer to learn temporal representation. The three paths are connected in parallel to avoid the vanishing gradient problem. Finally, to evaluate the effectiveness of the proposed model, experiments were conducted on four widely utilized HAR datasets. Additionally, the model’s performance was compared to several state-of-the-art deep learning models, which further validated its effectiveness. The experimental results show that the proposed HAR-DeepConvLG model performs better than the existing HAR deep learning–based models, achieving a competitive classification accuracy.

A deep learning self-attention cross residual network with Info-WGANGP for mitotic cell identification in HEp-2 medical microscopic images

BackgroundThe identification of human epithelial type-2 mitotic cell patterns in the indirect immunofluorescence images (IIF HEp-2) is a critical step for autoimmune diseases computer-aided diagnosis (CAD) systems. Recognition of HEp-2 cells in the mitotic phase is clinically vital for validating the HEp-2 samples and assisting in diagnosing specimens with mixed patterns. Typically, mitotic cells are rarely observed in the HEp-2 specimen images, resulting in a significant skewness of the available medical datasets towards the majority of interphase (non-mitotic) patterns. MethodsIn this paper, a deep learning framework is proposed based on a self-attention deep cross-residual network with an efficient generative adversarial network (GAN). The proposed framework consists of two consecutive steps. First, the problem of imbalanced data of minority mitotic against the majority interphase cells is remedied using Info-WGANGP approach in combination with the conventional data augmentation methods. Second, a downstream end-to-end deep learning Att-DCRNet is developed to classify the mitotic and interphase cell patterns of the IIF HEp-2 medical images. A comprehensive experimental study is performed to validate the effectiveness of the proposed framework against other state-of-the-art methods over the public medical dataset of UQ-SNP_HEp-2 Task-3. ResultsThe proposed framework demonstrates competitive classification results attaining a maximum performance of 84.10% F1-score, 84.70% Matthew's correlation coefficient (MCC), and 99.0% balanced class accuracy (BcA), which proves its applicability for automatically supporting an accurate diagnosis decision regarding the HEp-2 mitotic and interphase cell patterns. The source code is available at this link: https://github.com/Anaam-dl/AttDCRNet_with_InfoWGANGP.

Automatic explanation of the classification of Spanish legal judgments in jurisdiction-dependent law categories with tree estimators

Automatic legal text classification systems have been proposed in the literature to address knowledge extraction from judgments and detect their aspects. However, most of these systems are black boxes even when their models are interpretable. This may raise concerns about their trustworthiness. Accordingly, this work contributes with a system combining Natural Language Processing (nlp) with Machine Learning (ml) to classify legal texts in an explainable manner. We analyze the features involved in the decision and the threshold bifurcation values of the decision paths of tree structures and present this information to the users in natural language. This is the first work on automatic analysis of legal texts combining nlp and ml along with Explainable Artificial Intelligence techniques to automatically make the models’ decisions understandable to end users. Furthermore, legal experts have validated our solution, and this knowledge has also been incorporated into the explanation process as “expert-in-the-loop” dictionaries. Experimental results on an annotated data set in law categories by jurisdiction demonstrate that our system yields competitive classification performance, with accuracy values well above 90%, and that its automatic explanations are easily understandable even to non-expert users.

Multi-Label Classification via Adaptive Resonance Theory-Based Clustering.

This paper proposes a multi-label classification algorithm capable of continual learning by applying an Adaptive Resonance Theory (ART)-based clustering algorithm and the Bayesian approach for label probability computation. The ART-based clustering algorithm adaptively and continually generates prototype nodes corresponding to given data, and the generated nodes are used as classifiers. The label probability computation independently counts the number of label appearances for each class and calculates the Bayesian probabilities. Thus, the label probability computation can cope with an increase in the number of labels. Experimental results with synthetic and real-world multi-label datasets show that the proposed algorithm has competitive classification performance to other well-known algorithms while realizing continual learning.

Ensuring Competitive Advantages of University Graduates in the Regional Labor Market: A Didactic Model (On the Example of Training IT Graduates).

The article focuses on the problem of constructing a didactic model of training a graduate of a higher education institution aimed at ensuring his/her competitiveness. The theoretical model is built on the example of training IT graduates of a regional university, but it can be extended to other areas, provided with highquality professional standards of a specialist. In order to overcome the gap between the competencies of students, formed in the process of mastering the basic professional educational program of the university, and the requirements of key regional employers to the training of young specialists, the approach to the design and modernization of the educational program of the university is presented, taking into account the specifics of professional activity of the graduate on the example of IT-industry. To achieve the research goal which is the development and theoretical substantiation of the model, the concept of “competitiveness of a university graduate” is analyzed, the classification of human competitiveness as a subject of economic life is presented. The didactic model of ensuring competitiveness in the regional labor market of a graduate of IT-specialties on educational programs of higher education, orienting a graduate of a regional university to intra-professional competitiveness in the conditions of the organization of the production process, characteristic of the regional employer, has been developed. Taking into account that for modern IT-industry the project approach is the main one, the conclusion is formulated that the necessary level of competitiveness of the graduate of IT-directions of training will be achieved if while designing or modernizing the educational program to proceed from the necessity to prepare the graduate for activity at the lower level of the team model of project implementation and to introduce the project roles of the upper level. The article provides a brief description of project roles, compiled together with experts representing a key regional employer. It is noted that for successful activity in certain project roles, along with professional competencies, it is necessary for a young specialist to have personal qualities corresponding to their specifics. It should also be noted that the results of the experimental testing of the model will be presented in subsequent publications.

RLogist: Fast Observation Strategy on Whole-Slide Images with Deep Reinforcement Learning

Whole-slide images (WSI) in computational pathology have high resolution with gigapixel size, but are generally with sparse regions of interest, which leads to weak diagnostic relevance and data inefficiency for each area in the slide. Most of the existing methods rely on a multiple instance learning framework that requires densely sampling local patches at high magnification. The limitation is evident in the application stage as the heavy computation for extracting patch-level features is inevitable. In this paper, we develop RLogist, a benchmarking deep reinforcement learning (DRL) method for fast observation strategy on WSIs. Imitating the diagnostic logic of human pathologists, our RL agent learns how to find regions of observation value and obtain representative features across multiple resolution levels, without having to analyze each part of the WSI at the high magnification. We benchmark our method on two whole-slide level classification tasks, including detection of metastases in WSIs of lymph node sections, and subtyping of lung cancer. Experimental results demonstrate that RLogist achieves competitive classification performance compared to typical multiple instance learning algorithms, while having a significantly short observation path. In addition, the observation path given by RLogist provides good decision-making interpretability, and its ability of reading path navigation can potentially be used by pathologists for educational/assistive purposes. Our code is available at: https://github.com/tencent-ailab/RLogist.

Facilitated machine learning for image-based fruit quality assessment

Image-based machine learning models can be used to make the sorting and grading of agricultural products more efficient. In many regions, implementing such systems can be difficult due to the lack of centralization and automation of postharvest supply chains. Stakeholders are often too small to specialize in machine learning, and large training data sets are unavailable. We propose a machine learning procedure for images based on pre-trained Vision Transformers. It is easier to implement than the current standard approach of training Convolutional Neural Networks (CNNs) as we do not (re-)train deep neural networks. We evaluate our approach based on two data sets for apple defect detection and banana ripeness estimation. Our model achieves a competitive classification accuracy equal to or less than one percent below the best-performing CNN. At the same time, it requires three times fewer training samples to achieve a 90% accuracy.