Large Number Of Categories Research Articles

Preserving text space integrity for robust compositional zero-shot learning via mixture of pretrained experts

In the current landscape of Compositional Zero-Shot Learning (CZSL) methods that leverage CLIP, the predominant approach is based on prompt learning paradigms. These methods encounter significant computational complexity when dealing with a large number of categories. Additionally, when confronted with new classification tasks, there is a necessity to learn the prompts again, which can be both time-consuming and resource-intensive. To address these challenges, We present a new methodology, named the Mixture of Pretrained Expert (MoPE), for enhancing Compositional Zero-shot Learning through Logit-Level Fusion with Mutile Expert Fusion Module. The MoPE skillfully blends the benefits of extensive pre-trained models like CLIP, Bert, GPT-3 and Word2Vec for effectively tackling Compositional Zero-shot Learning. Firstly, we extract the text label space for each language model individually, then map the visual feature vectors to their respective text spaces. This maintains the integrity and structure of the original text space. During this process, the pre-trained expert parameters are kept frozen. The mapping of visual features to the corresponding text spaces is subject to learning and could be considered as multiple learnable visual experts. In the model fusion phase, we propose a new fusion strategy that features a gating mechanism that adjusts the contributions of various models dynamically. This enables our approach to adapt more effectively to a range of tasks and data sets. The method’s robustness is demonstrated by the fact that the language model is not tailored to specific downstream task datasets or losses. This preserves the larger model’s topology and expands the potential for application. Preliminary experiments conducted on the UT-Zappos, AO-Clever, and C-GQA datasets indicate that MoPE performs competitively when compared to existing techniques.

Arabic text classification based on analogical proportions

AbstractText classification is the process of labelling a given set of text documents with predefined classes or categories. Existing Arabic text classifiers are either applying classic Machine Learning algorithms such as k‐NN and SVM or using modern deep learning techniques. The former are assessed using small text collections and their accuracy is still subject to improvement while the latter are efficient in classifying big data collections and show limited effectiveness in classifying small corpora with a large number of categories. This paper proposes a new approach to Arabic text classification to treat small and large data collections while improving the classification rates of existing classifiers. We first demonstrate the ability of analogical proportions (AP) (statements of the form ‘x is to as is to ’), which have recently been shown to be effective in classifying ‘structured’ data, to classify ‘unstructured’ text documents requiring preprocessing. We design an analogical model to express the relationship between text documents and their real categories. Next, based on this principle, we develop two new analogical Arabic text classifiers. These rely on the idea that the category of a new document can be predicted from the categories of three others, in the training set, in case the four documents build together a ‘valid’ analogical proportion on all or on a large number of components extracted from each of them. The two proposed classifiers (denoted AATC1 and AATC2) differ mainly in terms of the keywords extracted for classification. To evaluate the proposed classifiers, we perform an extensive experimental study using five benchmark Arabic text collections with small or large sizes, namely ANT (Arabic News Texts) v2.1 and v1.1, BBC‐Arabic, CNN‐Arabic and AlKhaleej‐2004. We also compare analogical classifiers with both classical ML‐based and Deep Learning‐based classifiers. Results show that AATC2 has the best average accuracy (78.78%) over all other classifiers and the best average precision (0.77) ranked first followed by AATC1 (0.73), NB (0.73) and SVM (0.72) for the ANT corpus v2.1. Besides, AATC1 shows the best average precisions (0.88) and (0.92), respectively for the BBC‐Arabic corpus and AlKhaleej‐2004, and the best average accuracy (85.64%) for CNN‐Arabic over all other classifiers. Results demonstrate the utility of analogical proportions for text classification. In particular, the proposed analogical classifiers are shown to significantly outperform a number of existing Arabic classifiers, and in many cases, compare favourably to the robust SVM classifier.

A novel plant type, leaf disease and severity identification framework using CNN and transformer with multi-label method.

The growth of plants is threatened by numerous diseases. Accurate and timely identification of these diseases is crucial to prevent disease spreading. Many deep learning-based methods have been proposed for identifying leaf diseases. However, these methods often combine plant, leaf disease, and severity into one category or treat them separately, resulting in a large number of categories or complex network structures. Given this, this paper proposes a novel leaf disease identification network (LDI-NET) using a multi-label method. It is quite special because it can identify plant type, leaf disease and severity simultaneously using a single straightforward branch model without increasing the number of categories and avoiding extra branches. It consists of three modules, i.e., a feature tokenizer module, a token encoder module and a multi-label decoder module. The LDI-NET works as follows: Firstly, the feature tokenizer module is designed to enhance the capability of extracting local and long-range global contextual features by leveraging the strengths of convolutional neural networks and transformers. Secondly, the token encoder module is utilized to obtain context-rich tokens that can establish relationships among the plant, leaf disease and severity. Thirdly, the multi-label decoder module combined with a residual structure is utilized to fuse shallow and deep contextual features for better utilization of different-level features. This allows the identification of plant type, leaf disease, and severity simultaneously. Experiments show that the proposed LDI-NET outperforms the prevalent methods using the publicly available AI challenger 2018 dataset.

Investigating the Functioning of Rating Scales With Rasch Models.

The hypothesis implicit in the rating scale design is that the categories reflect increasing levels of the latent variable. Rasch models for ordered polytomous items include parameters, called thresholds, that allow for empirically testing this hypothesis. Failure of the thresholds to advance monotonically with the categories (a condition that is referred to as "threshold disordering") provides evidence that the rating scale is not functioning as intended. This work focuses on scales consisting of rather large numbers of categories, whose use is often recommended in the literature. Threshold disordering is observed in both an extended 8-point scale specially developed for the Patient Health Questionnaire-9 and the original 10-point scale of the Behavioral Religiosity Scale. The results of this work prompt practitioners not to take the functioning of the rating scale for granted, but to verify it empirically.

Finding Discriminative Subsequences Via a Coverage Measure and Mutual Information Selection Strategy for Multi-Class Time Series Classification

Time series classification (TSC) has attracted considerable attention from the data mining community over the past decades. One of the effective ways to handle this task is to find discriminative subsequences in time series to train a classifier. Obviously, how to measure the discriminative power of subsequences and find the optimal combination of subsequences is crucial to the accuracy of TSC. In this paper, we introduce a new method, CRMI, to find high-quality discriminative subsequences for multi-class time series classification (MC-TSC). Different from existing methods, there are two significant innovations in the work. At first, we propose a novel measure, named coverage ratio, to evaluate the discriminative power of a subsequence based on a coverage matrix which is figured out by the clustering technique. Second, a heuristic algorithm based on mutual information (MI) is proposed to find the optimal combination of subsequence candidates. The calculation of MI is also based on the coverage matrix. Extensive experiments were conducted on 54 UCR time series datasets with at least 3 categories, and the results show that (1) the proposed algorithm achieves the highest average accuracy and outperforms most of the existing shapelet-based TSC algorithms; (2) compared with existing methods, the proposed algorithm performs better on datasets with a large number of categories.

Compound Text-Guided Prompt Tuning via Image-Adaptive Cues

Vision-Language Models (VLMs) such as CLIP have demonstrated remarkable generalization capabilities to downstream tasks. However, existing prompt tuning based frameworks need to parallelize learnable textual inputs for all categories, suffering from massive GPU memory consumption when there is a large number of categories in the target dataset. Moreover, previous works require to include category names within prompts, exhibiting subpar performance when dealing with ambiguous category names. To address these shortcomings, we propose Compound Text-Guided Prompt Tuning (TGP-T) that significantly reduces resource demand while achieving superior performance. We introduce text supervision to the optimization of prompts, which enables two benefits: 1) releasing the model reliance on the pre-defined category names during inference, thereby enabling more flexible prompt generation; 2) reducing the number of inputs to the text encoder, which decreases GPU memory consumption significantly. Specifically, we found that compound text supervisions, i.e., category-wise and content-wise, is highly effective, since they provide inter-class separability and capture intra-class variations, respectively. Moreover, we condition the prompt generation on visual features through a module called Bonder, which facilitates the alignment between prompts and visual features. Extensive experiments on few-shot recognition and domain generalization demonstrate that TGP-T achieves superior performance with consistently lower training costs. It reduces GPU memory usage by 93% and attains a 2.5% performance gain on 16-shot ImageNet. The code is available at https://github.com/EricTan7/TGP-T.

Bayesian estimation of the Kullback-Leibler divergence for categorical systems using mixtures of Dirichlet priors.

In many applications in biology, engineering, and economics, identifying similarities and differences between distributions of data from complex processes requires comparing finite categorical samples of discrete counts. Statistical divergences quantify the difference between two distributions. However, their estimation is very difficult and empirical methods often fail, especially when the samples are small. We develop a Bayesian estimator of the Kullback-Leibler divergence between two probability distributions that makes use of a mixture of Dirichlet priors on the distributions being compared. We study the properties of the estimator on two examples: probabilities drawn from Dirichlet distributions and random strings of letters drawn from Markov chains. We extend the approach to the squared Hellinger divergence. Both estimators outperform other estimation techniques, with better results for data with a large number of categories and for higher values of divergences.

On clustering levels of a hierarchical categorical risk factor

Abstract Handling nominal covariates with a large number of categories is challenging for both statistical and machine learning techniques. This problem is further exacerbated when the nominal variable has a hierarchical structure. We commonly rely on methods such as the random effects approach to incorporate these covariates in a predictive model. Nonetheless, in certain situations, even the random effects approach may encounter estimation problems. We propose the data-driven Partitioning Hierarchical Risk-factors Adaptive Top-down algorithm to reduce the hierarchically structured risk factor to its essence, by grouping similar categories at each level of the hierarchy. We work top-down and engineer several features to characterize the profile of the categories at a specific level in the hierarchy. In our workers’ compensation case study, we characterize the risk profile of an industry via its observed damage rates and claim frequencies. In addition, we use embeddings to encode the textual description of the economic activity of the insured company. These features are then used as input in a clustering algorithm to group similar categories. Our method substantially reduces the number of categories and results in a grouping that is generalizable to out-of-sample data. Moreover, we obtain a better differentiation between high-risk and low-risk companies.

An improved multi-stage framework for large-scale hierarchical text classification problems using a modified feature hashing and bi-filtering strategy

The classification of large-scale textual dataset is associated with a huge number of instances and millions of features which must be discriminated between large numbers of categories. The task requires the utilization of a defined hierarchy structure and tools that automatically classify instances within the hierarchy known as Large Scale Hierarchical Text Classification (LSHTC). Predicting the labels of instances by the employed classifiers is challenging due to the high number of features. Furthermore, the existing Dimensional Reduction (DR) approaches in cooperation with the LSHTC framework are still quite inefficient. In such a problem, an effective Hierarchical Dimensional Reduction approach can be advantageous in improving the performance of the LSHTC. Therefore, in this paper, we enhance the performance of LSHTC by proposing a Multi-stage Hierarchical Dimensional Reduction (MHDR) approach based on Modified Feature Hashing (MFH) and Hierarchical Bi-Filtering (HBF) method. In addition to alleviating bad collision and result discrepancy, experimental results show that the proposed approach has achieve the best performance in terms of micro-f1 and macro-f1 by recording average scores of 58.47% and 54.77% using TD-SVM, and average scores of 51.14% and 48.70% using TD-LR, respectively. The method also achieved 11% speed-up than the approaches compared.

Image Classification on Fashion Dataset Using Inception V3

Fashion is a form of self-expression that allows us to be able to manifest our personality and identity more confidently. One of the effects of Covid 19 is the economics of the industry, especially the fashion category as the largest category in the e-commerce industry. However, A large number of categories in each fashion brand allows shop owners to be misclassifications about the placement of items that have nearly the same clothing model. The other problem is sellers uploading pictures of products on the platform for the sale and the consequent manual tagging involved. In this paper, we proposed image classification on the fashion dataset using inception V3. The methodology of this paper consists of scrapping data from the official websites of five famous fashion brands, data preprocessing, and classification with the Inception V3 method. The accuracy and F1-Score values obtained using Inception V3 are 92.86% and 92.85%. The proposed method is the highest result of the comparison method and can differentiate between knitted with a scarf that is difficult to classify when compared to other comparison methods.

Метрика схожості категоріальних розподілів, що враховує спорідненість різних категорій

Estimating a level of similarity of two objects is a common problem in pattern recognition, clustering and classification. Among these problems can be reviewer recommendation, similar text documents analysis, human pose detection in video, species distribution clustering, recommendation in internet-shops etc. In case of categorical attributes an object is described as a distribution of membership degrees over categories. Similarity metrics of such distributions are usually defined as a superposition of objects’ similarities for each category. Most often it is a sum of similarities in separate categories. In addition to that each category is considered independently and in isolation from the others. Some practical problems have categories that are kinship. Therefore, it is expedient to consider objects’ similarity not only directly, as a similarity between equivalent categories, but it is also necessary to consider an indirect similarity, cross-similarity through kinship categories. It is such similarity metric of two categorical distributions that accounts for the kinship of different categories is proposed in this paper. The metric has two components. The first component is defined as Czekanowski metric. It defines a direct similarity of categorical distributions as a sum of intersection of distributions’ membership degrees of two objects. After the intersection the residuals are accounted for in the second component of the metric. The second metric’s component is defined as element-wise product of two matrices: matrix of residuals composition from membership degrees of two categorical distributions and matrix of categories’ paired kinship. It is assumed that kinship indices for each pair of categories are known. As a result, with a large number of categories the overall noisy contribution from weakly kinship categories is prominent. Therefore, it is proposed to filter the noise and account only for contribution from strongly kinship categories.

КОНЦЕПТУАЛЬНАЯ ОБЛАСТЬ PERSONALITY TRAITS: СТРУКТУРА И СРЕДСТВА ЯЗЫКОВОЙ РЕПРЕЗЕНТАЦИИ

The paper deals with conceptual and language representation of the character of a person and aims to explore the content and structure of the conceptual domain PERSONALITY TRAITS through describing the system of categories and subcategories that constitute the domain and to identify the units that represent the categories in the English language and discourse. The research methods used include conceptual analysis, dictionary and thesaurus analysis, discourse analysis, comparative analysis and the analytico-descriptive method. The article defines the scope of the conceptual domain and shows that it contains a large number of categories and subcategories that can be grouped into 22 thematic classes representing various aspects of a person’s character and attitude to other people and entities. The categories are expressed in discourse by numerous lexemes (nouns and adjectives) and phrases. It is shown that some of the categories, most notably those pertaining to a person’s attitude to norms of civilized behaviour, have a much greater number of means of expression than others. The findings contribute to the studies of the English language worldview and comparative analysis of different language worldviews. The findings can also be used in the cognitive studies of lexemes, synonyms and translation.

Learning From High-Cardinality Categorical Features in Deep Neural Networks

Some machine learning algorithms expect the input variables and the output variables to be numeric. Therefore, in an early stage of modelling, feature engineering is required when categorical variables present in the dataset. As a result, we must encode those attributes into an appropriate feature vector. However, categorical variables having more than 100 unique values are considered to be high-cardinality and there exists no straightforward methods to handle them. Besides, the majority of the work on categorical variable encoding in the literature assumes that the categories is limited, known beforehand, and made up of mutually-exclusive elements, inde-pendently from the data, which is not necessarily true for real-world applications. Feature engineering typically practices to tackle the high cardinality issues with data-cleaning techniques which they are time-consuming and often needs human intervention and domain expertise which are major costs in data science projects The most common methods of transform categorical variables is one-hot encoding and target encoding. To address the issue of encoding categorical variables in environments with a high cardinality, we also seek a general-purpose approach for statistical analysis of categorical entries that is capable of handling a very large number of catego-ries, while avoiding computational and statistical difficulties. Our proposed approach is low dimensional; thus, it is very efficient in processing time and memory, it can be computed in an online learning setting. Even though for this paper, we opt to utilize it in the input layer, dictionaries are typically architecture-independent and may be moved between different architectures or layers.

Sequence Embeddings Help Detect Insurance Fraud

Roughly 10 percent of the insurance industry’s incurred losses are estimated to stem from fraudulent claims. One solution is to use tabular data to construct models that can distinguish between claims that are legitimate and those that are fraudulent. However, while canonical tabular data models enable robust fraud detection, complex sequential data have been out of the insurance industry’s scope. For health insurance, we propose deep learning architectures that process insurance data consisting of sequential records of patient visits and characteristics. Both the sequential and tabular components improve the quality of the model, generating new insights into the detection of health insurance fraud. Empirical results derived using relevant data from a health insurance company show that our approach outperforms state-of-the-art models and can substantially improve the claims management process. We obtain a ROC AUC metric of 0.873, while the best competitor based on state-of-the-art models achieves 0.815. Moreover, we demonstrate that our architectures are more robust to data corruption. As more and more semi-structured event sequence data become available to insurers, our methods will be valuable for many similar applications, particularly when variables have a large number of categories, such as those from the International Classification of Disease (ICD) codes or other classification codes.

A classification method to classify bone marrow cells with class imbalance problem

Bone marrow cell morphology has long been used to diagnose blood diseases. However, it requires long-term experience from a suitable person. Furthermore, the outcomes of their recognition are subjective and no quantitative standard has been established yet. Consequently, developing a deep learning automatic system for classifying bone marrow cells is extremely important. However, real-life data sets, such as bone marrow cell data, constantly suffer from a long-tail distribution problem, owing to which the final trained classifier is biased toward a large number of categories. Thus, addressing this issue is crucial. The current research presents a class balance classification method (CBCM) for classifying 15 types of bone marrow cell data sets with a class imbalance problem. CBCM outperforms other balance approaches such as random over-sampling, synthetic minority over-sampling technique (SMOTE), random under-sampling, weighted random forest and weighted cross-entropy function, achieving precision, sensitivity, and specificity values of 84.53%, 84.44% and 99.29% respectively. A more extensive comparison between the baseline and CBCM, as well as the Grad-CAM and Guided Grad-CAM of CBCM, reveals that CBCM is a reliable and effective solution to address the long-tail distribution problem of the bone marrow cell data sets.

The number of response categories in ordered response models.

The choice of the number m of response categories is a crucial issue in categorization of a continuous response. The paper exploits the Proportional Odds Models' property which allows to generate ordinal responses with a different number of categories from the same underlying variable. It investigates the asymptotic efficiency of the estimators of the regression coefficients and the accuracy of the derived inferential procedures when m varies. The analysis is based on models with closed-form information matrices so that the asymptotic efficiency can be analytically evaluated without need of simulations. The paper proves that a finer categorization augments the information content of the data and consequently shows that the asymptotic efficiency and the power of the tests on the regression coefficients increase with m. The impact of the loss of information produced by merging categories on the efficiency of the estimators is also considered, highlighting its risks especially when performed in its extreme form of dichotomization. Furthermore, the appropriate value of m for various sample sizes is explored, pointing out that a large number of categories can offset the limited amount of information of a small sample by a better quality of the data. Finally, two case studies on the quality of life of chemotherapy patients and on the perception of pain, based on discretized continuous scales, illustrate the main findings of the paper.

Historical analysis of the development of powerlifting. Powerlifting. Stages of the learning process. Competition texchniques

At the present stage of the study, powerlifting is one of the priority sports in Ukraine among various segments of the population. This is a fairly new and modern sport with a huge history, which has played an important role in the formation and implementation of sports activities.
 The popularity of powerlifting is due to its simplicity, accessibility for a large number of categories of the population, fast results with the right technique and strong-willed qualities of character, it is also important to have a positive impact on the athlete's health. and the psychological state of man.
 Powerlifting classes increase muscle strength, strengthen ligaments and joints, help to develop endurance, flexibility and other physical qualities, cultivate willpower, self-confidence, increase the efficiency of the whole organism.
 Strength sports have always been popular among athletes and people who strive to lead a healthy lifestyle. Recently, this area of sport is developing particularly rapidly, such relatively new types as bodybuilding, powerlifting, bench press, arm wrestling, armlifting, workout have become widespread. The main problem faced by athletes and coaches is the almost complete lack of competent training systems for these power sports, as well as in the process of competitive training: the requirements and features of competitive activities. The reason is clear and quite objective - the relative novelty and lack of theoretical basis. Now the only possible way to create a training technique is mechanical borrowing from other sports, mainly from weightlifting.
 Powerlifting (from English. Powerlifting, power - strength, lifting - lifting) - power triathlon, includes three competitive exercises with a barbell: squats, bench press and traction.
 Powerlifting sports training is a process aimed at comprehensive and systematic improvement of all physical and moral qualities that are leading in this sport. Sports training in powerlifting includes training in the technique of competitive exercises (squats, bench press, traction position) and the development of physical qualities with special auxiliary exercises with a barbell and other types of weights.
 Despite the fact that powerlifting does not require special technical skills. The three movements of the competition look simple and, it would seem, with some physical strength you can show significant results. However, as in any other sport, powerlifting has many nuances and factors, despite which it is impossible to talk about any results. First of all, these are the general basics of technique, as well as individually selected techniques and training methods that take into account age, anatomical, biomechanical, psychological characteristics of the athlete, his physical fitness. Since all three movements in powerlifting are associated with a heavy load on the joint and musculoskeletal system, any serious powerlifting is impossible without constant medical supervision (at least once a month). Not the last role is played by the training and competitive equipment meeting the requirements of the international standards and the equipment of the athlete.

Identification of Multi-Class Drugs Based on Near Infrared Spectroscopy and Bidirectional Generative Adversarial Networks.

Drug detection and identification technology are of great significance in drug supervision and management. To determine the exact source of drugs, it is often necessary to directly identify multiple varieties of drugs produced by multiple manufacturers. Near-infrared spectroscopy (NIR) combined with chemometrics is generally used in these cases. However, existing NIR classification modeling methods have great limitations in dealing with a large number of categories and spectra, especially under the premise of insufficient samples, unbalanced samples, and sensitive identification error cost. Therefore, this paper proposes a NIR multi-classification modeling method based on a modified Bidirectional Generative Adversarial Networks (Bi-GAN). It makes full utilization of the powerful feature extraction ability and good sample generation quality of Bi-GAN and uses the generated samples with obvious features, an equal number between classes, and a sufficient number within classes to replace the unbalanced and insufficient real samples in the courses of spectral classification. 1721 samples of four kinds of drugs produced by 29 manufacturers were used as experimental materials, and the results demonstrate that this method is superior to other comparative methods in drug NIR classification scenarios, and the optimal accuracy rate is even more than 99% under ideal conditions.

An explainable attention network for fraud detection in claims management

Insurance companies must manage millions of claims per year. While most of these are not fraudulent, those that are nevertheless cost insurance companies and those they insure vast amounts of money. The ultimate goal is to develop a predictive model that can single out fraudulent claims and pay out non-fraudulent ones automatically. Health care claims have a peculiar data structure, comprising inputs of varying length and variables with a large number of categories. Both issues are challenging for traditional econometric methods. We develop a deep learning model that can handle these challenges by adapting methods from text classification. Using a large dataset from a private health insurer in Germany, we show that the model we propose outperforms a conventional machine learning model. With the rise of digitalization, unstructured data with characteristics similar to ours will become increasingly common in applied research, and methods to deal with such data will be needed.

Bringing Context Inside Process Research with Digital Trace Data

Context is usually conceptualized as external to a theory or model and treated as something to be controlled or eliminated in empirical research. We depart from this tradition and conceptualize context as permeating processual phenomena. This move is possible because digital trace data are now increasingly available, providing rich and fine-grained data about processes mediated or enabled by digital technologies. This paper introduces a novel method for including fine-grained contextual information from digital trace data within the description of process (e.g., who, what, when, where, why). Adding contextual information can result in a very large number of fine-grained categories of events, which are usually considered undesirable. However, we argue that a large number of categories can make process data more informative for theorizing and that including contextual detail enriches the understanding of processes as they unfold. We demonstrate this by analyzing audit trail data of electronic medical records using ThreadNet, an open source software application developed for the qualitative visualization and analysis of process data. The distinctive contribution of our approach is the novel way in which we contextualize events and action in process data. Providing new, usable ways to incorporate context can help researchers ask new questions about the dynamics of processual phenomena.