Multi-class Classification Scheme Research Articles

A comparative assessment of machine learning algorithms in the IoT-based network intrusion detection systems

The rapid increase in online risks is a reflection of the exponential growth of Internet of Things (IoT) networks. Researchers have proposed numerous intrusion detection techniques to mitigate the harm caused by these threats. Enterprises use intrusion detection systems (IDSs) and intrusion prevention systems (IPSs) to keep their networks safe, stable, and accessible. Network intrusion detection solutions have lately integrated powerful Machine Learning (ML) techniques to safeguard IoT networks. Selecting the proper data features for effectively training such ML models is critical to maximizing detection accuracy and computational efficiency. However, the efficiency of these systems degrades in high-dimensional data spaces, and it is crucial to have a suitable feature extraction method to eliminate extraneous data from the classification procedure. The detection accuracy and false positive rate of many ML-based IDSs also rise when the samples used to train the models are unbalanced. This study provides a detailed overview of the UNSW-NB15(DS-1) and NF-UNSWNB15(DS-2) datasets for intrusion detection, which will be utilized to develop and evaluate our models. In addition, this model uses the MaxAbsScaler algorithm to implement a filter-based feature scaling strategy . Then, use the condensed feature set to perform several ML techniques, including Support Vector Machines (SVM), K-nearest neighbors (KNN), Logistic Regression (LR), Naive Bayes (NB), Decision Tree (DT), and Random Forest (RF), considering multiclass classification. Accuracy tests for the multiclass classification scheme were improved from 60% to 94% using the MaxAbsScaler-based feature scaling method.

Deep Learning-Based Leaf Disease Detection in Crop Using Images for Agricultural Application

The "Leaf Disease Detection" system addresses the critical challenge of plant diseases in agriculture through the implementation of an automated solution leveraging deep learning techniques. In this comprehensive endeavor, convolutional neural networks (CNNs), specifically DenseNet-121, ResNet-50, VGG-16, and Inception V4, are fine-tuned for efficient and accurate identification of plant diseases. The project utilizes the Plant Village dataset, encompassing 54,305 images across 38 plant disease classes, to conduct a comparative analysis of model performance. DenseNet-121 emerged as the top-performing model, achieving an exceptional 99.81% classification accuracy, surpassing other state-of-the-art models. The system's methodology strategically employs transfer learning to overcome computational challenges associated with training deep CNN layers. This approach, coupled with the multi-class classification strategy, proves robust in handling diverse plant species and diseases within each class. The results highlight the superior efficiency of transfer learning in comparison to building models from scratch, showcasing the potential for real-world applications in agriculture. The system's success is attributed to the careful optimization of hyper parameters and the adoption of advanced deep learning techniques, offering a promising avenue for automated and accurate plant disease detection, with implications for improving agricultural practices, minimizing economic losses, and ensuring global food security.

An Ensemble Framework Approach to Crop Type Prediction Using Feature Selection and Multiclass Classification

Crop type classification plays a crucial role in modern agriculture, aiding in yield prediction, resource management, and land-use planning. This paper presents a comprehensive framework for crop type classification utilizing a combination of feature selection techniques, robust classification Algorithm, and a Support Vector Machine (SVM)-based multiclass classification approach. The proposed framework begins with a novel feature selection process that identifies the most relevant attributes from the Agricultural Data and Rainfall data. This feature selection step is essential for reducing data dimensionality, enhancing classification accuracy, and improving model interpretability. Following feature selection, a state-of-the-art multiclass classification strategy based on Support Vector Machines is employed. SVMs are known for their capability to handle high-dimensional data and have demonstrated superior performance in various classification tasks. In this framework, SVMs are adapted to handle multiclass crop type classification efficiently. The model is trained on the selected features and optimized using hyperparameter tuning techniques to ensure robust performance.

Multiclass Sparse Centroids With Application to Fast Time Series Classification.

In this article, we propose an efficient multiclass classification scheme based on sparse centroids classifiers. The proposed strategy exhibits linear complexity with respect to both the number of classes and the cardinality of the feature space. The classifier we introduce is based on binary space partitioning, performed by a decision tree where the assignation law at each node is defined via a sparse centroid classifier. We apply the presented strategy to the time series classification problem, showing by experimental evidence that it achieves performance comparable to that of state-of-the-art methods, but with a significantly lower classification time. The proposed technique can be an effective option in resource-constrained environments where the classification time and the computational cost are critical or, in scenarios, where real-time classification is necessary.

Optimal classification and generalized prevalence estimates for diagnostic settings with more than two classes.

An accurate multiclass classification strategy is crucial to interpreting antibody tests. However, traditional methods based on confidence intervals or receiver operating characteristics lack clear extensions to settings with more than two classes. We address this problem by developing a multiclass classification based on probabilistic modeling and optimal decision theory that minimizes the convex combination of false classification rates. The classification process is challenging when the relative fraction of the population in each class, or generalized prevalence, is unknown. Thus, we also develop a method for estimating the generalized prevalence of test data that is independent of classification of the test data. We validate our approach on serological data with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) naïve, previously infected, and vaccinated classes. Synthetic data are used to demonstrate that (i) prevalence estimates are unbiased and converge to true values and (ii) our procedure applies to arbitrary measurement dimensions. In contrast to the binary problem, the multiclass setting offers wide-reaching utility as the most general framework and provides new insight into prevalence estimation best practices.

ECG Marker Evaluation for the Machine-Learning-Based Classification of Acute and Chronic Phases of Trypanosoma cruzi Infection in a Murine Model.

Chagas disease (CD) is a neglected parasitic disease caused by the protozoan Trypanosoma cruzi (T. cruzi). The disease has two clinical phases: acute and chronic. In the acute phase, the parasite circulates in the blood. The infection can be asymptomatic or can cause unspecific clinical symptoms. During the chronic phase, the infection can cause electrical conduction abnormalities and progress to cardiac failure. The use of an electrocardiogram (ECG) has been a methodology for diagnosing and monitoring CD, but it is necessary to study the ECG signals to better understand the behavior of the disease. The aim of this study is to analyze different ECG markers using machine-learning-based algorithms for the classification of the acute and chronic phases of T. cruzi infection in a murine experimental model. The presented methodology includes a statistical analysis of control vs. infected models in both phases, followed by an automatic selection of ECG descriptors and the implementation of several machine learning algorithms for the automatic classification of control vs. infected mice in acute and/or chronic phases (binomial classification), as well as a multiclass classification strategy (control vs. the acute group vs. the chronic group). Feature selection analysis showed that P wave duration, R and P wave voltages, and the QRS complex are some of the most important descriptors. The classifiers showed good results in detecting the acute phase of infection (with an accuracy of 87.5%), as well as in multiclass classification (control vs. the acute group vs. the chronic group), with an accuracy of 91.3%. These results suggest that it is possible to detect infection at different phases, which can help in experimental and clinical studies of CD.

Joint Decision Tree and Visual Feature Rate Control Optimization for VVC UHD Coding.

In this paper, a joint decision tree and visual feature optimization rate control scheme for ultrahigh-definition (UHD) versatile video coding (VVC) is proposed. First, we design a new rate-distortion (R-D) model for UHD videos, and we establish a decision-tree-based multiclass classification scheme to improve the prediction accuracy of the R-D model by fully considering visual features. Second, based on the proposed R-D model, the globally optimal solution is obtained through convex optimization. Finally, we embed our algorithm into the latest VVC reference software, VTM 10.2. According to our experimental results, compared with the latest algorithm in VTM 10.2 and other state-of-the-art algorithms, our method can achieve significant bit rate reductions while maintaining a given peak signal-to-noise ratio (PSNR) or structural similarity index measure (SSIM).

A deep learning technique for intrusion detection system using a Recurrent Neural Networks based framework

In recent years, the spike in the amount of information transmitted through communication infrastructures has increased due to the advances in technologies such as cloud computing, vehicular networks systems, the Internet of Things (IoT), etc. As a result, attackers have multiplied their efforts for the purpose of rendering network systems vulnerable. Therefore, it is of utmost importance to improve the security of those network systems. In this study, an IDS framework using Machine Learning (ML) techniques is implemented. This framework uses different types of Recurrent Neural Networks (RNNs), namely, Long-Short Term Memory (LSTM), Gated Recurrent Unit (GRU) and Simple RNN. To assess the performance of the proposed IDS framework, the NSL-KDD and the UNSW-NB15 benchmark datasets are considered. Moreover, existing IDSs suffer from low test accuracy scores in detecting new attacks as the feature dimension grows. In this study, an XGBoost-based feature selection algorithm was implemented to reduce the feature space of each dataset. Following that process, 17 and 22 relevant attributes were picked from the UNSW-NB15 and NSL-KDD, respectively. The accuracy obtained through the test subsets was used as the main performance metric in conjunction with the F1-Score, the validation accuracy, and the training time (in seconds). The results showed that for the binary classification tasks using the NSL-KDD, the XGBoost-LSTM achieved the best performance with a test accuracy (TAC) of 88.13%, a validation accuracy (VAC) of 99.49% and a training time of 225.46 s. For the UNSW-NB15, the XGBoost-Simple-RNN was the most efficient model with a TAC of 87.07%. For the multiclass classification scheme, the XGBoost-LSTM achieved a TAC of 86.93% over the NSL-KDD and the XGBoost-GRU obtained a TAC of 78.40% over the UNSW-NB15 dataset. These results demonstrated that our proposed IDS framework performed optimally in comparison to existing methods.

Some methods for addressing errors in static AIS data records

The Automatic Identification System (AIS) provides essential services in support of maritime domain awareness. Accurate AIS values for hull dimension and type are often critical for safe and efficient management of ship traffic, and for development of new artificial intelligence maritime algorithms. AIS variables are subject to faults from multiple sources, ranging from bad weather to human error. New heuristic methods for correcting ship draft, beam, and class were developed and evaluated, using AIS data in the vicinity of large Florida ports as a test bed. Novel low order polynomials for 8 broad functional vessel classes yielded predicted values for draft and beam as functions of vessel length. The majority of relative differences between predicted and reported values were <0.1. A logistic regression (LR) multiclass classification scheme using the residuals from these polynomial predictions generally showed good agreement between estimated and reported vessel class. The LR scheme demonstrated skill in verifying AIS-transmitted classification, detecting incorrectly classified vessels, and flagging those with incorrect draft or operating near an extreme draft. A diagnostic of reports whose classification had very low and very high confidence suggested directions for further improvement of the algorithm. A new hierarchy for processed AIS data is proposed.

Ensemble Filter technique for Detection and Classification of attacks in Cloud Computing

In all technologies, including traditional computing and cloud computing, security has always been the primary concern. In recent years, cloud computing has become widely accepted on a global scale. Cyber attacks aimed at it have increased along with its widespread acceptance. Although ample research is done in the security domain and cloud computing is based on rigid security fundamentals, the advancing network security attacks create the need for an advanced security mechanism. Also, the multiclass classification strategy has received very little attention, and classification accuracy can yet be improved. Hence, this work proposes an Ensemble Filter-based Intrusion Detection System (EFIDS) to address the limitations of previous research work. It not only identifies malicious traffic but also categorizes the attempted attacks (multiclass classification). The famous intrusion detection benchmark dataset, NSL KDD, is used to evaluate the model. Using the model, it was possible to enhance the classification accuracy of both binary and multiclass approaches up to 99.85 percent and 99.63 percent, respectively. Additionally, both forms of classification have shown a 65–70% improvement in training time.

A Cloud Based Optimization Method for Zero-Day Threats Detection Using Genetic Algorithm and Ensemble Learning

This article presents a cloud-based method to classify 0-day attacks from a novel dataset called UGRansome1819. The primary objective of the research is to classify potential unknown threats using Machine Learning (ML) algorithms and cloud services. Our study contribution uses a novel anomaly detection dataset that carries 0-day attacks to train and test ML algorithms using Amazon Web Services such as S3 bucket and SageMaker. The proposed method used Ensemble Learning with a Genetic Algorithm (GA) optimizer having three ML algorithms such as Naive Bayes (NB), Random Forest (RF), and Support Vector Machine (SVM). These algorithms analyze the dataset by combining each classifier and assessing the classification accuracy of 0-day threats. We have implemented several metrics such as Accuracy, F1-Score, Confusion Matrix, Recall, and Precision to evaluate the performance of the selected algorithms. We have then compared the UGRansome1819 performance complexity with existing datasets using the same optimization settings. The RF implementation (before and after optimization) remains constant on the UGRansome1819 that outperformed the CAIDA and UNSWNB-15 datasets. The optimization technique only improved in Accuracy on the UNSWNB-15 and CAIDA datasets but sufficient performance was achieved in terms of F1-Score with UGRansome1819 using a multi-class classification scheme. The experimental results demonstrate a UGRansome1819 classification ratio of 1% before and after optimization. When compared to the UNSWNB-15 and CAIDA datasets, UGRansome1819 attains the highest accuracy value of 99.6% (prior optimization). The Genetic Algorithm was used as a feature selector and dropped five attributes of the UGRansome1819 causing a decrease in the computational time and over-fitting. The straightforward way to improve the model performance to increase its accuracy after optimization is to add more data samples to the training data. Doing so will add more details to the data and fine-tune the model will result in a more accurate and optimized performance. The experiments demonstrate the instability of single classifiers such as SVM and NB and suggest the proposed optimized validation technique which can aggregate weak classifiers (e.g., SVM and NB) into an ensemble of the genetic optimizer to enhance the classification performance. The UGRansome1819 model’s specificity and sensitivity were estimated to be 100% with three predictors of threatening classes (Signature, Synthetic Signature, and Anomaly). Lastly, the test classification accuracy of the SVM model improved by 6% after optimization.

The Use of C-Band and X-Band SAR with Machine Learning for Detecting Small-Scale Mining

Illicit small-scale mining occurs in many tropical regions and is both environmentally and socially hazardous. The aim of this study was to determine whether the classification of Synthetic Aperture Radar (SAR) imagery could detect and map small-scale mining in Ghana by analyzing multi-temporal filtering applied to three SAR datasets and testing five machine-learning classifiers. Using an object-based image analysis approach, we were successful in classifying water bodies associated with small-scale mining. The multi-temporally filtered Sentinel-1 dataset was the most reliable, with kappa coefficients at 0.65 and 0.82 for the multi-class classification scheme and binary-water classification scheme, respectively. The single-date Sentinel-1 dataset has the highest overall accuracy, at 90.93% for the binary water classification scheme. The KompSAT-5 dataset achieved the lowest accuracy at an overall accuracy of 80.61% and a kappa coefficient of 0.61 for a binary-water classification scheme. The experimental results demonstrated that it is possible to classify water as a proxy to identify illegal mining activities and that SAR is a potentially accurate and reliable solution for the detection of SSM in tropical regions such as Ghana. Therefore, using SAR can assist local governments in regulating small-scale mining activities by providing specific spatial information on the whereabouts of small-scale mining locations.

Risk assessment in COVID-19 patients: A multiclass classification approach

Understanding SARS-CoV-2 infection that causes COVID-19 disease among the population was fundamental to determine the risk factors associated with severe cases or even death. Amidst the study of the pandemic, Artificial Intelligence (AI) and Machine Learning (ML) have been successfully applied in many areas such as biomedicine. Using a dataset from the Mexican Ministry of Health, we performed a multiclass classification scheme for the detection of risks in COVID-19 patients and implemented three Machine Learning algorithms achieving the following accuracy measures: Random Forest (89.86%), GBM (89.37%) XGBoost (89.97%). The key findings are the identification of relevant components associated with different severities of COVID-19 disease. Among these factors, we found sex, age, days elapsed from the beginning of symptoms, symptoms such as dyspnea and polypnea; and other comorbidities such as diabetes and hypertension. This setting allows us to establish predicting algorithms to model the risk that an individual or a specific group of people face after contracting COVID-19 and the factors associated with developing complications or receiving appropriate treatment.

DeepMC-iNABP: Deep learning for multiclass identification and classification of nucleic acid-binding proteins

Nucleic acid-binding proteins (NABPs), including DNA-binding proteins (DBPs) and RNA-binding proteins (RBPs), play vital roles in gene expression. Accurate identification of these proteins is crucial. However, there are two existing challenges: one is the problem of ignoring DNA- and RNA-binding proteins (DRBPs), and the other is a cross-predicting problem referring to DBP predictors predicting DBPs as RBPs, and vice versa. In this study, we proposed a computational predictor, called DeepMC-iNABP, with the goal of solving these difficulties by utilizing a multiclass classification strategy and deep learning approaches. DBPs, RBPs, DRBPs and non-NABPs as separate classes of data were used for training the DeepMC-iNABP model. The results on test data collected in this study and two independent test datasets showed that DeepMC-iNABP has a strong advantage in identifying the DRBPs and has the ability to alleviate the cross-prediction problem to a certain extent. The web-server of DeepMC-iNABP is freely available at http://www.deepmc-inabp.net/. The datasets used in this research can also be downloaded from the website.

A computed tomography (CT)-derived radiomics approach for predicting primary co-mutations involving TP53 and epidermal growth factor receptor (EGFR) in patients with advanced lung adenocarcinomas (LUAD).

Epidermal growth factor receptor (EGFR) co-mutated with TP53 could reduce responsiveness to tyrosine kinase inhibitors (TKIs) and worsen patients' prognosis compared to TP53 wild type patients in. mutated lung adenocarcinomas (LUAD). To identify this genetically unique subset prior to treatment through computed tomography (CT) images had not been reported yet. Stage III and IV LUAD with known mutation status of EGFR and TP53 from The First Affiliated Hospital of Sun Yat-sen University (May 1, 2017 to June 1, 2020) were collected. Characteristics of pretreatment enhanced-CT images were analyzed. One-versus-one was used as the multiclass classification strategy to distinguish the three subtypes of co-mutations: EGFR + & TP53 +, EGFR + & TP53 -, EGFR -. The clinical model, semantic model, radiomics model and integrated model were built. Area under the receiver-operating characteristic curves (AUCs) were used to evaluate the prediction efficacy. A total of 199 patients were enrolled, including 83 (42%) cases of EGFR -, 55 (28%) cases of EGFR + & TP53 +, 61 (31%) cases of EGFR + & TP53 -. Among the four different models, the integrated model displayed the best performance for all the three subtypes of co-mutations: EGFR - (AUC, 0.857; accuracy, 0.817; sensitivity, 0.998; specificity, 0.663), EGFR + & TP53 + (AUC, 0.791; accuracy, 0.758; sensitivity, 0.762; specificity, 0.783), EGFR + & TP53 - (AUC, 0.761; accuracy, 0.813; sensitivity, 0.594; specificity, 0.977). The radiomics model was slightly inferior to the integrated model. The results for the clinical and the semantic models were dissatisfactory, with AUCs less than 0.700 for all the three subtypes. CT imaging based artificial intelligence (AI) is expected to distinguish co-mutation status involving TP53 and EGFR. The proposed integrated model may serve as an important alternative marker for preselecting patients who will be adaptable to and sensitive to TKIs.

A Novel Coarse-to-Fine Method of Ship Detection in Optical Remote Sensing Images Based on a Deep Residual Dense Network

Automatic ship detection in optical remote sensing images is of great significance due to its broad applications in maritime security and fishery control. Most ship detection algorithms utilize a single-band image to design low-level and hand-crafted features, which are easily influenced by interference like clouds and strong waves and not robust for large-scale variation of ships. In this paper, we propose a novel coarse-to-fine ship detection method based on discrete wavelet transform (DWT) and a deep residual dense network (DRDN) to address these problems. First, multi-spectral images are adopted for sea-land segmentation, and an enhanced DWT is employed to quickly extract ship candidate regions with missing alarms as low as possible. Second, panchromatic images with clear spatial details are used for ship classification. Specifically, we propose the local residual dense block (LRDB) to fully extract semantic feature via local residual connection and densely connected convolutional layers. DRDN mainly consists of four LRDBs and is designed to further remove false alarms. Furthermore, we exploit the multiclass classification strategy, which can overcome the large intra-class difference of targets and identify ships of different sizes. Extensive experiments demonstrate that the proposed method has high robustness in complex image backgrounds and achieves higher detection accuracy than other state-of-the-art methods.

Imbalance learning for variable star classification

ABSTRACT The accurate automated classification of variable stars into their respective subtypes is difficult. Machine learning–based solutions often fall foul of the imbalanced learning problem, which causes poor generalization performance in practice, especially on rare variable star subtypes. In previous work, we attempted to overcome such deficiencies via the development of a hierarchical machine learning classifier. This ‘algorithm-level’ approach to tackling imbalance yielded promising results on Catalina Real-Time Survey (CRTS) data, outperforming the binary and multiclass classification schemes previously applied in this area. In this work, we attempt to further improve hierarchical classification performance by applying ‘data-level’ approaches to directly augment the training data so that they better describe underrepresented classes. We apply and report results for three data augmentation methods in particular: Randomly Augmented Sampled Light curves from magnitude Error (RASLE), augmenting light curves with Gaussian Process modelling (GpFit) and the Synthetic Minority Oversampling Technique (SMOTE). When combining the ‘algorithm-level’ (i.e. the hierarchical scheme) together with the ‘data-level’ approach, we further improve variable star classification accuracy by 1–4 per cent. We found that a higher classification rate is obtained when using GpFit in the hierarchical model. Further improvement of the metric scores requires a better standard set of correctly identified variable stars, and perhaps enhanced features are needed.

A deep learning method with wrapper based feature extraction for wireless intrusion detection system

In the past decade, wired and wireless computer networks have substantially evolved because of the rapid development of technologies such as the Internet of Things (IoT), wireless handled devices, vehicular networks, 4G and 5G, cyber-physical systems, etc. These technologies exchange large amount of data, and as a result, they are prone to several malicious actions, attacks and security threats that can compromise the availability and integrity of information or services. Therefore, the security and protection of the various communication infrastructures using an intrusion detection system (IDS) is of critical importance. In this research, we propose a Feed-Forward Deep Neural Network (FFDNN) wireless IDS system using a Wrapper Based Feature Extraction Unit (WFEU). The extraction method of the WFEU uses the Extra Trees algorithm in order to generate a reduced optimal feature vector. The effectiveness and efficiency of the WFEU-FFDNN is studied based on the UNSW-NB15 and the AWID intrusion detection datasets. Furthermore, the WFEU-FFDNN is compared to standard machine learning (ML) algorithms that include Random Forest (RF), Support Vector Machine (SVM), Naïve Bayes (NB), Decision Tree (DT) and k-Nearest Neighbor (kNN). The experimental studies include binary and multiclass types of attacks. The results suggested that the proposed WFEU-FFDNN has greater detection accuracy than other approaches. In the instance of the UNSW-NB15, the WFEU generated an optimal feature vector consisting of 22 attributes. Using this input vector; our approach achieved overall accuracies of 87.10% and 77.16% for the binary and multiclass classification schemes, respectively. In the instance of the AWID, a reduced input vector of 26 attributes was generated by the WFEU, and the experiments demonstrated that our method obtained overall accuracies of 99.66% and 99.77% for the binary and the multiclass classification configurations, respectively.

Multi-site structural damage identification using a multi-label classification scheme of machine learning

This study proposes a new method to locate multi-site structural damage using a data-driven multi-label classification (MLC) method. Differing from the multiclass classification (MCC) scheme that sets each damage case as a category, the MLC method denotes each damage case with multiple labels, with each label indicating the occurrence of damage at a certain location. The MLC method considers the physical correlation between damage cases sharing damage locations which is neglected in the MCC method. This paper uses the instance differentiation algorithm to implement MLC. Damage identification results using numerical and experimental data indicate that the MLC can identify multi-site damage with good accuracy, even for damage cases that are not covered by the training dataset. Through comparison, it is demonstrated that the MLC method outperforms the MCC and binary classification methods for multi-site damage identification. Moreover, MLC preserves generality when tested on unobserved multi-site damage cases.

Comparing Multiclass, Binary, and Hierarchical Machine Learning Classification schemes for variable stars

ABSTRACTUpcoming synoptic surveys are set to generate an unprecedented amount of data. This requires an automatic framework that can quickly and efficiently provide classification labels for several new object classification challenges. Using data describing 11 types of variable stars from the Catalina Real-Time Transient Survey (CRTS), we illustrate how to capture the most important information from computed features and describe detailed methods of how to robustly use information theory for feature selection and evaluation. We apply three machine learning algorithms and demonstrate how to optimize these classifiers via cross-validation techniques. For the CRTS data set, we find that the random forest classifier performs best in terms of balanced accuracy and geometric means. We demonstrate substantially improved classification results by converting the multiclass problem into a binary classification task, achieving a balanced-accuracy rate of ∼99 per cent for the classification of δ Scuti and anomalous Cepheids. Additionally, we describe how classification performance can be improved via converting a ‘flat multiclass’ problem into a hierarchical taxonomy. We develop a new hierarchical structure and propose a new set of classification features, enabling the accurate identification of subtypes of Cepheids, RR Lyrae, and eclipsing binary stars in CRTS data.