Class-specific Models Research Articles

An enzyme-inspired specificity in deep learning model for sleep stage classification using multi-channel PSG signals input: Separating training approach and its performance on cross-dataset validation for generalizability

Numerous automatic sleep stage classification systems have been developed, but none have become effective assistive tools for sleep technicians due to issues with generalization. Four key factors hinder the generalization of these models are instruments, montage of recording, subject type, and scoring manual factors. This study aimed to develop a deep learning model that addresses generalization problems by integrating enzyme-inspired specificity and employing separating training approaches. Subject type and scoring manual factors were controlled, while the focus was on instruments and montage of recording factors. The proposed model consists of three sets of signal-specific models including EEG-, EOG-, and EMG-specific model. The EEG-specific models further include three sets of channel-specific models. All signal-specific and channel-specific models were established with data manipulation and weighted loss strategies, resulting in three sets of data manipulation models and class-specific models, respectively. These models were CNNs. Additionally, BiLSTM models were applied to EEG- and EOG-specific models to obtain temporal information. Finally, classification task for sleep stage was handled by ‘the-last-dense’ layer. The optimal sampling frequency for each physiological signal was identified and used during the training process. The proposed model was trained on MGH dataset and evaluated using both within dataset and cross-dataset. For MGH dataset, overall accuracy of 81.05 %, MF1 of 79.05 %, Kappa of 0.7408, and per-class F1-scores: W (84.98 %), N1 (58.06 %), N2 (84.82 %), N3 (79.20 %), and REM (88.17 %) can be achieved. Performances on cross-datasets are as follows: SHHS1 200 records reached 79.54 %, 70.56 %, and 0.7078; SHHS2 200 records achieved 76.77 %, 66.30 %, and 0.6632; Sleep-EDF 153 records gained 78.52 %, 72.13 %, and 0.7031; and BCI-MU (local dataset) 94 records achieved 83.57 %, 82.17 %, and 0.7769 for overall accuracy, MF1, and Kappa respectively. Additionally, the proposed model has approximately 9.3 M trainable parameters and takes around 26 s to process one PSG record. The results indicate that the proposed model demonstrates generalizability in sleep stage classification and shows potential as a feasibility tool for real-world applications. Additionally, enzyme-inspired specificity effectively addresses the challenges posed by varying montage of recording, while the identified optimal frequencies mitigate instrument-related issues.

Heterogeneity in mode choice behavior: A spatial latent class approach based on accessibility measures

We propose a method to estimate mode choice models, where preference parameters are sensitive to the spatial context of the trip origin, challenging traditional assumptions of spatial homogeneity in the relationship between travel modes and the built environment. The framework, called Spatial Latent Classes (SLC), is based on the integrated choice and latent class approach, although instead of defining classes for the decision maker, it estimates the probability of a location belonging to a class, as a function of spatial attributes. For each Spatial Latent Class, a different mode choice model is specified, and the resulting behavioral model for each location is a weighted average of all class-specific models, which is estimated to maximize the likelihood of reproducing observed travel behavior. We test our models with data from Portland, Oregon, specifying spatial class membership models as a function of local and regional accessibility measures. Results show the SLC increases model fit when compared with traditional methods and, more importantly, allows segmenting urban space into meaningful zones, where predominant travel behavior patterns can be easily identified. We believe this is a very intuitive way to spatially analyze travel behavior trends, allowing policymakers to identify target areas of the city and the accessibility levels required to attain desired modal splits.

An Empirical Investigation on Vulnerability for Software Companies

This research analyzes software vulnerability information from the perspective of software companies. A total of 13019 vulnerabilities from 136 software companies were collected from a public vulnerability database. A latent class model classifies the companies into three classes based on vulnerability information during a five-year period, and then three class-specific models pinpoint the most significant key features of vulnerabilities for each class. A class I company can reduce vulnerability level if it puts focus on "boundary condition errors," "input validation errors," and "exception handling errors." A class II company needs to emphasize "access validation errors" and "race condition errors." Interestingly, a class III company needs to avoid any potential "origin validation errors," "boundary condition errors," "design errors," and "access validation errors." With these significant key features information, software companies can effectively reduce vulnerability by managing related errors throughout the development and testing process.

Recurrent-based regression of Sentinel time series for continuous vegetation monitoring

Dense time series of optical satellite imagery describing vegetation activity provide essential information for the efficient and regular monitoring of vegetation. Nevertheless, the temporal resolution of optical sensors is strongly affected by cloud cover, resulting in significant missing information. The use of complementary acquisitions, such as Synthetic Aperture Radar (SAR) data, opens the door to the development of new multi-sensor methodologies aiming at the reconstruction of missing information. However, the joint exploitation of new radar and optical missions, such as the Sentinel, raises new challenges given the different nature and response of the two data sources. In this work, the SenRVM methodology is proposed as a new multi-sensor approach to regress SAR time series towards Normalized Difference Vegetation Index (NDVI). A deep Recurrent Neural Network architecture which integrates SAR acquisitions and ancillary data is adopted. The regression task permits a continuous optical temporal resolution of 6 days. Multiple experiments are carried out to assess the SenRVM framework by studying two large-scale areas in France. Through an extensive interpretation of the results, SenRVM is evaluated on three main vegetation types (grasslands, crops, and forests). High accurate results (R2 > 0.83 and MAE < 0.05) over more than 140,000 vegetation polygons are obtained, both for multi-class and class-specific models. The importance of the SenRVM input features is discussed through an ablation study, highlighting that the relevance of the features differs for the various classes. The good performances of SenRVM are assessed by evaluating the quality of the reconstruction over short- and long-term data gaps. Results are compared to five state-of-the-art methodologies. Finally, some preliminary experiments are first carried out to show how SenRVM results could be used to improve the existing cloud & shadow masks. The recovering of time series breaks caused by vegetation cover changes is also illustrated.

Safety performance functions for low-volume rural minor collector two-lane roadways

The Fixing America's Surface Transportation Act (FAST Act) highlights a data-driven method to improve traffic safety on all public paved roads in the U.S. The first edition of the Highway Safety Manual (HSM) is a widely used tool that provides crash predictive models in the form of safety performance functions (SPFs). There are no specific SPFs for low-volume roadways in the HSM. It is important to know that low-volume roadways are the major roadway types in terms of total mileage. This study used 2015–2019 crash data from Texas, incorporating with other relevant geometric and traffic variables, to develop SPFs for a specific low-volume roadway type (rural minor collector two-lane roadways). This study proposed a rules-based SPF developed approach that makes the prediction accuracies higher compared to the full model. The R2 values range from 0.18 to 0.22 for all data (without splitting) for different injury level models. The prediction accuracies are improved in the decision tree-based models. For different class specific models (based on injury levels), the R2 values range from 0.25 to 0.41. Three SPF groups are developed based on crash injury types. The SPFs can provide guidance in refining the prediction accuracies of rural minor collectors.

Classification of platinum nanoparticle catalysts using machine learning

Computer simulations and machine learning provide complementary ways of identifying structure/property relationships that are typically targeting toward predicting the ideal singular structure to maximize the performance in a given application. This can be inconsistent with experimental observations that measure the collective properties of entire samples of structures that contain distributions or mixture of structures, even when synthesized and processed with care. Metallic nanoparticle catalysts are an important example. In this study, we have used a multi-stage machine learning workflow to identify the correct structure/property relationships of Pt nanoparticles relevant to oxygen reduction, hydrogen oxidation, and hydrogen evolution reactions. By including classification prior to regression, we identified two distinct classes of nanoparticles and subsequently generated the class-specific models based on experimentally relevant criteria that are consistent with observations. These multi-structure/multi-property relationships, predicting properties averaged over a large sample of structures, provide a more accessible way to transfer data-driven predictions into the lab.

CLASENTI

Arabic text sentiment analysis suffers from low accuracy due to Arabic-specific challenges (e.g., limited resources, morphological complexity, and dialects) and general linguistic issues (e.g., fuzziness, implicit sentiment, sarcasm, and spam). The limited resources problem requires efforts to build new and improved Arabic corpora and lexica. We propose a class-specific sentiment analysis (CLASENTI) framework. The framework includes a new annotation approach to build multi-faceted Arabic corpus and lexicon allowing for simultaneous annotation of different facets, including domains, dialects, linguistic issues, and polarity strengths. Each of these facets has multiple classes (e.g., the nine classes representing dialects found in the Arab world). The new corpus and lexicon annotations facilitate the development of new class-specific classification models and polarity strength calculation. For the new sentiment classification models, we propose a hybrid model combining corpus-based and lexicon-based models. The corpus-based model has two interrelated phases to build; (1) full-corpus classification models for all facets; and (2) class-specific models trained on filtered subsets of the corpus according to the performances of the full-corpus models. To calculate polarity strengths, the lexicon-based model filters the annotated lexicon based on the specific classes of the domain and dialect. As a case study, we collect and annotate 15274 reviews from various sources, including surveys, Facebook comments, and Twitter posts, pertaining to governmental services. In addition, we develop a new web-based application to apply the proposed framework on the case study. CLASENTI framework reaches up to 95% accuracy and 93% F1-Score surpassing the best-known sentiment classifiers implemented in Scikit-learn library that achieve 82% accuracy and 81% F1-Score for Arabic when tested on the same dataset.

Dual-extrusion 3D printing of anatomical models for education.

Two material 3D printing is becoming increasingly popular, inexpensive and accessible. In this paper, freely available printable files and dual extrusion fused deposition modelling were combined to create a number of functional anatomical models. To represent muscle and bone FilaFlex3D flexible filament and polylactic acid (PLA) filament were extruded respectively via a single 0.4 mm nozzle using a Big Builder printer. For each filament, cubes (5 mm3 ) were printed and analyzed for X, Y, and Z accuracy. The PLA printed cubes resulted in errors averaging just 1.2% across all directions but for FilaFlex3D printed cubes the errors were statistically significantly greater (average of 3.2%). As an exemplar, a focus was placed on the muscles, bones and cartilage of upper airway and neck. The resulting single prints combined flexible and hard structures. A single print model of the vocal cords was constructed which permitted movement of the arytenoids on the cricoid cartilage and served to illustrate the action of intrinsic laryngeal muscles. As University libraries become increasingly engaged in offering inexpensive 3D printing services it may soon become common place for both student and educator to access websites, download free models or 3D body parts and only pay the costs of print consumables. Novel models can be manufactured as dissectible, functional multi-layered units and offer rich possibilities for sectional and/or reduced anatomy. This approach can liberate the anatomist from constraints of inflexible hard models or plastinated specimens and engage in the design of class specific models of the future. Anat Sci Educ 11: 65-72. © 2017 American Association of Anatomists.

LipidFrag: Improving reliability of in silico fragmentation of lipids and application to the Caenorhabditis elegans lipidome.

Lipid identification is a major bottleneck in high-throughput lipidomics studies. However, tools for the analysis of lipid tandem MS spectra are rather limited. While the comparison against spectra in reference libraries is one of the preferred methods, these libraries are far from being complete. In order to improve identification rates, the in silico fragmentation tool MetFrag was combined with Lipid Maps and lipid-class specific classifiers which calculate probabilities for lipid class assignments. The resulting LipidFrag workflow was trained and evaluated on different commercially available lipid standard materials, measured with data dependent UPLC-Q-ToF-MS/MS acquisition. The automatic analysis was compared against manual MS/MS spectra interpretation. With the lipid class specific models, identification of the true positives was improved especially for cases where candidate lipids from different lipid classes had similar MetFrag scores by removing up to 56% of false positive results. This LipidFrag approach was then applied to MS/MS spectra of lipid extracts of the nematode Caenorhabditis elegans. Fragments explained by LipidFrag match known fragmentation pathways, e.g., neutral losses of lipid headgroups and fatty acid side chain fragments. Based on prediction models trained on standard lipid materials, high probabilities for correct annotations were achieved, which makes LipidFrag a good choice for automated lipid data analysis and reliability testing of lipid identifications.

Blind Domain Adaptation With Augmented Extreme Learning Machine Features.

In practical applications, the test data often have different distribution from the training data leading to suboptimal visual classification performance. Domain adaptation (DA) addresses this problem by designing classifiers that are robust to mismatched distributions. Existing DA algorithms use the unlabeled test data from target domain during training time in addition to the source domain data. However, target domain data may not always be available for training. We propose a blind DA algorithm that does not require target domain samples for training. For this purpose, we learn a global nonlinear extreme learning machine (ELM) model from the source domain data in an unsupervised fashion. The global ELM model is then used to initialize and learn class specific ELM models from the source domain data. During testing, the target domain features are augmented with the reconstructed features from the global ELM model. The resulting enriched features are then classified using the class specific ELM models based on minimum reconstruction error. Extensive experiments on 16 standard datasets show that despite blind learning, our method outperforms six existing state-of-the-art methods in cross domain visual recognition.

Estimating class-specific parametric models using finite mixtures: an application to a hedonic model of wine prices

Hedonic price models are commonly used in the study of markets for various goods, most notably those for wine, art, and jewelry. These models were developed to estimate implicit prices of product attributes within a given product class, where in the case of some goods, such as wine, substantial product differentiation exists. To address this issue, recent research on wine prices employs local polynomial regression clustering (LPRC) for estimating regression models under class uncertainty. This study demonstrates that a superior empirical approach – estimation of a mixture model – is applicable to a hedonic model of wine prices, provided only that the dependent variable in the model is rescaled. The present study also catalogues several of the advantages over LPRC modeling of estimating mixture models.

Deep Reconstruction Models for Image Set Classification

Image set classification finds its applications in a number of real-life scenarios such as classification from surveillance videos, multi-view camera networks and personal albums. Compared with single image based classification, it offers more promises and has therefore attracted significant research attention in recent years. Unlike many existing methods which assume images of a set to lie on a certain geometric surface, this paper introduces a deep learning framework which makes no such prior assumptions and can automatically discover the underlying geometric structure. Specifically, a Template Deep Reconstruction Model (TDRM) is defined whose parameters are initialized by performing unsupervised pre-training in a layer-wise fashion using Gaussian Restricted Boltzmann Machines (GRBMs). The initialized TDRM is then separately trained for images of each class and class-specific DRMs are learnt. Based on the minimum reconstruction errors from the learnt class-specific models, three different voting strategies are devised for classification. Extensive experiments are performed to demonstrate the efficacy of the proposed framework for the tasks of face and object recognition from image sets. Experimental results show that the proposed method consistently outperforms the existing state of the art methods.

Prediction of Henry’s Law Constants via group-specific quantitative structure property relationships

Henry’s Law Constants (HLCs) for several hundred organic compounds in water at 25°C were predicted by Quantitative Structure Property Relationship (QSPR) models, with the division of organic compounds into specific classes to yield more accurate models than generalised ones. Both multiple linear regression (MLR) and artificial neural network (ANN) versions of models were produced for three general cases, encompassing the entire data set; one used the six best descriptors, as determined by maximising the correlation coefficient; another used the twelve best descriptors in a similar manner, whilst the third used the same twelve descriptors as English and Carroll (2001). These achieved, respectively, root-mean square errors (RMSEs) of 0.719, 0.52 and 0.607 log(Hcc) units for the MLR version and 0.601, 0.394 and 0.431 for the test set of the ANN models, where Hcc is the ratio of the compound’s concentration in the vapour phase to that in the liquid phase. These were compared with models for six specific chemical classes: (i) alkanes, (ii) cyclic alkanes, (iii) alkenes, (iv) halogenated compounds, (v) aldehydes, ketones and esters grouped together, and (vi) monoaromatics. These group-specific models had RMSEs of 0.153, 0.141. 0.097, 0.168, 0.122 and 0.104 respectively for the MLR versions and 0.684, 0.719, 0.856, 0.784, 0.875 and 0.861 for the test set of the ANN models. It was found that the class-specific models achieved lower RMSEs than the general models, when using MLR models. The use of ANN was found to improve the predictive accuracy of the general models but failed to improve that for the class-specific models vis-à-vis MLR.

Gas chromatographic retention index as a basis for predicting evaporation rates of complex mixtures

Models that predict the fate of petroleum fuels in the environment are often required for effective remediation of fuel-contaminated sites. In this research, an environmental fuel spill was simulated by means of a diesel/water microcosm, in which the temporal changes in composition were assessed during evaporation by gas chromatography–mass spectrometry (GC–MS). First-order kinetic rate constants were calculated for 51 selected compounds and utilized to develop predictive models for evaporation rate constants, using GC retention indices on a nonpolar stationary phase. Models were initially developed to predict rates of evaporation of compounds from individual classes (normal alkane, branched alkane, alkyl benzene, and polycyclic hydrocarbon) and then expanded to include all compounds (comprehensive model). Using the comprehensive model, the rate constants were predicted with a mean absolute percent error (MAPE) of 10%, whereas the class-specific models resulted in less error (4–8%). These models were employed to predict the fraction remaining of the total fuel (6% error) as well as the fraction remaining of individual compounds (13% MAPE). Accurate models such as these will facilitate remediation of environmental releases of petroleum products.

A linear solvation energy relationship model of organic chemical partitioning to particulate organic carbon in soils and sediments

Predicting the association of contaminants with particulate organic matter in the environment is critical in determining the fate and bioavailability of chemicals. A ubiquitous measure of contaminant association with soil and sediment particulate organic matter is the organic carbon partition coefficient K(OC) . Chemical class-specific models relating the K(OC) to the octanol-water partition coefficient K(OW) have been used to predict the partitioning to organic carbon in the water column and sediment for nonpolar hydrophobic pollutants and some polar pollutants. A single linear solvation energy relationship (LSER) is proposed as a simpler and chemically based alternative for predicting K(OC) for a more diverse set of compounds. A chemically diverse set of K(OC) data is used to obtain a more robust and more universally representative model of organic carbon partitioning than previously available LSER models. The resulting model has a root mean square error (RMSE) of prediction for log K(OC) of RMSE = 0.48 for the fitted data set and RMSE = 0.55 for an independent data set. An analysis of LSER coefficients highlights the relative importance of hydrogen bonding interactions.

Multimedia environmental chemical partitioning from molecular information

The prospect of assessing the environmental distribution of chemicals directly from their molecular information was analyzed. Multimedia chemical partitioning of 455 chemicals, expressed in dimensionless compartmental mass ratios, was predicted by SimpleBox 3, a Level III Fugacity model, together with the propagation of reported uncertainty for key physicochemical and transport properties, and degradation rates. Chemicals, some registered in priority lists, were selected according to the availability of experimental property data to minimize the influence of predicted information in model development. Chemicals were emitted in air or water in a fixed geographical scenario representing the Netherlands and characterized by five compartments (air, water, sediments, soil and vegetation). Quantitative structure–fate relationship (QSFR) models to predict mass ratios in different compartments were developed with support vector regression algorithms. A set of molecular descriptors, including the molecular weight and 38 counts of molecular constituents were adopted to characterize the chemical space. Out of the 455 chemicals, 375 were used for training and testing the QSFR models, while 80 were excluded from model development and were used as an external validation set. Training and test chemicals were selected and the domain of applicability (DOA) of the QSFRs established by means of self-organizing maps according to structural similarity. Best results were obtained with QSFR models developed for chemicals belonging to either the class [C] and [C; O], or the class with at least one heteroatom different than oxygen in the structure. These two class-specific models, with respectively 146 and 229 chemicals, showed a predictive squared coefficient of q 2 ≥ 0.90 both for air and water, which respectively dropped to q 2 ≈ 0.70 and 0.40 for outlying chemicals. Prediction errors were of the same order of magnitude as the deviations associated to the uncertainty of the physicochemical and transport properties, and degradation rates.

Classification of sport videos using edge-based features and autoassociative neural network models

In this paper, we propose a method for classification of sport videos using edge-based features, namely edge direction histogram and edge intensity histogram. We demonstrate that these features provide discriminative information useful for classification of sport videos, by considering five sports categories, namely, cricket, football, tennis, basketball and volleyball. The ability of autoassociative neural network (AANN) models to capture the distribution of feature vectors is exploited, to develop class-specific models using edge-based features. We show that combining evidence from complementary edge features results in improved classification performance. Also, combination of evidence from different classifiers like AANN, hidden Markov model (HMM) and support vector machine (SVM) helps improve the classification performance. Finally, the performance of the classification system is examined for test videos which do not belong to any of the above five categories. A low rate of misclassification error for these test videos validates the effectiveness of edge-based features and AANN models for video classification.

Efficient class-specific models for autoregressive processes with slowly varying amplitude in white noise

This paper describes an efficient model to describe an autoregressive (AR) signal with slowly-varying amplitude in additive white Gaussian noise (WGN). Even a simple low-order AR model becomes complicated by varying amplitude and additive white noise. However, by approximating the signal amplitude as piecewise-constant, an efficient filtering approach can be applied in order to compute the maximum likelihood (ML) estimate for the entire data record. The model is efficient both in terms of having a compact set of parameters and in the computational sense. Simulation results are provided. The algorithm has applications in signal modeling for underwater acoustic signals, particularly active wideband signals such as explosive sources.

Growing a multi-class classifier with a reject option

In many classification problems objects should be rejected when the confidence in their classification is too low. An example is a face recognition problem where the faces of a selected group of people have to be classified, but where all other faces and non-faces should be rejected. These problems are typically solved by estimating the class densities and assigning an object to the class with the highest posterior probability. The total probability density is thresholded to detect the outliers. Unfortunately, this procedure does not easily allow for class-dependent thresholds, or for class models that are not based on probability densities but on distances. In this paper we propose a new heuristic to combine any type of one-class models for solving the multi-class classification problem with outlier rejection. It normalizes the average model output per class, instead of the more common non-linear transformation of the distances. It creates the possibility to adjust the rejection threshold per class, and also to combine class models that are not (all) based on probability densities and to add class models without affecting the boundaries of existing models. Experiments show that for several classification problems using class-specific models significantly improves the performance.

General and class specific models for prediction of soil sorption using various physicochemical descriptors.

Diverse chemical descriptors were explored for use in QSAR models aimed to screen the soil sorption potential of organic compounds. The descriptors included logP, HyperChem QSARProperties descriptors, a combination of connectivity indices, geometrical, and quantum chemical measures, and two sets from the DRAGON and CODESSA program packages, respectively. Generally, the univariate logP models were capable of capturing most of the variation and give an indication of the sorption potential. The multivariate models required refined variable selection procedures but were shown to include crucial descriptors for modeling compound classes with specific chemical characteristics.