Prototype-based Model Research Articles

Prototype-based Models for Real Estate Valuation: A Machine Learning Model That Explains Prices

Prototype-based Models for Real Estate Valuation: A Machine Learning Model That Explains Prices

CPCL: Conceptual prototypical contrastive learning for Few-Shot text classification

As prototype-based Few-Shot Learning methods, Prototypical Network generates prototypes for each class in a low-resource state and classify by a metric module. Therefore, the quality of prototypes matters but they are inaccurate from the few support instances, and the domain-specific information of training data are harmful to the generalizability of prototypes. We propose a Conceptual Prototype (CP), which contains both rich instance and concept features. The numerous query data can inspire the few support instances. An interactive network is designed to leverage the interrelation between support set and query-detached set to acquire a rich Instance Prototype which is typical on the whole data. Besides, class labels are introduced to prototype by prompt engineering, which makes it more conceptual. The label-only concept makes prototype immune to domain-specific information in training phase to improve its generalizability. Based on CP, Conceptual Prototypical Contrastive Learning (CPCL) is proposed where PCL brings instances closer to its corresponding prototype and pushes away from other prototypes. “2-way 5-shot” experiments show that CPCL achieves 92.41% accuracy on ARSC dataset, 2.30% higher than other prototype-based models. Meanwhile, the 0-shot performance of CPCL is comparable to Induction Network in the 5-shot way, indicating that our model is adequate for 0-shot tasks.

Multiobjective Evolutionary Optimization for Prototype-Based Fuzzy Classifiers

Evolving intelligent systems (EISs), particularly, the zero-order ones have demonstrated strong performance on many real-world problems concerning data stream classification, while offering high model transparency and interpretability thanks to their prototype-based nature. Zero-order EISs typically learn prototypes by clustering streaming data online in a “one pass” manner for greater computation efficiency. However, such identified prototypes often lack optimality, resulting in less precise classification boundaries, thereby hindering the potential classification performance of the systems. To address this issue, a commonly adopted strategy is to minimise the training error of the models on historical training data or alternatively, to iteratively minimise the intra-cluster variance of the clusters obtained via online data partitioning. This recognises the fact that the ultimate classification performance of zero-order EISs is driven by the positions of prototypes in the data space. Yet, simply minimising the training error may potentially lead to overfitting, whilst minimising the intra-cluster variance does not necessarily ensure the optimised prototype-based models to attain improved classification outcomes. To achieve better classification performance whilst avoiding overfitting for zero-order EISs, this paper presents a novel multi-objective optimisation approach, enabling EISs to obtain optimal prototypes via involving these two disparate but complementary strategies simultaneously. Five decision-making schemes are introduced for selecting a suitable solution to deploy from the final non-dominated set of the resulting optimised models. Systematic experimental studies are carried out to demonstrate the effectiveness of the proposed optimisation approach in improving the classification performance of zero-order EISs.

Wind turbine blade icing detection: a federated learning approach

Wind farms are often located at high latitudes, which entails a high risk of icing for wind turbine blades. Traditional anti-icing methods rely primarily on manual observation, the use of special materials, or external sensors/tools, but these methods are limited by human experience, additional costs, and understanding of the mechanical mechanism. Model-based approaches rely heavily on prior knowledge and are subject to misinterpretation. Data-driven approaches can deliver promising solutions but require large datasets for training, which might face significant challenges with respect to data management, e.g., privacy protection and ownership. To address these issues, this paper proposes a federated learning (FL) based model for blade icing detection. The proposed approach first creates a prototype-based model for each client and then aggregates all client models into a globally weighted model. The clients use a prototype-based modeling method to address the data imbalance problem, while using the FL-based learning method to ensure data security and safety. The proposed model is comprehensively evaluated using data from two wind farms, with 70 wind turbines. The results validate the effectiveness of the proposed prototype-based client model for feature extraction, and the superiority over the five baselines in terms of icing detection accuracy. In addition, the experiment demonstrates the promising result of online blade icing detection, with almost 100% accuracy.

Probabilistic pragmatics explains gradience and focality in natural language quantification

An influential view in philosophy and linguistics equates the meaning of a sentence to the conditions under which it is true. But it has been argued that this truth-conditional view is too rigid and that meaning is inherently gradient and revolves around prototypes. Neither of these abstract semantic theories makes direct predictions about quantitative aspects of language use. Hence, we compare these semantic theories empirically by applying probabilistic pragmatic models as a link function connecting linguistic meaning and language use. We consider the use of quantity words (e.g., "some," "all"), which are fundamental to human language and thought. Data from a large-scale production study suggest that quantity words are understood via prototypes. We formulate and compare computational models based on the two views on linguistic meaning. These models also take into account cognitive factors, such as salience and numerosity representation. Statistical and empirical model comparison show that the truth-conditional model explains the production data just as well as the prototype-based model, when the semantics are complemented by a pragmatic module that encodes probabilistic reasoning about the listener's uptake.

Quantum-inspired learning vector quantizers for prototype-based classification

Prototype-based models like the Generalized Learning Vector Quantization (GLVQ) belong to the class of interpretable classifiers. Moreover, quantum-inspired methods get more and more into focus in machine learning due to its potential efficient computing. Further, its interesting mathematical perspectives offer new ideas for alternative learning scenarios. This paper proposes a quantum computing-inspired variant of the prototype-based GLVQ for classification learning. We start considering kernelized GLVQ with real- and complex-valued kernels and their respective feature mapping. Thereafter, we explain how quantum space ideas could be integrated into a GLVQ using quantum bit vector space in the quantum state space {mathcal {H}}^{n} and show the relations to kernelized GLVQ. In particular, we explain the related feature mapping of data into the quantum state space {mathcal {H}}^{n}. A key feature for this approach is that {mathcal {H}}^{n} is an Hilbert space with particular inner product properties, which finally restrict the prototype adaptations to be unitary transformations. The resulting approach is denoted as Qu-GLVQ. We provide the mathematical framework and give exemplary numerical results.

Prototype-based classification and error analysis under bootstrapping strategy

A prototype-based classification is proposed to select handfuls of class data for learning rules and prediction. A class point is considered as a prototype if it forms a hypersphere that represents a part of class area measured by any distance metric and class label. The prototype selection algorithm, formulated by a set covering optimisation, selects the number of within-class points that is as small as possible, while preserving class covering regions for the unknown data distribution. The upper bound of the error is analysed to compare the effectiveness of the prototype-based classification with the Bayes classifier. Under a bootstrapping strategy and the 0/1 loss, the bias and variance components are driven from a generalisation error without assuming the unknown distribution of a given problem. This analysis provides a way to evaluate prototype-based models and select the optimal model estimate for any standard classifier. The experiments show that the proposed approach is very competitive when compared to the nearest neighbour and the Bayes classifier and efficient in choosing prototypes in terms of class covering regions, data size and computation time.

Prototype-based classification and error analysis under bootstrapping strategy

A prototype-based classification is proposed to select handfuls of class data for learning rules and prediction. A class point is considered as a prototype if it forms a hypersphere that represents a part of class area measured by any distance metric and class label. The prototype selection algorithm, formulated by a set covering optimisation, selects the number of within-class points that is as small as possible, while preserving class covering regions for the unknown data distribution. The upper bound of the error is analysed to compare the effectiveness of the prototype-based classification with the Bayes classifier. Under a bootstrapping strategy and the 0/1 loss, the bias and variance components are driven from a generalisation error without assuming the unknown distribution of a given problem. This analysis provides a way to evaluate prototype-based models and select the optimal model estimate for any standard classifier. The experiments show that the proposed approach is very competitive when compared to the nearest neighbour and the Bayes classifier and efficient in choosing prototypes in terms of class covering regions, data size and computation time.

Retrofits for Energy Efficient Office Buildings: Integration of Optimized Photovoltaics in the Form of Responsive Shading Devices

This study presents a retrofit strategy: integrating optimized photovoltaics (PV) in the form of responsive shading devices using a dual-axis solar tracking system. A prototype-based model was fabricated to compare the efficiency of PV in this implementation with the conventional fixed installation. The office building, T1 Empire World in Erbil, was selected as a retrofit case study and for the application of the proposed integration system. In order to assess the effectiveness of the proposed retrofit method, the energy performance of the base case is simulated to be compared later with the energy performance simulations after the integration technique. The amount of generated electricity from the PV surfaces of the integrated shading elements is calculated. The energy simulations were performed using OpenStudio® (NREL, Washington, DC, USA), EnergyPlusTM (NREL, Washington, DC, USA), and Grasshopper/ Ladybug tools in which the essential results were recorded for the baseline reference, as well as the energy performance of the retrofitted building. The results emphasize that the PV-integrated responsive shading devices can maximize the efficiency of PV cells by 36.8% in comparison to the fixed installation. The integrated system can provide approximately 15.39% of the electricity demand for operating the building. This retrofit method has reduced the total site energy consumption by 33.2% compared to the existing building performance. Total electricity end-use of the various utilities was lowered by 33.5%, and the total natural gas end-use of heating demand was reduced by 30.9%. Therefore, the percentage reduction in electricity cooling demand in July and August is 42.7% due to minimizing the heat gain in summer through blocking the sun’s harsh rays from penetrating into interior spaces of the building. In general, this system has multiple benefits, starting with being extremely efficient and viable in generating sustainable alternative energy—which is the global growing concern of today’s sustainable development—providing thermal comfort for occupants, and granting a dynamic appearance to the building when the PV-integrated elements rotate according to the sun’s position in the sky.

Conceptual spaces in object-oriented framework

The aim of this paper is to show that the middle level of mental representations in a conceptual spaces framework is consistent with the OOP paradigm. We argue that conceptual spaces framework together with vague prototype theory of categorization appears to be the most suitable solution for modeling the cognitive apparatus of humans, and that the OOP paradigm can be easily and intuitively reconciled with this framework. First, we show that the prototype-based OOP approach is consistent with Gärdenfors’ model in terms of structural coherence. Second, we argue that the product of cloning process in a prototype-based model is in line with the structure of categories in Gärdenfors’ proposal. Finally, in order to make the fuzzy object-oriented model consistent with conceptual space, we demonstrate how to define membership function in a more cognitive manner, i.e. in terms of similarity to prototype.

Impaired generalization of speaker identity in the perception of familiar and unfamiliar voices.

In 2 behavioral experiments, we explored how the extraction of identity-related information from familiar and unfamiliar voices is affected by naturally occurring vocal flexibility and variability, introduced by different types of vocalizations and levels of volitional control during production. In a first experiment, participants performed a speaker discrimination task on vowels, volitional (acted) laughter, and spontaneous (authentic) laughter from 5 unfamiliar speakers. We found that performance was significantly impaired for spontaneous laughter, a vocalization produced under reduced volitional control. We additionally found that the detection of identity-related information fails to generalize across different types of nonverbal vocalizations (e.g., laughter vs. vowels) and across mismatches in volitional control within vocalization pairs (e.g., volitional laughter vs. spontaneous laughter), with performance levels indicating an inability to discriminate between speakers. In a second experiment, we explored whether personal familiarity with the speakers would afford greater accuracy and better generalization of identity perception. Using new stimuli, we largely replicated our previous findings: whereas familiarity afforded a consistent performance advantage for speaker discriminations, the experimental manipulations impaired performance to similar extents for familiar and unfamiliar listener groups. We discuss our findings with reference to prototype-based models of voice processing and suggest potential underlying mechanisms and representations of familiar and unfamiliar voice perception. (PsycINFO Database Record

Prototype-based Models for the Supervised Learning of Classification Schemes

AbstractAn introduction is given to the use of prototype-based models in supervised machine learning. The main concept of the framework is to represent previously observed data in terms of so-called prototypes, which reflect typical properties of the data. Together with a suitable, discriminative distance or dissimilarity measure, prototypes can be used for the classification of complex, possibly high-dimensional data. We illustrate the framework in terms of the popular Learning Vector Quantization (LVQ). Most frequently, standard Euclidean distance is employed as a distance measure. We discuss how LVQ can be equipped with more general dissimilarites. Moreover, we introduce relevance learning as a tool for the data-driven optimization of parameterized distances.

Prototype-based models in machine learning.

An overview is given of prototype-based models in machine learning. In this framework, observations, i.e., data, are stored in terms of typical representatives. Together with a suitable measure of similarity, the systems can be employed in the context of unsupervised and supervised analysis of potentially high-dimensional, complex datasets. We discuss basic schemes of competitive vector quantization as well as the so-called neural gas approach and Kohonen's topology-preserving self-organizing map. Supervised learning in prototype systems is exemplified in terms of learning vector quantization. Most frequently, the familiar Euclidean distance serves as a dissimilarity measure. We present extensions of the framework to nonstandard measures and give an introduction to the use of adaptive distances in relevance learning.

Efficient rejection strategies for prototype-based classification

Due to intuitive training algorithms and model representation, prototype-based models are popular in settings where on-line learning and model interpretability play a major role. In such cases, a crucial property of a classifier is not only which class to predict, but also if a reliable decision is possible in the first place, or whether it is better to reject a decision. While strong theoretical results for optimum reject options in the case of known probability distributions or estimations thereof are available, there do not exist well-accepted reject strategies for deterministic prototype-based classifiers. In this contribution, we present simple and efficient reject options for prototype-based classification, and we evaluate their performance on artificial and benchmark data sets using the example of learning vector quantization. We demonstrate that the proposed reject options improve the accuracy in most cases, and their performance is comparable to an optimal reject option of the Bayes classifier in cases where the latter is available. Further, we show that the results are comparable to a well established reject option for support vector machines in cases where learning vector quantization classifiers are suitable for the given classification task, even providing better results in some cases.

Prototype-Based Modeling for <newline/>Facial Expression Analysis

Automatic facial expression analysis systems are aiming towards the application of computer vision techniques in human computer interaction, emotion analysis, and even medical care via a space mapping between the continuous emotion and a set of discrete expression categories. The main difficulty with these systems is the inherent problem of facial alignment due to person-specific appearance. Beside the facial representation problem, the same displayed facial expression may vary differently across humans; this can be true even for the same person in different contexts. To cope with these variable factors, we introduce the concept of prototype-based model as anchor modeling through a SIFT-flow registration. A set of prototype facial expression models is generated as a reference space of emotions on which face images are projected to generate a set of registered faces. To characterize the facial expression appearance, oriented gradients are processed on each registered image. We obtained the best results 87% with the person–independent evaluation strategy on JAFFE dataset (7-class expression recognition problem), and 83% on the complex setting of the GEMEP-FERA database (5-class problem).

Learning interpretable kernelized prototype-based models

Since they represent a model in terms of few typical representatives, prototype based learning such as learning vector quantization (LVQ) constitutes a directly interpretable machine learning technique. Recently, several LVQ schemes have been extended towards a kernelized or dissimilarity based version which can be applied if data are represented by pairwise similarities or dissimilarities only. This opens the way towards its application in domains where data are typically not represented in vectorial form. Albeit kernel LVQ still represents models by typical prototypes, interpretability is usually lost this way: since no vector space model is available, prototypes are represented indirectly in terms of combinations of data. In this contribution, we extend a recent kernel LVQ scheme by sparse approximations to overcome this problem: instead of the full coefficient vectors, few exemplars which represent the prototypes can be directly inspected by practitioners in the same way as data in this case. For this purpose, we investigate different possibilities to approximate a prototype by a sparse counterpart during or after training relying on different heuristics or approximation algorithms, respectively, in particular sparsity constraints while training, geometric approaches, orthogonal matching pursuit, and core techniques for the minimum enclosing ball problem. We discuss the behavior of these methods in several benchmark problems as concerns quality, sparsity, and interpretability, and we propose different measures how to quantitatively evaluate the performance of the approaches.

A hybrid class- and prototype-based object model to support language-neutral structural intercession

ContextDynamic languages have turned out to be suitable for developing specific applications where runtime adaptability is an important issue. Although .Net and Java platforms have gradually incorporated features to improve their support of dynamic languages, they do not provide intercession for every object or class. This limitation is mainly caused by the rigid class-based object model these platforms implement, in contrast to the flexible prototype-based model used by most dynamic languages. ObjectiveOur approach is to provide intercession for any object or class by defining a hybrid class- and prototype-based object model that efficiently incorporates structural intercession into the object model implemented by the widespread .Net and Java platforms. MethodIn a previous work, we developed and evaluated an extension of a shared-source implementation of the .Net platform. In this work, we define the formal semantics of the proposed reflective model, and modify the existing implementation to include the hybrid model. Finally, we assess its runtime performance and memory consumption, comparing it to existing approaches. ResultsOur platform shows a competitive runtime performance compared to 9 widespread systems. On average, it performs 73% and 61% better than the second fastest system for short- and long-running applications, respectively. Besides, it is the JIT-compiler approach that consumes less average memory. The proposed approach of including a hybrid object-model into the virtual machine involves a 444% performance improvement (and 65% less memory consumption) compared to the existing alternative of creating an extra software layer (the DLR). When none of the new features are used, our platform requires 12% more execution time and 13% more memory than the original .Net implementation. ConclusionOur proposed hybrid class- and prototype-based object model supports structural intercession for any object or class. It can be included in existing JIT-compiler class-based platforms to support common dynamic languages, providing competitive runtime performance and low memory consumption.

Six-year-old Children's Ability on Category Learning: Category Representation, Attention and Learning Strategy

This paper explores 6-year-old children's category representation and learning strategies. Category learning is a fundamental ability through which human beings acquire and organize new knowledge about the world (Ashby, 2005), and is critical for normal cognitive development. There are three major theories or models of how categories are represented: Rule-based, Prototype-based, and Exemplar-based models. Rule-based models assume that category learning is a process of discovering an explicit rule to maximize accuracy (Ashby, 2005; Seger, 2006). Prototype-based models assume that stimuli are categorized on the basis of their similarity to category prototypes stored in memory (Rosch Mervis, 1975; Smith, Chapman, Redford, 2010; Coutinho, Redford, Smith, 2010). A category prototype is generally defined as the average, or most typical, member of a category. Exemplar-based models assume that the categorization of a new exemplar is based on the similarity of the new exemplar to the representations of all previously encountered exemplars stored in memory (Medin Schaffer, 1978; Kruschke, 1992; Nosofsky, 1992). Previous studies suggest that 6-year-old children have developed some ability to use category knowledge to solve problems (Wilburn Feeney, 2008; Sloutsky Lo, 1999; Sloutsky Fisher, 2001). Furthermore, several critical aspects of category learning are acquired at this age. Fang, Fang, Xi (1991) pointed out that 6-year-old is a critical period for children to learn to understand the relation between the whole and the part of a subject. Yin (1996) further suggested that 6 years is an important age to learn superordinate categories (for example, the category "furniture"). Two category structures were used in this study. Experiment 1 used the "5/4 category structure" from Medin and Schaffer (1978) and experiment 2 used the "3/3 category structure" from Yamauchi, Love, Markman (2002). The category structures were adapted in order to be able to identify which kinds of representation the children were forming: rule, exemplar or prototype. 62 6-year-old children took part in the experiments. During each trial, an individual exemplar was presented, the participant was asked to infer and indicate which category (A or B) the exemplar belonged to, and feedback as to whether the subject was right or wrong was provided. After a number of such trials of inference and feedback, participants reached the learning criterion and were considered to have formed new category knowledge. A mathematical technique of "Model Fitting" was introduced to analyze the data from two experiments. Different models were used to examine whether childrens’ responses were best fit by exemplar or prototype models, to identify which features the children paid attention to, and to identify which classification strategy children used. Experiment 1 showed that 6-year-old children were able to learn the 5/4 categorization task and reach criterion. Model fitting analyses of category representation found that these children tended to form exemplar representations rather than prototype representations. On measures of distribution of attention, 6-year-old children could identify and pay more attention to the more typical dimensions. Finally, when learning strategy was examined, 6-year-old children used either a single-dimension or rule-plus-exception strategy to classify the items. Experiment 2 using the 3/3 task found similar results to experiment 1, but further found that 6-year-old children could not integrate their processing of the most important distinctive dimensions across the two categories. This result was consistent with the findings of Inhelder Piaget (1958), and Anaki Bentin (2009) that 6-year-old children could not process information across different categories.

The Expectation Effect of the Sample Size in Category Learning

This paper explores the effects of category size expectations on category learning. The expectation effect is the finding that category learning is improved when subjects are told how many items or exemplars are in each category in advance. There are three major theories or models of how categories are represented: Rule-based, Prototype-based, and Exemplar-based. Rule-based models assume that category learning is a process of discovering an explicit, verbalizable rule that maximizes categorization accuracy (Ashby, 2005; Seger Cincotta, 2006). Prototype-based models assume that stimuli are categorized on the basis of their similarity to category prototypes stored in memory (Rosch Mervis, 1975; Smith, Chapman, Redford, 2010; Coutinho, Redford, Smith, 2010). A category prototype is generally defined as the average, or most typical, member of a category. Exemplar-based models assume that the categorization of a new exemplar is based on the similarity of the new exemplar to the representations of all previously encountered exemplars stored in memory (Medin Schaffer, 1978; Kruschke, 1992; Nosofsky, 1992). According to Rule-based and Prototype-based models, people abstract the rule or prototype as their final representation without regard to the total number of exemplars in each relevant category; therefore, knowing how many exemplars are in each category should not affect learning. However, according to Exemplar-based models categories are represented as all the, specific exemplars that have been previously experienced. This implies that knowledge of category size may improve exemplar based categorization learning. Two learning conditions, Known condition (KC) and Unknown condition (UC) were compared in this experiment. In KC participants were instructed as to how many total exemplars (9) they would see across both categories. In UC participants were given no information about category size. The "5-4 category structure" from Medin and Schaffer (1978) was adapted in order to be able to identify which kinds of representation the participants were forming: rule, exemplar or prototype. 106 undergraduate students took part in the experiment. During each trial, an individual exemplar was presented, the participant was then asked to decide and indicate which category (A or B) the exemplar belonged to, and finally feedback as to whether the response was right or wrong was provided. Training continued until participants reached a learning criterion of three consecutive blocks with a combined accuracy of 90%, or until they completed 40 blocks (360 trials). A mathematical technique of "Model Fitting" was introduced to analyze the data from experiment. Different models were used to examine whether participants’ responses were best fit by exemplar or prototype models, to identify which features the participants paid attention to, and to identify which categorization strategy participants used. The results showed that the expectation effect for category size was significant. Participants who knew the sample size (KC) at the beginning of learning required fewer blocks on average to reach the criterion than the participants who did not know the sample size (UC) in advance; 22 and 27 blocks respectively, t (68)=2.088, p0.05. This result is consistent with the predictions from the Exemplar-based models but not the Rule-based or Prototype-based models. To explore whether the prototype or exemplar model provided the better account of the participant’s representation, we adopted a mathematical method of parameter estimation (Minda Smith, 2002) and fitted two models to each participant’s data: exemplar processing was assessed via a five-parameter version of the General Context Model (GCM), and prototype processing was assessed using the Multiplicative Prototype Model (MPM). We found that the fit of the GCM was quantitatively superior to the MPM model for both learning conditions. We also found that the KC group was more sensitive to the diagnostic dimensions of the category than the UC group. Across the blocks of training, the KC group showed three distinct phases of learning: an early phase in which overall accuracy was consistent with a single-rule strategy, followed by a phase in which accuracy was consistent with a rule-plus-exception strategy, and finally a phase in which accuracy was consistent with an information-integration strategies. This three phase pattern was not present in the UC group.

A Prototype-Based Model of Adult Attachment for Clinicians

Models of adult attachment have proven to be useful for understanding illness behavior, stress responses, susceptibility to disease processes, and psychotherapeutic approaches to difficult patients. Two methods of assessing patterns of attachment, using self-report instruments and using techniques such as the Adult Attachment Interview (AAI), are only weakly related and each has drawbacks for clinical use. We have previously assessed commonalities and differences in the descriptions of attachment patterns that emerge from these schools and synthesized them in empirically based attachment prototypes. In this companion article, we describe a prototype-based model of attachment. This model defines dimensions of attachment anxiety and attachment avoidance as composites of particular aspects of internal working models of self and other, behavior in current close relationships, patterns of expression of affect, and narrative coherence. The model emphasizes the clinical importance of the severity of attachment insecurity, defined as a dimension which incorporates problems in the previously listed domains of attachment as well as deficits in mentalizing, self-agency, and resolution of trauma. The model locates the central tendencies of prototypic ("textbook") descriptions of four patterns of attachment (secure, dismissing, preoccupied, and fearful/disorganized) while avoiding definitions of the boundaries between categories of attachment. We compare the prototype-based model to the two most prominent current models of attachment, the 4-category, 2-dimension model derived from self-report methods of assessment and categories of "attachment states of mind" derived from the Adult Attachment Interview. Finally, we discuss limitations of the prototype-based model and areas requiring further research.