Probabilistic Knowledge Research Articles

Probabilistic Knowledge Transfer for Lightweight Deep Representation Learning.

Knowledge-transfer (KT) methods allow for transferring the knowledge contained in a large deep learning model into a more lightweight and faster model. However, the vast majority of existing KT approaches are designed to handle mainly classification and detection tasks. This limits their performance on other tasks, such as representation/metric learning. To overcome this limitation, a novel probabilistic KT (PKT) method is proposed in this article. PKT is capable of transferring the knowledge into a smaller student model by keeping as much information as possible, as expressed through the teacher model. The ability of the proposed method to use different kernels for estimating the probability distribution of the teacher and student models, along with the different divergence metrics that can be used for transferring the knowledge, allows for easily adapting the proposed method to different applications. PKT outperforms several existing state-of-the-art KT techniques, while it is capable of providing new insights into KT by enabling several novel applications, as it is demonstrated through extensive experiments on several challenging data sets.

The Physics of Possibility: Victorian Fiction, Science, and Gender

The Physics of Possibility: Victorian Fiction, Science, and Gender

O jogo pedagógico “brincando com a probabilidade” para os anos iniciais do ensino fundamental

O presente trabalho tem como objetivo analisar as tarefas elaboradas para um jogo pedagógico relacionadas ao conceito de espaço amostral segundo a Teoria Antropológica do Didático (TAD). Tomando como referência a Base Nacional Comum Curricular – BNCC, proposta curricular brasileira no que diz respeito aos conteúdos e habilidades a serem trabalhados na unidade temática “Estatística e Probabilidade” para os anos iniciais do Ensino Fundamental, elaboramos cartas Perguntas (?) para o jogo, que são tarefas baseadas em situações problemas cujo objetivo é favorecer a apreensão do conhecimento probabilístico. Mostrou-se a possibilidade de desenvolver um trabalho pedagógico baseado em jogos e resolução de problemas que envolva conteúdos probabilísticos, criando um recurso que favoreça o repensar sobre os métodos estratégicos, redimensionando-os a fim de minimizar o hiato existente entre as atividades lúdicas cotidianas realizadas pelos alunos, espontaneamente, e o trabalho desencadeado em sala de aula.

A Method to Learn Embedding of a Probabilistic Medical Knowledge Graph: Algorithm Development.

BackgroundKnowledge graph embedding is an effective semantic representation method for entities and relations in knowledge graphs. Several translation-based algorithms, including TransE, TransH, TransR, TransD, and TranSparse, have been proposed to learn effective embedding vectors from typical knowledge graphs in which the relations between head and tail entities are deterministic. However, in medical knowledge graphs, the relations between head and tail entities are inherently probabilistic. This difference introduces a challenge in embedding medical knowledge graphs.ObjectiveWe aimed to address the challenge of how to learn the probability values of triplets into representation vectors by making enhancements to existing TransX (where X is E, H, R, D, or Sparse) algorithms, including the following: (1) constructing a mapping function between the score value and the probability, and (2) introducing probability-based loss of triplets into the original margin-based loss function.MethodsWe performed the proposed PrTransX algorithm on a medical knowledge graph that we built from large-scale real-world electronic medical records data. We evaluated the embeddings using link prediction task.ResultsCompared with the corresponding TransX algorithms, the proposed PrTransX performed better than the TransX model in all evaluation indicators, achieving a higher proportion of corrected entities ranked in the top 10 and normalized discounted cumulative gain of the top 10 predicted tail entities, and lower mean rank.ConclusionsThe proposed PrTransX successfully incorporated the uncertainty of the knowledge triplets into the embedding vectors.

Error Processing During the Online Retrieval of Probabilistic Sequence Knowledge

Abstract. Adaptive behavior involves rapid error processing and action evaluation. However, it has not been clarified how errors contribute to automatic behaviors that can be retrieved to successfully adapt to our complex environment. Automatic behaviors strongly rely on the process of probabilistic sequence learning and memory. Therefore, the present study investigated error processing during the online retrieval of probabilistic sequence knowledge. Twenty-four healthy young adults acquired and continuously retrieved a repeating stimulus sequence reflected by reaction time (RT) changes on a rapid forced-choice RT task. Performance was compared with a baseline that denoted the processing of random stimuli embedded in the probabilistic sequence. At the neurophysiological level, event-related brain potentials synchronized to responses were measured. Error processing was tracked by the error negativity (Ne) and the error positivity (Pe). The mean amplitude of the Ne gradually decreased as the task progressed, similarly for the sequence retrieval and the embedded baseline process. The mean amplitude of the Pe increased over time, likewise, irrespective of the type of the stimuli. Accordingly, we propose that automatic error detection (Ne) and conscious error evaluation (Pe) are not sensitive to sequence learning and retrieval. Overall, the present study provides insight into how error processing takes place for the retrieval of sequence knowledge in a probabilistic environment.

The Active Sensing of Control Difference.

SummaryIn everyday life, people control objects in the world around them to varying degrees. The processes people actively use to establish their control, while interacting with an environment containing large ambiguity, remain unknown. This study examines how people explore their control over the environment and how they detect small differences in control among objects. In the experimental task, participants moved three dots on a screen and identified one dot over which the level of control is different from that of the other two. The results support a two-step behavior mechanism underlying the sensing of control difference: People first explore their overall control in the environment, and then the results of the initial exploration are used to selectively tune the direction (i.e., either more or less) of the detected control difference, ensuring efficient and rapid detection of the type of control difference that is potentially important for further action selections.

Lifelong generative modeling

Lifelong learning is the problem of learning multiple consecutive tasks in a sequential manner, where knowledge gained from previous tasks is retained and used to aid future learning over the lifetime of the learner. It is essential towards the development of intelligent machines that can adapt to their surroundings. In this work we focus on a lifelong learning approach to unsupervised generative modeling, where we continuously incorporate newly observed distributions into a learned model. We do so through a student-teacher Variational Autoencoder architecture which allows us to learn and preserve all the distributions seen so far, without the need to retain the past data nor the past models. Through the introduction of a novel cross-model regularizer, inspired by a Bayesian update rule, the student model leverages the information learned by the teacher, which acts as a probabilistic knowledge store. The regularizer reduces the effect of catastrophic interference that appears when we learn over sequences of distributions. We validate our model’s performance on sequential variants of MNIST, FashionMNIST, PermutedMNIST, SVHN and Celeb-A and demonstrate that our model mitigates the effects of catastrophic interference faced by neural networks in sequential learning scenarios.

Summary

Probabilistic Knowledge argues that credences can be knowledge (Moss, 2018). Say you have 0.5 credence that a coin landed heads, 0.3 credence that a dog is about to bite you, or 0.9 credence that a criminal defendant is guilty. Each of these credences can constitute knowledge in just the same way that your full beliefs can constitute knowledge. Traditional epistemology has focused on the epistemic status of full beliefs in propositions, such as the proposition that you are not dreaming, or that you have hands, or that the sun will rise tomorrow. But in addition to having knowledge of black-and-white propositions, we have knowledge that comes in every shade of grey. My book is about credences, but not just about credences. More generally, it is about probabilistic beliefs. In addition to point-valued credences, your probabilistic beliefs may include the belief that it is at least 0.9 likely that a defendant is guilty, or that Jones is more likely to smoke than Smith, or that it is 0.7 likely that Smith smokes if Jones does. In addition to these beliefs, you have probabilistic beliefs that do not supervene on your credences – including, I argue, the beliefs you express using epistemic modals and indicative conditionals. All of these beliefs can constitute probabilistic knowledge.

Probabilistic Knowledge in Action

Probabilistic Knowledge in Action

Replies to Edgington, Pavese, and Campbell-Moore and Konek

Edgington raises concerns about four central subjects of my book – my complex content account of credences, my treatment of nested epistemic modals and probability operators, my semantics for indicative conditionals, and the factivity of probabilistic knowledge. In what follows, I discuss and answer her concerns in this same order. Even dogs have probabilistic beliefs. At dinnertime, Fido sits near the children, not the adults, because he believes that the children are more likely to drop food onto the floor. Edgington worries that this fact makes trouble for my complex content account of probabilistic belief. As she puts it, believing a probabilistic content appears to require more ‘self-consciousness and conceptual apparatus’ than believing a proposition, and Fido ‘doesn’t have the concept of probability’ that seems required to believe a probabilistic content. However, Edgington is mistaken to think that you must have a concept of probability in order to believe probabilistic contents. When it comes to concept possession, the complex content account is no more demanding than traditional accounts of belief. You do not need to grasp the concept of a possible world in order to believe the traditional content represented by the set of worlds where Jones smokes. For just the same reason, you do not need to grasp probability concepts in order to believe the content represented by the set of probability spaces that assign .6 to Jones smoking. Whether the theorist models ordinary beliefs using possible worlds or using probability spaces, subjects can have ordinary beliefs without grasping the concepts that the theorist uses to talk about them.

Credence, Conditionals, Knowledge and Truth

Epistemology, strictly speaking, is the theory of knowledge: what it is, when we have it, why it is valuable, what we can do with it. But the term has come to be used more widely, to include the theory of reasonable belief, rational belief change, non-conclusive inference, etc., in cases where knowledge may not be obtainable. Especially under the heading ‘formal epistemology’, probability theory typically plays a prominent role here, while it typically does not play much of a role in traditional, knowledge-centred epistemology, because, it is natural to think, probability theory typically governs something weaker than knowledge. It is about the management of uncertainty. Sarah Moss’s book Probabilistic Knowledge is revolutionary. It shares with traditional epistemology the claim that knowledge is the central concept. On the other hand, it is probabilistic through and through. In short, our ordinary probability judgements can be knowledge. There are different kinds of probability. Maybe we can know objective chances, but that is not Moss’s target. Maybe we can know that a certain body of evidence renders probable a conclusion, but that is not the target either. Our ordinary probability judgements are not statements about their relation to a body of evidence. The subject may not be capable of such second-order judgements. ‘Fido can believe that you are probably about to take him outside without being capable of any second-order belief about what is consistent with his evidence’ (23).1 The target is our ordinary credences – degrees of closeness to certainty, expressed by ‘probably’, ‘more likely than not’, etc. Recurrent examples are ‘Jones probably smokes’ and ‘it is 0.6 likely that Jones smokes’.

Regression and progression in stochastic domains

Reasoning about degrees of belief in uncertain dynamic worlds is fundamental to many applications, such as robotics and planning, where actions modify state properties and sensors provide measurements, both of which are prone to noise. With the exception of limited cases such as Gaussian processes over linear phenomena, belief state evolution can be complex and hard to reason with in a general way, especially when the agent has to deal with categorical assertions, incomplete information such as disjunctive knowledge, as well as probabilistic knowledge. Among the many approaches for reasoning about degrees of belief in the presence of noisy sensing and acting, the logical account proposed by Bacchus, Halpern, and Levesque is perhaps the most expressive, allowing for such belief states to be expressed naturally as constraints. While that proposal is powerful, the task of how to plan effectively is not addressed. In fact, at a more fundamental level, the task of projection, that of reasoning about beliefs effectively after acting and sensing, is left entirely open.To aid planning algorithms, we study the projection problem in this work. In the reasoning about actions literature, there are two main solutions to projection: regression and progression. Both of these have proven enormously useful for the design of logical agents, essentially paving the way for cognitive robotics. Roughly, regression reduces a query about the future to a query about the initial state. Progression, on the other hand, changes the initial state according to the effects of each action and then checks whether the formula holds in the updated state. In this work, we show how both of these generalize in the presence of degrees of belief, noisy acting and sensing. Our results allow for both discrete and continuous probability distributions to be used in the specification of beliefs and dynamics.

Bag-of-Concepts representation for document classification based on automatic knowledge acquisition from probabilistic knowledge base

Text representation, a crucial step for text mining and natural language processing, concerns about transforming unstructured textual data into structured numerical vectors to support various machine learning and data mining algorithms. For document classification, one classical and commonly adopted text representation method is Bag-of-Words (BoW) model. BoW represents document as a fixed-length vector of terms, where each term dimension is a numerical value such as term frequency or tf-idf weight. However, BoW simply looks at surface form of words. It ignores the semantic, conceptual and contextual information of texts, and also suffers from high dimensionality and sparsity issues. To address the aforementioned issues, we propose a novel document representation scheme called Bag-of-Concepts (BoC), which automatically acquires useful conceptual knowledge from external knowledge base, then conceptualizes words and phrases in the document into higher level semantics (i.e. concepts) in a probabilistic manner, and eventually represents a document as a distributed vector in the learned concept space. By utilizing background knowledge from knowledge base, BoC representation is able to provide more semantic and conceptual information of texts, as well as better interpretability for human understanding. We also propose Bag-of-Concept-Clusters (BoCCl) model which clusters semantically similar concepts together and performs entity sense disambiguation to further improve BoC representation. In addition, we combine BoCCl and BoW representations using an attention mechanism to effectively utilize both concept-level and word-level information and achieve optimal performance for document classification.

Context affects implicit learning of spatial bias depending on task relevance.

Recent studies on the probability cueing effect have shown that a spatial bias emerges toward a location where a target frequently appears. In the present study, we explored whether such spatial bias can be flexibly shifted when the target-frequent location changes depending on the given context. In four consecutive experiments, participants performed a visual search task within two distinct contexts that predicted the visual quadrant that was more likely to contain a target. We found that spatial attention was equally biased toward two target-frequent quadrants, regardless of context (context-independent spatial bias), when the context information was not mandatory for accurate visual search. Conversely, when the context became critical for the visual search task, the spatial bias shifted significantly more to the target-frequent quadrant predicted by the given context (context-specific spatial bias). These results show that the task relevance of context determines whether probabilistic knowledge can be learned flexibly in a context-specific manner.

Semi-Supervised Boosting Using Similarity Learning Based on Modular Sparse Representation With Marginal Representation Learning of Graph Structure Self-Adaptive

The purpose of semi-supervised boosting strategy is to improve the classification performance of one given classifier for a large number of unlabeled data. In the semi-supervised boosting strategy, the unlabeled samples are assigned for pseudo labels according to similarities between the labeled samples and the unlabeled samples, and the unlabeled samples with high confidences of pseudo labels are selected as labeled samples at the same time. Good similarities help to assign more appropriate pseudo labels to the unlabeled samples. These selected samples with pseudo labels will be used as the labeled samples to train the new ensemble classifier. Therefore, good and distinguishable similarities learning between unlabeled samples and labeled samples has shown remarkable importance due to its promising performance for semi-supervised boosting strategy. This article presents semi-supervised boosting framework using similarity learning based on modular sparse representation by employing a marginal regression function with probabilistic graphical structure adaptation. In this article, distinguishable regression targets analysis, graph structure adaptation, robust modular sparse representation and semi-supervised boosting learning are seamlessly incorporated into a joint framework. This framework learns marginal regression targets from data rather than exploiting the conventional zero-one matrix that greatly hinders the freedom of regression fitness and degrades the performance of regression results to improve the interclass separation of the learned representation. Meanwhile, a regularization term based on probabilistic connection knowledge is used to construct a graph regularization with adaptive optimization, which improves the intra-class compactness of the learned representation. Additionally, modular sparse representation learning is used to improve the robustness of the learned representation. The experimental results on four datasets including face and object show that the recognition rates of the proposed method are significantly better than other state-of-the-art methods.

МАЙБУТНЄ ЯК ФІЛОСОФСЬКА. ПРОБЛЕМА Частина ІІ

З проблемою детермінованості майбутнього пов’язана проблема меж і можливостей нашого пізнання майбутнього. Інший аспект проблеми стосується способів пізнання майбутнього. На цьому напрямі напрацьований великий досвід і він потребує осмислення.Майбутнє для людини є чимось більшим ніж пізнавальним образом, воно є екзистенційним станом людини. Людина постійно перебуває в майбутньому, залишаючись при цьому в теперішньому і, частково, в минулому. Цей суб’єктивний стан майбутнього людини є особливою темою філософського осмислення.

A distance-based approach for merging probabilistic knowledge bases

A distance-based approach for merging probabilistic knowledge bases

Acting on Probabilistic Knowledge

Acting on Probabilistic Knowledge

Normative uncertainty and probabilistic moral knowledge

The aim of this paper is to examine whether it would be advantageous to introduce knowledge norms instead of the currently assumed rational credence norms into the debate about decision making under normative uncertainty. There is reason to think that this could help us better accommodate cases in which agents are rationally highly confident in false moral views. I show how Moss’ (Probabilistic knowledge. Oxford University Press, Oxford, 2018) view of probabilistic knowledge can be fruitfully employed to develop a decision theory that delivers plausible verdicts in these cases. I also argue that, for this new view to be better than existing alternatives, it must adopt a particular solution to the new evil demon problem, which asks whether agents and their BIV-counterparts are equally justified. In order to get an attractive decision theory for cases of moral uncertainty, we must reject the claim that agents and their BIV-counterparts are equally justified. Moreover, the resulting view must be supplemented with a moral epistemology that explains how it is possible to be rationally morally uncertain. This is especially challenging if we assume that moral truths are knowable a priori.

The Perceptual Prediction Paradox.

From the noisy information bombarding our senses, our brains must construct percepts that are veridical - reflecting the true state of the world - and informative - conveying what we did not already know. Influential theories suggest that both challenges are met through mechanisms that use expectations about the likely state of the world to shape perception. However, current models explaining how expectations render perception either veridical or informative are mutually incompatible. While the former propose that perceptual experiences are dominated by events we expect, the latter propose that perception of expected events is suppressed. To solve this paradox we propose a two-process model in which probabilistic knowledge initially biases perception towards what is likely and subsequently upweights events that are particularly surprising.