Probabilistic Knowledge Research Articles

A Corpus-Based Multifactorial Study of Help/help to Alternation in Learners’ Language: From the Perspective of Probabilistic Grammar

This corpus-based multifactorial study delves deeper into the well-known alternation between bare and full infinitive complements, specifically regarding the help concordances. It extends the line of research to learners’ language productions with a focus on comparing and contrasting the probabilistic grammatical knowledge reflected in the help/ help to choices, which are constrained by a complex network of factors. Regression modeling indicates that constraints such as the horror aequi and certain morphological forms of the verb help are comparatively stable among groups of Chinese English language learners and native English speakers. Therefore, the principle of cognitive complexity and the avoidance strategy fueling infinitive variations are further tested. While the object typology differs in effect strength, it also prompts consideration of cultural influences on the part of Chinese English learners. In addition, two different patterns of interaction effects are explored in each dataset, which implies that learners’ language choice is usage-based, determined by an unconscious assessment of multiple clues that are learned from language input and shaped by language experiences.

Online probabilistic knowledge distillation on cryptocurrency trading using Deep Reinforcement Learning

Leveraging Deep Reinforcement Learning (DRL) for training agents for financial trading has gained significant attention in recent years. However, training these agents in noisy financial environments remains challenging and unstable, significantly impacting their performance as trading agents, as the recent literature has also showcased. This paper introduces a novel distillation method for DRL agents, aiming to improve the training stability of DRL agents. The proposed method transfers knowledge from a teacher ensemble to a student model, incorporating both the action probability distribution knowledge from the output layer, as well as the knowledge from the intermediate layers of the teacher’s network. Furthermore, the proposed method also works in an online fashion, allowing for eliminating the separate teacher training process typically involved in many DRL distillation pipelines, simplifying the distillation process. The proposed method is extensively evaluated on a large-scale cryptocurrency trading setup, demonstrating its ability to both lead to significant improvements in trading accuracy and obtained profit, as well as increase the stability of the training process.

Degrees of truthlikeness, independence, equality and order in probabilistic propositional knowledge representation

Abstract Probabilistic propositional knowledge representation, as here conceived, deals with knowledge representation in the form of probabilistic (p-)valuations of the atomic propositions of a propositional language, of probability (p-)distributions over propositional state descriptions and with their relation. The aim of the paper is conceptual: introducing and illustrating a number of related, normalized concepts regarding both valuations and distributions: degrees of truthlikeness, (internal) independence, equality and order. The backbone of these concepts is the relevant sum of absolute distances. All degrees are in the unit interval and will be illustrated at the relevant place by data about the co-morbidity of psychiatric syndromes (Van Loo et al., 2016). In a new section we illustrate all concepts by an example of weather conditions, to be precise: (not) windy, (not) hot, (not) rainy days in March 2023, in De Bilt (NL). We also briefly discuss some other approaches: other distance measures, covariance as a measure of mutual dependence and various degrees of equality. In the final section we enlist several further research questions and issues, among which the question whether this paper is relevant for propositional probability logics. The paper is highly inspired by the rich paper of Gustavo Cevolani and Roberto Festa (2021).

Reliability of a probabilistic knowledge structure

Indexes for estimating the overall reliability of a test in the framework of knowledge space theory (KST) are proposed and analyzed. First, the possibility of applying in KST the existing classical test theory (CTT) methods, based on the ratio between the true score variance and the total variance of the measure, has been explored. However, these methods are not suitable because in KST error and true score are not independent. Therefore, two new indexes based on the concepts of entropy and conditional entropy are developed. One index is used to estimate the reliability of the response pattern given the knowledge state, while the second one refers to the reliability of the estimated knowledge state of a person. Some theoretical considerations as well as simulations and an empirical example on real data are provided within a study of the behavior of these indexes under a certain number of different conditions.

A world model: On the political logics of generative AI

The computational logics of large language models (LLMs) or generative AI – from the early models of CLIP and BERT to the explosion of text and image generation via ChatGPT and DALL-E − are increasingly penetrating the social and political world. Not merely in the direct sense that generative AI models are being deployed to govern difficult problems, whether decisions on the battlefield or responses to pandemic, but also because generative AI is shaping and delimiting the political parameters of what can be known and actioned in the world. Contra the promise of a generalizable “world model” in computer science, the article addresses how and why generative AI gives rise to a model of the world, and with it a set of political logics and governing rationalities that have profound and enduring effects on how we live today. The article traces the genealogies of generative AI models, how they have come into being, and why some concepts and techniques that animate these models become durable forms of knowledge that actively shape the world, even long after a specific material commercial GPT model has moved on to a new iteration. Though generative AI retains significant traces of former scientific and computational regimes – in statistical practices, probabilistic knowledge, and so on – it is also dislocating epistemological arrangements and opening them to novel ways of perceiving, characterising, classifying, and knowing the world. Four defining aspects of the political logic of generative AI are elaborated: i) generativity as something more than the capacity to generate image or text outputs, so that a generative logic acts upon the world understood as estimates of “underlying distributions” in data; ii) latency as a political logic of compression in which (by contrast with claims to reduction or distortion) the thing that is hidden, unknown or latent becomes surfaced and amenable to being governed; iii) broken and parallelized sequences as the ordering device of the political logic of generative AI, where attention frameworks radically change the possibilities for governing non-linear problems; iv) pre-training and fine-tuning as a computational logic of generative AI that simultaneously shapes a “zero shot politics” oriented towards unencountered data and new tasks. Across each of the four aspects, the article maps the emerging contemporary political logic of generative AI.

Legal proof: why knowledge matters and knowing does not

I discuss the knowledge account of legal proof in Moss (2023) and develop an alternative. The unifying thread throughout this article are reflections on the beyond reasonable doubt (BRD) standard of proof. In Section 1, I will introduce the details of Moss’s account and how she motivates it via the BRD standard. In Section 2, I will argue that there are important disanalogies between BRD and knowledge that undermine Moss’s argument. There is however another motivation for the knowledge account: combined with auxiliary claims, that is probabilistic knowledge and moral encroachment, it can provide a general solution to the puzzle of statistical evidence. Section 3 spells out the details. In Section 4, I suggest to combine the knowledge account with pragmatic encroachment, instead of moral encroachment, in order to stay clear of the thorny issues whether corporations have moral rights. In Section 5, I argue that the verdicts of Moss’s account in cases of false justified beliefs and non-luminous knowledge conflict with the BRD standard and thus call for abandoning the account. Based on the social function of the BRD standard, I suggest a replacement for the knowledge account that is also just as potent as a general solution for the puzzle of statistical evidence. While I will grant that knowledge is neither always necessary nor always sufficient for convictions, I will argue that the concept of knowledge nonetheless plays a significant and ineliminable role in legal decision-making.

A probabilistic knowledge graph for target identification.

Early identification of safe and efficacious disease targets is crucial to alleviating the tremendous cost of drug discovery projects. However, existing experimental methods for identifying new targets are generally labor-intensive and failure-prone. On the other hand, computational approaches, especially machine learning-based frameworks, have shown remarkable application potential in drug discovery. In this work, we propose Progeni, a novel machine learning-based framework for target identification. In addition to fully exploiting the known heterogeneous biological networks from various sources, Progeni integrates literature evidence about the relations between biological entities to construct a probabilistic knowledge graph. Graph neural networks are then employed in Progeni to learn the feature embeddings of biological entities to facilitate the identification of biologically relevant target candidates. A comprehensive evaluation of Progeni demonstrated its superior predictive power over the baseline methods on the target identification task. In addition, our extensive tests showed that Progeni exhibited high robustness to the negative effect of exposure bias, a common phenomenon in recommendation systems, and effectively identified new targets that can be strongly supported by the literature. Moreover, our wet lab experiments successfully validated the biological significance of the top target candidates predicted by Progeni for melanoma and colorectal cancer. All these results suggested that Progeni can identify biologically effective targets and thus provide a powerful and useful tool for advancing the drug discovery process.

From P Versus NP to Probabilistic and Zero Knowledge Proof Systems

From P Versus NP to Probabilistic and Zero Knowledge Proof Systems

Glass box machine learning for retrospective cohort studies using many patient records. The complex example of bleeding peptic ulcer

Glass Box Machine Learning is, in this study, a type of partially supervised data mining and prediction technique, like a neural network in which each weight or pattern of mutually relevant weights is now replaced by a meaningful "probabilistic knowledge element." We apply it to retrospective cohort studies using large numbers of structured medical records to help select candidate patients for future cohort studies and similar clinical trials. Here it is applied to aid analysis of approaches to aid Deep Learning, but the method lends itself well to direct computation of odds with “explainability” in study design that can complement “Black Box” Deep Learning. Cohort studies and clinical trials traditionally involved at least one 2 × 2 contingency table, but in the age of emerging personalized medicine and the use of machine learning to discover and incorporate further relevant factors, these tables can extend into many extra dimensions as a 2 × 2 x 2 × 2 x ….data structure by considering different conditional demographic and clinical factors of a patient or group, as well as variations in treatment. We consider this in terms of multiple 2 × 2 x 2 data substructures where each one is summarized by an appropriate measure of risk and success called DOR*. This is the diagnostic odds ratio DOR for a specified disease conditional on a favorable outcome divided by the corresponding DOR conditional on an unfavorable outcome. Bleeding peptic ulcer was chosen as a complex disease with many influencing factors, one that is still subject to controversy and that highlights the challenges of using Real World Data.

Semi-supervised Federated Learning for Digital Twin 6G-enabled IIoT: A Bayesian estimated approach

IntroductionIn recent years, the proliferation of Industrial Internet of Things (IIoT) devices has resulted in a substantial increase in data generation across various domains, including the nascent 6G networks. Digital Twins (DTs), serving as virtual replicas of physical entities, have gained popularity within the realm of IoT due to their capacity to simulate and optimize physical systems in a cost-effective manner. Nonetheless, the security of DTs and the safeguarding of the sensitive data they generate have emerged as paramount concerns. Fortunately, the Federated Fearning (FL) system has emerged as a promising solution to address the challenge of data privacy within DTs. Nonetheless, the requisite acquisition of a significant volume of labeled data for training purposes poses a formidable challenge, particularly in a DT environment that blends real and virtual data. ObjectivesTo tackle this challenge, this study presents an innovative Semi-supervised FL (SSFL) framework designed to overcome the scarcity of labeled data through the strategic utilization of pseudo-labels. MethodsSpecifically, our proposed SSFL algorithm, named SSFL-MBE, introduces a novel approach by combining Mix data augmentation and Bayesian Estimation consistency regularization loss, thereby integrating robust augmentation techniques to enhance model generalization. Furthermore, we introduce a Bayesian-estimated pseudo-label loss that leverages prior probabilistic knowledge to enhance model performance. Our investigation focuses particularly on a demanding scenario where labeled and unlabeled data are segregated across disparate locations, specifically, the server and various clients. ResultsComprehensive evaluations conducted on CIFAR-10 and MNIST datasets conclusively demonstrate that our proposed algorithm consistently surpasses mainstream SSFL baseline models, exhibiting an enhancement in model performance ranging from 0.5% to 1.5%. ConclusionOverall, this work contributes to the development of more efficient and secure approaches for model training in DT-empowered FL settings, which is crucial for the deployment of IIoTs in 6G-enabled environments.

Residual Fusion Probabilistic Knowledge Distillation for Speech Enhancement

Residual Fusion Probabilistic Knowledge Distillation for Speech Enhancement

Temporal dynamics of statistical learning in children’s song contributes to phase entrainment and production of novel information in multiple cultures

Statistical learning is thought to be linked to brain development. For example, statistical learning of language and music starts at an early age and is shown to play a significant role in acquiring the delta-band rhythm that is essential for language and music learning. However, it remains unclear how auditory cultural differences affect the statistical learning process and the resulting probabilistic and acoustic knowledge acquired through it. This study examined how children’s songs are acquired through statistical learning. This study used a Hierarchical Bayesian statistical learning (HBSL) model, mimicking the statistical learning processes of the brain. Using this model, I conducted a simulation experiment to visualize the temporal dynamics of perception and production processes through statistical learning among different cultures. The model learned from a corpus of children’s songs in MIDI format, which consists of English, German, Spanish, Japanese, and Korean songs as the training data. In this study, I investigated how the probability distribution of the model is transformed over 15 trials of learning in each song. Furthermore, using the probability distribution of each model over 15 trials of learning each song, new songs were probabilistically generated. The results suggested that, in learning processes, chunking and hierarchical knowledge increased gradually through 15 rounds of statistical learning for each piece of children’s songs. In production processes, statistical learning led to the gradual increase of delta-band rhythm (1–3 Hz). Furthermore, by combining the acquired chunks and hierarchy through statistical learning, statistically novel music was generated gradually in comparison to the original songs (i.e. the training songs). These findings were observed consistently, in multiple cultures. The present study indicated that the statistical learning capacity of the brain, in multiple cultures, contributes to the acquisition and generation of delta-band rhythm, which is critical for acquiring language and music. It is suggested that cultural differences may not significantly modulate the statistical learning effects since statistical learning and slower rhythm processing are both essential functions in the human brain across cultures. Furthermore, statistical learning of children’s songs leads to the acquisition of hierarchical knowledge and the ability to generate novel music. This study may provide a novel perspective on the developmental origins of creativity and the importance of statistical learning through early development.

A bucket fill factor estimation method in construction environments by fusing deep learning and machine vision

The need for high productivity in the construction field is becoming increasingly urgent, and the efficiency of loading and unloading materials by construction vehicles is one of the main influencing factors. The fill factor, which refers to the percentage of the volume of materials in the shovel bucket to the capacity of the shovel bucket, can significantly affect the productivity of construction vehicles, and thereby affect the environmental pollution caused by these vehicles. Therefore, an effective estimation of the fill factor is a prerequisite for improving productivity and suppressing pollution in construction vehicles. However, previous studies have been conducted under ideal environments, the actual harsh environments of the construction sites are not in the purview. Therefore, in this paper, the fill factor of a loader is estimated in nine construction environments, including six ideal environments and three harsh environments. First, the corresponding image processing pipelines are proposed for a variety of environmental characteristics. Then, a stereo segmentation map is constructed based on the loader bucket trajectory and the knowledge of different color space characteristics, which not only addresses the environmental limitations but also significantly improves the accuracy. Finally, an improved CNN is combined with probabilistic knowledge for the fill factor estimation, and transfer learning is also applied to improve the training speed and accuracy. The results show that a satisfactory accuracy of 96.89% is achieved for the abovementioned construction environments, which results in a more efficient and cleaner production of construction vehicles.

Teaching and learning of probability

In this paper, we develop a personal synthesis of the most outstanding research on the teaching and learning of probability in the past years. We conducted a systematic search to examine publications on this topic in mathematics education, statistics education, education, and psychology journals. This exploration was complemented by additional studies published in conference proceedings or books. We classified these papers to highlight the main recent research tendencies in the field, according to the theme studied and considering the research objectives. Epistemological analyses suggest that informal inference based on simulation diminishes the topic abstraction but reduces probability to its frequentist view. Topics receiving particular attention include children’s probabilistic knowledge, the effect of visualizations on solving conditional probability problems, teachers’ education and probability modelling. In the final section, we recommend relevant points in which more investigation is needed to complete our knowledge about teaching and learning. In particular, we miss research on teachers’ mathematical knowledge of many probability concepts and on their didactic knowledge.

Exploiting the implicit independence assumption for learning directed graphical models

Bayesian network classifiers (BNCs) provide a sound formalism for representing probabilistic knowledge and reasoning with uncertainty. Explicit independence assumptions can effectively and efficiently reduce the size of the search space for solving the NP-complete problem of structure learning. Strong conditional dependencies, when added to the network topology of BNC, can relax the independence assumptions, whereas the weak ones may result in biased estimates of conditional probability and degradation in generalization performance. In this paper, we propose an extension to the k-dependence Bayesian classifier (KDB) that achieves the bias/variance trade-off by verifying the rationality of implicit independence assumptions implicated. The informational and probabilistic dependency relationships represented in the learned robust topologies will be more appropriate for fitting labeled and unlabeled data, respectively. The comprehensive experimental results on 40 UCI datasets show that our proposed algorithm achieves competitive classification performance when compared to state-of-the-art BNC learners and their efficient variants in terms of zero-one loss, root mean square error (RMSE), bias and variance.

MAPS-KB: A Million-Scale Probabilistic Simile Knowledge Base

The ability to understand and generate similes is an imperative step to realize human-level AI. However, there is still a considerable gap between machine intelligence and human cognition in similes, since deep models based on statistical distribution tend to favour high-frequency similes. Hence, a large-scale symbolic knowledge base of similes is required, as it contributes to the modeling of diverse yet unpopular similes while facilitating additional evaluation and reasoning. To bridge the gap, we propose a novel framework for large-scale simile knowledge base construction, as well as two probabilistic metrics which enable an improved understanding of simile phenomena in natural language. Overall, we construct MAPS-KB, a million-scale probabilistic simile knowledge base, covering 4.3 million triplets over 0.4 million terms from 70 GB corpora. We conduct sufficient experiments to justify the effectiveness and necessity of the methods of our framework. We also apply MAPS-KB on three downstream tasks to achieve state-of-the-art performance, further demonstrating the value of MAPS-KB. Resources of MAPS-KB are publicly available at https://github.com/Abbey4799/MAPS-KB.

On the progression of belief

Based on weighted possible-world semantics, Belle and Lakemeyer recently proposed the logic DS, a probabilistic extension of a modal variant of the situation calculus with a model of belief. The logic has many desirable properties like full introspection and it is able to precisely capture the beliefs of a probabilistic knowledge base in terms of the notion of only-believing. While the proposal is intuitively appealing, it is unclear how to do planning with such logic. The reason behind this is that the logic lacks projection reasoning mechanisms and projection lies at the heart of planning. Projection reasoning, in general, is to decide what holds after actions. Two main solutions to projection exist: regression and progression. Roughly, regression reduces a query about the future to a query about the initial state while progression, on the other hand, changes the initial state according to the effects of actions and then checks whether the formula holds in the updated state. In this paper, we study projection by progression in the logic DS. It is known that the progression of a categorical knowledge base wrt a noise-free action corresponds to what is only-known after that action. We show how to progress a type of probabilistic knowledge base wrt noisy actions by the notion of only-believing after actions. Our notion of only-believing is closely related to Lin and Reiter's notion of progression.

A web application for reasoning on probabilistic description logics knowledge bases

AbstractThe aim of the Semantic Web is making information and resources from the Web automatically processable by machines. Usually, the uncertainty characterizing much of this information is addressed by means of a probabilistic semantics. Following the vision of a “Probabilistic Semantic Web”, a plethora of probabilistic semantics have been proposed: some of them change the syntax and/or the semantics itself of the knowledge representation language, others allow one to annotate axioms of a knowledge base with a probability value. Among the latter, the DISPONTE semantics exploits probabilistic annotations to extend query answering with the capability of returning the probability of a query being true in a domain. In order to promote the adoption of Probabilistic Semantic Web we first developed BUNDLE, a framework that can exploit different underlying (probabilistic and non‐probabilistic) reasoners to perform probabilistic inference under the DISPONTE semantics. In this paper we present a web application for BUNDLE, to show how DISPONTE is easily usable even in already established applications and systems. It allows users to query a DISPONTE knowledge base written or uploaded directly in the application interface by using just a web browser, without the need to install any software on their machine. It is accessible on the web at https://bundle.ml.unife.it/ and also provides some examples for familiarizing with the application. The results of a usability evaluation involving human participants are also reported, showing the relevance and the practical impact of the tool and possible ways for improvement.

Research on visualization in probability problem solving

The importance of visual representations in education and mathematics is well known. Probabilities are a domain in mathematics that uses many visual representations since their theory consists of a variety of diagrams and graphs. In the past, many studies have shown that the use of various representations in teaching probabilities can greatly improve learning. Of course, the use of a visual representation or a visual tool when teaching or solving an exercise can have a variety of roles. The present work is based on the ancillary and informative role of the image. The following research examines the extent to which students, by solving a probability problem, have the need to use a visual representation or image. Additionally, the differences in student performance are investigated, given the role of the image in the activity. This knowledge can improve the teaching methods of probabilities and, with their appropriate use, school textbooks. The results show that there are more perspectives but also needs for research on the use of visual representations for the teaching of probabilities. The presence of the image works as a motivating factor for children to solve a problem with a lack of probabilistic knowledge, but the role of the image affects their final performance.

Probabilistic knowledge infusion through symbolic features for context-aware activity recognition

In the general machine learning domain, solutions based on the integration of deep learning models with knowledge-based approaches are emerging. Indeed, such hybrid systems have the advantage of improving the recognition rate and the model’s interpretability. At the same time, they require a significantly reduced amount of labeled data to reliably train the model. However, these techniques have been poorly explored in the sensor-based Human Activity Recognition (HAR) domain. The common-sense knowledge about activity execution can potentially improve purely data-driven approaches. While a few knowledge infusion approaches have been proposed for HAR, they rely on rigid logic formalisms that do not take into account uncertainty. In this paper, we propose P-NIMBUS, a novel knowledge infusion approach for sensor-based HAR that relies on probabilistic reasoning. A probabilistic ontology is in charge of computing symbolic features that are combined with the features automatically extracted by a CNN model from raw sensor data and high-level context data. In particular, the symbolic features encode probabilistic common-sense knowledge about the activities consistent with the user’s surrounding context. These features are infused within the model before the classification layer. We experimentally evaluated P-NIMBUS on a HAR dataset of mobile devices sensor data that includes 14 different activities performed by 25 users. Our results show that P-NIMBUS outperforms state-of-the-art neuro-symbolic approaches, with the advantage of requiring a limited amount of training data to reach satisfying recognition rates (i.e., more than 80% of F1-score with only 20% of labeled data).