Inference Tasks Research Articles

Dynamic inferences of crash risks in freeway merging zones: a spatio-temporal deep learning model

Freeway merging zones are critical for freeway operations and management due to potential crashes arising from complex vehicle merging behaviours. This paper investigates the application of a spatio-temporal deep learning model to infer crash risks in the zones. We first introduce a crash risk index based on Time-To-Collision and vehicle merging patterns. An innovation is that the developed spatio-temporal transformer model can analyze the evolving risk index. This model effectively captures dynamic risk features through a multi-head attention mechanism within its spatio-temporal learning components. Numerical experiments on nine inference tasks with varying spatial resolutions show improved performance of the model with lower resolution. Moreover, the ST-Transformer model is benchmarked against three advanced deep learning models, which consistently demonstrates its superiority in capturing spatio-temporal dependence in risk sequences. This investigation significantly contributes to a richer understanding of proactive traffic safety, providing valuable insights for advanced freeway management and driver assistance systems.

Calibrating Bayesian Generative Machine Learning for Bayesiamplification

Abstract Recently, combinations of generative and Bayesian deep learning have been introduced in particle physics for both fast detector simulation and inference tasks.
These neural networks aim to quantify the uncertainty on the generated distribution originating from limited training statistics.
The interpretation of a distribution-wide uncertainty however remains ill-defined.
We show a clear scheme for quantifying the calibration of Bayesian generative machine learning models.
For a Continuous Normalizing Flow applied to a low-dimensional toy example, we evaluate the calibration of Bayesian uncertainties from either a mean-field Gaussian weight posterior, or Monte Carlo sampling network weights, to gauge their behaviour on unsteady distribution edges. 
Well calibrated uncertainties can then be used to roughly estimate the number of uncorrelated truth samples that are equivalent to the generated sample and clearly indicate data amplification for smooth features of the distribution.

A Joint Survey in Decentralized Federated Learning and TinyML: A Brief Introduction to Swarm Learning

TinyML/DL is a new subfield of ML that allows for the deployment of ML algorithms on low-power devices to process their own data. The lack of resources restricts the aforementioned devices to running only inference tasks (static TinyML), while training is handled by a more computationally efficient system, such as the cloud. In recent literature, the focus has been on conducting real-time on-device training tasks (Reformable TinyML) while being wirelessly connected. With data processing being shift to edge devices, the development of decentralized federated learning (DFL) schemes becomes justified. Within these setups, nodes work together to train a neural network model, eliminating the necessity of a central coordinator. Ensuring secure communication among nodes is of utmost importance for protecting data privacy during edge device training. Swarm Learning (SL) emerges as a DFL paradigm that promotes collaborative learning through peer-to-peer interaction, utilizing edge computing and blockchain technology. While SL provides a robust defense against adversarial attacks, it comes at a high computational expense. In this survey, we emphasize the current literature regarding both DFL and TinyML/DL fields. We explore the obstacles encountered by resource-starved devices in this collaboration and provide a brief overview of the potential of transitioning to Swarm Learning.

Robust and energy-efficient RPL optimization algorithm with scalable deep reinforcement learning for IIoT

The increasing complexity and quantity of the Industrial Internet of Things (IIoT) pose new challenges to the traditional routing protocol for low-power and lossy networks (RPL) in terms of dynamic management, data transmission reliability, and energy efficiency optimization. This paper proposes a scalable deep reinforcement learning (DRL) algorithm with a multi-attention actor double critic model for routing optimization (MADC) to meet the requirements of IIoT for efficient and intelligent routing decisions while improving data transmission reliability and energy efficiency. Specifically, MADC employs the centralized training and decentralized execution (CTDE) learning paradigm to decouple the model’s training and inference tasks, which reduces the difficulty and computational cost of model learning and improves the training efficiency. In addition, a lightweight actor network based on multi-scale convolutional attention mechanism is designed in MADC, which can provide intelligent and real-time decision-making capabilities for resource-constrained nodes with low computational and storage complexities. Moreover, a scalable critic network utilizing multiple attention mechanisms is proposed. It is not only suitable for dynamic and changing network environments but also can more comprehensively and accurately evaluate local observation states, providing more accurate and efficient guidance for model optimization. Furthermore, MADC incorporates a double critic network architecture to mitigate potential overestimation issues during training, thereby ensuring the model’s robustness and reliability. Simulation results demonstrate that MADC outperforms existing RPL optimization algorithms in terms of energy efficiency, data transmission reliability, and adaptability.

Lost in the Shuffle: Testing Power in the Presence of Errorful Network Vertex Labels

Two-sample network hypothesis testing is an important inference task with applications across diverse fields such as medicine, neuroscience, and sociology. Many of these testing methodologies operate under the implicit assumption that the vertex correspondence across networks is a priori known. This assumption is often untrue, and the power of the subsequent test can degrade when there are misaligned/label-shuffled vertices across networks. This power loss due to shuffling is theoretically explored in the context of random dot product and stochastic block model networks for a pair of hypothesis tests based on Frobenius norm differences between estimated edge probability matrices or between adjacency matrices. The loss in testing power is further reinforced by numerous simulations and experiments, both in the stochastic block model and in the random dot product graph model, where the power loss across multiple recently proposed tests in the literature is considered. Lastly, the impact that shuffling can have in real-data testing is demonstrated in a pair of examples from neuroscience and from social network analysis.

Multifaceted Natural Language Processing Task-Based Evaluation of Bidirectional Encoder Representations From Transformers Models for Bilingual (Korean and English) Clinical Notes: Algorithm Development and Validation.

The bidirectional encoder representations from transformers (BERT) model has attracted considerable attention in clinical applications, such as patient classification and disease prediction. However, current studies have typically progressed to application development without a thorough assessment of the model's comprehension of clinical context. Furthermore, limited comparative studies have been conducted on BERT models using medical documents from non-English-speaking countries. Therefore, the applicability of BERT models trained on English clinical notes to non-English contexts is yet to be confirmed. To address these gaps in literature, this study focused on identifying the most effective BERT model for non-English clinical notes. In this study, we evaluated the contextual understanding abilities of various BERT models applied to mixed Korean and English clinical notes. The objective of this study was to identify the BERT model that excels in understanding the context of such documents. Using data from 164,460 patients in a South Korean tertiary hospital, we pretrained BERT-base, BERT for Biomedical Text Mining (BioBERT), Korean BERT (KoBERT), and Multilingual BERT (M-BERT) to improve their contextual comprehension capabilities and subsequently compared their performances in 7 fine-tuning tasks. The model performance varied based on the task and token usage. First, BERT-base and BioBERT excelled in tasks using classification ([CLS]) token embeddings, such as document classification. BioBERT achieved the highest F1-score of 89.32. Both BERT-base and BioBERT demonstrated their effectiveness in document pattern recognition, even with limited Korean tokens in the dictionary. Second, M-BERT exhibited a superior performance in reading comprehension tasks, achieving an F1-score of 93.77. Better results were obtained when fewer words were replaced with unknown ([UNK]) tokens. Third, M-BERT excelled in the knowledge inference task in which correct disease names were inferred from 63 candidate disease names in a document with disease names replaced with [MASK] tokens. M-BERT achieved the highest hit@10 score of 95.41. This study highlighted the effectiveness of various BERT models in a multilingual clinical domain. The findings can be used as a reference in clinical and language-based applications.

Tree sequences as a general-purpose tool for population genetic inference.

As population genetics data increases in size new methods have been developed to store genetic information in efficient ways, such as tree sequences. These data structures are computationally and storage efficient, but are not interchangeable with existing data structures used for many population genetic inference methodologies such as the use of convolutional neural networks (CNNs) applied to population genetic alignments. To better utilize these new data structures we propose and implement a graph convolutional network (GCN) to directly learn from tree sequence topology and node data, allowing for the use of neural network applications without an intermediate step of converting tree sequences to population genetic alignment format. We then compare our approach to standard CNN approaches on a set of previously defined benchmarking tasks including recombination rate estimation, positive selection detection, introgression detection, and demographic model parameter inference. We show that tree sequences can be directly learned from using a GCN approach and can be used to perform well on these common population genetics inference tasks with accuracies roughly matching or even exceeding that of a CNN-based method. As tree sequences become more widely used in population genetics research we foresee developments and optimizations of this work to provide a foundation for population genetics inference moving forward.

Do young American children essentialize ethnicity? Examining inductive inferences about Hispanic/Latinx individuals in an ethnically diverse sample

AbstractHispanic and Latinx individuals represent one of the largest and fastest growing ethnic groups in the United States. Yet research has not investigated whether young children hold essentialist beliefs about this prevalent social category. The present study addressed this issue by examining whether children used this ethnic category to make inductive inferences about novel individuals, one dimension of essentialism. A total of 108 children, 5 to 7 years of age (54 female; 56 Hispanic, 46 non‐Hispanic), completed a forced‐choice inference task. Children did not expect members of the same ethnic group to share properties, and this did not vary with their own ethnic group membership. This suggests that in the US, the belief that ethnicity is causally informative undergoes a protracted developmental trajectory, as has been observed for essentialist beliefs about race.

Watch your tune! On the role of intonation for scalar diversity

Recent research has highlighted that lexical scales vary in their likelihood of giving rise to a scalar inference—a finding labeled scalar diversity. The current paper examines the role of intonation for this phenomenon, which has thus far primarily been studied using written materials. A specific focus in this regard was on the so-called rise-fall-rise contour, which has been argued to (i) convey uncertainty, which could have an influence on scalar inference calculation, and (ii) be sensitive to properties of lexical scales, which could interact with factors driving scalar diversity. Experiment 1 combined production with an inference task to assess the likelihood of different intonational contours, as well as how a given contour affects scalar inference rates. Production of the rise-fall-rise varied across lexical scales, as expected, and led to an increase in scalar inference derivation relative to a fall. The latter finding was further confirmed in Experiment 2, which explicitly manipulated intonational contours in the inference task. The results, thus, show the importance of taking intonation into account when studying scalar diversity and scalar inference more generally, and they also have implications for theories of the rise-fall-rise contour. Additionally, the experiments revealed a contour that is prosodically similar to the so-called Contradiction Contour, but appears to serve a different pragmatic function.

Resource‐adaptive and OOD‐robust inference of deep neural networks on IoT devices

AbstractEfficiently executing inference tasks of deep neural networks on devices with limited resources poses a significant load in IoT systems. To alleviate the load, one innovative method is branching that adds extra layers with classification exits to a pre‐trained model, enabling inputs with high‐confidence predictions to exit early, thus reducing inference cost. However, branching networks, not originally tailored for IoT environments, are susceptible to noisy and out‐of‐distribution (OOD) data, and they demand additional training for optimal performance. The authors introduce BrevisNet, a novel branching methodology designed for creating on‐device branching models that are both resource‐adaptive and noise‐robust for IoT applications. The method leverages the refined uncertainty estimation capabilities of Dirichlet distributions for classification predictions, combined with the superior OOD detection of energy‐based models. The authors propose a unique training approach and thresholding technique that enhances the precision of branch predictions, offering robustness against noise and OOD inputs. The findings demonstrate that BrevisNet surpasses existing branching techniques in training efficiency, accuracy, overall performance, and robustness.

Multiple network embedding for anomaly detection in time series of graphs

The problem of anomaly detection in time series of graphs is considered, focusing on two related inference tasks: the detection of anomalous graphs within a time series and the detection of temporally anomalous vertices. These tasks are approached via the adaptation of multiple adjacency spectral embedding (MASE), a statistically principled method for joint graph inference. The effectiveness of the method is demonstrated for these inference tasks, and its performance is assessed based on the nature of detectable anomalies. Theoretical justification is provided, along with insights into its use. The approach identifies anomalous vertices beyond just large degree changes when applied to the Enron communication graph, a large-scale commercial search engine time series, and a larval Drosophila connectome.

Tree sequences as a general-purpose tool for population genetic inference.

As population genetics data increases in size new methods have been developed to store genetic information in efficient ways, such as tree sequences. These data structures are computationally and storage efficient, but are not interchangeable with existing data structures used for many population genetic inference methodologies such as the use of convolutional neural networks (CNNs) applied to population genetic alignments. To better utilize these new data structures we propose and implement a graph convolutional network (GCN) to directly learn from tree sequence topology and node data, allowing for the use of neural network applications without an intermediate step of converting tree sequences to population genetic alignment format. We then compare our approach to standard CNN approaches on a set of previously defined benchmarking tasks including recombination rate estimation, positive selection detection, introgression detection, and demographic model parameter inference. We show that tree sequences can be directly learned from using a GCN approach and can be used to perform well on these common population genetics inference tasks with accuracies roughly matching or even exceeding that of a CNN-based method. As tree sequences become more widely used in population genetics research we foresee developments and optimizations of this work to provide a foundation for population genetics inference moving forward.

Demonstration of 4-quadrant analog in-memory matrix multiplication in a single modulation

Analog in-memory computing (AIMC) leverages the inherent physical characteristics of resistive memory devices to execute computational operations, notably matrix-vector multiplications (MVMs). However, executing MVMs using a single-phase reading scheme to reduce latency necessitates the simultaneous application of both positive and negative voltages across resistive memory devices. This degrades the accuracy of the computation due to the dependence of the device conductance on the voltage polarity. Here, we demonstrate the realization of a 4-quadrant MVM in a single modulation by developing analog and digital calibration procedures to mitigate the conductance polarity dependence, fully implemented on a multi-core AIMC chip based on phase-change memory. With this approach, we experimentally demonstrate accurate neural network inference and similarity search tasks using one or multiple cores of the chip, at 4 times higher MVM throughput and energy efficiency than the conventional four-phase reading scheme.

SCL: A sustainable deep learning solution for edge computing ecosystem in smart manufacturing

Edge computing empowered Deep Learning (DL) solutions have risen as the foremost facilitators of automation in a multitude of smart manufacturing applications. These models are implemented on edge devices with frozen learning capabilities to execute DL inference task(s). Nevertheless, the data they process is susceptible to intermittent alterations amidst the ever-changing landscape of dynamic smart manufacturing ecosystem. It sparks the demand for model maintenance solution(s) to address adaptability and dynamism issues to enhance the efficiency of smart manufacturing solutions. Moreover, additional issue(s), such as the non-availability of comprehensive data (or the availability of solely contemporary data), near-real-time execution of DL model maintenance task, etc., imposes daunting obstructions in devising an efficient DL model maintenance strategy. This work proposes a novel approach that encompasses the merits of Continual Learning (CL) and Split Learning (SL) driven by edge intelligence, amalgamating them into a hybrid solution aptly named Split-based Continual Learning (SCL). CL ensures the sustained performance of the DL model amidst constraints related to data availability. At the same time, SL empowers near-real-time execution at the edge to achieve improved efficiency. An extension of the SCL scheme, termed as Extended SCL (ESCL), is implemented to addresses the interaction soundness aspects among the mobile edge devices in a collaborative execution environment. Evaluation of a vision-based product-quality inspection use case in an emulated hardware test-bed setup signifies that the performance of SCL and ESCL schemes have the potential to meet the needs of smart manufacturing. SCL attains an appreciable reduction in the model maintenance cost in the range of 21 to 48 and 12 to 29 percent compared to the ECN-only and basic-SL schemes. The ESCL scheme further improved the performance by 18 to 34 and 20 to 36 percent respectively over the basic-SL and SCL.

Natural Language Inference with Transformer Ensembles and Explainability Techniques

Natural language inference (NLI) is a fundamental and quite challenging task in natural language processing, requiring efficient methods that are able to determine whether given hypotheses derive from given premises. In this paper, we apply explainability techniques to natural-language-inference methods as a means to illustrate the decision-making procedure of its methods. First, we investigate the performance and generalization capabilities of several transformer-based models, including BERT, ALBERT, RoBERTa, and DeBERTa, across widely used datasets like SNLI, GLUE Benchmark, and ANLI. Then, we employ stacking-ensemble techniques to leverage the strengths of multiple models and improve inference performance. Experimental results demonstrate significant improvements of the ensemble models in inference tasks, highlighting the effectiveness of stacking. Specifically, our best-performing ensemble models surpassed the best-performing individual transformer by 5.31% in accuracy on MNLI-m and MNLI-mm tasks. After that, we implement LIME and SHAP explainability techniques to shed light on the decision-making of the transformer models, indicating how specific words and contextual information are utilized in the transformer inferences procedures. The results indicate that the model properly leverages contextual information and individual words to make decisions but, in some cases, find difficulties in inference scenarios with metaphorical connections which require deeper inferential reasoning.

Energy-Efficient Industrial Internet of Things in Green 6G Networks

The research problem of this systematic review was whether green 6G networks can integrate energy-efficient Industrial Internet of Things (IIoT) in terms of distributed artificial intelligence, green 6G pervasive edge computing communication networks and big-data-based intelligent decision algorithms. We show that sensor data fusion can be carried out in energy-efficient IoT smart industrial urban environments by cooperative perception and inference tasks. Our analyses debate on 6G wireless communication, vehicular IoT intelligent and autonomous networks, and energy-efficient algorithm and green computing technologies in smart industrial equipment and manufacturing environments. Mobile edge and cloud computing task processing capabilities of decentralized network control and power grid system monitoring were thereby analyzed. Our results and contributions clarify that sustainable energy efficiency and green power generation together with IoT decision support and smart environmental systems operate efficiently in distributed artificial intelligence 6G pervasive edge computing communication networks. PRISMA was used, and with its web-based Shiny app flow design, the search outcomes and screening procedures were integrated. A quantitative literature review was performed in July 2024 on original and review research published between 2019 and 2024. Study screening, evidence map visualization, and data extraction and reporting tools, machine learning classifiers, and reference management software were harnessed for qualitative and quantitative data, collection, management, and analysis in research synthesis. Dimensions and VOSviewer were deployed for data visualization and analysis.

A geometrical solution underlies general neural principle for serial ordering

A general mathematical description of how the brain sequentially encodes knowledge remains elusive. We propose a linear solution for serial learning tasks, based on the concept of mixed selectivity in high-dimensional neural state spaces. In our framework, neural representations of items in a sequence are projected along a “geometric” mental line learned through classical conditioning. The model successfully solves serial position tasks and explains behaviors observed in humans and animals during transitive inference tasks amidst noisy sensory input and stochastic neural activity. This approach extends to recurrent neural networks performing motor decision tasks, where the same geometric mental line correlates with motor plans and modulates network activity according to the symbolic distance between items. Serial ordering is thus predicted to emerge as a monotonic mapping between sensory input and behavioral output, highlighting a possible pivotal role for motor-related associative cortices in transitive inference tasks.

Natural Frequencies Improve Public Understanding of Medical Test Results: An Experimental Study on Various Bayesian Inference Tasks with Multiple Scoring Methods and Non-Bayesian Reasoning Strategies.

It is well established that the natural frequencies (NF) format is cognitively more beneficial for Bayesian inference than the conditional probabilities (CP) format. However, empirical studies have suggested that the NF facilitation effect might be limited to specific groups of individuals. Unlike previous studies that focused on a limited number of Bayesian inference problems evaluated by a single scoring method, it was essential to examine multiple Bayesian problems using various scoring metrics. This study also explored the impact of numeracy on Bayesian inference and assessed non-Bayesian cognitive strategies using the numerical information in problem solving. In a Web-based experimental survey, 175 South Korean adults were randomly assigned to 1 of 2 format groups (NF v. CP). After completing numeracy scales, participants were asked to estimate 4 Bayesian inference problems and document the numerical information used in their problem-solving process. Four scoring methods-strict rounding, loose rounding, absolute deviation, and 50-Split-were used to evaluate participants' estimations. The NF format generally outperformed the CP format across all problems, except in a chorionic villus sampling test problem when evaluated using the 50-Split method. In addition, numeracy levels significantly influenced Bayesian inference; participants with higher numeracy demonstrated better performance. In addition, participants used various non-Bayesian strategies influenced by the format and the nature of the problems. The NF facilitation effect was consistently observed across multiple Bayesian problems and scoring methods. Individuals with higher numeracy levels benefited more from the NF format. The use of various non-Bayesian strategies varied with the formats and nature of specific tasks. The natural frequencies (NF) format is known to foster understanding of medical test results compared with the conditional probabilities (CP) format, but some studies have reported that this benefit is either nonexistent or limited to specific groups.This study aims to replicate previous empirical studies using various Bayesian problems using multiple scoring methods.The NF format fosters understanding of medical test results across all Bayesian problems by all scoring methods, except in the CVS problem when using a 50-Split scoring method.Participants with high numeracy perform better Bayesian inference than those with lower numeracy. Particularly, higher numerates benefit more in the NF format than lower numerates do. In addition, the public tend to use various non-Bayesian reasoning strategies depending on the format and the nature of the tasks.

Variability of theory of mind versus pragmatic ability in typical and atypical development

IntroductionNumerous studies have linked deficits in Theory of Mind (ToM) with language problems. We aimed to explore the similarities and differences between children's patterns of performance on a ToM task that requires minimal linguistic skill and a pragmatic inference task that relies on both ToM and language. We assessed variability in pragmatic inference skills and ToM across populations of children (8–14 years) displaying varying cognitive profiles. We further compared the sensitivity of ToM versus pragmatic ability to core language skills, memory and executive functioning (EF). MethodToM was tested using the Social Attribution Task (SAT-MC-II). Pragmatic ability was assessed in an implicature comprehension task. Receptive vocabulary, grammar comprehension, short-term and working memory (STM and WM) capacity and EF were measured using Hungarian adaptations of standard tasks and tests developed by the authors’ lab. In addition to typically developing (TD) children (n = 33), we included children with neurodevelopmental disorders where ToM and/or language abilities are vulnerable: autism spectrum disorder (ASD, n = 26), attention deficit hyperactivity disorder (ADHD, n = 25) and developmental language disorder (DLD, n = 18). ResultsResults revealed a significant but only moderate positive correlation between pragmatic inference and ToM indicating that the two abilities are related but distinct. The ASD group showed impairments in both ToM and pragmatic inference ability but no significant deficit was observed in ADHD or DLD relative to TD children in either skill. However, while SAT-MC-II results were only affected by verbal WM and vocabulary measures, pragmatic performance was associated with STM, verbal WM, EF, grammatical skills and vocabulary. ConclusionOur findings indicate that disentangling the contributions of different cognitive skills to ToM tasks may help clarify the role of ToM in language skills and identify distinct patterns of ToM and pragmatic skills in developmental disorders.

Lower confidence and increased error sensitivity in OCD patients while learning under volatility

A decoupling between confidence and action could relate to compulsive behaviour as seen in obsessive-compulsive disorder (OCD). The link between confidence and action in OCD has been investigated in clinical case-control studies and in the general population with discrepant findings. The generalizability of findings from highly-compulsive general population samples to clinical OCD samples has been questioned. Here, we investigate action-confidence coupling for 38 OCD patients compared to 37 healthy controls (HC), using a predictive inference task. We compared those results to a comparison between matched high and low compulsive individuals from the general population. Action-updating, confidence and their coupling were compared between the groups. Moreover, computational modeling was performed to compare groups on error sensitivity and environmental parameters. OCD patients showed lower confidence and higher learning rates in reaction to (small) prediction errors than HC, signaling hyperactive error signaling and lower confidence estimation. No evidence was found for differences in action-confidence coupling between groups. In contrast high the compulsive group showed higher confidence and stronger decoupling than the low compulsive group, both of which were related to symptoms. The underlying mechanisms of obsessive-compulsive behaviour might differ between clinical and highly-compulsive general population samples, resulting in different (meta)cognitive profiles.