e16525 Background: Immuno-oncology (IO) initiated a major shift in the cancer treatment paradigm, as therapies no longer target cancer cells directly but rather restore and exploit a patient’s own antitumor immunity. However at present, there are no conclusive diagnostic tools capable of predicting combination immunotherapy response with high accuracy in renal cancer. We’ve developed an ex vivo IO model and multivariate analysis to predict clinical drug efficacy by combining patient tumor samples, functional assays, artificial intelligence and omics. This model recapitulates each patient’s tumor-immune microenvironment (TiME) and allows time-course bulk and single-cell resolution analyses of functional metrics in 3D. Methods: Human renal tumor resections and matched blood (N=20 REC approved) were processed for spatial transcriptomics and single cell isolation of tumor and PBMCs. Tumor-dissociated (tumor, stromal, immune, etc.) cells and PBMC subsets were characterized by flow cytometry (FCm). Target cells (tumor) and effector cells (PBMCs and subsets thereof, CD8+) were stained with different fluorescent dyes including viability probes (caspase 3/7, SYTOX) and encapsulated in hydrogels that recapitulate human TiME physiology. Cultures were treated with approved regimens including immune checkpoint (ipilimumab, pembrolizumab) and receptor-tyrosine kinase (TKI)(cabozantinib, lenvatinib) inhibitors. Cells were tracked over 7 days using 3D time-course confocal microscopy. Computer vision analysis detected and quantified behaviors such as immune infiltration, immune/tumor cell migration, T cell-mediated tumor killing and tumor viability in response to treatments. Results: Our preliminary data shows that treatment with pembrolizumab resulted in a 26% increase in the infiltration of CD8- PBMC fraction into the microtumor core and a 14% increase in infiltration of the CD8+ subset. Tumor cell death was 15% higher in pembro-treated samples compared to untreated and 30% higher compared to tumor cultures alone (no PBMCs). Migration speed of immune cells was found to be higher as cells invaded and slower through engagement/killing, peaking at day 1 (3 µm/min) and slowing to 2 and 1.5 µm/min for CD8+ and CD8- cells by day 3. FCm was used to characterize tumor cell subpopulations (cancer, endothelial, immune) and the expression of targets in each patient. Treatment with TKIs led to reduced phosphorylation of VEGFR, PDGFRβ and HGFR (N=3). Conclusions: Our platform allows time-course analysis of functional cell response metrics. The model tests treatment combinations across multiple modes of action and quantifies cell response including viability, death, migration, infiltration and immune-surveillance. Future work aims to further develop the platform to match/predict patient responses in breast (NCT05435352), kidney, and other tumors.

Memorising vocabulary is an important aspect of formal foreign-language learning. Advances in cognitive psychology have led to the development of adaptive learning systems that make vocabulary learning more efficient. One way these computer-based systems optimize learning is by measuring learning performance in real time to create optimal repetition schedules for individual learners. While such adaptive learning systems have been successfully applied to word learning using keyboard-based input, they have thus far seen little application in word learning where spoken instead of typed input is used. Here we present a framework for speech-based word learning using an adaptive model that was developed for and tested with typing-based word learning. We show that typing- and speech-based learning result in similar behavioral patterns that can be used to reliably estimate individual memory processes. We extend earlier findings demonstrating that a response-time based adaptive learning approach outperforms an accuracy-based, Leitner flashcard approach in learning efficiency (demonstrated by higher average accuracy and lower response times after a learning session). In short, we show that adaptive learning benefits transfer from typing-based learning, to speech based learning. Our work provides a basis for the development of language learning applications that use real-time pronunciation assessment software to score the accuracy of the learner’s pronunciations. We discuss the implications for our approach for the development of educationally relevant, adaptive speech-based learning applications.

Changes of Mind are a striking example of our ability to flexibly reverse decisions and change our own actions. Previous studies largely focused on Changes of Mind in decisions about perceptual information. Here we report reversals of decisions that require integrating multiple classes of information: 1) Perceptual evidence, 2) higher-order, voluntary intentions, and 3) motor costs. In an adapted version of the random-dot motion task, participants moved to a target that matched both the external (exogenous) evidence about dot-motion direction and a preceding internally-generated (endogenous) intention about which colour to paint the dots. Movement trajectories revealed whether and when participants changed their mind about the dot-motion direction, or additionally changed their mind about which colour to choose. Our results show that decision reversals about colour intentions are less frequent in participants with stronger intentions (Exp. 1) and when motor costs of intention pursuit are lower (Exp. 2). We further show that these findings can be explained by a hierarchical, multimodal Attractor Network Model that continuously integrates higher-order voluntary intentions with perceptual evidence and motor costs. Our model thus provides a unifying framework in which voluntary actions emerge from a dynamic combination of internal action tendencies and external environmental factors, each of which can be subject to Change of Mind.

Imbuing robots with personality has been shown to be an effective design approach in HRI, promoting user trust and acceptance. We explore personality design in a non-anthropomorphic voice-assisted home robot. Our design approach developed three distinct robot personas: Butler, Buddy, and Sidekick, intended to differ in proactivity and emotional impact. Persona differences were signaled to users by a combination of humanoid (speech, intonation), and indirect cues (colors and movement). We use Big Five personality theory to evaluate perceived differences between personas in an exploratory Wizard of Oz study. Participants were largely able to recognize underlying personality traits expressed through these cue combinations in ways that were consistent with our design goals. The proactive Buddy persona was judged as more Extravert than the more passive Sidekick persona, and the Butler persona was perceived as more Conscientious and less Neurotic than either Buddy or Butler personas. Users also had clear preferences between different personas; they wanted robots that mimicked but accentuated their own personality. Results suggest that future designs might exploit abstract cues to signal personality traits.

AbstractFunctional recovery after brain damage varies widely and depends on many factors, including lesion site and extent. When a neuronal system is damaged, recovery may occur by engaging residual (e.g., perilesional) components. When damage is extensive, recovery depends on the availability of other intact neural structures that can reproduce the same functional output (i.e., degeneracy). A system’s response to damage may occur rapidly, require learning or both. Here, we simulate functional recovery from four different types of lesions, using a generative model of word repetition that comprised a default premorbid system and a less used alternative system. The synthetic lesions (i) completely disengaged the premorbid system, leaving the alternative system intact, (ii) partially damaged both premorbid and alternative systems, and (iii) limited the experience-dependent plasticity of both. The results, across 1000 trials, demonstrate that (i) a complete disconnection of the premorbid system naturally invoked the engagement of the other, (ii) incomplete damage to both systems had a much more devastating long-term effect on model performance and (iii) the effect of reducing learning capacity within each system. These findings contribute to formal frameworks for interpreting the effect of different types of lesions.

Blockchain is a revolutionary technology due to its security and transparency . A digital identity system based on blockchain technology can play an important role in securing the asset ownership information and protecting it against any discrepancy. However, there has been no work that combines the advantages of blockchain technology and an individual’s biometric information in creating a digital identity that can be used for ownership system, thereby preventing any fraud in ownership system. In this paper, a Blockchain-based Ownership and Value Added Tax (VAT) management system have been proposed depending on a digital identity system that is implemented on blockchain technology using individuals’ biometric information. The Ownership system along with VAT management was implemented using Ethereum smart contract. The experimental results of testing our system indicate that an intruder cannot perform any modification illegally to ownership data in the system. Any adversarial attempt is aborted instantly and thereby the security of citizen’s ownership information is ensured. Moreover, appropriate amount of VAT is automatically assigned to the owner while changing the ownership of suitable products. Our proposed system can be very useful for protecting the digital ownership information of citizens of a country and ensuring fair payment of VAT for suitable products.

Simulation of population dynamics is a central research theme in computational biology, which contributes to understanding the interactions between predators and preys. Conventional mathematical tools of this theme, however, are incapable of accounting for several important attributes of such systems, such as the intelligent and adaptive behavior exhibited by individual agents. This unrealistic setting is often insufficient to simulate properties of population dynamics found in the real-world. In this work, we leverage multi-agent deep reinforcement learning, and we propose a new model of large-scale predator-prey ecosystems. Using different variants of our proposed environment, we show that multi-agent simulations can exhibit key real-world dynamical properties. To obtain this behavior, we firstly define a mating mechanism such that existing agents reproduce new individuals bound by the conditions of the environment. Furthermore, we incorporate a real-time evolutionary algorithm and show that reinforcement learning enhances the evolution of the agents' physical properties such as speed, attack and resilience against attacks.

AbstractHuman perception and experience of time is strongly influenced by ongoing stimulation, memory of past experiences, and required task context. When paying attention to time, time experience seems to expand; when distracted, it seems to contract. When considering time based on memory, the experience may be different than in the moment, exemplified by sayings like “time flies when you’re having fun”. Experience of time also depends on the content of perceptual experience – rapidly changing or complex perceptual scenes seem longer in duration than less dynamic ones. The complexity of interactions between attention, memory, and perceptual stimulation is a likely reason that an overarching theory of time perception has been difficult to achieve. Here, we introduce a model of perceptual processing and episodic memory that makes use of hierarchical predictive coding, short-term plasticity, spatio-temporal attention, and episodic memory formation and recall, and apply this model to the problem of human time perception. In an experiment with ~ 13, 000 human participants we investigated the effects of memory, cognitive load, and stimulus content on duration reports of dynamic natural scenes up to ~ 1 minute long. Using our model to generate duration estimates, we compared human and model performance. Model-based estimates replicated key qualitative biases, including differences by cognitive load (attention), scene type (stimulation), and whether the judgement was made based on current or remembered experience (memory). Our work provides a comprehensive model of human time perception and a foundation for exploring the computational basis of episodic memory within a hierarchical predictive coding framework.Author summaryExperience of the duration of present or past events is a central aspect of human experience, the underlying mechanisms of which are not yet fully understood. In this work, we combine insights from machine learning and neuroscience to propose a combination of mathematical models that replicate human perceptual processing, long-term memory, attention, and duration perception. Our computational implementation of this framework can process information from video clips of ordinary life scenes, record and recall important events, and report the duration of these clips. To assess the validity of our proposal, we conducted an experiment with ~ 13, 000 human participants. Each was shown a video between 1-64 seconds long and reported how long they believed it was. Reports of duration by our computational model qualitatively matched these human reports, made about the exact same videos. This was true regardless of the video content, whether time was actively judged or based on memory of the video, or whether the participants focused on a single task or were distracted - all factors known to influence human time perception. Our work provides the first model of human duration perception to incorporate these diverse and complex factors and provides a basis to probe the deep links between memory and time in human experience.

We propose a novel neural network architecture, named the Global Workspace Network (GWN), which addresses the challenge of dynamic and unspecified uncertainties in multimodal data fusion. Our GWN is a model of attention across modalities and evolving through time, and is inspired by the well-established Global Workspace Theory from the field of cognitive science. The GWN achieved average F1 score of 0.92 for discrimination between pain patients and healthy participants and average F1 score = 0.75 for further classification of three pain levels for a patient, both based on the multimodal EmoPain dataset captured from people with chronic pain and healthy people performing different types of exercise movements in unconstrained settings. In these tasks, the GWN significantly outperforms the typical fusion approach of merging by concatenation. We further provide extensive analysis of the behaviour of the GWN and its ability to address uncertainties (hidden noise) in multimodal data.