Computational Model Research Articles

The Effect of Reward Magnitude on Different Types of Exploration in Human Reinforcement Learning

AbstractHow humans resolve the explore–exploit dilemma in complex environments is an important open question. Previous studies suggested that environmental richness may affect the degree of exploration in a type-specific manner and reduce random exploration while increasing uncertainty-based exploration. Our study examined this possibility by extending a recently developed two-armed bandit task that can dissociate the uncertainty and novelty of stimuli. To extract the pure effect of environmental richness, we manipulated the reward by its magnitude, not its probability, across blocks because reward probability affects outcome controllability. Participants (N = 198) demonstrated increased optimal choices when the relative reward magnitude was higher. A behavioral analysis with computational modeling revealed that a higher reward magnitude reduced the degree of random exploration but had little effect on the degree of uncertainty- and novelty-based exploration. These results suggest that humans modulate their degree of random exploration depending on the relative level of environmental richness. Combined with findings from previous studies, our findings indicate the possibility that outcome controllability also influences the exploration–exploitation balance in human reinforcement learning.

Design of optical performance for self-luminous pavement materials

The lower visibility in the nighttime environment makes higher accident risks during driving. Using self-luminous materials in pavement construction can effectively improve the luminance of load surface, thus significantly improve nighttime driving safety. This study has designed three types of self-luminous pavement materials (SLPM) with different optical properties by incorporating long-persistent luminescent aggregates into the mixture and using polymer binders with varying degrees of light transmittance, such as transparent polyurethane, transparent asphalt, and traditional asphalt. On this basis, utilizing digital image processing techniques to compare the optical performance differences among these three SLPM and established corresponding computational models based on the obtained data to enable efficient and accurate evaluation of the optical properties of pavement materials. The experimental results indicate that the durability performance of SLPM gradually diminishes over time, and this degree of attenuation is negatively correlated with the light transmittance of the binder. Specifically, a higher light transmittance of the binder results in a smaller decline in SLPM durability, whereas a lower transmittance leads to a more significant reduction. The optical performance calculation model established by this research can predict the reflective performance of SLPM through the optical parameters of aggregate, binders, and the doping amount of long persistent luminescent materials. In addition, the model can accurately calculate the minimum content of self-luminous materials according to the optical requirements required by the project, which can provide a reference for the performance evaluation of self-luminous pavement materials and the selection of optimal proportioning scheme.

Lumbar spine marker placement errors and soft tissue artifact during dynamic flexion/extension and lateral bending in individuals with chronic low back pain

Quantitative in vivo biomechanical assessments are typically performed with optoelectronic motion capture (MoCap) using retroreflective markers attached to skin. This technique inherently contains measurement errors from both marker placement on palpated bony landmarks and skin motion relative to the underlying bone (i.e., soft tissue artifact (STA)). Research on lumbar spine STA is scarce and limited to young, healthy participants in static positions. This study aimed to evaluate static placement errors, lumbar spine STA from MoCap marker clusters (MMC), and linear relationships between STA and patient characteristics. Thirty-nine participants with cLBP performed three trials each of flexion/extension and lateral bending while imaged simultaneously by MoCap (120 Hz) and dynamic biplane radiography DBR (20 Hz). MMCs were placed 29.5 ± 18.0 mm and 27.1 ± 13.4 mm superior to the most prominent aspect of the L1 and L5 spinous process, respectively. L1 relative to L5 STA was larger during flexion/extension (8.6 ± 5.7°) than lateral bending (4.5 ± 2.1°) (p < 0.001). After correcting for marker placement errors, components of the L1 and L5 STA averaged as much as 16.3 mm and 11.4° during flexion/extension, but only 4.0 mm and 4.8° or less during lateral bending. On average, STA for individual L1 and L5 vertebrae increased as participants moved away from the upright neutral position. STA was participant-dependent, however, age and BMI did not model STA well. Given the inaccuracy in marker placement and wide range of patterns of STA, caution is urged when making clinical decisions or when using computational models to estimate spine tissue loading based upon lumbar spine kinematics obtained from skin-mounted markers.

Influenza A virus within-host evolution and positive selection in a densely sampled household cohort over three seasons

Abstract While influenza A virus (IAV) antigenic drift has been documented globally, in experimental animal infections, and in immunocompromised hosts, positive selection has generally not been detected in acute infections. This is likely due to challenges in distinguishing selected rare mutations from sequencing error, a reliance on cross-sectional sampling, and/or the lack of formal tests of selection for individual sites. Here, we sequenced IAV populations from 346 serial, daily nasal swabs from 143 individuals collected over three influenza seasons in a household cohort. Viruses were sequenced in duplicate, and intrahost single nucleotide variants (iSNV) were identified at a 0.5% frequency threshold. Within-host populations exhibited low diversity, with &amp;gt;75% mutations present at &amp;lt;2% frequency. Children (0-5 years) had marginally higher within-host evolutionary rates than adolescents (6-18 years) and adults (&amp;gt;18 years, 4.4x10-6 vs. 9.42x10-7 and 3.45x10-6, p &amp;lt;0.001). Forty-five iSNV had evidence of parallel evolution, but were not overrepresented in HA and NA. Several increased from minority to consensus level, with strong linkage among iSNV across segments. A Wright Fisher Approximate Bayesian Computational model identified positive selection at 23/256 loci (9%) in A(H3N2) specimens and 19/176 loci (11%) in A(H1N1)pdm09 specimens, and these were infrequently found in circulation. Overall, we found that within-host IAV populations were subject to genetic drift and purifying selection, with only subtle differences across seasons, subtypes, and age strata. Positive selection was rare and inconsistently detected.

From computational models of the splicing code to regulatory mechanisms and therapeutic implications.

Since the discovery of RNA splicing and its role in gene expression, researchers have sought a set of rules, an algorithm or a computational model that could predict the splice isoforms, and their frequencies, produced from any transcribed gene in a specific cellular context. Over the past 30 years, these models have evolved from simple position weight matrices to deep-learning models capable of integrating sequence data across vast genomic distances. Most recently, new model architectures are moving the field closer to context-specific alternative splicing predictions, and advances in sequencing technologies are expanding the type of data that can be used to inform and interpret such models. Together, these developments are driving improved understanding of splicing regulatory mechanisms and emerging applications of the splicing code to the rational design of RNA- and splicing-based therapeutics.

Gradient-enhanced deep Gaussian processes for multifidelity modeling

Multifidelity models integrate data from multiple sources to produce a single approximator for the underlying process. Dense low-fidelity samples are used to reduce interpolation error, while sparse high-fidelity samples are used to compensate for bias or noise in the low-fidelity samples. Deep Gaussian processes (GPs) are attractive for multifidelity modeling as they are non-parametric, robust to overfitting, perform well for small datasets, and, critically, can capture nonlinear and input-dependent relationships between data of different fidelities. Many datasets naturally contain gradient data, most commonly when they are generated by computational models that have adjoint solutions or are built in automatic differentiation frameworks.Principally, this work extends deep GPs to incorporate gradient data. We demonstrate this method on an analytical test problem and two realistic aerospace problems: one focusing on a hypersonic waverider with an inviscid gas dynamics truth model and another focusing on the canonical ONERA M6 wing with a viscous Reynolds-averaged Navier-Stokes truth model.In both examples, the gradient-enhanced deep GP outperforms a gradient-enhanced linear GP model and their non-gradient-enhanced counterparts.

A Drosophila computational brain model reveals sensorimotor processing

The recent assembly of the adult Drosophila melanogaster central brain connectome, containing more than 125,000 neurons and 50 million synaptic connections, provides a template for examining sensory processing throughout the brain1,2. Here we create a leaky integrate-and-fire computational model of the entire Drosophila brain, on the basis of neural connectivity and neurotransmitter identity3, to study circuit properties of feeding and grooming behaviours. We show that activation of sugar-sensing or water-sensing gustatory neurons in the computational model accurately predicts neurons that respond to tastes and are required for feeding initiation4. In addition, using the model to activate neurons in the feeding region of the Drosophila brain predicts those that elicit motor neuron firing5—a testable hypothesis that we validate by optogenetic activation and behavioural studies. Activating different classes of gustatory neurons in the model makes accurate predictions of how several taste modalities interact, providing circuit-level insight into aversive and appetitive taste processing. Additionally, we applied this model to mechanosensory circuits and found that computational activation of mechanosensory neurons predicts activation of a small set of neurons comprising the antennal grooming circuit, and accurately describes the circuit response upon activation of different mechanosensory subtypes6–10. Our results demonstrate that modelling brain circuits using only synapse-level connectivity and predicted neurotransmitter identity generates experimentally testable hypotheses and can describe complete sensorimotor transformations.

Neural Encoding of Bodies for Primate Social Perception.

Primates, as social beings, have evolved complex brain mechanisms to navigate intricate social environments. This review explores the neural bases of body perception in both human and nonhuman primates, emphasizing the processing of social signals conveyed by body postures, movements, and interactions. Early studies identified selective neural responses to body stimuli in macaques, particularly within and ventral to the superior temporal sulcus (STS). These regions, known as body patches, represent visual features that are present in bodies but do not appear to be semantic body detectors. They provide information about posture and viewpoint of the body. Recent research using dynamic stimuli has expanded the understanding of the body-selective network, highlighting its complexity and the interplay between static and dynamic processing. In humans, body-selective areas such as the extrastriate body area (EBA) and fusiform body area (FBA) have been implicated in the perception of bodies and their interactions. Moreover, studies on social interactions reveal that regions in the human STS are also tuned to the perception of dyadic interactions, suggesting a specialized social lateral pathway. Computational work developed models of body recognition and social interaction, providing insights into the underlying neural mechanisms. Despite advances, significant gaps remain in understanding the neural mechanisms of body perception and social interaction. Overall, this review underscores the importance of integrating findings across species to comprehensively understand the neural foundations of body perception and the interaction between computational modeling and neural recording.

MicroRNAs provide negative feedback and stability in gene regulatory network models of cell-state transitions

The development of multicellular organisms occurs through a series of cell state transitions controlled by gene regulatory networks. Central to these networks are transcription factors (TFs) which bind enhancers and activate the expression of other genes, some of which are also TFs. Gene regulatory networks (GRN) connect TFs and enhancers in a nonlinear circuit capable of producing complex behavior such as bifurcations between stable cell states. Our dynamic network modelling of the Embryonic Stem Cell (ESC) to Definitive Endoderm (DE) transition requires an as yet unknown negative feedback mechanism for stability. Here, we show that cell state specific microRNAs (miRNAs) can provide this negative feedback by inactivating other cell lineage determining TFs (ESC or DE) during the transition. Our model provides a mechanism to maintain stable cell states without requiring a large set of cell-type-specific repressive TFs, of which there are fewer known examples than activators. In support of this model, we use computational models and analyze gene and miRNA expression and chromatin accessibility data from human cell lines to detect enhancers activating the miRNAs consistent with our network model. Our analysis highlights the interplay between TFs and miRNAs during ESC to DE transition and proposes a novel model for gene regulation.

Neuron-Glial Interactions: Implications for Plasticity, Behavior, and Cognition.

The traditional view of glial cells as mere supportive tissue has shifted, due to advances in technology and theoretical conceptualization, to include a diversity of other functions, such as regulation of complex behaviors. Astrocytes, the most abundant glial cells in the central nervous system (CNS), have been shown to modulate synaptic functions through gliotransmitter-mediated neurotransmitter reuptake, influencing neuronal signaling and behavioral functions. Contemporary studies further highlight astrocytes' involvement in complex cognitive functions. For instance, inhibiting astrocytes in the hippocampus can lead to memory deficits, suggesting their integral role in memory processes. Moreover, astrocytic calcium activity and astrocyte-neuron metabolic coupling have been linked to changes in synaptic strength and learning. Microglia, another type of glial cell, also extend beyond their supportive roles, contributing to learning and memory processes, with microglial reductions impacting these functions in a developmentally dependent manner. Oligodendrocytes, traditionally thought to have limited roles postdevelopment, are now recognized for their activity-dependent modulation of myelination and plasticity, thus influencing behavioral responses. Recent advancements in technology and computational modeling have expanded our understanding of glial functions, particularly how astrocytes influence neuronal circuits and behaviors. This review underscores the importance of glial cells in CNS functions and the need for further research to unravel the complexities of neuron-glia interactions, the impact of these interactions on brain functions, and potential implications for neurological diseases.

Hydrogen enrichment in methanol SI engine at varying injection timing during compression stroke

This study investigates the effect of hydrogen enrichment in a direct-injected methanol-fuelled SI engine, with combustion and emission performance analysed by varying the methanol injection timing from 150° to 60° CA bTDC. A three-dimensional computational fluid dynamic model of the hydrogen-enriched methanol engine was developed to predict the engine performance, in-cylinder, and emission characteristics. The numerical model was solved using three-dimensional Reynolds-Averaged Navier Stokes, the RNG k-epsilon model for turbulence, the O'Rourke and Amsden sub-model for heat transfer, the extended Zeldovich mechanism for nitric oxide emissions, and the Hiroyasu-NSC model for soot emissions. The results indicated that retarding the injection timing resulted in a decrease in the indicated specific CO and soot emissions along with a rise in the indicated specific NOx emissions. Furthermore, hydrogen enrichment with methanol enhanced the hydroxyl radical concentration, reduced the combustion duration, reduced also the indicated specific CO and soot emissions while increasing the indicated specific NOx emissions. The study indicates that hydrogen enrichment could extend the late injection timing limit of methanol by enhancing fuel-air mixing, which improves and controls the combustion process more effectively.

De novo design of mini-protein binders broadly neutralizing Clostridioides difficile toxin B variants

Clostridioides difficile toxin B (TcdB) is the key virulence factor accounting for C. difficile infection-associated symptoms. Effectively neutralizing different TcdB variants with a universal solution poses a significant challenge. Here we present the de novo design and characterization of pan-specific mini-protein binders against major TcdB subtypes. Our design successfully binds to the first receptor binding interface (RBI-1) of the varied TcdB subtypes, exhibiting affinities ranging from 20 pM to 10 nM. The cryo-electron microscopy (cryo-EM) structures of the mini protein binder in complex with TcdB1 and TcdB4 are consistent with the computational design models. The engineered and evolved variants of the mini-protein binder and chondroitin sulfate proteoglycan 4 (CSPG4), another natural receptor that binds to the second RBI (RBI-2) of TcdB, better neutralize major TcdB variants both in cells and in vivo, as demonstrated by the colon-loop assay using female mice. Our findings provide valuable starting points for the development of therapeutics targeting C. difficile infections (CDI).

Device simulation study of multilayer MoS2Schottky barrier field-effect transistors.

Molybdenum disulfide (MoS2) is a representative two-dimensional layered transition-metal dichalcogenide semiconductor. Layer-number-dependent electronic properties are attractive in the development of nanomaterial-based electronics for a wide range of applications including sensors, switches, and amplifiers. MoS2field-effect transistors (FETs) have been studied as promising future nanoelectronic devices with desirable features of atomic-level thickness and high electrical properties. When a naturally n-doped MoS2is contacted with metals, a strong Fermi-level pinning effect adjusts a Schottky barrier and influences its electronic characteristics significantly. In this study, we investigate multilayer MoS2Schottky barrier FETs (SBFETs), emphasizing the metal-contact impact on device performance via computational device modeling. We find that p-type MoS2 SBFETs may be built with appropriate metals and gate voltage control. Furthermore, we propose ambipolar multilayer MoS2 SBFETs with asymmetric metal electrodes, which exhibit gate-voltage dependent ambipolar transport behavior through optimizing metal contacts in MoS2 device. Introducing a dual-split gate geometry, the MoS2SBFETs can further operate in four distinct configurations: p - p, n - n, p - n, and n - p. Electrical characteristics are calculated, and improved performance of a high rectification ratio can be feasible as an attractive feature for efficient electrical and photonic devices.

Intelligence analysis of membrane distillation via machine learning models for pharmaceutical separation

This study investigates simulation of pharmaceutical separation via membrane distillation process by computational simulation and machine learning modeling strategy. The efficacy of three regression models, i.e., Multi-layer Perceptron (MLP), Gamma Regression, and Support Vector Regression (SVR) in predicting the solute concentration, C(mol/m³), was evaluated. The hyper-parameters were optimized by fine-tuning the models using the Red Deer Algorithm (RDA). Computational analyses were carried out for removal of pharmaceuticals from solution by membrane distillation in continuous mode. Mass transfer and machine learning models were implemented focusing on concentration of solute in the feed section of membrane. Results indicate that the Multi-layer Perceptron model achieved great accuracy with an R2 of 0.9955, an MAE of 0.0084, and an RMSE of 0.0148, effectively capturing complex nonlinear relationships in the data. Gamma Regression also performed acceptably, with fitting R2 of 0.9214, showing its suitability for positively skewed data. The Support Vector Regression model, while capturing the general trend, showed the lowest performance with an R2 of 0.8710. These findings suggest that the Multi-layer Perceptron is the most accurate model for this dataset, followed by Gamma Regression and Support Vector Regression. This underscores the importance of careful model selection and optimization in regression analysis in combination with computational simulation of membrane processes.

Molecular docking analysis and in vivo assessment of zinc oxide nanoparticle toxicity in zebrafish larvae

The zinc oxide nanoparticles (ZnO-NPs) being widely employed in several industries and consumer products, are raising concerns about their safety on aquatic biota and human health. This study aims to investigate the possible toxicological effects of ZnO-NPs through a combined in vivo and in silico approach. Zebrafish embryos were exposed to several ZnO-NPs concentrations and morphological alterations and lipid peroxidation (MDA) were investigated. Furthermore, molecular docking simulations were applied to study the intermolecular interactions of ZnO-NPs against critical embryonic proteins namely zebrafish hatching enzyme1 (ZHE1) as well as the superoxide dismutase (SOD1). Treatment with ZnO-NPs resulted in an increase in MDA concentration and a decrease in antioxidant enzyme levels. Besides a significant decrease in mRNA expression of key enzymes of ROS detoxification genes, a modulation of inflammatory genes with a low downregulation of tnf-α, and an upregulation of il-1β were observed. Docking study suggests that the delayed hatching and increased cellular oxidative stress in zebrafish embryos may occur through a synergistic mechanism based on the ZnO-NP—dependent inhibition of ZHE1 and SOD1 enzymes. The integration of in vivo assessments with in silico computational modeling provided a more comprehensive evaluation of potential physiological risks in zebrafish embryos associated with nanomaterial exposure.

Challenges and Approaches in the Study of Neural Entrainment.

When exposed to rhythmic stimulation, the human brain displays rhythmic activity across sensory modalities and regions. Given the ubiquity of this phenomenon, how sensory rhythms are transformed into neural rhythms remains surprisingly inconclusive. An influential model posits that endogenous oscillations entrain to external rhythms, thereby encoding environmental dynamics and shaping perception. However, research on neural entrainment faces multiple challenges, from ambiguous definitions to methodological difficulties when endogenous oscillations need to be identified and disentangled from other stimulus-related mechanisms that can lead to similar phase-locked responses. Yet, recent years have seen novel approaches to overcome these challenges, including computational modeling, insights from dynamical systems theory, sophisticated stimulus designs, and study of neuropsychological impairments. This review outlines key challenges in neural entrainment research, delineates state-of-the-art approaches, and integrates findings from human and animal neurophysiology to provide a broad perspective on the usefulness, validity, and constraints of oscillatory models in brain-environment interaction.

Towards a computational model for higher orders of Theory of Mind in social agents

Effective communication between humans and machines requires artificial tools to adopt a human-like social perspective. The Theory of Mind (ToM) enables understanding and predicting mental states and behaviours, crucial for social interactions from childhood through adulthood. Artificial agents with ToM skills can better coordinate actions, such as in warehouses or healthcare. Incorporating ToM in AI systems can revolutionise our interactions with intelligent machines. This proposal emphasises the current focus on first-order ToM models in the literature and investigates the potential of creating a computational model for higher-order ToM.

Barrier-free, open-top microfluidic chip for generating two distinct, interconnected 3D microvascular networks

Developing microphysiological cell culture platforms with a three-dimensional (3D) microenvironment has been a significant advancement from traditional monolayer cultures. Still, most of the current microphysiological platforms are limited in closed designs, i.e. are not accessible after 3D cell culture loading. Here, we report an open-top microfluidic chip which enables the generation of two sequentially loaded 3D cell cultures without physical barriers restricting the nurture, gas exchange and cellular communication. As a proof-of-concept, we demonstrated the formation of two 3D vasculatures, one in the upper and the other in the lower compartment, under three distinct flow conditions: asymmetric side-to-center, symmetric side-to-center and symmetric center-to-side. We used computational modelling to characterize initial flow pressures in cell culture compartments. We showed prominent vessel formation and branched vasculatures in upper and lower cell culture compartments with interconnecting, lumenized vessels with in vivo-relevant diameter in all flow conditions. With advanced image processing, we quantified and compared the overall vascular network volume and the total length formed in asymmetric side-to-center, symmetric side-to-center and symmetric center-to-side flow conditions. Our results indicate that the developed chip can house two distinct 3D cell cultures with merging vessels between compartments and by providing asymmetric side-to-center or symmetric center-to-side flow vascular morphogenesis is enhanced in terms of overall network length. The developed open-top microfluidic chip may find various applications in generation of tissue-specific 3D-3D co-cultures for studying cellular interactions in vascularized tissues and organs.

Selective maintenance of the complex system considering maintenance time uncertainty for system components with multiple repairpersons

AbstractThis research presents an innovative selected maintenance model for complex systems that considers the uncertainty in maintenance time (MT) for system components with multiple repairpersons. The computational model of uncertain MT for system components is established. An imperfect maintenance model is introduced, which has many imperfect maintenance levels not only considering do nothing, minimal repair, and replacement but also considering multiple intermediate maintenance levels. Furthermore, the system components maintenance assignment algorithm with multiple repairpersons is proposed to addresses the problem of how to assign multiple maintenance tasks to multiple repairpersons in order to minimize system MT. And it is innovatively integrated into the particle swarm optimization (PSO) to solve the proposed selective maintenance model, which enables heuristic algorithm to efficiently assign the multiple maintenance tasks with multiple repairpersons. The effectiveness and advantages of the proposed model and algorithm are verified by numerical experiments.

Raloxifene Prevents Chemically-Induced Ferroptotic Neuronal Death In Vitro and In Vivo.

Ferroptosis, a regulated form of cell death characterized by excessive iron-dependent lipid peroxidation, can be readily induced in cultured cells by chemicals such as erastin and RSL3. Protein disulfide isomerase (PDI) has been identified as an upstream mediator of chemically induced ferroptosis and also a target for ferroptosis protection. In this study, we discovered that raloxifene (RAL), a selective estrogen receptor modulator known for its neuroprotective actions in humans, can effectively inhibit PDI function and provide robust protection against chemically induced ferroptosis in cultured HT22 neuronal cells. Specifically, RAL can bind directly to PDI both in vitro and in intact neuronal cells and inhibit its catalytic activity. Computational modeling analysis reveals that RAL can tightly bind to PDI through forming a hydrogen bond with its His256 residue, and biochemical analysis further shows that when PDI's His256 is mutated to Ala256, RAL loses its inhibition of PDI's catalytic activity. This inhibition of PDI by RAL significantly reduces the dimerization of both the inducible and neuronal nitric oxide synthases and the accumulation of nitric oxide, both of which have recently been shown to play a crucial role in mediating chemically induced ferroptosis through subsequent induction of ROS and lipid-ROS accumulation. In vivo behavioral analysis shows that mice treated with RAL are strongly protected against kainic acid-induced memory deficits and hippocampal neuronal damage. In conclusion, this study demonstrates that RAL is a potent inhibitor of PDI and can effectively prevent chemically induced ferroptosis in hippocampal neurons both in vitro and in vivo. These findings offer a novel estrogen receptor-independent mechanism for RAL's neuroprotective actions in animal models and humans.