Dynamic Learning Model Research Articles

The Contribution of R&D İnvestments to Technological Learning Capability: Evidence from Manufacturing İndustry Sectors of the Turkish Economy

Learning-by-searching is the process of experience in the emergence of new high-value-added products with the new knowledge and skills that an enterprise achieves through the network system it establishes with its own R&D unit, other enterprises, universities, and other information sources. The most important factor that contributes to learning-by-searching is undoubtedly the investments made in research and development in economic sectors. A review of the literature on the subject reveals that R&D expenditures are only used as the second independent variable in linear learning curve models. For this reason, in the existing applied studies, only a fixed representative learning rate, which is the average of the learning rates in the period in question, is determined. In this study, a two-factor dynamic learning curve model is used to measure the cost-reducing effect of R&D expenditures. Thus, in addition to learning-by-doing ratios, the evolution of learning-by-searching ratios over time is also taken into account. The findings of the study support the view in the literature that the learning-by-doing rates in the one-factor dynamic learning model are biased. In addition, in the two-factor dynamic learning curve model, it was also found that in periods when learning-by-doing loses its effect, learning-by-searching increases its effect.

Application of deep learning in cloud cover prediction using geostationary satellite images

ABSTRACT Predicting cloud cover is essential in fields, such as agriculture, climatology, and meteorology, where accurate weather forecasting can significantly impact decision-making. Traditional methods for cloud cover prediction encounter significant limitations in capturing complete spatial and temporal cloud dynamics. To address the problem, this study employs high-resolution data from the new-generation geostationary satellite Geostationary Korea Multi-Purpose Satellite 2A (GEO-KOMPSAT-2A; GK2A) and enables more accurate and timely predictions when combined with advanced deep learning techniques. We explore the effectiveness of advanced deep learning techniques – specifically 3D Convolutional Neural Networks, Long Short-Term Memory networks, and Convolutional Long Short-Term Memory (ConvLSTM) – using GK2A cloud detection data, which provides updates every 10 minutes at 2 km spatial resolution. Our model utilizes training sequences of four past hourly images to predict cloud cover up to 4 hours ahead. For improved computational efficiency, each image is divided into four patches during training. Notably, this research incorporates a dynamic learning model, continuously updating with the most recent data, in contrast with static models which do not update their parameters with new data once trained. Results show that ConvLSTM tends to exhibit stable and relatively higher performance across prediction times compared to the other models in August 2021. While transformer models, such as Video Swin Transformer and TimeSformer, showed strong training performance, they struggled with overfitting, particularly on smaller datasets. In contrast, ConvLSTM demonstrated better generalization to test data, highlighting its suitability for tasks with limited training data and simpler structures. Year-long validation demonstrates the robustness of the ConvLSTM model, which consistently outperforms the other models in all major metrics, including a precision of 0.79, recall of 0.80, F1-score of 0.80 (which balances precision and recall), and accuracy of 0.78 throughout 2021. However, results show that the model’s performance in terms of F1-score, recall, and precision is positively correlated with cloud fraction, with a slight tendency for higher accuracy during the summer compared to winter, indicating sensitivity to seasonal cloud cover variations.

The Role of Religious Research in Shaping an Outstanding Generation of Islamic Education at Universitas Negeri Surabaya

The study aimed to analyze the effectiveness of religious research as a catalyst in shaping an outstanding generation of Islamic Education (PAI) students at Universitas Negeri Surabaya (UNESA) enabling them to not only master religious theories but also apply them in relevant social contexts. This approach is expected to enhance students’ critical thinking, deepen their understanding of the material, and encourage active participation in research-based PAI learning. The study discusses the implications of applying religious research to students in UNESA. How can the effectiveness of religious research improve students’ abilities and skills? The study employs a mixed-methods approach, with quantitative data obtained through pre-tests and post-tests administered to students, while qualitative data is gathered through interviews, classroom observations, and student assignment analysis. The results reveal that after implementing religious research, there was a significant improvement in students’ critical thinking skills, understanding of PAI materials, active participation in discussions, and the quality of research assignments. Students also demonstrated an enhanced understanding of PAI materials in a broader and more practical context. Based on pre-test and post-test results, the interval results indicated an increase of approximately 30% - 40% based on measured indicators. The religious research approach was able to create a more dynamic and relevant PAI learning model in line with current needs, proving effective in shaping students who are not only academically excellent but also possess strong contextual abilities. This suggests that religious research can serve as a sustainable learning model to improve the quality of PAI education at the university level.

A Deep Dynamic Causal Learning Model to Study Changes in Dynamic Effective Connectivity during Brain Development.

Brain dynamic effective connectivity (dEC), characterizes the information transmission patterns between brain regions that change over time, which provides insight into the biological mechanism underlying brain development. However, most existing methods predominantly capture fixed or temporally invariant EC, leaving dEC largely unexplored. Herein we propose a deep dynamic causal learning model specifically designed to capture dEC. It includes a dynamic causal learner to detect time-varying causal relationships from spatio-temporal data, and a dynamic causal discriminator to validate these findings by comparing original and reconstructed data. Our model outperforms established baselines in the accuracy of identifying dynamic causalities when tested on the simulated data. When applied to the Philadelphia Neurodevelopmental Cohort, the model uncovers distinct patterns in dEC networks across different age groups. Specifically, the evolution process of brain dEC networks in young adults is more stable than in children, and significant differences in information transfer patterns exist between them. This study highlights the brain's developmental trajectory, where networks transition from undifferentiated to specialized structures with age, in accordance with the improvement of an individual's cognitive and information processing capability. The proposed model consists of the identification and verification of dynamic causality, utilizing the spatio-temporal fusing information from fMRI. As a result, it can accurately detect dEC and characterize its evolution over age.

Inter-temporal dynamic joint learning model considering intra- and inter-day mutable correlations for blood glucose level prediction

Accurate and low clinical risk blood glucose level (BGL) prediction plays an important role in blood glucose management of type 1 diabetes (T1D). Due to the diurnal variation in physiological regulation and the influence of external life events, blood glucose patterns change over time and events within a day. The fact that humans have an inherent biological cycle of approximately one day and an individual has similar responses to the same life events leading to day-to-day similarities in blood glucose trajectories. However, the daily schedules are not entirely same under free-living conditions so that similar glycemic patterns may occur at different periods on each day. Thus, the mutable intra-day and inter-day temporal correlations in multi-day BGL data challenge the BGL prediction. To this end, we propose an inter-temporal dynamic joint learning method, named as iTDJL. Firstly, considering the variability of the association between days in the free-living, we design a discovery strategy based on discrete cross-correlation to dynamically select the highly correlated series slices from previous days. Then, based on the multi-head self-attention mechanism, the intra-day and inter-day attention layers are developed and connected in sequence to capture the correlation relationships between different time spans that varies with input. Finally, the future multi-step predictions are obtained through a multi-layer perception layer. We evaluate the iTDJL model on the real-world OhioT1DM dataset. Compared to other advanced methods, the iTDJL produces more reliable BGL predictions without requiring meal announcement and other lifestyle information from the patients under free-living conditions. Our proposed method may promote the T1D management, such as clinical decision support assisting insulin injection and abnormal glucose warning.

DySurv: dynamic deep learning model for survival analysis with conditional variational inference.

Machine learning applications for longitudinal electronic health records often forecast the risk of events at fixed time points, whereas survival analysis achieves dynamic risk prediction by estimating time-to-event distributions. Here, we propose a novel conditional variational autoencoder-based method, DySurv, which uses a combination of static and longitudinal measurements from electronic health records to estimate the individual risk of death dynamically. DySurv directly estimates the cumulative risk incidence function without making any parametric assumptions on the underlying stochastic process of the time-to-event. We evaluate DySurv on 6 time-to-event benchmark datasets in healthcare, as well as 2 real-world intensive care unit (ICU) electronic health records (EHR) datasets extracted from the eICU Collaborative Research (eICU) and the Medical Information Mart for Intensive Care database (MIMIC-IV). DySurv outperforms other existing statistical and deep learning approaches to time-to-event analysis across concordance and other metrics. It achieves time-dependent concordance of over 60% in the eICU case. It is also over 12% more accurate and 22% more sensitive than in-use ICU scores like Acute Physiology and Chronic Health Evaluation (APACHE) and Sequential Organ Failure Assessment (SOFA) scores. The predictive capacity of DySurv is consistent and the survival estimates remain disentangled across different datasets. Our interdisciplinary framework successfully incorporates deep learning, survival analysis, and intensive care to create a novel method for time-to-event prediction from longitudinal health records. We test our method on several held-out test sets from a variety of healthcare datasets and compare it to existing in-use clinical risk scoring benchmarks. While our method leverages non-parametric extensions to deep learning-guided estimations of the survival distribution, further deep learning paradigms could be explored.

HELMET: A Hybrid Machine Learning Framework for Real-Time Prediction of Edema Trajectory in Large Middle Cerebral Artery Stroke.

Malignant cerebral edema occurs when brain swelling displaces and compresses vital midline structures within the first week of a large middle cerebral artery stroke. Early interventions such as hyperosmolar therapy or surgical decompression may reverse secondary injury but must be administered judiciously. To optimize treatment and reduce secondary damage, clinicians need strategies to frequently and quantitatively assess the trajectory of edema using updated, relevant information. However, existing risk assessment tools are limited by the absence of structured records capturing the evolution of edema and typically estimate risk at a single time point early in the admission, therefore failing to account for changes in variables over the following hours or days. To address this, we developed and validated dynamic machine learning models capable of accurately predicting the severity of midline structure displacement, an established indicator of malignant edema, in real-time. Our models can provide updated estimations as frequently as every hour, using data from structured time-varying patient records, radiographic text, and human-curated neurological characteristics. Our work resulted in two novel multi-class classification models, collectively named Hybrid Ensemble Learning Models for Edema Trajectory (HELMET), predicting the progression of midline shift over 8-hour (HELMET-8) and 24-hour windows (HELMET-24), respectively. HELMET combines transformer-based large language models with supervised ensemble learning, demonstrating the value of merging human expertise and multimodal health records in developing clinical risk scores. Both models were trained on a retrospective cohort of 15,696 observations from 623 patients hospitalized with large middle cerebral artery ischemic stroke and were externally validated using 3,713 observations from 60 patients at a separate hospital system. Our HELMET models are accurate and generalize effectively to diverse populations, achieving a cross-validated mean area under the receiver operating characteristic score of 96.6% in the derivation cohort and 92.5% in the external validation cohort. Moreover, our approach provides a framework for developing hybrid risk prediction models that integrate both human-extracted and algorithm-derived multi-modal inputs. Our work enables accurate estimation of complex, dynamic, and highly specific clinical targets, such as midline shift, in real-time, even when relevant structured information is limited in electronic health record databases.

Simulation of policies to promote pro‐environmental behavior in a dynamic learning model of social networks: Cases of Brazil, the United States, Japan, Germany, China, and India

AbstractPro‐environmental behavior does not seem to diffuse sufficiently in Brazil, China, Germany, Japan, India, or the United States. Based on the current distribution of carbon footprints and learning patterns (DeGroot learning, best practice, and free riding) in social networks in the six countries, what will happen to the diffusion of pro‐environmental behaviors in each country? What policy measures will be most effective? This study aims to simulate the diffusion of pro‐environmental behavior and to test the effectiveness of various policy measures in a dynamic social network model in the six countries. The distribution of pro‐environmental behaviors is generated by global gridded carbon footprint data. The distribution of learning patterns in social networks is estimated in each country, using data from the World Values Survey. This study considers three policy measures: increasing the number of people pursuing best pro‐environmental behavior practices (best practice promotion, [BPP]), decreasing the number of free riders (free‐riding reduction, [FRR]), and converting the learning patterns of people with high degrees of centrality into best practice (centrality‐based best practice promotion, [CBPP]). Interesting findings were threefold. First, there are large differences in the estimated degrees of individual pro‐environmental behavior and the proportions of learning patterns in the six countries. Second, FRR and BPP are effective in promoting pro‐environmental behavior, but the choice of FRR or BPP depends on the initial proportion of free riders. Third, CBPP is the most effective policy of the three in all countries. Centrality in social networks must be considered when implementing intervention.

Tracking the impact of heavy metals on human health and ecological environments in complex coastal aquifers using improved machine learning optimization.

The rising heavy metal (HM) pollution in coastal aquifers in rapidly urbanizing areas such as Dammam leads to significant risks to public health and environmental sustainability, challenging compliance with Environmental Protection Agency (EPA) guidelines, World Health Organization (WHO) standards, and Sustainable Development Goals (SDGs) related to clean water and life on land. This study developed the predictive-based monitoring of HM concentrations, including cadmium (Cd), chromium (Cr), and mercury (Hg) in the coastal aquifers of Dammam, influenced by industrial, agricultural, and urban activities. For this purpose, dynamic system identification and machine learning (ML) models integrated with three ensemble techniques, namely, simple averaging (SAE), weighted averaging (WAE), and neuro-ensemble (N-ESB), were employed to enhance the accuracy, reliability, and efficiency of environmental monitoring systems. The experimental data were calibrated and validated in addition to k-fold cross-validation to ensure the predictive skills of the models. The methodology integrates extensive data collection across varied land uses in Dammam and accurate model calibration and validation phases to develop highly accurate predictive models. The findings proved that the N-ESB and Hammerstein-Wiener (HW) models surpassed other models in predicting the concentrations of all HM. For Cd, the N-ESB model achieved a root mean square error (RMSE = 0.0010mg/kg). Similarly, Cr demonstrated superior performance (RMSE = 0.0179mg/kg). Further numerical results indicated that the HW algorithm proved the most effective for Hg, with RMSE = 0.0000mg/kg. The quantitative comparison suggested that the N-ESB model's consistently high performance and low error rates make it an optimal choice for real-time, precise monitoring and management of HM pollution in coastal aquifers. The outcomes of this research highlighted the importance of integrating advanced predictive modeling techniques in environmental science, providing significant and practical implications for policymaking and ecological management.

Mediated learning: A computational rendering of ketamine-induced symptoms.

This article explores the contribution of the double error dynamic asymptote computational associative learning model to understanding the role of mediated learning mechanisms in the generation of spurious associations, as those postulated to characterize schizophrenia. Three sets of simulations for mediated conditioning, mediated extinction, and a mediated enhancement of latent inhibition, a unique model prediction, are presented. For each set of simulations, a parameter that modulates the impact of associative memory retrieval and the dissipation of nonperceptual activated representations through the network was manipulated. The effect of this operation is analyzed and compared to ketamine-induced effects on associative memories and mediated learning. The model's potential to predict these effects and present a plausible error-correction associative mechanism is discussed in the context of animal models of schizophrenia. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Dynamic machine learning model to forecast patient availability for clinical trials.

e13641 Background: Patient eligibility is a routine concept in clinical research, comprised of simple or complex set of inclusion or exclusion criteria applied in a specific sequence. Our methodology introduces the concept of ‘availability’ as a narrower term that is closer to the actual number that will be considered for a specific study and who may decide to participate. This specific research develops Machine Learning (ML) model that dynamically predicts likelihood of Multiple Myeloma progression and departure from targeted outcomes to sustain therapy and to indicate potential availability for clinical trials – to reduce screening efforts and improve screening precision of clinical research organizations. Methods: The cohort for training these models was drawn from the ConcertAI Oncology Research database, further enriched by key variables derived from unstructured data sources. Eligible patients for the analysis had a diagnosis of Multiple Myeloma made after Jan 1, 2020. Progressions were labelled using International Myeloma Working Group Criteria. Patients that did not satisfy the criteria were censored. Labels were assigned to patients after initial diagnosis of MM, and the duration between diagnosis to label was computed to build time to event (TTE) models. Over 2000 features were created to build this model, e.g., laboratory values, vitals, biomarkers, medications, ECOG, stage, etc. Variable creation preserved temporality of lab variables and vitals and noted presence of other variables. Several ML methods were employed to build the TTE models, with gradient boosting resulting in the highest c-index (0.8) based on 12 clinically relevant features. A temporal validation of the model was performed on a simulated index date. Results: Cohort used for training the model comprised of N = 4315 patients with average age at diagnosis of 72.3 years, and ~11% of the patients labelled as progressing. Temporal validation was performed on N = 1874 patients, with ~9% progression rate. The area under the curve (AUC) and weighted F1 score for this cohort was 87% and 89% respectively. Conclusions: Patient availability assessments are complex, not supported effectively by existing solutions and often require 100’s of hours from highly trained clinical personnel. An AI model predicting Multiple Myeloma progression built using a large-scale real-world dataset demonstrated 2-4x improvement in forecasting patient availability over other approaches. Integrating these solutions into clinical workflows can improve the effectiveness of research teams; enable more patients to have clinical trials as viable options at the right time; and allow care teams to have forward visibility to choices of standard-of-care and clinical trial alternatives in the best interest of patient outcomes. [Table: see text]

Cyber Risk Assessment Framework for the Construction Industry Using Machine Learning Techniques

Construction 4.0 integrates digital technologies that increase vulnerability to cyber threats. A dedicated cyber risk assessment framework is essential for proactive risk mitigation. However, existing studies on this subject within the construction sector are scarce, with most discussions still in the preliminary stages. This study introduces a cyber risk assessment framework that integrates machine learning techniques, pioneering a data-driven approach to quantitatively assess cyber risks while considering industry-specific vulnerabilities. The framework builds on over 20 literature reviews related to construction cybersecurity and semi-structured interviews with two industry experts, ensuring both rigor and alignment with practical industrial needs. This study also addresses the challenges of data collection and proposes potential solutions, such as a standardized data collection format with preset fields that computers can automatically populate using data from construction companies. Additionally, the framework proposes dynamic machine learning models that adjust based on new data, facilitating continuous risk monitoring tailored to industry needs. Furthermore, this study explores the potential of advanced language models in cybersecurity management, positioning them as intelligent cybersecurity consultants that provide answers to security inquiries. Overall, this study develops a conceptual machine learning framework aimed at creating a robust, off-the-shelf cyber risk management system for industry practitioners.

Advancing urban traffic accident forecasting through sparse spatio-temporal dynamic learning

Traffic accidents have emerged as one of the most public health safety matters, raising concerns from both the public and urban administrators. The ability to accurately predict traffic accident not only supports the governmental decision-making in advance but also enhances public confidence in safety measures. However, the efficacy of traditional spatio-temporal prediction models are compromised by the skewed distributions and sparse labeling of accident data. To this end, we propose a Sparse Spatio-Temporal Dynamic Hypergraph Learning (SST-DHL) framework that captures higher-order dependencies in sparse traffic accidents by combining hypergraph learning and self-supervised learning. The SST-DHL model incorporates a multi-view spatiotemporal convolution block to capture local correlations and semantics of traffic accidents, a cross-regional dynamic hypergraph learning model to identify global spatiotemporal dependencies, and a two-supervised self-learning paradigm to capture both local and global spatiotemporal patterns. Through experimentation on New York City and London accident datasets, we demonstrate that our proposed SST-DHL exhibits significant improvements compared to optimal baseline models at different sparsity levels. Additionally, it offers enhanced interpretability of results by elucidating complex spatio-temporal dependencies among various traffic accident instances. Our study demonstrates the effectiveness of the SST-DHL framework in accurately predicting traffic accidents, thereby enhancing public safety and trust.

396 Creating a High-Precision, Dynamic Machine Learning Model Using Time Series Data to Predict Cerebral Vasospsm in Aneurysmal Subarachnoid Hemorrhage

INTRODUCTION: Cerebral vasospasm (CV) occurs in up to one-third of patients who suffer from aneurysmal subarachnoid hemorrhage (aSAH) and can result in significant morbidity. Prior attempts to predict CV after aSAH have failed because they use static data collected on the day of presentation. We hypothesize that trending clinical and radiographic data including transcranial dopplers (TCDs) throughout 14 intensive care unit (ICU) days can be used to create a highly predictive, dynamic machine learning model for CV. METHODS: We conducted an IRB-approved retrospective study including 428 patients with aSAH hemorrhage treated at our institution between June 2014 and December 2018. For each patient, 74 features were trended including vital signs, lab values and ventriculostomy output. TCD measurements included anterior and posterior circulation velocities, Lindegaard Ratios and pulsatility indices. We employed a generative transformer-based model that uses patient representations from aggregate prior time points to predict the likelihood of CV. RESULTS: Of the 428 aSAH patients, 85 had angiographic-proven CV. There was no difference in demographic features including age, gender distribution and BMI between the CV and no-CV groups. Throughout 14 ICU days, the CV patients consistently displayed elevated mean arterial pressure and cerebral perfusion pressure. All TCD measurements were higher for the CV group. Initial logistic regression models based on static data from post-ictus days 1 through 4 showed steadily increasing F1 scores from 0.67 to 0.76. Feature analysis identified Lindegaard Ratios as the most heavily weighted features. CONCLUSIONS: Daily TCDs paired with additional longitudinal clinical data can be used to create a high-precision dynamic machine learning model for predicting CV after aSAH. These methods could also permit identification of patients at low risk of CV for reduced ICU length of stay.

Best Social Entrepreneurship Teaching and Learning Strategies for Promoting Students' Social Entrepreneurial Minds: A Scoping Review

This paper aims to explore the best teaching and learning strategies for promoting students' social entrepreneurial minds. At the initial stage, the scoping review draws from 198 secondary data from various works in the social entrepreneurship education context, including studies on education programs, curricular development, knowledge-acquiring mechanisms, and social entrepreneurial exploration processes. It also examines instructional methods and strategies that foster the development of potential social entrepreneur mindsets. Finally, 26 included papers present a Top-Down and Bottom-Up Approach, Experiential Learning, Action Learning, Community-Based Project, and Problem-Based Learning (PBL) as effective social entrepreneurship teaching and learning approaches to promote social entrepreneurial mindsets. The review concludes that a combination of all of the aforementioned teaching and learning strategies is among the top teaching approaches for effective social entrepreneurship education. The paper proposes that dynamic social entrepreneurship curricula and learning models that emphasize hands-on experience and experience-based learning can be effective strategies for promoting students' social entrepreneurial minds.

Contrastive representation learning on dynamic networks

Representation learning for dynamic networks is designed to learn the low-dimensional embeddings of nodes that can well preserve the snapshot structure, properties and temporal evolution of dynamic networks. However, current dynamic network representation learning methods tend to focus on estimating or generating observed snapshot structures, paying excessive attention to network details, and disregarding distinctions between snapshots with larger time intervals, resulting in less robustness for sparse or noisy networks. To alleviate these challenges, this paper proposes a contrastive mechanism for temporal representation learning on dynamic networks, inspired by the success of contrastive learning in visual and static network representation learning. This paper proposes a novel Dynamic Network Contrastive representation Learning (DNCL) model. Specifically, contrast objective functions are constructed using intra-snapshot and inter-snapshot contrasts to capture the network topology, node feature information, and network evolution information, respectively. Rather than estimating or generating ground-truth network features, the proposed approach maximizes mutual information between nodes from different time steps and views generated. The experimental results of link prediction, node classification, and clustering on several real-world and synthetic networks demonstrate the superiority of DNCL over state-of-the-art methods, indicating the effectiveness of the proposed approach for dynamic network representation learning.

Multimodal subspace identification for modeling discrete-continuous spiking and field potential population activity

Objective. Learning dynamical latent state models for multimodal spiking and field potential activity can reveal their collective low-dimensional dynamics and enable better decoding of behavior through multimodal fusion. Toward this goal, developing unsupervised learning methods that are computationally efficient is important, especially for real-time learning applications such as brain–machine interfaces (BMIs). However, efficient learning remains elusive for multimodal spike-field data due to their heterogeneous discrete-continuous distributions and different timescales. Approach. Here, we develop a multiscale subspace identification (multiscale SID) algorithm that enables computationally efficient learning for modeling and dimensionality reduction for multimodal discrete-continuous spike-field data. We describe the spike-field activity as combined Poisson and Gaussian observations, for which we derive a new analytical SID method. Importantly, we also introduce a novel constrained optimization approach to learn valid noise statistics, which is critical for multimodal statistical inference of the latent state, neural activity, and behavior. We validate the method using numerical simulations and with spiking and local field potential population activity recorded during a naturalistic reach and grasp behavior. Main results. We find that multiscale SID accurately learned dynamical models of spike-field signals and extracted low-dimensional dynamics from these multimodal signals. Further, it fused multimodal information, thus better identifying the dynamical modes and predicting behavior compared to using a single modality. Finally, compared to existing multiscale expectation-maximization learning for Poisson–Gaussian observations, multiscale SID had a much lower training time while being better in identifying the dynamical modes and having a better or similar accuracy in predicting neural activity and behavior. Significance. Overall, multiscale SID is an accurate learning method that is particularly beneficial when efficient learning is of interest, such as for online adaptive BMIs to track non-stationary dynamics or for reducing offline training time in neuroscience investigations.

LiteTransNet: An interpretable approach for landslide displacement prediction using transformer model with attention mechanism

Accurate landslide displacement prediction is crucial for effective early warning systems to mitigate hazards. The importance of historical information varies with time during prediction due to the underlying landslide deformation mechanism. Despite advances in dynamic machine learning models like Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), they struggle to fully capture and interpret the varying importance of historical information during prediction, resulting in decreased accuracy and limited understanding of landslide physical mechanisms. Additionally, these approaches rely on manual feature selection, which is independent of the learning process and therefore is prone to estimation errors. To address these challenges, we propose LiteTransNet, a Lightweighted Transformer Network tailored for landslide displacement prediction. Built on the revolutionary Transformer model for sequential data, LiteTransNet leverages localized self-attention to selectively focus on relevant timestamps and provides interpretable results through its attention heatmap. Furthermore, LiteTransNet performs end-to-end time series modeling without extensive feature engineering. We validate LiteTransNet on two landslides in China's Three Gorges Reservoir Area and demonstrate its improved accuracy over recurrent baselines utilizing feature selection methods. Notably, the attention heatmap generated by LiteTransNet provides interpretable insights into the model's temporal dependency learning. It uncovers the varying importance of historical information at different timestamps, showcasing how LiteTransNet's attention aligns with specific periods of the external environment that cause significant disruptions in the landslide system. Our findings reveal that these disruptions continuously impact displacement prediction until their effects fade or are replaced by subsequent intense disturbances, which traditional recurrent methods are unable to capture. Further experiment shows that LiteTransNet enhances efficiency through its streamlined architecture and its parallelization capability. Overall, LiteTransNet innovates landslide prediction by providing accuracy, interpretability, and efficiency, enhancing understanding of deformation mechanisms to facilitate effective early warning systems.

Algebraic Dynamical Systems in Machine Learning

We introduce an algebraic analogue of dynamical systems, based on term rewriting. We show that a recursive function applied to the output of an iterated rewriting system defines a formal class of models into which all the main architectures for dynamic machine learning models (including recurrent neural networks, graph neural networks, and diffusion models) can be embedded. Considered in category theory, we also show that these algebraic models are a natural language for describing the compositionality of dynamic models. Furthermore, we propose that these models provide a template for the generalisation of the above dynamic models to learning problems on structured or non-numerical data, including ‘hybrid symbolic-numeric’ models.

Ideonamic: An integrative computational dynamic model of ideomotor learning and effect-based action control.

According to ideomotor theory, actions are represented, controlled, and retrieved in terms of the perceptual effects that these actions experientially engender. When agents perform a motor action, they observe its subsequent perceptual effects and establish action-effect associations. When they want to achieve this effect at a later time, they use the action-effect associations to preactivate the action by internally activating the effect representation. Ideomotor theory has received extensive support in recent years. To capture this particular effect-based view on action control and goal-directed behavior, we developed IDEONAMIC, an integrative computational model based on dynamic field theory that represents the specific components of the action control process as dynamic neural fields. We show that IDEONAMIC applies conveniently to different types of experimental ideomotor settings, simulates key findings, generates novel predictions from the dynamics of data, and allows reapproaching the underlying cognitive mechanisms from a computational point of view. We encourage the application of IDEONAMIC to more types of ideomotor settings to gain insights into effect-based action control. The model is available at https://osf.io/hbc6n. (PsycInfo Database Record (c) 2024 APA, all rights reserved).