Temporal Encoding Research Articles

Reflexive neural circuits and the origin of language and music codes

Conscious activity is grounded on the reflexive self-awareness in sense perception, through which the codes signifying sensual perceptive events operate and constrain human behavior. These codes grow via the creative generation of hypertextual statements. We apply the model of Vladimir Lefebvre (Lefebvre, V.A., 1987, J. Soc. Biol. Struct. 10, 129-175) to reveal the underlying structures on which the perception and creative development of language and music codes are based. According to this model, the reflexive structure of conscious subject is grounded on three thermodynamic cycles united by the control of the basic functional cycle by the second one, and resulting in the internal action that it turn is perceived by the third cycle evaluating this action. In this arrangement, the generative language structures are formed and the frequencies of sounds that form musical phrases and patterns are selected. We discuss the participation of certain neural brain structures and the establishment of reflexive neural circuits in the ad hoc transformation of perceptive signals and show the similarities between the processes of perception and of biological self-maintenance and morphogenesis. We trace the peculiarities of the temporal encoding of emotions in music and musical creativity, as well as the principles of sharing musical information between the performing and the perceiving individuals.

Fixational eye movements enhance the precision of visual information transmitted by the primate retina

Fixational eye movements alter the number and timing of spikes transmitted from the retina to the brain, but whether these changes enhance or degrade the retinal signal is unclear. To quantify this, we developed a Bayesian method for reconstructing natural images from the recorded spikes of hundreds of retinal ganglion cells (RGCs) in the macaque retina (male), combining a likelihood model for RGC light responses with the natural image prior implicitly embedded in an artificial neural network optimized for denoising. The method matched or surpassed the performance of previous reconstruction algorithms, and provides an interpretable framework for characterizing the retinal signal. Reconstructions were improved with artificial stimulus jitter that emulated fixational eye movements, even when the eye movement trajectory was assumed to be unknown and had to be inferred from retinal spikes. Reconstructions were degraded by small artificial perturbations of spike times, revealing more precise temporal encoding than suggested by previous studies. Finally, reconstructions were substantially degraded when derived from a model that ignored cell-to-cell interactions, indicating the importance of stimulus-evoked correlations. Thus, fixational eye movements enhance the precision of the retinal representation.

LSTGCN: Inductive Spatial Temporal Imputation Using Long Short Term Dependencies

Spatial temporal forecasting of urban sensors is essentially important for many urban systems, such as intelligent transportation and smart cities. However, due to the problem of hardware failure or network failure, there are some missing values or missing monitoring sensors that need to be interpolated. Recent research on deep learning has made substantial progress on imputation problem, especially temporal aspect (i.e., time series imputation), while little attention has been paid to spatial aspect (both dynamic and static) and long term temporal dependencies. In this paper, we proposed a spatial temporal imputation model, named Long Short Term Graph Convolution Networks (LSTGCN), which includes gated temporal extraction (GTE) module, multi-head attention based temporal capture (MHAT) module, long term periodic temporal encoding (LPTE) module and bidirectional spatial graph convolution (BSGC) module. The GTE adopts a gated mechanism to filter short-term temporal information, while the MHAT utilizes position encoding to enhance the difference of each time stamps, then use multi-head attention to capture short term temporal dependency. The BSGC is adopted to handle with spatial relationships between sensor nodes. And we design a periodic encoding technique to process long term temporal dependencies. The BSGC handles spatial relationships between sensor nodes, and a periodic encoding technique is used to process long-term temporal dependencies. Our experimental analysis includes completion and forecasting tasks, as well as transfer and ablation analyses. The results show that our proposed model outperforms state-of-the-art baselines on real-world datasets.

TADGCN: A Time-Aware Dynamic Graph Convolution Network for long-term traffic flow prediction

Traffic flow prediction is a crucial component of the Intelligent Transportation System (ITS) with substantial implications for traffic management and daily travel. While short-term traffic forecasting has shown promise, long-term prediction presents unique challenges. The existing short-term approaches fall short in long-term scenarios due to two primary issues: (1) ignoring modeling the interaction relationships among dynamic spatial dependencies, and (2) extracting long-term temporal dependencies without capturing the time information in-depth. To mitigate the above issues, we propose a novel model called TADGCN, a Time-Aware Dynamic Graph Convolution Network, to effectively perform long-term traffic flow prediction. TADGCN captures latent interactions among dynamic spatial dependencies using Attention-Driven Dynamic Adaptive Graph Convolution Network (ADAGCN) modules. The time-aware capability is effectively enhanced using Time-Aware Joint Multi-View Temporal Encoder (TJMTE) modules due to the construction of a time-aware matrix. We conduct diversified experiments on three real-world datasets (England, PEMS04, and PEMS08) to evaluate the performance of our proposed method TADGCN. The results reveal that TADGCN achieves the best accuracy in all cases against 16 competitive state-of-the-art baselines.

Spatio-temporal learning and explaining for dynamic functional connectivity analysis: Application to depression

BackgroundFunctional connectivity has been shown to fluctuate over time. The present study aimed to identifying major depressive disorders (MDD) with dynamic functional connectivity (dFC) from resting-state fMRI data, which would be helpful to produce tools of early depression diagnosis and enhance our understanding of depressive etiology. MethodsThe resting-state fMRI data of 178 subjects were collected, including 89 MDD and 89 healthy controls. We propose a spatio-temporal learning and explaining framework for dFC analysis. A yet effective spatio-temporal model is developed to classifying MDD from healthy controls with dFCs. The model is a stacking neural network model, which learns network structure information by a multi-layer perceptron based spatial encoder, and learns time-varying patterns by a Transformer based temporal encoder. We propose to explain the spatio-temporal model with a two-stage explanation method of importance feature extracting and disorder-relevant pattern exploring. The layer-wise relevance propagation (LRP) method is introduced to extract the most relevant input features in the model, and the attention mechanism with LRP is applied to extract the important time steps of dFCs. The disorder-relevant functional connections, brain regions, and brain states in the model are further explored and identified. ResultsWe achieved the best classification performance in identifying MDD from healthy controls with dFC data. The top important functional connectivity, brain regions, and dynamic states closely related to MDD have been identified. LimitationsThe data preprocessing may affect the classification performance of the model, and this study needs further validation in a larger patient population. ConclusionsThe experimental results demonstrate that the proposed spatio-temporal model could effectively classify MDD, and uncover structural and temporal patterns of dFCs in depression.

Automatic detection of sleep apnea from a single-lead ECG signal based on spiking neural network model

BackgroundSleep apnea (SLA) is a commonly encountered sleep disorder characterized by repetitive cessation of respiration while sleeping. In the past few years, researchers have focused on developing less complex and more cost-effective diagnostic approaches for identifying SLA recipients, in contrast to the cumbersome, complicated, and expensive conventional methods. MethodThis study presents a biologically plausible learning approach of spiking neural networks (SNN) with temporal coding and a tempotron learning model for diagnosing SLA disorder using single-lead electrocardiogram (ECG) data information. The proposed framework utilizes temporal encoding and the leaky integrate and fire model to transform the ECG signal into spikes for capturing the signal's dynamic pattern nature and to simulate input response behaviors. The tempoton learning technique, a spike-based algorithm, trains the SNN model to identify SLA event patterns from encoded output spike trains. This study utilized ECG data to extract heart rate variability (HRV) and ECG-derived respiration (EDR) signals from 1-min segment data of ECG records for input to SNN model. Thirty-five recordings of both released and withheld data from the Apnea-ECG databases from Physionet have been applied to train the SNN model and validate the model's efficacy in identifying SLA occurrences. ResultsThe proposed method demonstrated substantial improvements compared to other SLA detection techniques, achieving a significant accuracy of 94.63 % for per-segment detection, along with specificity, sensitivity, F1-score and AUC values of 96.21 %, 92.04 %, 0.9285, and 0.9851 respectively. The accuracy for per-recording detection achieved 100 %, with a correlation coefficient value of 0.986. Additionally, the experiment used UCD data for validation methods, achieving an accuracy of 84.573 %. ConclusionsThese results suggest the effectiveness and accessibility of the presented approach for accurately identifying SLA cases. The suggested model enhances the performance of SLA detection when contrasted with various techniques based on feature engineering and feature learning.

Quantitative Stock Selection Model Using Graph Learning and a Spatial–Temporal Encoder

In the rapidly evolving domain of finance, quantitative stock selection strategies have gained prominence, driven by the pursuit of maximizing returns while mitigating risks through sophisticated data analysis and algorithmic models. Yet, prevailing models frequently neglect the fluid dynamics of asset relationships and market shifts, a gap that undermines their predictive and risk management efficacy. This oversight renders them vulnerable to market volatility, adversely affecting investment decision quality and return consistency. Addressing this critical gap, our study proposes the Graph Learning Spatial–Temporal Encoder Network (GL-STN), a pioneering model that seamlessly integrates graph theory and spatial–temporal encoding to navigate the intricacies and variabilities of financial markets. By harnessing the inherent structural knowledge of stock markets, the GL-STN model adeptly captures the nonlinear interactions and temporal shifts among assets. Our innovative approach amalgamates graph convolutional layers, attention mechanisms, and long short-term memory (LSTM) networks, offering a comprehensive analysis of spatial–temporal data features. This integration not only deciphers complex stock market interdependencies but also accentuates crucial market insights, enabling the model to forecast market trends with heightened precision. Rigorous evaluations across diverse market boards—Main Board, SME Board, STAR Market, and ChiNext—underscore the GL-STN model’s exceptional ability to withstand market turbulence and enhance profitability, affirming its substantial utility in quantitative stock selection.

Multi-Encoder Spatio-Temporal Feature Fusion Network for Electric Vehicle Charging Load Prediction

Electric vehicles (EVs) have been initiated as a preference for decarbonizing road transport. Accurate charging load prediction is essential for the construction of EV charging facilities systematically and for the coordination of EV energy demand with the requisite peak power supply. It is noted that the charging load of EVs exhibits high complexity and randomness due to temporal and spatial uncertainties. Therefore, this paper proposes a SEDformer-based charging road prediction method to capture the spatio-temporal characteristics of charging load data. As a deep learning model, SEDformer comprises multiple encoders and a single decoder. In particular, the proposed model includes a Temporal Encoder Block based on the self-attention mechanism and a Spatial Encoder Block based on the channel attention mechanism with sequence decomposition, followed by an aggregated decoder for information fusion. It is shown that the proposed method outperforms various baseline models on a real-world dataset from Palo Alto, U.S., demonstrating its superiority in addressing spatio-temporal data-driven load forecasting problems.

MEDi-SOL: Multi Ensemble Distribution Model for Estimating Sleep Onset Latency.

Sleep onset latency (SOL) is an important factor relating to the sleep quality of a subject. Therefore, accurate prediction of SOL is useful to identify individuals at risk of sleep disorders and to improve sleep quality. In this study, we estimate SOL distribution and falling asleep function using an electroencephalogram (EEG), which can measure the electric field of brain activity. We proposed a Multi Ensemble Distribution model for estimating Sleep Onset Latency (MEDi-SOL), consisting of a temporal encoder and a time distribution decoder. We evaluated the performance of the proposed model using a public dataset from the Sleep Heart Health Study. We considered four distributions, Normal, log-Normal, Weibull, and log-Logistic, and compared them with a survival model and a regression model. The temporal encoder with the ensemble log-Logistic and log-Normal distribution showed the best and second-best scores in the concordance index (C-index) and mean absolute error (MAE). Our MEDi-SOL, multi ensemble distribution with combining log-Logistic and log-Normal distribution, shows the best score in C-index and MAE, with a fast training time. Furthermore, our model can visualize the process of falling asleep for individual subjects. As a result, a distribution-based ensemble approach with appropriate distribution is more useful than point estimation.

Odour source distance is predictable from a time history of odour statistics for large scale outdoor plumes.

Odour plumes in turbulent environments are intermittent and sparse. Laboratory-scaled experiments suggest that information about the source distance may be encoded in odour signal statistics, yet it is unclear whether useful and continuous distance estimates can be made under real-world flow conditions. Here, we analyse odour signals from outdoor experiments with a sensor moving across large spatial scales in desert and forest environments to show that odour signal statistics can yield useful estimates of distance. We show that achieving accurate estimates of distance requires integrating statistics from 5 to 10 s, with a high temporal encoding of the olfactory signal of at least 20 Hz. By combining distance estimates from a linear model with wind-relative motion dynamics, we achieved source distance estimates in a 60 × 60 m2 search area with median errors of 3-8 m, a distance at which point odour sources are often within visual range for animals such as mosquitoes.

Dual-Rule-Based Weighted Fuzzy Interpolative Reasoning Module and Temporal Encoder–Decoder Bayesian Network for Reagent Addition Control

Dual-Rule-Based Weighted Fuzzy Interpolative Reasoning Module and Temporal Encoder–Decoder Bayesian Network for Reagent Addition Control

Spatio-Temporal Articulation & Coordination Co-attention Graph Network for human motion prediction

Human motion prediction, the task of forecasting future poses from a given observed sequence, is crucial for advancing human-centric computer vision. Significant progress has been made by employing fixed or adaptively learned spatiotemporal Graph Neural Networks (GNNs) for skeleton sequences. However, many existing GNN-based methods often yield counterfactual predictions by overlooking the inherent correlations between body parts essential for balance and coordination. In this study, we propose an end-to-end Spatio-Temporal Articulation & Coordination Co-attention Graph Network (ST-A&CCGN) that leverages both the physical and kinematic regularities of the human motion system. Specifically, we construct articulated and coordination graphs as prior knowledge to explicitly model the connectivities among articulated body parts and their coordination. Subsequently, we design a consistent module employing a parameter-sharing strategy to extract embeddings consistently representing both physical articulation and dynamic coordination. An attention mechanism is introduced to simultaneously learn adaptive weights for articulated, coordinated, and consistent embeddings. Finally, we utilize a multi-head temporal encoding module to capture temporal dependencies among spatial features. Extensive qualitative and quantitative evaluations conducted on three widely used benchmarks, namely Human3.6M, AMASS, and 3DPW, validate the effectiveness of the proposed framework. Additionally, ablation studies are conducted to emphasize the importance of integrating various modules. Remarkably, our model surpasses state-of-the-art (SOTA) competitors.

RN‐Net: Reservoir Nodes‐Enabled Neuromorphic Vision Sensing Network

Neuromorphic computing systems promise high energy efficiency and low latency. In particular, when integrated with neuromorphic sensors, they can be used to produce intelligent systems for a broad range of applications. An event‐based camera is such a neuromorphic sensor, inspired by the sparse and asynchronous spike representation of the biological visual system. However, processing the event data requires either using expensive feature descriptors to transform spikes into frames, or using spiking neural networks (SNNs) that are expensive to train. In this work, a neural network architecture is proposed, reservoir nodes‐enabled neuromorphic vision sensing network (RN‐Net), based on dynamic temporal encoding by on‐sensor reservoirs and simple deep neural network (DNN) blocks. The reservoir nodes enable efficient temporal processing of asynchronous events by leveraging the native dynamics of the node devices, while the DNN blocks enable spatial feature processing. Combining these blocks in a hierarchical structure, the RN‐Net offers efficient processing for both local and global spatiotemporal features. RN‐Net executes dynamic vision tasks created by event‐based cameras at the highest accuracy reported to date at one order of magnitude smaller network size. The use of simple DNN and standard backpropagation‐based training rules further reduces implementation and training costs.

Photoacoustic Tomography with Temporal Encoding Reconstruction (PATTERN) for cross-modal individual analysis of the whole brain

Cross-modal analysis of the same whole brain is an ideal strategy to uncover brain function and dysfunction. However, it remains challenging due to the slow speed and destructiveness of traditional whole-brain optical imaging techniques. Here we develop a new platform, termed Photoacoustic Tomography with Temporal Encoding Reconstruction (PATTERN), for non-destructive, high-speed, 3D imaging of ex vivo rodent, ferret, and non-human primate brains. Using an optimally designed image acquisition scheme and an accompanying machine-learning algorithm, PATTERN extracts signals of genetically-encoded probes from photobleaching-based temporal modulation and enables reliable visualization of neural projection in the whole central nervous system with 3D isotropic resolution. Without structural and biological perturbation to the sample, PATTERN can be combined with other whole-brain imaging modalities to acquire the whole-brain image with both high resolution and morphological fidelity. Furthermore, cross-modal transcriptome analysis of an individual brain is achieved by PATTERN imaging. Together, PATTERN provides a compatible and versatile strategy for brain-wide cross-modal analysis at the individual level.

SSTE: Syllable-Specific Temporal Encoding to FORCE-learn audio sequences with an associative memory approach

The circuitry and pathways in the brains of humans and other species have long inspired researchers and system designers to develop accurate and efficient systems capable of solving real-world problems and responding in real-time. We propose the Syllable-Specific Temporal Encoding (SSTE) to learn vocal sequences in a reservoir of Izhikevich neurons, by forming associations between exclusive input activities and their corresponding syllables in the sequence. Our model converts the audio signals to cochleograms using the CAR-FAC model to simulate a brain-like auditory learning and memorization process. The reservoir is trained using a hardware-friendly approach to FORCE learning. Reservoir computing could yield associative memory dynamics with far less computational complexity compared to RNNs. The SSTE-based learning enables competent accuracy and stable recall of spatiotemporal sequences with fewer reservoir inputs compared with existing encodings in the literature for similar purpose, offering resource savings. The encoding points to syllable onsets and allows recalling from a desired point in the sequence, making it particularly suitable for recalling subsets of long vocal sequences. The SSTE demonstrates the capability of learning new signals without forgetting previously memorized sequences and displays robustness against occasional noise, a characteristic of real-world scenarios. The components of this model are configured to improve resource consumption and computational intensity, addressing some of the cost-efficiency issues that might arise in future implementations aiming for compactness and real-time, low-power operation. Overall, this model proposes a brain-inspired pattern generation network for vocal sequences that can be extended with other bio-inspired computations to explore their potentials for brain-like auditory perception. Future designs could inspire from this model to implement embedded devices that learn vocal sequences and recall them as needed in real-time. Such systems could acquire language and speech, operate as artificial assistants, and transcribe text to speech, in the presence of natural noise and corruption on audio data.

Abstract PO4-07-03: Machine learning breast cancer risk prediction using sequential past mammograms - a pilot study

Abstract Background and Aim Some studies have reported the use of mammogram AI deep learning algorithms to accurately predict the risk of future breast cancer development in women. In these studies, random and unrelated mammograms were independently used for training of the AI model. We propose a novel deep learning system using sequential past mammograms instead of random studies and aimed to determine if it can further improve risk prediction in a pilot study. Methods Pre-training of the backbone of a Swin-transformer machine learning model was performed by applying region-of-interest lesional masks of benign and malignant on a training set of 1644 mammograms derived from the open-access Curated Breast Imaging Subset of Digital Database for Screening Mammography dataset. To further develop the model, deep learning training and validation were performed on the Cohort of Screen-age Women (CSAW) open-access screening mammogram set which was derived from the long term data of 873 women who developed breast cancer for the first time and 7850 non-cancer women. We then compared the diagnostic accuracy of a model using sequential past mammograms for training against the standard, non-sequential model. For the sequential model, imaging segmentation of the potential cancer site on mammography was performed on all past, pre-cancer mammograms obtained within 60 to 2555 days of cancer diagnosis in the pre-processing stage for breast cancer cases. AI training was then carried out by combining the pre-cancer mammograms belonging to the same woman in a temporal sequence using a transformer encoder. The learning rate started from 1e-6 and decayed 10 times every 10 epochs. For the non-sequential model, we removed the temporal encoder and used random, unrelated mammogram studies for training. Validation was performed on a dataset consisting of 75 pre-cancer mammograms from breast cancer cases and 69 mammograms from non-cancer cases. Results Sensitivity, specificity and overall accuracy on the validation set were 77.3% (58/75), 68.1% (47/69) and (105/144) 72.9% respectively for the sequential model compared to 69.3% (52/75), 69.6% (48/69) and 69.4% (100/144) respectively for the non-sequential model. Conclusion The study showed that deep learning breast cancer risk prediction can be further improved by using sequential past mammograms instead of random mammograms for AI model training. This can also potentially enhance other AI risk prediction models that employ combined mammogram and traditional breast cancer clinical risk factors. A larger validation study will be useful. Contact information: email Lester Leong at lester.leong.c.h@singhealth.com.sg ROC curve for breast cancer risk prediction using AI deep learning training with sequential past mammograms. Citation Format: Lester Leong, Mingjie Xu, Weimin Huang, Erli Zhang, Engracia Loh, Sze Yiun Teo, Geok Hoon Lim, Veronique Kiak Mien Tan, Yirong Sim, Fredrik Strand, Ryan Tan. Machine learning breast cancer risk prediction using sequential past mammograms - a pilot study [abstract]. In: Proceedings of the 2023 San Antonio Breast Cancer Symposium; 2023 Dec 5-9; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2024;84(9 Suppl):Abstract nr PO4-07-03.

Temporally transferable crop mapping with temporal encoding and deep learning augmentations

Detailed maps on the spatial and temporal distribution of crops are key for a better understanding of agricultural practices and for food security management. Multi-temporal remote sensing data and deep learning (DL) have been extensively studied for deriving accurate crop maps. However, strategies to solve the problem of transferring crop classification models over time, e.g., training the model with data for a recent year and mapping back to the past, have not been fully explored. This is due to the lack of a generalized method for aggregating optical data with regard to the irregularity in annual clear sky observations and the scarcity of multi-annual crop reference data to support a more generalized DL model. In this study, we tackled these challenges by introducing a method namely Temporal Encoding (TE) to capture the irregular phenological information. Subsequently, we adapted and integrated two methods, i.e., Random Observations Selection (ROS) and Random Day Shifting (RDS) to simulate the variability of temporal sparsity as well as the shifts of crop phenology over different years. We tested this approach with a 1-dimensional Convolutional Neural Network (1D-CNN) and a Transformer Network models. Our results for both classifiers showed that models trained with crop reference data from 2018 and a dense time series of Landsat 7/8 and Sentinel-2 A/B data can be transferred with little decreases in accuracy to map 12 consecutive years from 2010 to 2021. The Transformer Network was slightly more accurate, while the 1D-CNN was much three times faster. Furthermore, the proposed models could achieve similar performances in the same years with and without fully available satellite information. The TE with ROS and RDS appears well suited for improving temporal transferability to support long term historic crop mapping.

Traffic Prediction with Self-Supervised Learning: A Heterogeneity-Aware Model for Urban Traffic Flow Prediction Based on Self-Supervised Learning

Accurate traffic prediction is pivotal when constructing intelligent cities to enhance urban mobility and to efficiently manage traffic flows. Traditional deep learning-based traffic prediction models primarily focus on capturing spatial and temporal dependencies, thus overlooking the existence of spatial and temporal heterogeneities. Heterogeneity is a crucial inherent characteristic of traffic data for the practical applications of traffic prediction. Spatial heterogeneities refer to the differences in traffic patterns across different regions, e.g., variations in traffic flow between office and commercial areas. Temporal heterogeneities refer to the changes in traffic patterns across different time steps, e.g., from morning to evening. Although existing models attempt to capture heterogeneities through predefined handcrafted features, multiple sets of parameters, and the fusion of spatial–temporal graphs, there are still some limitations. We propose a self-supervised learning-based traffic prediction framework called Traffic Prediction with Self-Supervised Learning (TPSSL) to address this issue. This framework leverages a spatial–temporal encoder for the prediction task and introduces adaptive data masking to enhance the robustness of the model against noise disturbances. Moreover, we introduce two auxiliary self-supervised learning paradigms to capture spatial heterogeneities and temporal heterogeneities, which also enrich the embeddings of the primary prediction task. We conduct experiments on four widely used traffic flow datasets, and the results demonstrate that TPSSL achieves state-of-the-art performance in traffic prediction tasks.

MMSeaIce: a collection of techniques for improving sea ice mapping with a multi-task model

Abstract. The AutoICE challenge, organized by multiple national and international agencies, seeks to advance the development of near-real-time sea ice products with improved spatial resolution, broader spatial and temporal coverage, and enhanced consistency. In this paper, we present a detailed description of our solutions and experimental results for the challenge. We have implemented an automated sea ice mapping pipeline based on a multi-task U-Net architecture, capable of predicting sea ice concentration (SIC), stage of development (SOD), and floe size (FLOE). The AI4Arctic dataset, which includes synthetic aperture radar (SAR) imagery, ancillary data, and ice-chart-derived label maps, is utilized for model training and evaluation. Among the submissions from over 30 teams worldwide, our team achieved the highest combined score of 86.3 %, as well as the highest scores on SIC (92.0 %) and SOD (88.6 %). Notably, the result analysis and ablation studies demonstrate that instead of model architecture design, a collection of strategies/techniques we employed led to substantial enhancement in accuracy, efficiency, and robustness within the realm of deep-learning-based sea ice mapping. Those techniques include input SAR variable downscaling, input feature selection, spatial–temporal encoding, and the choice of loss functions. By highlighting the various techniques employed and their impacts, we aim to underscore the scientific advancements achieved in our methodology.

BERT4ST:: Fine-tuning pre-trained large language model for wind power forecasting

Accurate forecasting of wind power generation is essential for ensuring power safety, scheduling various energy sources, and improving energy utilization. However, the elusive nature of wind, influenced by various meteorological and geographical factors, greatly complicates the wind power forecasting task. To improve the forecasting accuracy of wind power (WP), we propose a BERT-based model for spatio-temporal forecasting (BERT4ST), which is the first approach to fine-tune a large language model for the spatio-temporal modeling of WP. To deal with the inherent characteristics of WP, BERT4ST exploits the individual spatial and temporal dependency of patches and redesigns a set of spatial and temporal encodings. By well analyzing the connection between bidirectional attention networks and WP spatio-temporal data, BERT4ST employs a pre-trained BERT encoder as the backbone network to learn the individual spatial and temporal dependency of patches of WP data. Additionally, BERT4ST fine-tunes the pre-trained backbone in a multi-stage manner, i.e., first aligning the language model with the spatio-temporal data and then fine-tuning the downstream tasks while maintaining the stability of the backbone network. Experimental results demonstrate that our BERT4ST achieves desirable performance compared to some state-of-the-art methods.