Temporal Input Research Articles

Thermal fields induced by oscillating laser beams versus non-oscillating equivalent laser beams — A numerical study

In high power laser material processing, like laser transformation hardening, laser welding or laser cladding, tailoring the laser intensity profile is a method to improve stability of the laser material process and/or improve the processing results. A method to tailor the laser intensity profile in real-time is using a Galvanometer Scanner to quickly and iteratively oscillate the focal spot of the laser beam over a predefined path. Due to the oscillating nature of this beam shaping method, the temporal heat input will differ from the temporal heat input by an equivalent non-oscillated laser intensity distribution. Mathematically this equivalent non-oscillated beam profile is the convolution of the laser intensity distribution of the oscillated focal spot and the predefined oscillation path. This work presents simulation results of laser induced thermal fields. The thermal fields induced by oscillated laser beams at different frequencies are compared with the thermal field induced by the equivalent non-oscillated beam. These comparisons show that even at the highest oscillation frequencies the oscillated and equivalent non-oscillated laser beams induce significantly different in temperature fields (e.g. up to 325 K peak temperature difference) and different cooling rates (105 K/s and 5500 K/s respectively).

Multi-site solar irradiance prediction based on hybrid spatiotemporal graph neural network

Constructing accurate spatiotemporal correlations is a challenging task in joint prediction of multiple photovoltaic sites. Some advanced algorithms for incorporating other surrounding site information have been proposed, such as graph neural network-based methods, which are usually based on static or dynamic graphs to build spatial dependencies between sites. However, the possibility of the simultaneous existence of multiple spatial dependencies is not considered. This paper establishes a spatiotemporal prediction model based on hybrid spatiotemporal graph neural network. In this model, we apply adaptive hybrid graph learning to learn composite spatial correlations among multiple sites. A temporal convolution module with multi-subsequence temporal data input is used to extract local semantic information to better predict future nonlinear temporal dependencies. A spatiotemporal adaptive fusion module is added to address the issue of integrating diverse spatiotemporal trends among multiple sites. To assess the model's predictive performance, nine solar radiation observation stations were selected in two different climatic environments. The average root mean square error (RMSE) of the constructed model was 38.51 and 49.90 W/m2, with average mean absolute error (MAE) of 14.72 and 23.06 W/m2, respectively. Single-site and multi-site prediction models were selected as baseline models. Compared with the baseline models, the RMSE and MAE reduce by 3.1%–20.8% and 8.9%–32.8%, respectively, across all sites. The proposed model demonstrates the effectiveness of improving accuracy in forecasting solar irradiance through multi-site predictions.

Identification of Optimal Data Augmentation Techniques for Multimodal Time-Series Sensory Data: A Framework

Recently, the research community has shown significant interest in the continuous temporal data obtained from motion sensors in wearable devices. These data are useful for classifying and analysing different human activities in many application areas such as healthcare, sports and surveillance. The literature has presented a multitude of deep learning models that aim to derive a suitable feature representation from temporal sensory input. However, the presence of a substantial quantity of annotated training data is crucial to adequately train the deep networks. Nevertheless, the data originating from the wearable devices are vast but ineffective due to a lack of labels which hinders our ability to train the models with optimal efficiency. This phenomenon leads to the model experiencing overfitting. The contribution of the proposed research is twofold: firstly, it involves a systematic evaluation of fifteen different augmentation strategies to solve the inadequacy problem of labeled data which plays a critical role in the classification tasks. Secondly, it introduces an automatic feature-learning technique proposing a Multi-Branch Hybrid Conv-LSTM network to classify human activities of daily living using multimodal data of different wearable smart devices. The objective of this study is to introduce an ensemble deep model that effectively captures intricate patterns and interdependencies within temporal data. The term “ensemble model” pertains to fusion of distinct deep models, with the objective of leveraging their own strengths and capabilities to develop a solution that is more robust and efficient. A comprehensive assessment of ensemble models is conducted using data-augmentation techniques on two prominent benchmark datasets: CogAge and UniMiB-SHAR. The proposed network employs a range of data-augmentation methods to improve the accuracy of atomic and composite activities. This results in a 5% increase in accuracy for composite activities and a 30% increase for atomic activities.

“TimeOntoImperfection”: An OWL 2 ontology to representing temporal data imperfection using imperfection theories

Dealing with temporal data imperfection is a crucial issue in several application domains. In fact, failure to handle these imperfections can have significant consequences and lead to incorrect analysis and decision-making. This is particularly true when handling imperfect temporal data inputs in applications for Alzheimer’s patients as a real example. In this context, there is a need for a global ontology that provides a semantic representation of temporal data imperfection. In the literature, there is a big number of ontologies that represent data. Some represent only perfect temporal data. Some others represent imperfect data but not temporal ones. To the best of our knowledge, there is no ontology that represents temporal data imperfection. In this paper, we represent “TimeOntoImperfection”, a usable global ontology that represents four types of imperfection: imprecision, uncertainty, both uncertainty and imprecision and conflict. We describe the structure of “TimeOntoImperfection”, then we conduct a case a study in which we illustrate the usefulness of our ontology. Finally, we introduce the validation part in the context of CAPTAIN MEMO - an ontology based memory prothesis dedicated to alzheimer patients- and we discuss the encouraging results derived from the evaluation step.

Predictive process model monitoring using long short-term memory networks

The field of predictive process monitoring focuses on case-level models to predict a single specific outcome such as a particular objective, (remaining) time, or next activity/remaining sequence. Recently, a longer-horizon, model-wide approach has been proposed in the form of process model forecasting, which predicts the future state of a whole process model through the forecasting of all activity-to-activity relations at once using time series forecasting.This paper introduces the concept of predictive process model monitoring which sits in the middle of both predictive process monitoring and process model forecasting. Concretely, by modelling a process model as a set of constraints being present between activities over time, we can capture more detailed information between activities compared to process model forecasting, while being compatible with typical predictive process monitoring objectives which are often expressed in the same language as these constraints. To achieve this, Processes-As-Movies (PAM) is introduced, i.e., a novel technique capable of jointly mining and predicting declarative process constraints between activities in various windows of a process’ execution. PAM predicts what declarative rules hold for a trace (objective-based), which also supports the prediction of all constraints together as a process model (model-based). Various recurrent neural network topologies inspired by video analysis tailored to temporal high-dimensional input are used to model the process model evolution with windows as time steps, including encoder–decoder long short-term memory networks, and convolutional long short-term memory networks. Results obtained over real-life event logs show that these topologies are effective in terms of predictive accuracy and precision.

Gaussian process-based state derivative estimator with temporal input in incremental flight control design

In this paper, a state derivative estimation technique is proposed using a derivative of Gaussian process regression method and applied to an incremental dynamics-based controller. The state derivative estimates of interest are with respect to time, and to obtain them, the temporal index array is used for Gaussian process state input. Using a Gaussian process, assumptions such as numerical differentiation of the state, parametric basis functions common in adaptive approaches, and the use of upper-bound information of the state derivative commonly used in robust methods are avoided. The proposed approach is compared with the backward difference formula (BDF) differentiator, first-order low-pass filter, second-order low-pass filter, and high-order sliding mode differentiator. Monte Carlo simulation with 1000 runs is performed under four different noise level measurement scenarios. The proposed Gaussian process-based differentiator with temporal index input resulted in better robustness and adaptability in state derivative estimation with similar or better performance.

3D-integrated multilayered physical reservoir array for learning and forecasting time-series information

A wide reservoir computing system is an advanced architecture composed of multiple reservoir layers in parallel, which enables more complex and diverse internal dynamics for multiple time-series information processing. However, its hardware implementation has not yet been realized due to the lack of a high-performance physical reservoir and the complexity of fabricating multiple stacks. Here, we achieve a proof-of-principle demonstration of such hardware made of a multilayered three-dimensional stacked 3 × 10 × 10 tungsten oxide memristive crossbar array, with which we further realize a wide physical reservoir computing for efficient learning and forecasting of multiple time-series data. Because a three-layer structure allows the seamless and effective extraction of intricate three-dimensional local features produced by various temporal inputs, it can readily outperform two-dimensional based approaches extensively studied previously. Our demonstration paves the way for wide physical reservoir computing systems capable of efficiently processing multiple dynamic time-series information.

Multimodal sensorimotor investigation of audio-visual integration in cochlear implant users

Rhythm is an omnipresent element of many daily activities. Numerous studies in cognitive sciences have highlighted that humans exhibit greater precision in synchronizing their movements with auditory rhythmic stimuli compared to visual ones. Deaf individuals were shown to excel in synchronizing with visual cues, surpassing those with normal hearing (NH). Furthermore, it was demonstrated that cochlear implant (CI) users were able to move in time to the beat of music, although not as well as NH controls. This study aims to investigate whether CI users retain a visual synchronization advantage from their pre-implant deafness, while maintaining auditory synchronization skills comparable to those of NH individuals, or if the neural reorganization post-implantation negates the visual synchronization advantage acquired pre-implantation. Specifically, we assessed both unimodal and multimodal auditory and visual abilities in CI users compared to NH controls using a standard sensorimotor synchronization paradigm. Results revealed that CI users exhibit comparable auditory rhythmic synchronization abilities to NH individuals, which is consistent with existing research, while not displaying a superior ability in synchronizing with visual rhythms, likely due to neural reorganization following implantation. This shift in audio-visual integration among CI users suggests that the post-implant reorganization of their auditory cortex might hinder the effective integration of temporal auditory input from the implant with visual information.

Reservoir Computing With Dynamic Reservoir using Cascaded DNA Memristors.

This article proposes molecular and DNA memristors where the state is defined by a single output variable. In past molecular and DNA memristors, the state of the memristor was defined based on two output variables. These memristors cannot be cascaded because their input and output sizes are different. We introduce a different definition of state for the molecular and DNA memristors. This change allows cascading of memristors. The proposed memristors are used to build reservoir computing (RC) models that can process temporal inputs. An RC system consists of two parts: reservoir and readout layer. The first part projects the information from the input space into a high-dimensional feature space. We also study the input-state characteristics of the cascaded memristors and show that the cascaded memristors retain the memristive behavior. The cascade connections in a reservoir can change dynamically; this allows the synthesis of a dynamic reservoir as opposed to a static one in the prior work. This reduces the number of memristors significantly compared to a static reservoir. The inputs to the readout layer correspond to one molecule per state instead of two; this significantly reduces the number of molecular and DSD reactions for the readout layer. A DNA RC system consisting of DNA memristors and a DNA readout layer is used to detect seizures from intra-cranial electroencephalogram (iEEG). We also demonstrate that a DNA RC system consisting of three cascaded DNA memristors and a DNA readout layer can be used to solve the time-series prediction task. The proposed approach can reduce the number of DNA strand displacement (DSD) reactions by three to five times compared to prior approaches.

Snowfall deposition in mountainous terrain: a statistical downscaling scheme from high-resolution model data on simulated topographies

One of the primary causes of non-uniform snowfall deposition on the ground in mountainous regions is the preferential deposition of snow, which results from the interaction of near-surface winds with topography and snow particles. However, producing high-resolution snowfall deposition patterns can be computationally expensive due to the need to run full atmospheric models. To address this, we developed two statistical downscaling schemes that can efficiently downscale near-surface, low-resolution snowfall data to fine-scale snow deposition accounting for the effect of preferential deposition in mountainous regions. Our approach relies on a comprehensive, model database generated using 3D wind fields from an atmospheric model and a preferential deposition model on several thousand simulated topographies covering a broad range in terrain characteristics. Both snowfall downscaling schemes rely on fine-scale topographic scaling parameters and low-resolution wind speed as input. While one scheme, referred to as the “wind scheme”, further necessitates fine-scale vertical wind components, a second scheme, referred to as the “aspect scheme”, does not require fine-scale temporal input. We achieve this by additionally downscaling near-surface vertical wind speed solely using topographic scaling parameters and low-resolution wind direction. We assess the performance of our downscaling schemes using an independent subset of the model database on simulated topographies, model data on actual terrain, and spatially measured new snow depth obtained through a photogrammetric drone survey following a snowfall event on previously snow-free ground. While the assessments show that our downscaling schemes perform well (relative errors ≤ ±3% with modeled and ≤ ±6% with measured snowfall deposition), they also demonstrate comparable results to benchmark downscaling models. However, our schemes notably outperform the benchmark models in representing fine-scale patterns. Our downscaling schemes possess several key features, including high computational efficiency, versatility enabled by the comprehensive model database, and independence from fine-scale temporal input data (aspect scheme), indicating their potential for widespread applicability. Therefore, our downscaling schemes for near-surface snowfall and vertical wind speed can be beneficial for various applications at fine grid resolutions such as in atmospheric and climate sciences, snow hydrology, glaciology, remote sensing, and avalanche sciences.

SegMorph: Concurrent Motion Estimation and Segmentation for Cardiac MRI Sequences.

We propose a novel recurrent variational network, SegMorph, to perform concurrent segmentation and motion estimation on cardiac cine magnetic resonance image (CMR) sequences. Our model establishes a recurrent latent space that captures spatiotemporal features from cine-MRI sequences for multitask inference and synthesis. The proposed model follows a recurrent variational auto-encoder framework and adopts a learnt prior from the temporal inputs. We utilise a multi-branch decoder to handle bi-ventricular segmentation and motion estimation simultaneously. In addition to the spatiotemporal features from the latent space, motion estimation enriches the supervision of sequential segmentation tasks by providing pseudo-ground truth. On the other hand, the segmentation branch helps with motion estimation by predicting deformation vector fields (DVFs) based on anatomical information. Experimental results demonstrate that the proposed method performs better than state-of-the-art approaches qualitatively and quantitatively for both segmentation and motion estimation tasks. We achieved an 81% average Dice Similarity Coefficient (DSC) and a less than 3.5 mm average Hausdorff distance on segmentation. Meanwhile, we achieved a motion estimation Dice Similarity Coefficient of over 79%, with approximately 0.14% of pixels displaying a negative Jacobian determinant in the estimated DVFs.

Reconstruction of Supersonic Flowfield Using Physical Neural Network Based on Channel Interaction

The evolution of shock waves in a combustor is a complex process comprising multiple time series, which results in a long computational cycle for unsteady numerical simulations. Current data-driven methods integrating physical information provide novel opportunities to rapidly and accurately predict the flowfield in the combustor. In this study, we construct a dataset under Mach numbers from 2 to 5 based on a self-designed scramjet combustor model and propose a physical convolutional neural network flowfield reconstruction method based on channel interaction (PICNNCI). This method embeds Euler equations that characterize the flow characteristics of inviscid fluids as physical constraints to solve the low-accuracy problem of traditional convolutional neural networks arising due to insufficient prior information. The model first performs dimensional transformations on temporal and spatial information input and converts a one-dimensional column vector into a two-dimensional vector to meet the convolution operation rules. Then, convolution is used for feature extraction to output the predicted flowfield, including velocity, pressure, and density fields. Finally, the predicted flowfield is optimized using the Euler equation and compared with the results obtained by numerical calculation and traditional neural network methods. The average peak signal-to-noise ratio of PICNNCI reaches 34.01 dB, and the linear correlation reaches 0.994.

A reservoir of timescales emerges in recurrent circuits with heterogeneous neural assemblies.

The temporal activity of many physical and biological systems, from complex networks to neural circuits, exhibits fluctuations simultaneously varying over a large range of timescales. Long-tailed distributions of intrinsic timescales have been observed across neurons simultaneously recorded within the same cortical circuit. The mechanisms leading to this striking temporal heterogeneity are yet unknown. Here, we show that neural circuits, endowed with heterogeneous neural assemblies of different sizes, naturally generate multiple timescales of activity spanning several orders of magnitude. We develop an analytical theory using rate networks, supported by simulations of spiking networks with cell-type specific connectivity, to explain how neural timescales depend on assembly size and show that our model can naturally explain the long-tailed timescale distribution observed in the awake primate cortex. When driving recurrent networks of heterogeneous neural assemblies by a time-dependent broadband input, we found that large and small assemblies preferentially entrain slow and fast spectral components of the input, respectively. Our results suggest that heterogeneous assemblies can provide a biologically plausible mechanism for neural circuits to demix complex temporal input signals by transforming temporal into spatial neural codes via frequency-selective neural assemblies.

CNN-GRU based method for peak location of reflected Terahertz signals from thermal barrier coatings

ABSTRACT Due to the limitation of spraying conditions, the microstructure of Thermal Barrier Coatings (TBCs) is not homogeneous, so it is important to extract the peak information of the reflected signal efficiently and accurately using Terahertz (THz) non-destructive detection technique for online thickness measurement. To this end, a hybrid model peak information extraction method based on a convolutional neural network (CNN) and a gated recurrent unit (GRU) is proposed. Firstly, a theoretical model of the THz signal is used to generate the simulated signal; secondly, a 1D-CNN is used to extract features adaptively from the temporal signal input, and a GRU is constructed to learn the temporal information between feature vectors to complete the extraction of peak information; finally, a calibration strategy is employed to determine the highest value around the original localisation result in order to eliminate localisation error and achieve peak location. The proposed CNN-GRU model predicts evaluation metrics that are 2–4 times smaller than those of other methods, and the running time is reduced to 45.71% compared to networks with close prediction accuracy. The method can reduce manual involvement and time expenses while maintaining prediction accuracy and that it provides a new intelligent way for online thickness measuring.

Early deficits in dentate circuit and behavioral pattern separation after concussive brain injury

Traumatic brain injury leads to cellular and circuit changes in the dentate gyrus, a gateway to hippocampal information processing. Intrinsic granule cell firing properties and strong feedback inhibition in the dentate are proposed as critical to its ability to generate unique representation of similar inputs by a process known as pattern separation. Here we evaluate the impact of brain injury on cellular decorrelation of temporally patterned inputs in slices and behavioral discrimination of spatial locations in vivo one week after concussive lateral fluid percussion injury (FPI) in mice. Despite posttraumatic increases in perforant path evoked excitatory drive to granule cells and enhanced ΔFosB labeling, indicating sustained increase in excitability, the reliability of granule cell spiking was not compromised after FPI. Although granule cells continued to effectively decorrelate output spike trains recorded in response to similar temporally patterned input sets after FPI, their ability to decorrelate highly similar input patterns was reduced. In parallel, encoding of similar spatial locations in a novel object location task that involves the dentate inhibitory circuits was impaired one week after FPI. Injury induced changes in pattern separation were accompanied by loss of somatostatin expressing inhibitory neurons in the hilus. Together, these data suggest that the early posttraumatic changes in the dentate circuit undermine dentate circuit decorrelation of temporal input patterns as well as behavioral discrimination of similar spatial locations, both of which could contribute to deficits in episodic memory.

MFAN: Mixing Feature Attention Network for trajectory prediction

Accurate trajectory prediction of surrounding agents is essential for autonomous vehicles, where the key challenge is to understand the complex interactions among agents. Previous works treat all interacted features between agents equally in modeling interaction, while neglecting their different importance to the interaction, thus inevitably limiting the interaction modeling ability. Besides, existing methods suffer from significant performance degradation when domain shifts, resulting in severely deviant prediction from reality. To address these issues, we propose a novel prediction framework, dubbed Mixing Feature Attention Network (MFAN). Specifically, the proposed mixing feature attention is a parallel design to adaptively determine the importance of different interacted features and simultaneously capture the global interaction feature to improve interaction modeling. Meanwhile, the spatial global interaction is modeled from a spatial edge-featured graph input to capture the enhanced spatial interaction. The temporal motion pattern is modeled from a temporal edge-featured graph input to enhance the domain adaption. Finally, we estimate the parameters of bivariant Gaussian distribution for trajectory prediction. Experimental results show that our method achieves superior performance in trajectory prediction while maintaining low computational complexity and performs accurate prediction even when domain shifts.

What, if anything, is coincidence detection?

Neural delays and coincidence detection are neural operations posited as key elements in temporal processing, in the auditory system and beyond. Although longstanding physiological evidence for coincidence detectors exists at a phenomenological level, mechanistic examinations have only recently been performed. We review intracellular recordings from two cell types which are traditionally regarded as prototypes of coincidence detectors. Neurons in the medial superior olive (MSO) are sensitive to “coincidences” in action potentials from the ipsi- and contralateral ear, making the neurons sensitive to interaural temporal differences. Octopus cells in the cochlear nucleus are sensitive to “coincidences” of action potentials across inputs tuned to a wide range of frequencies. We found that in both cases, the spike output of neurons does not simply reflect coincidences of input spikes. When inputs vary in strength, their sequence of activation is an important determinant of response strength. Maximal output is not just determined by the maximal number of coincidences, but reflects temporal input structure in interaction with intrinsic (membrane) properties. These observations suggest a simpler mechanism of temporal sensitivity than traditional lag and coincidence proposals.

A Deep Learning Data Fusion Model Using Sentinel-1/2, SoilGrids, SMAP, and GLDAS for Soil Moisture Retrieval

Abstract We develop a deep learning–based convolutional-regression model that estimates the volumetric soil moisture content in the top ∼5 cm of soil. Input predictors include Sentinel-1 (active radar) and Sentinel-2 (multispectral imagery), as well as geophysical variables from SoilGrids and modeled soil moisture fields from SMAP and GLDAS. The model was trained and evaluated on data from ∼1000 in situ sensors globally over the period 2015–21 and obtained an average per-sensor correlation of 0.707 and ubRMSE of 0.055 m3 m−3, and it can be used to produce a soil moisture map at a nominal 320-m resolution. These results are benchmarked against 14 other soil moisture evaluation research works at different locations, and an ablation study was used to identify important predictors. Significance Statement Soil moisture is a key variable in various agriculture and water management systems. Accurate and high-resolution estimates of soil moisture have multiple downstream benefits such as reduced water wastage by better understanding and managing the consumption of water, utilizing smarter irrigation methods and effective canal water management. We develop a deep learning–based model that estimates the volumetric soil moisture content in the top ∼5 cm of soil at a nominal 320-m resolution. Our results demonstrate that machine learning is a useful tool for fusing different modalities with ease, while producing high-resolution models that are not location specific. Future work could explore the possibility of using temporal input sources to further improve model performance.

Ultralow‐Power Compact Artificial Synapse Based on a Ferroelectric Fin Field‐Effect Transistor for Spatiotemporal Information Processing

Artificial synapses are key elements in building bioinspired, neuromorphic computing systems. Ferroelectric field‐effect transistors (FeFETs) with excellent controllability and complementary metal oxide semiconductor (CMOS) compatibility are favorable to achieving synaptic functions with low power consumption and high scalability. However, because of the only nonvolatile ferroelectric (Fe) characteristics in the FeFET, it is difficult to develop bioplausible short‐term synaptic elements for spatiotemporal information processing. By judiciously combining defects (DE) and Fe domains in gate stacks, a compact artificial synapse featuring spatiotemporal information processing on a single Fe–DE fin FET (FinFET) is proposed. The devices are designed to work in a separate DE mode to induce short‐term plasticity by spontaneous charge detrapping, and a hybrid Fe–DE mode to trigger long‐term plasticity through the coupling of defects and Fe domains. The capability of the compact synapse is demonstrated by differentiating 16 temporal inputs. Moreover, the highly controllable static electricity of advanced FinFETs leads to an ultralow power of 2 fJ spike−1. An all Fe–DE FinFET reservoir computing (RC) system is then constructed that achieves a recognition accuracy of 97.53% in digit classification. This work enables constructing RC systems with fully advanced CMOS‐compatible devices featuring highly energy‐efficient and low‐hardware systems.

Real-Time Forecasting of Dockless Scooter-Sharing Demand: A Spatio-Temporal Multi-Graph Transformer Approach

Accurately forecasting the real-time travel demand for dockless scooter-sharing is crucial for the planning and operations of transportation systems. Deep learning models provide researchers with powerful tools to achieve this task, but research in this area is still lacking. This paper thus proposes a novel deep learning architecture named Spatio-Temporal Multi-Graph Transformer (STMGT) to forecast the real-time spatiotemporal dockless scooter-sharing demand. The proposed model uses a graph convolutional network (GCN) based on adjacency graph, functional similarity graph, demographic similarity graph, and transportation supply similarity graph to attach spatial dependency to temporal input (i.e., historical demand). The output of GCN is subsequently processed with weather condition information by the Transformer to capture temporal dependency. Then, a convolutional layer is used to generate the final prediction. The proposed model is evaluated for two real-world case studies in Washington, D.C. and Austin, TX, respectively, and the results show that for both case studies, STMGT significantly outperforms all the selected benchmark models, and the most important model component is the weather information. The proposed model can help micromobility operators develop optimal vehicle rebalancing schemes and guide cities to better manage dockless scooter-sharing operations.