Multiple Time Steps Research Articles

Digital twin-driven graph domain adaptation neural network for remaining useful life prediction of rolling bearing

Remaining useful life (RUL) prediction is significant for the healthy operation of machinery. In order to accurately identify the bearing degeneration states, it is necessary to collect massive full lifecycle data. However, the bearing lifecycle data is insufficient for effectively training a RUL prediction model in engineering practice. In this paper, a digital twin-driven graph domain adaptation method is proposed. First, a full lifecycle dynamic twin model of bearings is constructed to generate abundant twin data, in which the surface morphology evolution and roller relative slip at different stages are simulated to generate vibration responses. Second, a novel multi-layered cross-domain gated graph convolutional network (MGGCN) is developed, in which a new graph domain adaptation model is designed to solve the problem that traditional domain adaptation methods are not effective in processing the non-Euclidean data. The spatial and temporal features are extracted by multiple nonlinear transformations and previous time-step hidden state incorporation, respectively. In addition, a graph Laplacian regularized maximum mean discrepancy (GLMMD) is designed and applied in the training of model to enhance the capability of discerning graph domain differences. The experimental results confirm that the proposed method can achieve effective performance even in scenarios with limited actual data.

A novel extreme adaptive GRU for multivariate time series forecasting

Multivariate time series forecasting is a critical problem in many real-world scenarios. Recent advances in deep learning have significantly enhanced the ability to tackle such problems. However, a primary challenge in time series forecasting comes from the imbalanced time series data that include extreme events. Despite being a small fraction of the data instances, extreme events can have a negative impact on forecasting as they deviate from the majority. However, many recent time series forecasting methods neglect this issue, leading to suboptimal performance. To address these challenges, we introduce a novel model, the Extreme Event Adaptive Gated Recurrent Unit (eGRU), tailored explicitly for forecasting tasks. The eGRU is designed to effectively learn both normal and extreme event patterns within time series data. Furthermore, we introduce a time series data segmentation technique that divides the input sequence into segments, each comprising multiple time steps. This segmentation empowers the eGRU to capture data patterns at different time step resolutions while simultaneously reducing the overall input length. We conducted comprehensive experiments on four real-world benchmark datasets to evaluate the eGRU’s performance. Our results showcase its superiority over vanilla RNNs, LSTMs, GRUs, and other state-of-the-art RNN variants in multivariate time series forecasting. Additionally, we conducted ablation studies to demonstrate the consistently superior performance of eGRU in generating accurate forecasts while incorporating a diverse range of labeling results.

Integrated surrogate framework for reactive transport simulation of uranium in situ leaching with generative models

The heterogeneity of uranium-bearing aquifers plays a crucial role in governing the migration and leaching of uranium. Uncertainty quantification (UQ) based on the Monte Carlo method is usually used to deal with the issue of aquifer heterogeneity. However, the substantial computational costs of this method arise from a large number of calls to the uranium reactive transport simulation; therefore, surrogate models are developed to reduce these costs. This study leverages a conditional deep convolutional generative adversarial network (cDC-GAN) and an adversarial autoencoder-based kriging regression (AAE-KRG) to develop an integrated surrogate framework for uranium reactive transport simulation. cDC-GAN and AAE-KRG are designed to achieve the cross-domain mapping of the permeability coefficient field to the uranium spatial distribution and leaching concentration over time, respectively. The results demonstrate a structural similarity exceeding 0.9 between the cDC-GAN and the numerical simulation, with a mean square error generally less than 0.002. These findings suggest that the cDC-GAN exhibits high accuracy and stability in terms of replacing the numerical simulation to obtain the spatial-temporal evolution of the uranium plume. Additionally, cDC-GAN is a dynamic surrogate model with the capability of interpolation because uranium spatial distributions with multiple time steps are used as training data during the training process. AAE-KRG also exhibits high accuracy and stability in terms of predicting the leaching concentration by permeability coefficient fields, as evidenced by determination coefficients generally exceeding 0.95. By comparing the computational costs, it is clear that cDC-GAN and AAE-KRG are suitable for processing spatial distribution and time series data, respectively. The trained surrogate models are used to undertake UQ of the uranium leaching area and leaching concentration. The results indicate that the uncertainty of the leaching area in the field-test model increases with time, and the increasing rate reaches a maximum in the middle period (201–300 d). The uncertainty of the leaching concentration increases with time in the early period, decreases after peaking at 137 d, and stabilizes in the middle and late periods. It is observed that the change characteristics of the uncertainty are consistent with the values of the leaching area or concentration. Furthermore, the spatial distribution pattern of uranium pollution is determined based on stochastic simulation results. The uncertainty intervals and the pollution risk status at multiple time steps are quickly obtained by surrogate models, which is helpful for scheme optimization in uranium mining. Owing to its characteristics of being data-driven and highly adaptable to nonlinear data, the integrated surrogate framework has applications beyond uranium in situ leaching.

Soil Moisture Prediction Model Based on Improved GRU Recurrent Neural Network

Soil moisture plays a crucial role in land water and energy cycles, and has a certain impact on weather and climate change. In agricultural production, crop moisture status can be determined based on soil moisture, and timely and effective irrigation strategies can be formulated to ensure grain yield while saving water resources, maximizing the value of agricultural water resource utilization, and achieving sustainable development. Therefore, the accuracy of soil moisture prediction has important research value for many fields such as agriculture and climate. In this paper, the super parameters of GRU Recurrent neural network are optimized by intelligent seagull optimization algorithm using a small number of influencing factors, namely, atmospheric temperature, atmospheric humidity, rainfall and soil moisture data, and a soil moisture prediction model is established. The model was used to predict soil moisture for the next 12 hours, 24 hours, 36 hours, and 48 hours, respectively. The final experiment showed that the model in this paper had better predictive effect on soil moisture, with the best predictive evaluation index data being MAPE (12h) = 4.4120%, R2 (12h) = 0.94605, and RMSE (12h) = 1.9998; By comparing the prediction results of multiple time steps vertically, it was found that the prediction accuracy of the model in this paper decreased more smoothly, meeting the requirements of soil moisture prediction.

LEISN: A long explicit–implicit spatio-temporal network for traffic flow forecasting

Recent studies have shown that it is necessary to further improve the prediction accuracy of complex and dynamic traffic flow from two angles: temporal dependence and spatial dependence. Although many spatio-temporal networks are extracting features from both temporal and spatial dependencies, there are still two problems that need to be addressed. Firstly, existing methods often focus on modeling local spatial dependence relationship between adjacent nodes, but neglect non-local spatial dependence relationship between distant nodes. Secondly, while recent work has explored various methods such as convolutional and recurrent neural networks for modeling temporal dependence, they may not fully capture the long-term temporal dependence in traffic data. Therefore, addressing the challenges of long-term dependency, explicit spatial dependency, and implicit spatial dependency, this paper proposes a Long-term Explicit–Implicit Spatio-Temporal Network (LEISN) for traffic flow prediction. A Long-term Dependency Module has been designed to store hidden states generated from multiple previous time steps, facilitating the transmission of long-term features. Based on this, two graph convolution-based spatial feature extraction branches are designed to extract explicit spatial features based on the adjacency matrix generated by spatial topology and implicit spatial features based on the implicit adjacency matrix generated by trend similarity, respectively. Then all spatio-temporal features are fused to produce the next state. Additionally, a new encoder framework, LENSI-ED, was proposed based on LEISN. Comparative experiments are conducted on four datasets, and the results showed that our model has advantages over existing methods. The proposed model addresses the issues of local and non-local spatial dependence relationships and long-term temporal dependence in traffic flow forecasting.

Spatio-Temporal Joint Graph Convolutional Networks for Traffic Forecasting

Recent studies have shifted their focus towards formulating traffic forecasting as a spatio-temporal graph modeling problem. Typically, they constructed a static spatial graph at each time step and then connected each node with itself between adjacent time steps to create a spatio-temporal graph. However, this approach failed to explicitly reflect the correlations between different nodes at different time steps, thus limiting the learning capability of graph neural networks. Additionally, those models overlooked the dynamic spatio-temporal correlations among nodes by using the same adjacency matrix across different time steps. To address these limitations, we propose a novel approach called Spatio-Temporal Joint Graph Convolutional Networks (STJGCN) for accurate traffic forecasting on road networks over multiple future time steps. Specifically, our method encompasses the construction of both pre-defined and adaptive spatio-temporal joint graphs (STJGs) between any two time steps, which represent comprehensive and dynamic spatio-temporal correlations. We further introduce dilated causal spatio-temporal joint graph convolution layers on the STJG to capture spatio-temporal dependencies from distinct perspectives with multiple ranges. To aggregate information from different ranges, we propose a multi-range attention mechanism. Finally, we evaluate our approach on five public traffic datasets and experimental results demonstrate that STJGCN is not only computationally efficient but also outperforms 11 state-of-the-art baseline methods.

A hybrid deep learning approach to improve real-time effluent quality prediction in wastewater treatment plant

Wastewater treatment plant (WWTP) operation is usually intricate due to large variations in influent characteristics and nonlinear sewage treatment processes. Effective modeling of WWTP effluent water quality can provide valuable decision-making support to facilitate their operations and management. In this study, we developed a novel hybrid deep learning model by combining the temporal convolutional network (TCN) model with the long short-term memory (LSTM) network model to improve the simulation of hourly total nitrogen (TN) concentration in WWTP effluent. The developed model was tested in a WWTP in Jiangsu Province, China, where the prediction results of the hybrid TCN-LSTM model were compared with those of single deep learning models (TCN and LSTM) and traditional machine learning model (feedforward neural network, FFNN). The hybrid TCN-LSTM model could achieve 33.1 % higher accuracy as compared to the single TCN or LSTM model, and its performance could improve by 63.6 % comparing to the traditional FFNN model. The developed hybrid model also exhibited a higher power prediction of WWTP effluent TN for the next multiple time steps within eight hours, as compared to the standalone TCN, LSTM, and FFNN models. Finally, employing model interpretation approach of Shapley additive explanation to identify the key parameters influencing the behavior of WWTP effluent water quality, it was found that removing variables that did not contribute to the model output could further improve modeling efficiency while optimizing monitoring and management strategies.

Quasi-in-Situ Analysis of Electropolished Additively Manufactured Stainless Steel Surfaces

Progress in additive manufacturing is leading to the emergence of new areas of application. Laser Powder Bed Fusion (L-PBF) is increasingly used for the development of metallic medical implants, but for high-risk implants like vascular support structures (stents), surface quality is critical to ensure successful implantation without harming the surrounding tissue and ensure the patients’ health. Therefore, enhancing the surface quality is crucial. Electropolishing is a method for removing surface roughness by smoothing out micro-peaks and valleys. However, L-PBF structures have a high surface roughness due to metal particles adhering on the surface. To achieve a smooth surface for additively manufactured implants like stents using electropolishing, the removal of these particles needs to be studied in more detail.The objective of this study is to examine the electropolishing mechanism of 316L stainless steel samples additively manufactured through Laser Powder Bed Fusion (L-PBF). The main objective is to investigate the removal properties and surface characteristics during electropolishing. To achieve this, various surfaces were characterized for morphology and roughness during Hull cell experiments. Markings are utilized on the Hull cell sample surfaces to identify points of interest during quasi-in-situ measurements. The surfaces are then analyzed after multiple time steps, applying different currents to investigate particle dissolution. The surface characteristics are analyzed through scanning electron microscopy, and surface roughness is analyzed using laser scanning microscopy.The results show that the electropolishing process preferentially removes the adhering particles present on the surface of the samples. Increasing the current density results in faster particle dissolution and a smoother surface (see Figure 1a and b). The mechanism of material removal of various surface features, as shown in Figure 1 (red circle, yellow arrow and red square), was assessed based on the experimental results of the surface structures seen on the SEM images. It was found that different surface features were removed during the experiment at different polishing times and current densities. The amount of charge flowed was found to correlate with surface morphology.Based on the obtained results, various surface features (such as large adherent particles, agglomerates of smaller particles, and valleys) and their changes with increasing test duration and current density were observed by quasi-in situ analyses. A reduction in the diameter of round particles adhering to the surface was observed at both low and higher current densities (see Figure 1a red circle a). Increasing the polishing time resulted in leveling of both large particles and valleys (see Figure 1b red square). Also, dissolution of agglomerates of smaller particles occurred at different polishing times as a function of current density and polishing time (see Figure 1a yellow arrow) are observed.Smoothed surface structures can be observed in regions with equivalent surface charge density (see Figure 2). As a result, comparable surface morphologies may appear at the same area charge density, irrespective of a specific current density. So, it may be adequate to only consider the amount of charge flowed to describe the electropolishing of additive materials.In conclusion, comprehending the dissolution characteristics of particles on L-PBF surfaces is essential for attaining satisfactory surface finish in electropolishing. The results of this study offer valuable perspectives into the electropolishing mechanism of additively manufactured 316L stainless steel and can guide future investigations on surface finishing and polishing of additive manufactured implants like stents. Figure 1

A Fast Algorithm for Aperiodic Linear Stencil Computation using Fast Fourier Transforms

Stencil computations are widely used to simulate the change of state of physical systems across a multidimensional grid over multiple timesteps. The state-of-the-art techniques in this area fall into three groups: cache-aware tiled looping algorithms, cache-oblivious divide-and-conquer trapezoidal algorithms, and Krylov subspace methods. In this article, we present two efficient parallel algorithms for performing linear stencil computations. Current direct solvers in this domain are computationally inefficient, and Krylov methods require manual labor and mathematical training. We solve these problems for linear stencils by using discrete Fourier transforms preconditioning on a Krylov method to achieve a direct solver that is both fast and general. Indeed, while all currently available algorithms for solving general linear stencils perform Θ( NT ) work, where N is the size of the spatial grid and T is the number of timesteps, our algorithms perform o ( NT ) work. To the best of our knowledge, we give the first algorithms that use fast Fourier transforms to compute final grid data by evolving the initial data for many timesteps at once. Our algorithms handle both periodic and aperiodic boundary conditions and achieve polynomially better performance bounds (i.e., computational complexity and parallel runtime) than all other existing solutions. Initial experimental results show that implementations of our algorithms that evolve grids of roughly 10 7 cells for around 10 5 timesteps run orders of magnitude faster than state-of-the-art implementations for periodic stencil problems, and 1.3× to 8.5× faster for aperiodic stencil problems. Code Repository: https://github.com/TEAlab/FFTStencils

Advances in using mathematical optimization to manage floods with assessment of possible benefits using a case study

Abstract This paper presents the benefits of using mathematical optimization for reservoir operation based on the assumed availability of short-term runoff forecasts. The novelty is the inclusion of the SSARR hydrological routing as optimization constraints in multiple time step optimization, where the routing coefficients are adjusted dynamically as functions of the channel flows. The paper shows significant reduction of downstream peak flows in flood-prone areas even with a forecast horizon of only 2 days, and it also includes the results of testing the effects of different lengths of forecasting horizons on model results. The case study is conducted on the Damodar River Basin in the Indian State of West Bengal, where basin development started in the 1950s, with flood protection of the downstream river valley as the highest management priority, in addition to water supply and hydro power. The solution methodology and the model results presented in this paper pave the way for eventual introduction to automated management of reservoir outflows that could shape water resources industry in much the same way that auto-pilot and driverless cars are revolutionizing the transportation industry, assuming that runoff forecasting capabilities continue to improve.

An incremental singular value decomposition approach for large-scale spatially parallel & distributed but temporally serial data – applied to technical flows

The article presents a strategy and its algorithm to compile a simulation-accompanying, incremental Singular Value Decomposition (SVD) for time-evolving, spatially parallel discrete data sets. The framework addresses state-of-the-art PDE solvers for computational science and engineering applications. An important characteristic of such applications is that the spatial size of the data is often time-invariant and significantly exceeds the temporal size due to the large computational grid in 3D applications. Typical examples, which are also considered in this article, relate to results extracted from unsteady flow simulations. Herein, the flow data, which progresses over time, is frequently calculated spatially parallel based on domain decomposition strategies, which allow to parallelize the simulation on distributed memory machines following a Single Instruction Multiple Data (SIMD) concept.With a view to the memory-efficient reuse of (compressed) simulation results and their CPU time-saving, sufficiently accurate generation, the paper scrutinizes the efficiency of incremental/parallel SVD approaches for such simulation examples. To improve the computational efficiency, the introduction of a bunch matrix is proposed, which enables the aggregation of multiple time steps and SVD updates, and significantly increases the efficiency. The suggested strategy is verified and validated by simple 2D laminar single-phase flows and subsequently applied to more complex 2D and 3D turbulent two-phase flows. Emphasis is given to (a) the accuracy of SVD-based reconstruction, (b) the physical realizability of the reconstructed quantities, (c) the independence of domain partitioning, (d) an efficient snapshot bunching, and (e) related implementation aspects. In addition, the influence of lower and (adaptive) upper rank thresholds on the effort and accuracy is evaluated. A final application renders the practical benefits of the approach and refers to a merchant ship in head waves at Re = 1.4×107 and Fn = 0.26. The simulation involves 2880 processor cores and the related full-rank snapshot matrix has (108×104) entries. With a numerical overhead of O(10%), this snapshot matrix can be incrementally generated and compressed by O(95%). The compression is accompanied by only small errors in the integral force and local wave elevation of O(10−2%). This qualifies the method for an efficient subsequent data processing.

Research on the Prediction of Tire Radial Load Based on 1D CNN and BiGRU

As an important indicator of vehicle systems, tire load is a key factor in the structural design and safety assessment of vehicles. Direct measurement methods for tire loads are expensive and complicated, while conventional load identification methods are limited by low accuracy and poor robustness. This study aimed to propose a radial load identification method for rubber-tired vehicles based on a one-dimensional convolutional neural network (1D CNN) and bidirectional gated recurrent unit (BiGRU). Considering a priori information of the radial load data of tires and based on the observability of the vehicle vibration system, the proposed method selected feature sets and then retained the effective feature subsets through feature selection to construct samples with multiple time steps as input and with a single time step as output for network training. In doing so, the load prediction results were obtained, and the theoretical model was modified by integrating prediction accuracy, generalization performance, and robustness. Compared with traditional algorithms, the proposed method could effectively reduce the error of load identification, improve adaptability under different operating conditions, and handle the measurement error of different noise levels, which are of practical application value in the engineering field.

Stabilizing machine learning prediction of dynamics: Novel noise-inspired regularization tested with reservoir computing

Recent work has shown that machine learning (ML) models can skillfully forecast the dynamics of unknown chaotic systems. Short-term predictions of the state evolution and long-term predictions of the statistical patterns of the dynamics (“climate”) can be produced by employing a feedback loop, whereby the model is trained to predict forward only one time step, then the model output is used as input for multiple time steps. In the absence of mitigating techniques, however, this feedback can result in artificially rapid error growth (“instability”). One established mitigating technique is to add noise to the ML model training input. Based on this technique, we formulate a new penalty term in the loss function for ML models with memory of past inputs that deterministically approximates the effect of many small, independent noise realizations added to the model input during training. We refer to this penalty and the resulting regularization as Linearized Multi-Noise Training (LMNT). We systematically examine the effect of LMNT, input noise, and other established regularization techniques in a case study using reservoir computing, a machine learning method using recurrent neural networks, to predict the spatiotemporal chaotic Kuramoto–Sivashinsky equation. We find that reservoir computers trained with noise or with LMNT produce climate predictions that appear to be indefinitely stable and have a climate very similar to the true system, while the short-term forecasts are substantially more accurate than those trained with other regularization techniques. Finally, we show the deterministic aspect of our LMNT regularization facilitates fast reservoir computer regularization hyperparameter tuning.

SpikeSim: An End-to-End Compute-in-Memory Hardware Evaluation Tool for Benchmarking Spiking Neural Networks

Spiking Neural Networks (SNNs) are an active research domain towards energy efficient machine intelligence. Compared to conventional artificial neural networks (ANNs), SNNs use temporal spike data and bio-plausible neuronal activation functions such as Leaky-Integrate Fire/Integrate Fire (LIF/IF) for data processing. However, SNNs incur significant dot-product operations causing high memory and computation overhead in standard von-Neumann computing platforms. To this end, In-Memory Computing (IMC) architectures have been proposed to alleviate the “memory-wall bottleneck" prevalent in von-Neumann architectures. Although recent works have proposed IMC-based SNN hardware accelerators, the following key implementation aspects have been overlooked 1) the adverse effects of crossbar non-ideality on SNN performance due to repeated analog dot-product operations over multiple time-steps 2) hardware overheads of essential SNN-specific components such as the LIF/IF and data communication modules. To this end, we propose SpikeSim, a tool that can perform realistic performance, energy, latency and area evaluation of IMC-mapped SNNs. SpikeSim consists of a practical monolithic IMC architecture called SpikeFlow for mapping SNNs. Additionally, the non-ideality computation engine (NICE) and energy-latency-area (ELA) engine performs hardware-realistic evaluation of SpikeFlow-mapped SNNs. Based on 65nm CMOS implementation and experiments on CIFAR10, CIFAR100 and TinyImagenet datasets, we find that the LIF/IF neuronal module has significant area contribution (>11% of the total hardware area). To this end, we propose SNN topological modifications that leads to 1.24× and 10× reduction in the neuronal module’s area and the overall energy-delay-product value, respectively. Furthermore, in this work, we perform a holistic comparison between IMC implemented ANN and SNNs and conclude that lower number of time-steps are the key to achieve higher throughput and energy-efficiency for SNNs compared to 4-bit ANNs. The code repository for the SpikeSim tool is available at https://github.com/Intelligent-Computing-Lab-Yale/Quanitzation-aware-SNN-training-and-hardware-evaluation-for-IMC-Architectures

NTformer: Near Time Transformer for multi-step prediction of wellhead pressure in fracturing operations

Fracturing process technology is a major factor in the effectiveness of fracturing for increasing production. One of the most critical issues during the construction of fracturing is the control of pressure variation, as excessive pressure can lead to serious accidents such as fracturing fluid leakage and well-bore control loss. To prevent accidents and improve operational efficiency, it is necessary to predict the future multi-step fracturing pressure in advance. Since fracturing operations are a process based on continuous changes in time dimensions, and their feature information has a high correlation in time series, this article optimizes a Transformer network with a serialization structure for wellhead pressure prediction, called Near Time Transformer (NTformer), which can effectively forecast pressure values across multiple time steps without relying on intricate feature engineering or domain-specific knowledge. Firstly, data cleaning and temporal encoding are performed on the original data to reduce the complexity of features at different stages and solve the problem of data temporal misalignment. To better learn from features, a feature conversion mechanism is proposed that the data features after embedding are divided into domain feature and wave feature. Further, utilizing the attention concept to spatially correlate feature information, and using the Transformer structure to transform features to achieve forecasting of future multi-step pressure. Finally, the idea of regularization is used to limit the parameters of the model to avoid over-fitting. Experimental results show that the method has a high prediction accuracy and can satisfy the accuracy requirements of the field application, offering a new ancillary method for improving the efficiency of oil and gas well fracturing construction and preventing safety accidents.

A forecasting framework on fusion of spatiotemporal features for multi-station PM2.5

Persistent PM2.5 pollution poses a serious threat to human health. Developing an accurate urban regional PM2.5 forecasting is of practical significance for environmental protection. However, previous studies have mostly focused on individual monitoring stations, neglecting the influence of neighboring stations, which limits forecasting accuracy. Additionally, the PM2.5 of a single monitoring station cannot reflect the overall situation of a region. Therefore, this paper develops a novel PM2.5 spatiotemporal forecasting framework that combines graph convolutional module, temporal convolutional module, linear module. It enables the forecasting of PM2.5 concentrations at multiple stations and multiple time steps in the future. Concretely, we utilize a mixed graph convolutional network to extract the spatial features of PM2.5. Then, an improved temporal convolutional network, the second-order residual temporal convolutional network, is developed to capture complex temporal features. Following the classical “linear and non-linear” modeling strategy, a linear module is added to the forecasting framework. Experiments on the real air pollution dataset from Beijing demonstrate that our framework outperforms the state-of-the-art baselines.

A Homomorphic Encryption Framework for Privacy-Preserving Spiking Neural Networks

Machine learning (ML) is widely used today, especially through deep neural networks (DNNs); however, increasing computational load and resource requirements have led to cloud-based solutions. To address this problem, a new generation of networks has emerged called spiking neural networks (SNNs), which mimic the behavior of the human brain to improve efficiency and reduce energy consumption. These networks often process large amounts of sensitive information, such as confidential data, and thus privacy issues arise. Homomorphic encryption (HE) offers a solution, allowing calculations to be performed on encrypted data without decrypting them. This research compares traditional DNNs and SNNs using the Brakerski/Fan-Vercauteren (BFV) encryption scheme. The LeNet-5 and AlexNet models, widely-used convolutional architectures, are used for both DNN and SNN models based on their respective architectures, and the networks are trained and compared using the FashionMNIST dataset. The results show that SNNs using HE achieve up to 40% higher accuracy than DNNs for low values of the plaintext modulus t, although their execution time is longer due to their time-coding nature with multiple time steps.

NUTS-BSNN: A non-uniform time-step binarized spiking neural network with energy-efficient in-memory computing macro

This work introduces a network architecture NUTS-BSNN: A Non-uniform Time-step Binarized Spiking Neural Network. NUTS-BSNN is a fully binarized spiking neural network with all binary weights, including the input and output layers. In the input and output layers, the weights are represented as stochastic series of numbers, while in the hidden layers, they are approximated to binary values for using simple XNOR-based computations. To compensate for the information loss due to binarization, we increased the convolutions at the input layer sequentially computed over multiple time-steps. The results from these operations are accumulated before generating spikes for the subsequent layers to increase the overall performance. We chose 14 time-steps for accumulation to achieve a good tradeoff between performance and inference latency. The proposed technique was evaluated using three datasets by direct training method and using a surrogate gradient algorithm. We achieved classification accuracies of 93.25%, 88.71%, and 70.31% on the Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets, respectively. Further, we present an in-memory computing architecture for NUTS-BSNN, which limits resource and power consumption for hardware implementation.

Revealing the predictive power of neural operators for strain evolution in digital composites

The demand for high-performance materials, along with advanced synthesis technologies such as additive manufacturing and 3D printing, has spurred the development of hierarchical composites with superior properties. However, computational modelling of such composites using physics-based solvers, while enabling the discovery of optimal microstructures, have prohibitively high computational cost hindering their practical application. To this extent, we show that Neural Operators (NOs) can be used to learn and predict the strain evolution in 2D digital composites. Specifically, we consider three architectures, namely, Fourier NO (FNO), Wavelet NO (WNO), and Multi-wavelet NO (MWT). We demonstrate that by providing a few initial strain frames as input, NOs can accurately predict multiple future time steps in an extremely data-efficient fashion, especially WNO. Further, once trained, NOs forecast the strain trajectories for completely unseen boundary conditions. Among NOs, only FNO offers super-resolution capabilities for estimating strains at multiple length scales, which can provide higher material and pixel-wise resolution. We also show that NOs can generalize to arbitrary geometries with finer domain resolution without the need for additional training. Based on all the results presented, we note that the FNO exhibits the best performance among the NOs, while also giving minimum inference time that is almost three orders magnitude lower than the conventional finite element solutions. Thus, FNOs can be used as a surrogate for accelerated simulation of the strain evolution in complex microstructures toward designing the next composite materials.

Analytic prediction for the threshold of non-Markovian epidemic process on temporal networks

The transmission of pathogen between hosts and the interactions among hosts are two crucial factors for the spreading of epidemics. The former process is generally non-Markovian as the amount of the pathogen developed in hosts undergoes complicated biological process, while the latter one is time-varying due to the dynamic nature of modern society. Despite the abundant efforts working on the effects of the two aspects, a framework that integrates these two factors in a unified representation is still missing. In this paper, we develop a framework with tensorial description encoding non-Markovian process and temporal structure by introducing a super-matrix representation that incorporates multiple discrete time steps in a chronological order. Our proposed framework formulated with super-matrix representation allows a general analytical derivation of the epidemic threshold in terms of the spectral radius of the super-matrix. The accuracy of the approach is verified by different temporal network models. This framework could serve as an effective tool to offer novel understanding of integrated dynamics induced from non-Markovian individual processes and temporal interacting structures.