Length Of Input Sequence Research Articles

Comparison of time-series models for predicting physiological metrics under sedation.

This study presents a comprehensive comparison of multiple time-series models applied to physiological metric predictions. It aims to explore the effectiveness of both statistical prediction models and pharmacokinetic-pharmacodynamic prediction model and modern deep learning approaches. Specifically, the study focuses on predicting the bispectral index (BIS), a vital metric in anesthesia used to assess the depth of sedation during surgery, using datasets collected from real-life surgeries. The goal is to evaluate and compare model performance considering both univariate and multivariate schemes. Accurate BIS prediction is essential for avoiding under- or over-sedation, which can lead to adverse outcomes. The study investigates a range of models: The traditional mathematical models include the pharmacokinetic-pharmacodynamic model and statistical models such as autoregressive integrated moving average (ARIMA) and vector autoregression (VAR). The deep learning models encompass recurrent neural networks (RNNs), specifically Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), as well as Temporal Convolutional Networks (TCNs) and Transformer models. The analysis focuses on evaluating model performance in predicting the BIS using two distinct datasets of physiological metrics collected from actual surgical procedures. It explores both univariate and multivariate prediction schemes and investigates how different combinations of features and input sequence lengths impact model accuracy. The experimental findings reveal significant performance differences among the models: In univariate prediction scenarios for predicting BIS, the LSTM model demonstrates a 2.88% improvement over the second-best performing model. For multivariate predictions, the LSTM model outperforms others by 6.67% compared to the next best model. Furthermore, the addition of Electromyography (EMG) and Mean Arterial Pressure (MAP) brings significant accuracy improvement when predicting BIS. The study emphasizes the importance of selecting and building appropriate time-series models to achieve accurate predictions in biomedical applications. This research provides insights to guide future efforts in improving vital sign prediction methodologies for clinical and research purposes. Clinically, with improvements in the prediction of physiological parameters, clinicians can be informed of interventions if an anomaly is detected or predicted.

Predicting significant wave height in the South China Sea using the SAC-ConvLSTM model

IntroductionThe precise forecasting of Significant wave height(SWH) is vital to ensure the safety and efficiency of aquatic activities such as ocean engineering, shipping, and fishing.MethodsThis paper proposes a deep learning model named SAC-ConvLSTM to perform 24-hour prediction with the SWH in the South China Sea. The long-term prediction capability of the model is enhanced by using the attention mechanism and context vectors. The prediction ability of the model is evaluated by mean absolute error (MAE), root mean square error (RMSE), mean square error (MSE), and Pearson correlation coefficient (PCC).ResultsThe experimental results show that the optimal input sequence length for the model is 12. Starting from 12 hours, the SAC-ConvLSTM model consistently outperforms other models in predictive performance. For the 24-hour prediction, this model achieves RMSE, MAE, and PCC values of 0.2117 m, 0.1083 m, and 0.9630, respectively. In addition, the introduction of wind can improve the accuracy of wave prediction. The SAC-ConvLSTM model also has good prediction performance compared to the ConvLSTM model during extreme weather, especially in coastal areas.DiscussionThis paper presents a 24-hour prediction of SWH in the South China Sea. Through comparative validation, the SAC-ConvLSTM model outperforms other models. The inclusion of wind data enhances the model's predictive capability. This model also performs well under extreme weather conditions. In physical oceanography, variables related to SWH include not only wind but also other factors such as mean wave period and sea surface air pressure. In the future, additional variables can be incorporated to further improve the model's predictive performance.

Chemical Gas Source Localization with Synthetic Time Series Diffusion Data Using Video Vision Transformer

Gas source localization is vital in emergency scenarios to enable swift and effective responses. In this study, we introduce a gas source localization model leveraging the video vision transformer (ViViT). Utilizing synthetic time series diffusion data, the source grid is predicted by classifying the grid with the highest probability of gas occurrence within the diffusion data coverage. Through extensive experimentation using the NBC-RAMS simulator, we generate large datasets of gas diffusion under varied experimental conditions and meteorological environments, enabling comprehensive model training and evaluation. Our findings demonstrate that the ViViT outperforms other deep learning models in processing time series gas data, showcasing a superior estimation performance. Leveraging a transformer architecture, the ViViT exhibits a robust classification performance even in scenarios influenced by weather conditions or incomplete observations. Furthermore, we conduct an analysis of accuracy and parameter count across various input sequence lengths, revealing the ability of the ViViT to maintain high computational efficiency while achieving accurate source localization. These results underscore the effectiveness of the ViViT as a model for gas source localization, particularly in situations demanding a rapid response in real-world environments, such as gas leaks or attacks.

An efficient CNN-transformer hybrid approach for water turbine unit failure prediction

Water Turbine Unit is the core equipment of water power generation. It is the most important research object in hydroelectric power. The analysis of daily monitoring data of Water Turbines can evaluate its running state and avoid the loss caused by failure. In this paper, A hybrid neural network architecture which CNN combines Transformer is proposed to evaluate the health state of water turbines based on multivariate long time series data. It has two core components: (i) dilated convolution applies to capture low-level and local semantic information, then the output of convolutional layers is divided into subseries-level patches by time, these patches are regarded as input tokens of Transformer layers; (ii) utilizing self-attention of Transformer to extract high-level and global semantic information. Patching design naturally has benefit as follow: (i) local semantic information is retained in Transformer input tokens and high-level semantic confirm to common sense of human understanding through low-level construction; (ii) the length of Transformer input sequence is greatly shorted and attention can be more concentrated compared with point-wise form. Meanwhile, the computation and memory usage reduces at the same time. The experimental result indicates that the hybrid architecture can achieves excellent performance in time series understanding.

Short-term multi-step-ahead sector-based traffic flow prediction based on the attention-enhanced graph convolutional LSTM network (AGC-LSTM)

AbstractAccurate sector-based air traffic flow predictions are essential for ensuring the safety and efficiency of the air traffic management (ATM) system. However, due to the inherent spatial and temporal dependencies of air traffic flow, it is still a challenging problem. To solve this problem, some methods are proposed considering the relationship between sectors, while the complicated spatiotemporal dynamics and interdependencies between traffic flow of route segments related to the sector are not taken into account. To address this challenge, the attention-enhanced graph convolutional long short-term memory network (AGC-LSTM) model is applied to improve the short-term sector-based traffic flow prediction, in which spatial structures of route segments related to the sector are considered for the first time. Specifically, the graph convolutional networks (GCN)-LSTM network model was employed to capture spatiotemporal dependencies of the flight data, and the attention mechanism is designed to concentrate on the informative features from key nodes at each layer of the AGC-LSTM model. The proposed model is evaluated through a case study of the typical enroute sector in the central–southern region of China. The prediction results show that MAE reduces by 14.4% compared to the best performing GCN-LSTM model among the other five models. Furthermore, the study involves comparative analyses to assess the influence of route segment range, input and output sequence lengths, and time granularities on prediction performance. This study helps air traffic managers predict flight situations more accurately and avoid implementing overly conservative or excessively aggressive flow management measures for the sectors.

A Joint Time-Frequency Domain Transformer for multivariate time series forecasting

In order to enhance the performance of Transformer models for long-term multivariate forecasting while minimizing computational demands, this paper introduces the Joint Time-Frequency Domain Transformer (JTFT). JTFT combines time and frequency domain representations to make predictions. The frequency domain representation efficiently extracts multi-scale dependencies while maintaining sparsity by utilizing a small number of learnable frequencies. Simultaneously, the time domain (TD) representation is derived from a fixed number of the most recent data points, strengthening the modeling of local relationships and mitigating the effects of non-stationarity. Importantly, the length of the representation remains independent of the input sequence length, enabling JTFT to achieve linear computational complexity. Furthermore, a low-rank attention layer is proposed to efficiently capture cross-dimensional dependencies, thus preventing performance degradation resulting from the entanglement of temporal and channel-wise modeling. Experimental results on eight real-world datasets demonstrate that JTFT outperforms state-of-the-art baselines in predictive performance.

Improving respiratory signal prediction with a deep neural network and simple changes to the input and output data format

Objective. To improve respiratory gating accuracy and radiation treatment throughput, we developed a generalized model based on a deep neural network (DNN) for predicting any given patient’s respiratory motion. Approach. Our model uses long short-term memory (LSTM) based on a recurrent neural network (RNN), and improves upon common techniques. The first improvement is that the data input is not a one-dimensional sequence, but two-dimensional block data. This shortens the input sequence length, reducing computation time. Second, the output is not a scalar, but a sequence prediction. This increases the amount of available data, allowing improved prediction accuracy. For training and evaluation of our model, 434 sets of real-time position management data were retrospectively collected from clinical studies. The data were separated in a ratio of 4:1, with the larger set used for training models and the remaining set used for testing. We measured the accuracy of respiratory signal prediction and amplitude-based gating with prediction windows equaling 133, 333, and 533 ms. This new model was compared with the original LSTM and a non-recurrent DNN model. Main results. The mean absolute errors with the prediction window at 133, 333 and 533 ms were 0.036, 0.084, 0.119 with our model; 0.049, 0.14, 0.246 with the original LSTM-based model; and 0.041, 0.119, 0.16 with the non-recurrent DNN model, respectively. The computation time were 0.66 ms with our model; 0.63 ms the original LSTM-based model; 1.60 ms the non-recurrent DNN model, respectively. The accuracies of amplitude-based gating with the same prediction window settings and a duty cycle of approximately 50% were 98.3%, 95.8% and 92.7% with our model, 97.6%, 93.9% and 87.2% with the original LSTM-based model; and 97.9%, 94.3% and 89.5% with the non-recurrent DNN model, respectively. Significance. Our RNN algorithm for respiratory signal prediction successfully estimated tumor positions. We believe it will be useful in respiratory signal prediction technology.

Proxyformer: Nyström-Based Linear Transformer with Trainable Proxy Tokens

Transformer-based models have demonstrated remarkable performance in various domains, including natural language processing, image processing and generative modeling. The most significant contributor to the successful performance of Transformer models is the self-attention mechanism, which allows for a comprehensive understanding of the interactions between tokens in the input sequence. However, there is a well-known scalability issue, the quadratic dependency (i.e. O(n^2)) of self-attention operations on the input sequence length n, making the handling of lengthy sequences challenging. To address this limitation, there has been a surge of research on efficient transformers, aiming to alleviate the quadratic dependency on the input sequence length. Among these, the Nyströmformer, which utilizes the Nyström method to decompose the attention matrix, achieves superior performance in both accuracy and throughput. However, its landmark selection exhibits redundancy, and the model incurs computational overhead when calculating the pseudo-inverse matrix. We propose a novel Nyström method-based transformer, called Proxyformer. Unlike the traditional approach of selecting landmarks from input tokens, the Proxyformer utilizes trainable neural memory, called proxy tokens, for landmarks. By integrating contrastive learning, input injection, and a specialized dropout for the decomposed matrix, Proxyformer achieves top-tier performance for long sequence tasks in the Long Range Arena benchmark.

TS-Fastformer: Fast Transformer for Time-series Forecasting

Many real-world applications require precise and fast time-series forecasting. Recent trends in time-series forecasting models are shifting from LSTM-based models to Transformer-based models. However, the Transformer-based model has a limited ability to represent sequential relationships in time-series data. In addition, the transformer-based model suffers from slow training and inference speed due to the bottleneck incurred by a deep encoder and step-by-step decoder inference. To address these problems, we propose a time-series forecasting optimized Transformer model, called TS-Fastformer. TS-Fastformer introduces three new optimizations: First, we propose a Sub Window Tokenizer for compressing input in a simple manner. The Sub Window Tokenizer reduces the length of input sequences to mitigate the complexity of self-attention and enables both single and multi-sequence learning. Second, we propose Time-series Pre-trained Encoder to extract effective representations through pre-training. This optimization enables TS-Fastformer to capture both seasonal and trend representations as well as to mitigate bottlenecks of conventional transformer models. Third, we propose the Past Attention Decoder to forecast target by incorporating past long short-term dependency patterns. Furthermore, Past Attention Decoder achieves high performance improvement by removing a trend distribution that changes over a long period. We evaluate the efficiency of our model with extensive experiments using seven real-world datasets and compare our model to six representative time-series forecasting approaches. The results show that the proposed TS-Fastformer reduces MSE by 10.1% compared to state-of-the-art model and demonstrates 21.6% faster training time compared to the existing fastest transformer, respectively.

Improving Stock Price Prediction Accuracy with StacBi LSTM

This research aimed to enhance stock price prediction accuracy using the Stacked Bidirectional Long Short-Term Memory (StacBi LSTM) model. The study addressed the challenge of capturing long-term dependencies and temporal patterns inherent in stock price data. The research objectives were to evaluate the model's performance across different input sequence lengths and identify the optimal length for prediction. Leveraging a dataset from the Indonesian Stock Exchange, the model's predictions were evaluated using key metrics such as RMSE, MAE, MAPE, and R2. Results indicated that the StacBi LSTM model excelled in capturing stock price trends and demonstrated strengths over traditional methods. The optimal input sequence length was identified, balancing computational efficiency and prediction accuracy. This research contributes valuable insights into improving stock price prediction techniques and offers practical implications for traders and investors. Future research directions encompass hybrid models and integrating external factors to enhance predictive capabilities further.

Language model-based B cell receptor sequence embeddings can effectively encode receptor specificity.

High throughput sequencing of B cell receptors (BCRs) is increasingly applied to study the immense diversity of antibodies. Learning biologically meaningful embeddings of BCR sequences is beneficial for predictive modeling. Several embedding methods have been developed for BCRs, but no direct performance benchmarking exists. Moreover, the impact of the input sequence length and paired-chain information on the prediction remains to be explored. We evaluated the performance of multiple embedding models to predict BCR sequence properties and receptor specificity. Despite the differences in model architectures, most embeddings effectively capture BCR sequence properties and specificity. BCR-specific embeddings slightly outperform general protein language models in predicting specificity. In addition, incorporating full-length heavy chains and paired light chain sequences improves the prediction performance of all embeddings. This study provides insights into the properties of BCR embeddings to improve downstream prediction applications for antibody analysis and discovery.

Intelligent prediction for support resistance in working faces of coal mine: A research based on deep spatiotemporal sequence models

Accurately predicting support resistance in coal mine working faces is a pressing matter in the field of intelligent mining research, which holds immense importance in enhancing the safety of coal production and the overall level of intelligent mining. The current research in this field primarily focuses on utilizing deep temporal sequence models for prediction. While there have been promising advancements in this area, concerns persist regarding the dependability of predicted outcomes. In fact, the movement of overburden in mining faces varies over time and space, and the changes in the support resistance that reflect the instability of overburden movement also have temporal and spatial characteristics. Therefore, in order to improve the reliability of model prediction, this paper proposes the application of deep spatiotemporal sequence models for support resistance prediction, and identifies the PredRNN and PredRNN++ models as suitable options. The exploration on the application of deep spatiotemporal sequence models for predicting the support resistance has been conducted. The proposed application involves the establishment of a fully working face area grid model with support number and mining cycle as coordinate units. Different from deep temporal sequence models that take the studied grid features as observations, the 25 adjacent spatiotemporal feature grids corresponding to the studied grid are considered as observations for the deep spatiotemporal sequence models. Based on the predicted three-dimensional tensor, the required predicted support resistance data can be obtained. Based on the complete support resistance data from 9 working faces in a coal mine in Datong mining area from 2009 to 2018, a comprehensive dataset was established to experimentally analyze the predictive performance and reliability of the LSTM, PredRNN, and PredRNN++ models. The experimental results demonstrate significant improvements in predictive performance and reliability of deep spatiotemporal sequence models compared to the current mainstream deep temporal sequence models. In terms of predictive performance, the PredRNN and PredRNN++ models have improved their feature extraction ability by incorporating the convolution operations to extract the spatial correlation of the input data. Furthermore, through the incorporation of gated units and nonlinear neurons into the storage units, coupled with the integration of vertical memory state transitions, the depth of the network has significantly increased, resulting in the enhancement of the sequence modeling capabilities. In terms of reliability of prediction results: The acceptable threshold for prediction error was set at 1000 kN. According to the average prediction results of all samples in the test set, it was found that the deep temporal sequence model was unable to accurately predict support resistance ahead. While the deep spatiotemporal sequence model demonstrated a predictable sequence length of 8. According to the prediction results of all samples in the test set, the accurate prediction rate increased from 30 %∼35 % to 65 %∼85 % and the maximum MAE value decreased from 10000 ∼ 12500kN to 6500 ∼ 9000kN in more than 20,000 test samples. The PredRNN model with an input sequence length of 50 achieves an accurate prediction rate of 85.48 % and a maximum MAE value of 6698kN, making it the most reliable prediction model.

Nucleotide-level prediction of CircRNA-protein binding based on fully convolutional neural network.

Introduction: CircRNA-protein binding plays a critical role in complex biological activity and disease. Various deep learning-based algorithms have been proposed to identify CircRNA-protein binding sites. These methods predict whether the CircRNA sequence includes protein binding sites from the sequence level, and primarily concentrate on analysing the sequence specificity of CircRNA-protein binding. For model performance, these methods are unsatisfactory in accurately predicting motif sites that have special functions in gene expression. Methods: In this study, based on the deep learning models that implement pixel-level binary classification prediction in computer vision, we viewed the CircRNA-protein binding sites prediction as a nucleotide-level binary classification task, and use a fully convolutional neural networks to identify CircRNA-protein binding motif sites (CPBFCN). Results: CPBFCN provides a new path to predict CircRNA motifs. Based on the MEME tool, the existing CircRNA-related and protein-related database, we analysed the motif functions discovered by CPBFCN. We also investigated the correlation between CircRNA sponge and motif distribution. Furthermore, by comparing the motif distribution with different input sequence lengths, we found that some motifs in the flanking sequences of CircRNA-protein binding region may contribute to CircRNA-protein binding. Conclusion: This study contributes to identify circRNA-protein binding and provides help in understanding the role of circRNA-protein binding in gene expression regulation.

A semantically enhanced text retrieval framework with abstractive summarization

AbstractRecently, large pretrained language models (PLMs) have led a revolution in the information retrieval community. In most PLMs‐based retrieval frameworks, the ranking performance broadly depends on the model structure and the semantic complexity of the input text. Sequence‐to‐sequence generative models for question answering or text generation have proven to be competitive, so we wonder whether these models can improve ranking effectiveness by enhancing input semantics. This article introduces SE‐BERT, a semantically enhanced bidirectional encoder representation from transformers (BERT) based ranking framework that captures more semantic information by modifying the input text. SE‐BERT utilizes a pretrained generative language model to summarize both sides of the candidate passage and concatenate them into a new input sequence, allowing BERT to acquire more semantic information within the constraints of the input sequence's length. Experimental results from two Text Retrieval Conference datasets demonstrate that our approach's effectiveness increasing as the length of the input text increases.

A novel approach to attention mechanism using kernel functions: Kerformer.

Artificial Intelligence (AI) is driving advancements across various fields by simulating and enhancing human intelligence. In Natural Language Processing (NLP), transformer models like the Kerformer, a linear transformer based on a kernel approach, have garnered success. However, traditional attention mechanisms in these models have quadratic calculation costs linked to input sequence lengths, hampering efficiency in tasks with extended orders. To tackle this, Kerformer introduces a nonlinear reweighting mechanism, transforming maximum attention into feature-based dot product attention. By exploiting the non-negativity and non-linear weighting traits of softmax computation, separate non-negativity operations for Query(Q) and Key(K) computations are performed. The inclusion of the SE Block further enhances model performance. Kerformer significantly reduces attention matrix time complexity from O(N2) to O(N), with N representing sequence length. This transformation results in remarkable efficiency and scalability gains, especially for prolonged tasks. Experimental results demonstrate Kerformer's superiority in terms of time and memory consumption, yielding higher average accuracy (83.39%) in NLP and vision tasks. In tasks with long sequences, Kerformer achieves an average accuracy of 58.94% and exhibits superior efficiency and convergence speed in visual tasks. This model thus offers a promising solution to the limitations posed by conventional attention mechanisms in handling lengthy tasks.

Prediction of Cognitive Test Scores from Variable Length Multimodal Data in Alzheimer’s Disease

Alzheimer’s disease (AD) is a neurogenerative condition characterized by sharp cognitive decline with no confirmed effective treatment or cure. This makes it critically important to identify the symptoms of Alzheimer’s disease in its early stages before significant cognitive deterioration has taken hold and even before any brain morphology and neuropathology are noticeable. In this study, five different multimodal deep neural networks (MDNN), with different architectures, in search of an optimal model for predicting the cognitive test scores for the Mini-Mental State Examination (MMSE) and the modified Alzheimer’s Disease Assessment Scale (ADAS-CoG13) over a span of 60 months (5 years). The multimodal data utilized to train and test the proposed models were obtained from the Alzheimer’s Disease Neuroimaging Initiative study and includes cerebrospinal fluid (CSF) levels of tau and beta-amyloid, structural measures from magnetic resonance imaging (MRI), functional and metabolic measures from positron emission tomography (PET), and cognitive scores from the neuropsychological tests (Cog). The models developed herein delve into two main issues: (1) application merits of single-task vs. multitask for predicting future cognitive scores and (2) whether time-varying input data are better suited than specific timepoints for optimizing prediction results. This model yields a high of 90.27% (SD = 1.36) prediction accuracy (correlation) at 6 months after the initial visit to a lower 79.91% (SD = 8.84) prediction accuracy at 60 months. The analysis provided is comprehensive as it determines the predictions at all other timepoints and all MDNN models include converters in the CN and MCI groups (CNc, MCIc) and all the unstable groups in the CN and MCI groups (CNun and MCIun) that reverted to CN from MCI and to MCI from AD, so as not to bias the results. The results show that the best performance is achieved by a multimodal combined single-task long short-term memory (LSTM) regressor with an input sequence length of 2 data points (2 visits, 6 months apart) augmented with a pretrained Neural Network Estimator to fill in for the missing values.

Variational eligibility trace meta-reinforcement recurrent network for residual life prediction of space rolling bearings

Traditional sequence recurrent neural networks (SRNNs) have the defect of long time dependence in the prediction of time series, resulting in their poor generalization ability. Moreover, it is required to traverse the whole training data set to realize supervised learning by SRNNs, which increases the time complexity and leads to their low prediction accuracy and high computation cost in the residual life prediction of space rolling bearings in the ground simulated space environment. In view of this, a novel SRNN named variational eligibility trace meta-reinforcement recurrent network (VETMRRN) is proposed for achieving higher residual life prediction accuracy and lower computation cost. In the proposed VETMRRN, a new sequence recurrent network structure is constructed to increase the memory amount of historical information, thus improving the long-term memory capacity of VETMRRN. Then, a hyperparameter self-initialization meta-learning network with an oracle gate mechanism is designed to self-initialize the hyperparameters of VETMRRN for fast determination of the optimal review sequence length. Hence, VETMRRN can adapt to different input sequence lengths and avoid the defect of long time dependence of traditional SRNNs. Furthermore, a variational auto-encoding meta policy gradient learning algorithm with an eligibility trace operator is designed to improve the training speed and enhance the global optimization effect for VETMRRN parameters. Based on the above advantages of VETMRRN, a new residual life prediction method of space rolling bearings in the ground simulated space environment is proposed. Firstly, the time-frequency fusion features are extracted by Shapely-value feature fusion from the vibration acceleration data of space rolling bearing as the performance degradation features. Then, the performance degradation features are input into VETMRRN to predict the performance degradation feature trends of space rolling bearings. Finally, a Weibull-distribution reliability model is established based on the performance degradation feature trend values to predict the residual life of space rolling bearings. The effectiveness of the proposed VETMRRN-based prediction method is verified by the vibration acceleration data collected from the self-built vibration monitoring platform of space rolling bearings in the ground simulated space environment. The results indicate that compared to traditional SRNNs, deep sparse auto-encoding neural network (DSAE-NN), and multi-kernel least-square support vector machine (MK-LSSVM), the proposed method can improve the prediction accuracy and reduce the computation cost in the residual life prediction of space rolling bearings. In the future, the generalization performance of VETMRRN still needs to be further improved.

A Markov Chain Framework for Modeling the Statistical Properties of Stochastic Computing Finite-State Machines

A general methodology to derive analytically the statistical properties of Stochastic Computing Finite-State Machines (SFSM) is introduced. The SFSMs, expressed as Moore ones, are modeled using Markov Chains, enabling the derivation in closed form of their output sequences’ statistical properties, including their expected value, their auto-& cross-correlation, their auto-& cross-covariance, their variance and standard deviation as well as their mean squared error. A MC overflow/underflow probability model accompanies the methodology, allowing to calculate analytically the expected number of steps before overflows/underflows, setting the guidelines to select the register’s size that reduces erroneous bits originating from them. In the proposed methodology both the input sequence length and the number of the SFSMs’ states are considered as parameters, accelerating the overall design procedure as the necessity for multiple time-consuming numerical simulations is eliminated. The proposed methodology’s accurate modeling capabilities are demonstrated with its application in two SFSMs selected from the SC literature, while comparisons with the numerical experiments justify its correctness.

Cloud Removal in Remote Sensing Using Sequential-Based Diffusion Models

The majority of the optical observations collected via spaceborne optical satellites are corrupted by clouds or haze, restraining further applications of Earth observation; thus, exploring an ideal method for cloud removal is of great concern. In this paper, we propose a novel probabilistic generative model named sequential-based diffusion models (SeqDMs) for the cloud-removal task in a remote sensing domain. The proposed method consists of multi-modal diffusion models (MmDMs) and a sequential-based training and inference strategy (SeqTIS). In particular, MmDMs is a novel diffusion model that reconstructs the reverse process of denosing diffusion probabilistic models (DDPMs) to integrate additional information from auxiliary modalities (e.g., synthetic aperture radar robust to the corruption of clouds) to help the distribution learning of main modality (i.e., optical satellite imagery). In order to consider the information across time, SeqTIS is designed to integrate temporal information across an arbitrary length of both the main modality and auxiliary modality input sequences without retraining the model again. With the help of MmDMs and SeqTIS, SeqDMs have the flexibility to handle an arbitrary length of input sequences, producing significant improvements only with one or two additional input samples and greatly reducing the time cost of model retraining. We evaluate our method on a public real-world dataset SEN12MS-CR-TS for a multi-modal and multi-temporal cloud-removal task. Our extensive experiments and ablation studies demonstrate the superiority of the proposed method on the quality of the reconstructed samples and the flexibility to handle arbitrary length sequences over multiple state-of-the-art cloud removal approaches.

Dislocation Substructures Evolution and an Informer Constitutive Model for a Ti-55511 Alloy in Two-Stages High-Temperature Forming with Variant Strain Rates in β Region.

The high-temperature compression characteristics of a Ti-55511 alloy are explored through adopting two-stage high-temperature compressed experiments with step-like strain rates. The evolving features of dislocation substructures over hot, compressed parameters are revealed by transmission electron microscopy (TEM). The experiment results suggest that the dislocations annihilation through the rearrangement/interaction of dislocations is aggravated with the increase in forming temperature. Notwithstanding, the generation/interlacing of dislocations exhibit an enhanced trend with the increase in strain in the first stage of forming, or in strain rates at first/second stages of a high-temperature compressed process. According to the testing data, an Informer deep learning model is proposed for reconstructing the stress-strain behavior of the researched Ti-55511 alloy. The input series of the established Informer deep learning model are compression parameters (compressed temperature, strain, as well as strain rate), and the output series are true stresses. The optimal input batch size and sequence length are 64 and 2, respectively. Eventually, the predicted results of the proposed Informer deep learning model are more accordant with the tested true stresses compared to those of the previously established physical mechanism model, demonstrating that the Informer deep learning model enjoys an outstanding forecasted capability for precisely reconstructing the high-temperature compressed features of the Ti-55511 alloy.