Feedforward Network Research Articles

Optimization design of biomechanical parameters based on advanced mathematical modelling

In recent years the use of biomechanics in athletic training and performance has received a lot of attention, especially in university sports programs. Biomechanics is the study of the mechanical principles that control how biological things move or are constructed. It is critical for understanding the intricate relationships between physical performance, body mechanics, and injury prevention. The objective of this study is to establish how biomechanical variables can be designed and optimized in universities using mathematical modeling. In this study, a novel Emperor Penguin Search-driven Dynamic Feedforward Neural Network (EPSO-DFNN) is proposed to optimize the biomechanical parameters of athletes. Various biomechanical data are utilized from athletes participating in different sports. Biomechanical parameters include muscle activation patterns, joint angles, forces, and movement. The data was preprocessed using Z-score normalization from the obtained data. The Fast Fourier Transform (FFT) using features is extracted from preprocessed data. The proposed method is to identify the optimal configurations for athlete’s movements tailored to their sports and individual biomechanical profiles. The proposed method is the performance of various evaluation metrics such as F1-score (92.76%), precision (91.42%), accuracy (90.02%), and recall (89.69%). The result demonstrated that the proposed method effectively improved the performance in athletic capabilities compared to other traditional algorithms. This study demonstrates how mathematical modeling may be used to optimize biomechanical characteristics, providing insightful information that can be used to improve athletic performance and encourage safer behaviors in athletic settings.

Abstract PR019: Transfer learning for accurate tissue of origin classification from cfDNA methylation

Abstract Accurately predicting the tissue of origin (TOO) is required for blood-based multi-cancer screening to guide follow-up imaging and care when cancer is detected. However, precisely determining the TOO using cell-free DNA (cfDNA) methylation remains challenging due to low and variable concentrations of circulating tumor DNA and cfDNA and the complex, poorly characterized nature of tissue-specific methylation patterns. To address these challenges, we developed a transfer learning approach that leverages methylation profiles learned from tissue biopsy samples to predict the TOO of cfDNA cancer samples. Our model achieved 89% balanced accuracy (BA) in classifying the TOO across 10 cancer groups and outperformed models trained solely on plasma cfDNA data. Unmatched tissue biopsy (N=517) and blood (N=1,155, NCT05435066) samples were collected from treatment-naïve cancer patients across 21 tumor types consolidated into 17 cancer groups. All samples were analyzed using a custom targeted bisulfite sequencing hybrid capture assay. Five unique metrics were used to quantify informative methylation signal at each TOO region of interest. A feed-forward neural network was first trained on biopsy methylation features, where the cancer signal is strongest. Subsequently, the model was fine-tuned using plasma cfDNA methylation features, which have significantly lower signal-to-noise ratio, allowing the model to adapt its predictions to the noisier data. The Adam optimizer was employed for both training and fine-tuning, with early stopping implemented to prevent overfitting. Dropout layers were incorporated to enhance model generalization and performance was evaluated using 10-fold cross validation on plasma cfDNA samples across 10 cancer groups for which at least 25 plasma samples were available (N=1,041). The same architecture was trained exclusively on plasma cfDNA samples for comparison. Results are reported for samples with tumor content greater than 0.1% (N=415). Our TOO prediction achieved 73% BA in top-one predictions, where the true label matched the highest probability prediction, and 89% BA in top-two predictions, where the true label matched the highest or the second-highest probability prediction. Multi-class classification revealed our method significantly outperforms a model trained only on plasma cfDNA data. The plasma only model achieved 69% and 82% BA in top-one and top-two predictions, respectively. Accuracy per indication revealed that training on biopsy data significantly improves the detection of tumor types that shed DNA at low rates compared to training on plasma data alone (breast cancer +14%, prostate cancer +12%, and uterine cancer +20% points). In conclusion, our transfer learning approach, utilizing tissue biopsy methylation data, significantly improves cfDNA-based TOO detection accuracy. This approach highlights the capability of transfer learning to enable automated feature extraction from signal-rich data to enhance the analysis of more challenging, heterogeneous cfDNA samples, advancing non-invasive cancer diagnostics. Citation Format: Shiva Farashahi, Amirali Kia, Dorna Kashef, Esther Brown, Feras Hantash, Kieran I Chacko. Transfer learning for accurate tissue of origin classification from cfDNA methylation [abstract]. In: Proceedings of the AACR Special Conference: Liquid Biopsy: From Discovery to Clinical Implementation; 2024 Nov 13-16; San Diego, CA. Philadelphia (PA): AACR; Clin Cancer Res 2024;30(21_Suppl):Abstract nr PR019.

Vanilla Feedforward Neural Networks as a Discretization of Dynamical Systems

Vanilla Feedforward Neural Networks as a Discretization of Dynamical Systems

Abstract A053: Liquid biopsy-based detection of triple negative breast cancer using DNA methylation biomarkers

Abstract Triple-negative breast cancer (TNBC) presents significant clinical challenges due to its aggressive phenotype, rapid progression, and limited targeted treatment options. TNBC often mimics benign lesions on ultrasound and metastatic features can be missed in routine mammography screening leading to misdiagnosis and delayed treatment. These limitations led us to investigate whether a blood-based liquid biopsy could provide a non-invasive, sensitive method for early TNBC detection, potentially overcoming the constraints of conventional imaging techniques. To discover biomarkers for TNBC, we performed targeted bisulfite sequencing on a cohort of 46 breast cancer solid tissue biopsy samples, comprising 19 TNBC and 27 non-TNBC cases. We identified 150 differentially methylated regions (mean length 703bp) separating TNBC and non-TNBC samples. Upon linking each region to its nearest gene, we observed several genes previously associated with prognosis and survival (HOXB3, PAX9, SOX9) substantiating the clinical relevance of these biomarkers. Using public TCGA data (369 breast cancer samples), we integrated expression and methylation data and observed directionally consistent changes. For example, HOXB3 showed increased methylation at its associated differential regions and decreased expression in TNBC cases. To ascertain whether these biomarkers could be useful in a liquid biopsy approach, we performed targeted bisulfite- sequencing on cell-free DNA samples derived from prospectively collected patients with breast cancer. All samples had independent tissue-based assessment of ER, PR and HER2 using immunohistochemistry or in situ hybridization. We then built a feed-forward neural network classifier with classes TNBC and non-TNBC that used transfer-learning and the identified biomarkers to learn methylation signatures in the biopsy samples and map them to cell-free DNA. To emulate clinical workflow, where subtyping follows cancer diagnosis and tissue of origin identification, we evaluated cfDNA samples correctly identified as breast cancer using two independent machine learning models trained using 2,393 prospectively collected multi- indication cancer and non-cancer samples (NCT05435066). Subsequent TNBC subtyping of breast cancer-positive cases (N=56) accurately identified 77% (10/13) of TNBC samples and 91% (39/43) of non-TNBC cases, with overall accuracy of 84%. The estimated tumor content range for correctly predicted TNBC samples was 0.5-33% with the three mis-classified samples all showing very low tumor content (0.22%, 0.28%, 0.59%). Non-TNBC cases showed a tumor content range of 0.07-36% for correct and 2.3-14% for misclassified samples. In conclusion, we identified novel methylation biomarkers that distinguish TNBC in the cell-free DNA of patients without needing invasive tissue biopsies. The high sensitivity of our assay at low tumor fractions has the potential to improve outcomes through earlier intervention and may enable earlier screening in high-risk groups, monitoring of minimal residual disease and longitudinal monitoring during treatment. Citation Format: Jocelyn Charlton, Shiva Farashahi, Kade Pettie, Elie Massaad, Josh Hubbell, Feras Hantash, Kieran I Chacko. Liquid biopsy-based detection of triple negative breast cancer using DNA methylation biomarkers [abstract]. In: Proceedings of the AACR Special Conference: Liquid Biopsy: From Discovery to Clinical Implementation; 2024 Nov 13-16; San Diego, CA. Philadelphia (PA): AACR; Clin Cancer Res 2024;30(21_Suppl):Abstract nr A053.

DGTNet：dynamic graph attention transformer network for traffic flow forecasting

Abstract Graph-based traffic flow prediction plays a crucial role in urban traffic management and planning. In this paper, we propose a novel Dynamic Graph Attention Transformer Network (DGTnet), which is designed to address the issue of inadequately integrating of temporal and spatial dimensions in traditional models. DGTnet maintains temporal continuity while revealing the complex dynamic relationships between key nodes in the urban traffic system, capturing the periodic changes in the rhythm of city life. Specifically, this study adopts adaptive signal decomposition technology to decompose traffic data into multiple intrinsic mode functions (IMFs), effectively capturing the dynamic changes in traffic flow. This decomposition method is key to the implementation of DGTnet dynamic graph construction, enabling the analysis of traffic flow at different time scales, thereby providing a new perspective for traffic flow prediction research. In the traffic prediction module, we comprehensively consider node, edge, and graph structural information, adopting a multi-head self-attention mechanism to achieve direct cross-modeling in both temporal and spatial dimensions. Finally, we introduce a position-wise feed forward network layer to integrate different types of data and capture nonlinear relationships. The experimental results, based on the public transportation network data sets METR_LA, PEMS_BAY, PEMS03 and PEMS07, demonstrate that DGTNet exhibits notable enhancements in three evaluation indicators, namely the Mean Absolute Percentage Error (MAPE), the Root Mean Square Error (RMSE) and the Mean Absolute Error (MAE). The pertinent code has been made available for public access at https://github.com/chenjing0616/DGTNet.

Noise classification in three-level quantum networks by Machine Learning

Abstract We investigate a machine learning based classification of noise acting on a small quantum network with the aim of detecting spatial or multilevel correlations, and the interplay with Markovianity. We control a three-level system by inducing coherent population transfer exploiting different pulse amplitude combinations as inputs to train a feedforward neural network. We show that supervised learning can classify different types of classical dephasing noise affecting the system. Three non-Markovian (quasi-static correlated, anti-correlated and uncorrelated) and Markovian noises are classified with more than $99\%$ accuracy. On the contrary, correlations of Markovian noise cannot be discriminated with our method. Our approach is robust to statistical measurement errors and retains its effectiveness for physical measurements where only a limited number of samples is available making it very experimental-friendly. Our result paves the way for classifying spatial correlations of noise in quantum architectures.

Diversifying Multi-Head Attention in the Transformer Model

Recent studies have shown that, due to redundancy, some heads of the Transformer model can be pruned without diminishing the efficiency of the model. In this paper, we propose a constrained optimization algorithm based on Hebbian learning, which trains specific layers in the Transformer architecture in order to enforce diversification between the different heads in the multi-head attention module. The diversification of the heads is achieved through a single-layer feed-forward neural network that is added to the Transformer architecture and is trained with the proposed algorithm. We utilize the algorithm in three different architectural variations of the baseline Transformer model. In addition to the diversification of the heads, the proposed methodology can be used to prune the heads that capture redundant information. Experiments on diverse NLP tasks, including machine translation, text summarization, question answering and large language modeling, show that our proposed approach consistently improves the performance of baseline Transformer models.

Importance sampling for option pricing with feedforward neural networks

AbstractWe study the problem of reducing the variance of Monte Carlo estimators through performing suitable changes of the sampling measure computed by feedforward neural networks. To this end, building on the concept of vector stochastic integration, we characterise the Cameron–Martin spaces of a large class of Gaussian measures induced by vector-valued continuous local martingales with deterministic covariation. We prove that feedforward neural networks enjoy, up to an isometry, the universal approximation property in these topological spaces. We then prove that sampling measures generated by feedforward neural networks can approximate the optimal sampling measure arbitrarily well. We conclude with a comprehensive numerical study pricing path-dependent European options for asset price models that incorporate factors such as changing business activity, knock-out barriers, dynamic correlations and high-dimensional baskets.

Comparative study on deep and machine learning approaches for predicting wind pressures on tall buildings

Wind-structures interaction has been extensively examined in the last few decades using field measurements, full scale measurements and wind tunnel testing. These experimental approaches are considered costly and time consuming. The need for a reliable analytical approach that can be used for examining wind-effects on buildings is clear. Although Computational Fluid Dynamics (CFD) is one of the other alternative numerical options yet might not reached the level of confidence to be reliably used to finalize the structural design. On the other hand, a limited number of studies have been carried out using soft computing methods to examine wind-induced loads on structures. However, its promising results, more work is still required towards achieving the full analytical prediction of wind effects on structures. This study investigates the use of different soft-computing techniques in predicting wind pressures on tall buildings. Two deep learning methods viz deep belief network (DBN) and deep neural network (DNN), and five machine learning methods namely feedforward neural network, extreme learning machine, weighted extreme learning machine, random forest, and gradient boosting machine were evaluated, and compared in predicting the design wind pressures on tall buildings. Wind tunnel datasets, used in the current study to develop the proposed computing models, were collected from testing three tall buildings having the same full-scale horizontal dimensions of (40 m and 80 m) and different heights of (80 m, 120 m and 160 m). The buildings were tested at a scale of 1:400 in urban terrain exposure. Mean and fluctuating wind pressure coefficients on the building with the height of 120 m are herein predicted using the seven computing methods and the results were compared to the corresponding measured pressures. Overall, the examined methods performed well in the wind pressure prediction process. Furthermore, the employed DNN was found to have the best performance in predicting mean and fluctuating wind pressures with the highest correlation coefficients. Hence, the DNN was also used in predicting the mean and fluctuating wind pressures on the two other buildings with heights of 80 m and 160 m. Experimental results indicate that the employed DNN model can be effectively used in predicting wind-induced pressures on tall buildings.

Experimental Investigation of Indirect Tensile Strength of Hot Mix Asphalt with Varying Hydrated Lime Content at Low Temperatures and Prediction with Soft-Computing Models

If asphalt pavements are exposed to cold weather conditions and high humidity for long periods of time, cracking of the pavement is an inevitable consequence. In such cases, it would be a good decision to focus on the filler material, which plays an important role in the performance variation in the hot asphalt mixtures used in the pavement. Although the use of hydrated lime as a filler material in hot asphalt mixtures is a common method frequently recommended to eliminate the adverse effects of low temperature and to keep moisture sensitivity under control in asphalt pavements, the sensitivity of the quantities of the material cannot be ignored. Therefore, in this study, an amount of filler in the mixture was replaced with hydrated lime (HL) filler additive at different rates of 0%, 1%, 2%, 3% and 4%. These asphalt briquettes, designed according to the Marshall method, have optimum asphalt contents for samples with specified HL content. In this study, where the temperature effect was examined at five different levels of −10 °C, −5 °C, 0 °C, 5 °C and 25 °C, the samples were produced in two different groups, conditioned and unconditioned, in order to examine the effect of water. The indirect tensile strength (ITS) test was applied on the produced samples. Experimental study showed that HL additive strengthened the material at low temperatures and made it more resistant to cold weather conditions and humidity. In the second part of the study, two different prediction models with varying configurations were introduced using nonlinear regression and feed-forward neural networks (FFNNs) and the best prediction performance among these was investigated. Examination of the performance measures of the prediction models indicated that ITS can be accurately predicted using both methods. As a result of comparing the developed models with the experimental data, the model provides significant contributions to the evaluation of the relationship between the ITS values obtained with the specified conditioning, temperature changes and HL contents.

Metaplasticity and memory in multilevel recurrent feed-forward networks

Metaplasticity and memory in multilevel recurrent feed-forward networks

Heart Sound Classification Using Harmonic and Percussive Spectral Features from Phonocardiograms with a Deep ANN Approach

Cardiovascular diseases (CVDs) are a leading cause of mortality worldwide, with a particularly high burden in India. Non-invasive methods like Phonocardiogram (PCG) analysis capture the acoustic activity of the heart. This holds significant potential for the early detection and diagnosis of heart conditions. However, the complexity and variability of PCG signals pose considerable challenges for accurate classification. Traditional methods of PCG signal analysis, including time-domain, frequency-domain, and time-frequency domain techniques, often fall short in capturing the intricate details necessary for reliable diagnosis. This study introduces an innovative approach that leverages harmonic–percussive source separation (HPSS) to extract distinct harmonic and percussive spectral features from PCG signals. These features are then utilized to train a deep feed-forward artificial neural network (ANN), classifying heart conditions as normal or abnormal. The methodology involves advanced digital signal processing techniques applied to PCG recordings from the PhysioNet 2016 dataset. The feature set comprises 164 attributes, including the Chroma STFT, Chroma CENS, Mel-frequency cepstral coefficients (MFCCs), and statistical features. These are refined using the ROC-AUC feature selection method to ensure optimal performance. The deep feed-forward ANN model was rigorously trained and validated on a balanced dataset. Techniques such as noise reduction and outlier detection were used to improve model training. The proposed model achieved a validation accuracy of 93.40% with sensitivity and specificity rates of 82.40% and 80.60%, respectively. These results underscore the effectiveness of harmonic-based features and the robustness of the ANN in heart sound classification. This research highlights the potential for deploying such models in non-invasive cardiac diagnostics, particularly in resource-constrained settings. It also lays the groundwork for future advancements in cardiac signal analysis.

Data-driven multivariate time series prediction of in-vehicle equipment failure rates

Effectively predicting the failure rate of train-controlled on-board equipment is of great significance for rationally allocating equipment spares, drawing up maintenance plans, and reducing the occurrence of failures. In order to tackle the problem of the intricate attributes and limited predictive precision of the sample data pertaining to the failure rate of on-board equipment, in this paper, a multivariate time series prediction model for the failure rate of train-controlled on-board equipment is based on the combination of multivariate variational modal decomposition (MVMD) graph neural network (GNN) and transformer. First, the original failure rate time series, air temperature, humidity and sand and dust time series are modally decomposed using MVMD, and the intrinsic modal functions (IMFs) of each series are obtained; then, the dynamic graph of the GNN network is defined according to the IMFs, and Transformer’s self-attention mechanism is utilised to capture the dynamic graph’s temporal and spatial dependencies. Finally, the failure rate is output through the feed-forward neural network prediction value. Experiments using the historical fault data of CTCS3-300T train-controlled on-board equipment are carried out to confirm the efficacy of the suggested method, and it is contrasted with other conventional machine learning techniques. The outcomes of the experiment show that compared with other train-controlled on-board equipment failure prediction models, the proposed method has a very good superiority, as evidenced by its mean absolute error (MAE) of 0.0489 and root mean square error (RMSE) of 0.0510, which is of certain reference value for equipment operation and maintenance.

Towards industry-ready additive manufacturing: AI-enabled closed-loop control for 3D melt electrowriting.

Melt electrowriting (MEW) is an emerging high-resolution 3D printing technology used in biomedical engineering, regenerative medicine, and soft robotics. Its transition from academia to industry faces challenges such as slow experimentation, low printing throughput, poor reproducibility, and user-dependent operation, largely due to the nonlinear and multiparametric nature of the MEW process. To address these challenges, we applied computer vision and machine learning to monitor and analyze the process in real-time through imaging of the MEW jet between the nozzle-collector gap. To collect data for training we developed an automated data collection methodology that eases the experimental time from days to hours. A feedforward neural network, working in concert with optimization methods and a feedback loop, is used to develop closed-loop control ensuring reproducibility of the printed parts. We demonstrate that machine learning allows streamlining the MEW operation via closed-loop control of the highly nonlinear 3D printing technology.

Machine learning for high-performance solar radiation prediction

This research paper focuses on predicting surface solar radiation for 45 glaciers situated in two locations within the northern regions of Pakistan: Khyber Pakhtunkhwa (KPK) and Gilgit. The study investigates the relationship between input parameters, including date, temperature, pressure, precipitation, and aerosol, and solar radiation output parameters. The data used for this analysis is sourced from the CERES dataset, covering the period from 2001 to 2021. The study considers eight glaciers in KPK and thirty-seven glaciers in Gilgit to provide a comprehensive understanding of the surface solar radiation patterns in these regions. Various machine learning algorithms are employed for this prediction, including linear regression (LR), decision tree (DT), random forest (RF), feed-forward neural network (FFNN), and long short-term memory (LSTM). We considered the mean squared error (MSE), mean absolute error (MAE), normal root mean squared (nRMSE), and R-squared (R2) score as the performance metrics to assess the performance of the models. The results for the KPK location indicate that the FFNN algorithm achieved the highest accuracy with an MSE of 598.326, MAE of 18.9685, nRMSE of 0.06973, and R2 score of 0.916399. Similarly, the FFNN algorithm outperformed other models for the Gilgit location with an MSE of 738.78, MAE of 20.6887, nRMSE of 0.08071, and R2 score of 0.886703. This work contributes to understanding surface solar radiation patterns in the northern regions of Pakistan, specifically in KPK and Gilgit. This research has practical implications for renewable energy planning, climate change assessments, and glacier monitoring. By accurately predicting surface solar radiation, stakeholders can make informed decisions and develop sustainable strategies for these regions.

A Unified Model for Chinese Cyber Threat Intelligence Flat Entity and Nested Entity Recognition

In recent years, as cybersecurity threats have become increasingly severe and cyberattacks have occurred frequently, higher requirements have been put forward for cybersecurity protection. Therefore, the Named Entity Recognition (NER) technique, which is the cornerstone of Cyber Threat Intelligence (CTI) analysis, is particularly important. However, most existing NER studies are limited to recognizing single-layer flat entities, ignoring the possible nested entities in CTI. On the other hand, most of the existing studies focus on English CTIs, and the existing models performed poorly in a limited number of Chinese CTI studies. Given the above challenges, we propose in this paper a novel unified model, RBTG, which aims to identify flat and nested entities in Chinese CTI effectively. To overcome the difficult boundary recognition problem and the direction-dependent and distance-dependent properties in Chinese CTI NER, we use Global Pointer as the decoder and TENER as the encoder layer, respectively. Specifically, the Global Pointer layer solves the problem of the insensitivity of general NER methods to entity boundaries by utilizing the relative position information and the multiplicative attention mechanism. The TENER layer adapts to the Chinese CTI NER task by introducing an attention mechanism with direction awareness and distance awareness. Meanwhile, to cope with the complex feature capture of hierarchical structure and dependencies among Chinese CTI nested entities, the TENER layer solves the problem by following the structure of multiple self-attention layers and feed-forward network layers superimposed on each other in the Transformer. In addition, to fill the gap in the Chinese CTI nested entity dataset, we further apply the Large Language Modeling (LLM) technique and domain knowledge to construct a high-quality Chinese CTI nested entity dataset, CDTinee, which consists of six entity types selected from STIX, including nearly 4000 entity types extracted from more than 3000 threatening sentences. In the experimental session, we conduct extensive experiments on multiple datasets, and the results show that the proposed model RBTG outperforms the baseline model in both flat NER and nested NER.

Quantification of Size-Binned Particulate Matter in Electronic Cigarette Aerosols Using Multi-Spectral Optical Sensing and Machine Learning

To monitor health risks associated with vaping, we introduce a multi-spectral optical sensor powered by machine learning for real-time characterization of electronic cigarette aerosols. The sensor can accurately measure the mass of particulate matter (PM) in specific particle size channels, providing essential information for estimating lung deposition of vaping aerosols. For the sensor’s input, wavelength-specific optical attenuation signals are acquired for three separate wavelengths in the ultraviolet, red, and near-infrared range, and the inhalation pressure is collected from a pressure sensor. The sensor’s outputs are PM mass in three size bins, specified as 100–300 nm, 300–600 nm, and 600–1000 nm. Reference measurements of electronic cigarette aerosols, obtained using a custom vaping machine and a scanning mobility particle sizer, provided the ground truth for size-binned PM mass. A lightweight two-layer feedforward neural network was trained using datasets acquired from a wide range of puffing conditions. The performance of the neural network was tested using unseen data collected using new combinations of puffing conditions. The model-predicted values matched closely with the ground truth, and the accuracy reached 81–87% for PM mass in three size bins. Given the sensor’s straightforward optical configuration and the direct collection of signals from undiluted vaping aerosols, the achieved accuracy is notably significant and sufficiently reliable for point-of-interest sensing of vaping aerosols. To the best of our knowledge, this work represents the first instance where machine learning has been applied to directly characterize high-concentration undiluted electronic cigarette aerosols. Our sensor holds great promise in tracking electronic cigarette users’ puff topography with quantification of size-binned PM mass, to support long-term personalized health and wellness.

A spatiotemporal correlation and attention-based model for pipeline deformation prediction in foundation pit engineering

In foundation pit engineering, the deformation prediction of adjacent pipelines is crucial for construction safety. Existing approaches depend on constitutive models, grey correlation prediction, or traditional feedforward neural networks. Due to the complex hydrological and geological conditions, as well as the nonstationary and nonlinear characteristics of monitoring data, this problem remains a challenge. By formulating the deformation of monitoring points as multivariate time series, a deep learning-based prediction model is proposed, which utilizes the convolutional neural network to extract the spatial dependencies among various monitoring points, and leverages the bi-directional long-short memory unit network to extract temporal features. Notably, an attention mechanism is introduced to adjust the trainable weights of spatial-temporal features extracted in the prediction. The evaluation of a real-world subway project demonstrates that the proposed model has advantages compared with current models, particularly in long-term prediction. It improves the Adjusted R2 index averagely by from 19.4 to 61.6%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} compared with existing models. The proposed model also exhibits a decrease in mean absolute error ranging from 51.5 to 70.3%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} compared to others. Experiments and analyses verify that the spatial-temporal dependencies in time series and the attention learning for spatial-temporal features can improve the prediction of such engineering problems.

Enhanced local multi-windows attention network for lightweight image super-resolution

Since the global self-attention mechanism can capture long-distance dependencies well, Transformer-based methods have achieved remarkable performance in many vision tasks, including single-image super-resolution (SISR). However, there are strong local self-similarities in images, if the global self-attention mechanism is still used for image processing, it may lead to excessive use of computing resources on parts of the image with weak correlation. Especially in the high-resolution large-size image, the global self-attention will lead to a large number of redundant calculations. To solve this problem, we propose the Enhanced Local Multi-windows Attention Network (ELMA), which contains two main designs. First, different from the traditional self-attention based on square window partition, we propose a Multi-windows Self-Attention (M-WSA) which uses a new window partitioning mechanism to obtain different types of local long-distance dependencies. Compared with original self-attention mechanisms commonly used in other SR networks, M-WSA reduces computational complexity and achieves superior performance through analysis and experiments. Secondly, we propose a Spatial Gated Network (SGN) as a feed-forward network, which can effectively overcome the problem of intermediate channel redundancy in traditional MLP, thereby improving the parameter utilization and computational efficiency of the network. Meanwhile, SGN introduces spatial information into the feed-forward network that traditional MLP cannot obtain. It can better understand and use the spatial structure information in the image, and enhances the network performance and generalization ability. Extensive experiments show that ELMA achieves competitive performance compared to state-of-the-art methods while maintaining fewer parameters and computational costs.

Deep learning for prediction and classifying the dynamical behaviour of piecewise-smooth maps

This paper explores novel ways of predicting and classifying the dynamics of piecewise smooth maps using various deep learning models. Moreover, we have used machine learning models such as Decision Tree Classifier, Logistic Regression, K-Nearest Neighbor, Random Forest, and Support Vector Machine for predicting the border collision bifurcation in the 1D normal form map and the 1D tent map. The decision tree classifier best predicts the border collision bifurcation for the 1D normal form map, the random forest, and the 1D tent map. This study introduces a novel application of deep learning models to cobweb diagrams and phase portraits, which provides a new perspective for classifying regular and chaotic behaviour. Further, we classified the regular and chaotic behaviour of the 1D tent map and the 2D Lozi map using deep learning models like Convolutional Neural Network (CNN), ResNet50, and ConvLSTM via cobweb diagram and phase portraits, where CNN exhibits better performance than other models. We also classified the chaotic and hyperchaotic behaviour of the 3D piecewise smooth map using deep learning models such as the Feed Forward Neural Network (FNN), Long Short-Term Memory (LSTM), and Recurrent Neural Network (RNN). We have shown that LSTM performs best for classifying chaotic and hyperchaotic behaviour. Additionally, LSTM outperforms other models in accuracy and computational efficiency, making it highly effective for real-time analysis. Finally, deep learning models such as Long Short Term Memory (LSTM) and Recurrent Neural Network (RNN) are used for reconstructing the two-parameter bifurcation charts of 2D normal form map, in which LSTM is more precise than RNN in reconstructing the two-parametric charts.