Fully-connected Neural Network Research Articles

PR-FCNN: a data-driven hybrid approach for predicting PM2.5 concentration

The atmosphere’s fine articulate Matter (PM2.5) poses various health-related risks. Even though multiple efforts have been made to lower the emissions of these substances, the mortality rate is continuously increasing, requiring immediate inclination of the scientific community towards the design and development of advanced predictive models. Conventional statistical approaches have become dormant due to their limitations in capturing the innate relationships between the pollutants, particularly for predicting PM2.5 concentrations. In contrast, machine and deep learning techniques have shown great potential for forecasting air quality, providing more accuracy than its predecessor techniques. The present study investigates the utilization of hybrid approaches by integrating machine learning models with deep learning models to improve the prediction capabilities of PM2.5 concentration. It uses datasets from the World Air Quality Index (WAQI) and the State of Global Air (SOGA) to analyze the performance of the models on both the daily and annual data, respectively. This ensures the model’s effectiveness on a diversified dataset. The present study implements Random Forest (RF), Polynomial Regression (PR), XGBoost, and Extra Tree Regressor (ETR) coupled with Fully Connected Neural Network (FCNN), Long Short-Term Memory (LSTM), and Bi-directional LSTM (Bi-LSTM) for obtaining optimized results. Finally, after a thorough investigation, the hybrid PR model coupled with FCNN (PR-FCNN) is found to be the best model with improved R-squared (R2) values, portraying its potential for predicting PM2.5 concentration accurately. Based on the experimentation, the preset study recommends implementing hybrid approaches, offering better predictive accuracy in forecasting air pollutants, especially PM2.5.

Adaptive Beamforming for On-Orbit Satellite-Based ADS-B Based on FCNN

Digital multi-beam synthesis technology is generally used in the on-orbit satellite-based Automatic Dependent Surveillance–Broadcast (ADS-B) system. However, the probability of successfully detecting aircraft with uneven surface distribution is low. An adaptive digital beamforming method is proposed to improve the efficiency of aircraft detection probability. The current method has the problem of long operation time and is not suitable for on-orbit operation. Therefore, this paper proposes an adaptive beamforming method for the ADS-B system based on a fully connected neural network (FCNN). The simulation results show that the calculation time of this method is about 2.6 s when more than 15,000 sets of data are inputted, which is 15–80% better than the existing methods. Its detection success probability is 10% higher than those of existing methods, and it has better robustness against large amounts of data.

Fast and Smart State Characterization of Large-Format Lithium-Ion Batteries via Phased-Array Ultrasonic Sensing Technology

Lithium-ion batteries (LIBs) are widely used in electric vehicles and energy storage systems, making accurate state transition monitoring a key research topic. This paper presents a characterization method for large-format LIBs based on phased-array ultrasonic technology (PAUT). A finite element model of a large-format aluminum shell lithium-ion battery is developed on the basis of ultrasonic wave propagation in multilayer porous media. Simulations and comparative analyses of phased array ultrasonic imaging are conducted for various operating conditions and abnormal gas generation. A 40 Ah ternary lithium battery (NCMB) is tested at a 0.5C charge-discharge rate, with the state of charge (SOC) and ultrasonic data extracted. The relationship between ultrasonic signals and phased array images is established through simulation and experimental comparisons. To estimate the SOC, a fully connected neural network (FCNN) model is designed and trained, achieving an error of less than 4%. Additionally, phased array imaging, which is conducted every 5 s during overcharging and overdischarging, reveals that gas bubbles form at 0.9 V and increase significantly at 0.2 V. This research provides a new method for battery state characterization.

Development and validation of preeclampsia predictive models using key genes from bioinformatics and machine learning approaches

BackgroundPreeclampsia (PE) poses significant diagnostic and therapeutic challenges. This study aims to identify novel genes for potential diagnostic and therapeutic targets, illuminating the immune mechanisms involved.MethodsThree GEO datasets were analyzed, merging two for training set, and using the third for external validation. Intersection analysis of differentially expressed genes (DEGs) and WGCNA highlighted candidate genes. These were further refined through LASSO, SVM-RFE, and RF algorithms to identify diagnostic hub genes. Diagnostic efficacy was assessed using ROC curves. A predictive nomogram and fully Connected Neural Network (FCNN) were developed for PE prediction. ssGSEA and correlation analysis were employed to investigate the immune landscape. Further validation was provided by qRT-PCR on human placental samples.ResultFive biomarkers were identified with validation AUCs: CGB5 (0.663, 95% CI: 0.577-0.750), LEP (0.850, 95% CI: 0.792-0.908), LRRC1 (0.797, 95% CI: 0.728-0.867), PAPPA2 (0.839, 95% CI: 0.775-0.902), and SLC20A1 (0.811, 95% CI: 0.742-0.880), all of which are involved in key biological processes. The nomogram showed strong predictive power (C-index 0.873), while FCNN achieved an optimal AUC of 0.911 (95% CI: 0.732-1.000) in five-fold cross-validation. Immune infiltration analysis revealed the importance of T cell subsets, neutrophils, and NK cells in PE, linking these genes to immune mechanisms underlying PE pathogenesis.ConclusionCGB5, LEP, LRRC1, PAPPA2, and SLC20A1 are validated as key diagnostic biomarkers for PE. Nomogram and FCNN could credibly predict PE. Their association with immune infiltration underscores the crucial role of immune responses in PE pathogenesis.

A Privacy-Preserving Scheme for a Traffic Accident Risk Level Prediction System

Due to the expansion of Artificial Intelligence (AI), especially Machine Learning (ML), it is more common to face confidentiality regulations about using sensitive data in learning models generally hosted in cloud environments. Confidentiality regulations such as HIPAA and GDPR seek to guarantee the confidentiality and privacy of personal information. Input and output data of a learning model may include sensitive data that must be protected. Adversaries could intercept and exploit this data to infer more sensitive data or even to determine the structure of the prediction model. To guarantee data privacy, one option could be encrypting data and making inferences over encrypted data. This strategy would be challenging for learning models that now must receive encrypted data, make inferences over encrypted data, and deliver encrypted data. To address this issue, this paper presents a privacy-preserving machine learning approach using Fully Homomorphic Encryption (FHE) for a model that predicts risk levels of suffering a traffic accident. Despite the limitations of experimenting with FHE on machine learning models using a low-performance computer, limitations that are undoubtedly overcome by using high-performance computational infrastructure, we built some encrypted models. Among the encrypted models based on Decision Trees, Random Forests, XGBoost, and Fully Connected Neural Networks (FCNN), the model based on FCNN reached the highest accuracy (80.1%) for the lowest inference time (8.476 s).

Construction of a prediction model for induction of labor based on a small sample of clinical indicator data

Because of the diversity and complexity of clinical indicators, it is difficult to establish a comprehensive and reliable prediction model for induction of labor (IOL) outcomes with existing methods. This study aims to analyze the clinical indicators related to IOL and to develop and evaluate a prediction model based on a small-sample of data. The study population consisted of a total of 90 pregnant women who underwent IOL between February 2023 and January 2024 at the Shanghai First Maternity and Infant Healthcare Hospital, and a total of 52 clinical indicators were recorded. Maximal information coefficient (MIC) was used to select features for clinical indicators to reduce the risk of overfitting caused by high-dimensional features. Then, based on the features selected by MIC, the support vector machine (SVM) model based on small samples was compared and analyzed with the fully connected neural network (FCNN) model based on large samples in deep learning, and the receiver operating characteristic (ROC) curve was given. By calculating the MIC score, the final feature dimension was reduced from 55 to 15, and the area under curve (AUC) of the SVM model was improved from 0.872 before feature selection to 0.923. Model comparison results showed that SVM had better prediction performance than FCNN. This study demonstrates that SVM successfully predicted IOL outcomes, and the MIC feature selection effectively improves the model's generalization ability, making the prediction results more stable. This study provides a reliable method for predicting the outcome of induced labor with potential clinical applications.

GAO–FCNN–Enabled Beamforming of the RIS–Assisted Intelligent Communication System

The joint beamforming optimization from the perspective of the bit error rate (BER) in a reconfigurable intelligent surface (RIS)–assisted intelligent communication system is studied in this paper. A genetic algorithm (GA) is investigated to address the bottleneck of the system performance based on the dynamic adaptability theory. However, the bottleneck is caused by the interaction between the active and passive beamforming. To tackle the constraints of conventional optimization approaches, the hybrid scheme is proposed to combine the GA optimization (GAO) and fully connected neural network (FCNN) strategy. Specifically, the intelligent collaborative tuning of system parameters is achieved using this proposed technique. Simulation findings indicate that the hybrid scheme not only simplifies the calculation process to obtain the optimal network parameters, but also effectively optimizes the system structure by dynamically adjusting the RIS reflection configuration. Based on this, the signal transmission quality is improved, interference is reduced, and the stable and efficient operation of the RIS–assisted intelligent communication system is ensured in the complex wireless transmission scenario.

Neural network-augmented differentiable finite element method for boundary value problems

Classical numerical methods such as finite element method (FEM) face limitations due to their low efficiency when addressing large-scale problems. As a novel paradigm, the physics-informed neural network (PINN) has demonstrated significant potential to solve partial differential equations. However, conventional PINNs utilize meshless control at discrete sampling points, which limits their ability to effectively handle complex boundaries. Moreover, catastrophic failure may occur in the deep energy method (DEM, a specific type of PINN). To handle these challenges, this study proposes a Neural Network-augmented Differentiable Finite Element Method (NNDFEM) by combining PINN and finite element approximation. In NNDFEM, the neural network backend solely predicts nodal variables. Derivatives and complex boundary conditions can be well handled by the finite element frontend. The governing equation over the domain, Dirichlet, and Neumann boundary conditions are directly enforced on the finite element frontend. Thus, losses of boundary conditions in PINN are rendered unnecessary. The overfitting problem in DEM is also significantly mitigated. Fully connected neural network (FCNN), modified FCNN, and graph-convolutional network are tested as backends. NNDFEM circumvents nodal force calculation and matrix assembly in FEM. Functional losses of linear elasticity, finite strain nonlinear elasticity, heat conduction, and flow in porous media are validated. A systematic exploration unveils the role of 3D finite element mesh. For large-scale problems, a multi-fidelity learning strategy is employed. Thus, the three-dimensional case with over three million degrees of freedom trains well in two minutes. Benefiting from the fast inference of the neural network backend, the forward pass is 8,550 times faster than FEM.

Two-temperature thermochemical nonequilibrium model based on the coarse-grained treatment of molecular vibrational states.

Although the high-fidelity state-to-state (StS) model accurately describes high-temperature thermochemical nonequilibrium flows, its practical application is hindered by the prohibitively high computational cost. In this paper, we develop a reduced-order model that leverages the widelyused two-temperature (2T) framework and a coarse-grained treatment of molecular vibrational states to achieve accuracy comparable to the StS model while ensuring computational efficiency. We observe that the multigroup coarse-grained model (CGM), lumping vibrational energy levels into several groups, yields results close to the StS model for the high-temperature postshock oxygen flows, even using only two groups. However, theone-group CGM (CGM-1G), equivalent to the 2T model but using the StS kinetics, fails to approximate the StS results. Analysis of microscopic group properties reveals that the failure of the CGM-1G stems from the inability to capture the non-Boltzmann effects of mid-to-high vibrational levels, overestimating apparent dissociation rates and vibrational energy loss in the dissociation-dominated region. We then propose an analytical distribution function of vibrational groups by incorporating Treanor-like terms and an additional linear term (addressing the dissociation depletion of high-lying levels). Building upon this algebraic group distribution function and reconstructing vibrational levels within each group using the vibrational temperature, we develop a new 2T model called CG2T, which demonstrates accuracy much closer (than the CGM-1G) to the StS results for the postshock oxygen flows with varying degrees of thermochemical nonequilibrium. Moreover, a fullyconnected neural network is pretrained to substitute the module for the mass and vibrational energy source terms to enhance computational efficiency, achieving about 30-fold speedup in the CG2T model without sacrificing accuracy.

A machine learning paradigm for necessary observations to reduce uncertainties in aerosol climate forcing.

Uncertainties in estimates of climate cooling by anthropogenic aerosols have not decreased significantly in the last two decades, partly because observational constraints on crucial aerosol properties simulated in Earth System Models are insufficient. To help address this insufficiency in aerosol observations, we describe a paradigm for deriving higher-level aerosol properties with machine learning algorithms that use only lidar observations and reanalysis data as predictors. Our paradigm employs high-accuracy suborbital lidar and collocated in situ measurements to train and test two fully-connected neural network algorithms. We use two lidar data sets as input to our machine learning algorithms. The first data set consists of suborbital lidar observations not previously used in the training of the machine learning algorithms. The second data set consists of simulated UV-only observations to preview the algorithms' predictive capabilities in anticipation of data from the ATmospheric LIDar system on the EarthCARE satellite, which was launched in May 2024. Here we show that our algorithms predict two crucial aerosol properties, aerosol light absorption and cloud condensation nuclei concentrations with unprecedented accuracy, yielding mean relative errors of 21% and 13%, respectively, when suborbital lidar data are used as predictors. These errors represent significant improvements over conventional aerosol retrievals. Applied to future satellite missions, the paradigm presented here has great potential for constraining Earth System Models and reducing uncertainties in their estimates of aerosol climate forcing and future global warming.

Development of a unified framework of low-rank approximation and deep neural networks for predicting the spatial variability of SSC in `Spania' watermelons using vis/NIR hyperspectral imaging

Soluble Solids Content (SSC) is an important quality attribute that represents the internal quality of fruits. Visible and near-infrared (Vis/NIR) combined with chemometric algorithms are now popular methods for non-invasive measurement and visualization of SSC. However, in fruits with a thick rind and a large flesh volume, such as watermelon, SSC is not evenly distributed across the fruit. The variability in SSC across the fruit flesh may have an adverse effect on the accuracy of quality analysis algorithms. Thus, this paper presents an accurate and efficient approach for predicting the spatial variation of SSC in watermelons using a combined framework of low-rank approximation and deep neural networks. Watermelon ‘Spania’ was selected for this study, and hyperspectral images of each watermelon were taken from various views, including the top, bottom, and two lateral views. The low-rank property is employed as a constraint in this approach to eliminate the unwanted variations in the spectral data. The low-rank component of the spectral data, free of unwanted variations, is then fed into a fully connected neural network (FNN) for the prediction of watermelon SSC values. The proposed approach obtained optimal performance in calibration and prediction with RC2=0.982, RMSEP = 0.132, and RP2=0.945, RMSEP = 0.195 respectively. Further, the prediction outcomes of the proposed method were compared with state-of-the-art such as Partial Least Square Regression (PLSR), Support Vector Regression (SVR), Multiple Linear Regression (MLR), Decision Tree (DT), and Random Forest (RF). The overall results showed that combining HSI data with a low-rank and deep neural network framework is an efficient method for accurately predicting SSC in watermelons as well as correctly predicting spatial variability of SSC in individual fruits.

Spatial resolution improvement method of RDTS assisted by small-scale thermal region length recognition model

The Total Variation Deconvolution (TVD) algorithm plays an important role in signal reconstruction, however, when it is used to improve the spatial resolution of Raman Distributed Temperature Sensor (RDTS), there are certain challenges in parameter settings. This paper proposes to use Fully-Connected Neural Network to identify the length of small-scale thermal regions(SSTR), and based on the recognition results to set the TVD parameters automatically. We constructed training sets based on the periodic changes of SSTR signals in RDTS (which we call Thermal Region Response Modes, TRRM), to verify performance, we conducted comparative experiments between models obtained from a training set containing 100 types of TRRMs and 25 types of TRRMs, the Macro-F1 value of the former one is 0.2749 higher, reaching 0.9087, performed well in SSTR length recognition tasks. the traditional TVD assisted by this model can increase the spatial resolution of RDTS from 1.6 m to 0.4 m without manual intervention, which complements the lack of automation in applications of TVD and has practical value.

A Fully Connected Neural Network (FCNN) Model to Simulate Karst Spring Flowrates in the Umbria Region (Central Italy)

In the last decades, climate change has led to increasingly frequent drought events within the Mediterranean area, creating an urgent need of a more sustainable management of groundwater resources exploited for drinking and agricultural purposes. One of the most challenging issues is to provide reliable simulations and forecasts of karst spring discharges, whose reduced information, as well as the hydrological processes involving their feeding aquifers, is often a big issue for water service managers and researchers. In order to plan a sustainable water resource exploitation that could face future shortages, the groundwater availability should be assessed by continuously monitoring spring discharge during the hydrological year, using collected data to better understand the past behaviour and, possibly, forecast the future one in case of severe droughts. The aim of this paper is to understand the factors that govern different spring discharge patterns according to rainfall inputs and to present a model, based on artificial neural network (ANN) data training and cross-correlation analyses, to evaluate the discharge of some karst spring in the Umbria region (Central Italy). The model used is a fully connected neural network (FCNN) and has been used both for filling gaps in the spring discharge time series and for simulating the response of six springs to rainfall seasonal patterns from a 20-year continuous daily record, collected and provided by the Regional Environmental Protection Agency (ARPA) of the Umbria region.

Investigation on the optical properties of group-III nitride materials based on fully-connected neural network

In order to accurately simulate the optical properties of nitride optoelectronic devices, such as the reflection spectrum of distributed Bragg reflectors (DRBs), the optical constant is generally considered an important parameter. In this work, the fully-connected neural network is adopted to predict the real and imaginary parts of ordinary dielectric function (DF) of III-nitrides across the full composition range and wide spectral range. The input parameters include Al-component, Ga-component, In-component, and photon energy. The predicted dielectric constant is basically consistent with the results calculated by the analytical model reported in the literature. Then, the band gaps of 6 eV, 3.4 eV and 0.73 eV for AlN, GaN, and InN were determined by using the Tauc formula. The fitted bowing parameters are 0.94 eV, 4.3 eV, and 1.6 eV for AlGaN, InAlN, and InGaN alloys, respectively. Finally, using the predicted dielectric constant, the calculated reflection spectrum of the DBR structures is in agreement with the experimental results in the literature.

Multi-Step Forecasting Method For Influenza Pandemic Using Long Short-Term Memory Model To Improve Public Health Conditions

The flu spread is a major global public health problem that gets attention around the world because it can cause serious illness, cost a lot of money, and kill a lot of people every year. To avoid influenza-like disease (ILD) and run healthcare systems well, predicting when an influenza outbreak will happen is essential. Researchers have used Machine Learning (ML) techniques but have yet to find the best ways to see complicated, non-linear trends in sequential data about flu outbreaks. A Long Short-Term Memory (LSTM) and a Genetic Algorithm (GA) are used together in this study to show a new way to predict multi-step influenza breakouts. Every week, the study gathers information on ILD, which covers flu and other illnesses with signs similar to the flu. When measuring performance during busy times, the suggested model does much better than other advanced Machine Learning (ML) methods and Fully Connected Neural Networks (F-CNN).

Energy expenditure prediction in preschool children: a machine learning approach using accelerometry and external validation

Objective.This study aimed to develop convolutional neural networks (CNNs) models to predict the energy expenditure (EE) of children from raw accelerometer data. Additionally, this study sought to external validation of the CNN models in addition to the linear regression (LM), random forest (RF), and full connected neural network (FcNN) models published in Steenbocket al(2019J. Meas. Phys. Behav.294-102).Approach.Included in this study were 41 German children (3.0-6.99 years) for the training and internal validation who were equipped with GENEActiv, GT3X+, and activPAL accelerometers. The external validation dataset consisted of 39 Canadian children (3.0-5.99 years) that were equipped with OPAL, GT9X, GENEActiv, and GT3X+ accelerometers. EE was recorded simultaneously in both datasets using a portable metabolic unit. The protocols consisted of a semi-structured activities ranging from low to high intensities. The root mean square error (RMSE) values were calculated and used to evaluate model performances.Main results.(1) The CNNs outperformed the LM (13.17%-23.81% lower mean RMSE values), FcNN (8.13%-27.27% lower RMSE values) and the RF models (3.59%-18.84% lower RMSE values) in the internal dataset. (2) In contrast, it was found that when applied to the external Canadian dataset, the CNN models had consistently higher RMSE values compared to the LM, FcNN, and RF.Significance.Although CNNs can enhance EE prediction accuracy, their ability to generalize to new datasets and accelerometer brands/models, is more limited compared to LM, RF, and FcNN models.

Machine Learning-Based Remote Sensing Inversion of Non-Photosynthetic/Photosynthetic Vegetation Coverage in Desertified Areas and Its Response to Drought Analysis

Determining the responses of non-photosynthetic vegetation (NPV) and photosynthetic vegetation (PV) communities to climate change is crucial in illustrating the sensitivity and sustainability of these ecosystems. In this study, we evaluated the accuracy of inverting NPV and PV using Landsat imagery with random forest (RF), backpropagation neural network (BPNN), and fully connected neural network (FCNN) models. Additionally, we inverted MODIS NPV and PV time-series data using spectral unmixing. Based on this, we analyzed the responses of NPV and PV to precipitation and drought across different ecological regions. The main conclusions are as follows: (1) In NPV remote sensing inversion, the softmax activation function demonstrates greater advantages over the ReLU activation function. Specifically, the use of the softmax function results in an approximate increase of 0.35 in the R2 value. (2) Compared with a five-layer FCNN with 128 neurons and a three-layer BPNN with 12 neurons, a random forest model with over 50 trees and 5 leaf nodes provides better inversion results for NPV and PV (R2_RF-NPV = 0.843, R2_RF-PV = 0.861). (3) Long-term drought or heavy rainfall events can affect the utilization of precipitation by NPV and PV. There is a high correlation between extreme precipitation events following prolonged drought and an increase in PV coverage. (4) Under long-term drought conditions, the vegetation in the study area responded to precipitation during the last winter and growing season. This study provides an illustration of the response of semi-arid ecosystems to drought and wetting events, thereby offering a data basis for the effect evaluation of afforestation projects.

Short‐term load interval prediction with unilateral adaptive update strategy and simplified biased convex cost function

AbstractThis article proposes a unilateral Adaptive update strategy based Interval Prediction (AIP) model for short‐term load prediction, which is developed based on lower and upper bound estimation (LUBE) architecture. In traditional LUBE interval prediction model, the model training is usually trained by heuristic algorithms. In this article, the model training is formulated as a bi‐level optimization problem with the help of proposed unilateral adaptive update strategy and cost function. In lower‐level problem, a simplified biased convex cost function is developed to supervise the learning direction of basic prediction engines. The basic prediction engine utilizes Gated Recurrent Unit (GRU) to extract features and Full connected Neural Network (FNN) to generate interval boundary. In upper‐level problem, a unilateral adaptive update strategy with unilateral coverage rate is put forward. It iteratively tunes hyper‐parameters of cost function during training process. Comprehensive experiments based on residential load data are implemented and the proposed interval prediction model outperforms the tested state‐of‐the‐art algorithms, achieving a 15% reduction in prediction error and a 20% decrease in computational time.

Differentiable homotopy methods for gradually reinforcing the training of fully connected neural networks

Deep fully connected neural networks (FCNNs) are the workhorses of deep learning and are broadly applicable due to their “agnostic” structure. Generally, the learning capability of FCNNs improves with the increase in the number of layers and the width of each layer, which, however, comes at an increased computational cost in training. To alleviate this difficulty, in this paper, we develop a gradually reinforced differentiable homotopy (GRDH) method to train the FCNNs. Explicitly speaking, by introducing an extra variable t ranging between zero and one, we design a series of auxiliary functions, which are continuous and monotonically increasing in t. With the above functions, we formulate an optimization problem for training an artificial FCNN, which progressively incorporates more layers and nodes into the neural network as t changes from one to zero and eventually becomes an FCNN with a target number of layers or width of nodes. We prove that the set of solutions to the artificial problem contains an everywhere differentiable path, which starts from a uniquely given point at t=1 and ends at the weights and biases of the target FCNN as t goes to zero. The proposed GRDH method is a novel method that incorporates the differentiable homotopy methods into the training of deep learning methods, and retains the satisfactory theoretical convergence property the classical homotopy methods possess. To promote the application of the GRDH method, we implement it and another efficient method called HTA to train the same FCNNs and find that the GRDH method outperforms the HTA both in the computational time and number of iterations for obtaining a solution with similar (even higher) accuracy. Numerical results further confirm the effectiveness of the GRDH method to solve classification problems.

Imporved model for pavement performance prediction based on recurrent neural network using LTPP database

Accurate pavement performance prediction plays a critical role in formulating maintenance and repair strategies for transportation departments, enabling the achievement of better pavement performance with limited financial resources. However, due to the intricate influence of numerous factors on pavement performance deterioration, improving the accuracy of pavement performance prediction poses a challenge for conventional models. Therefore, the aim of this study is to establish a machine learning-based model pavement performance prediction model. First, this study considers five factors that affect pavement performance, including pavement initial performance indicators, traffic loads, weather, pavement structure, and maintenance measures, and identifies 15 specific indicators that affect pavement performance based on these five factors. Then, based on the Long-Term Pavement Performance (LTPP) database, the study screens and summarizes these indicators, obtaining 2464 high-quality pavement performance data for Pavement Conditions Index (PCI) prediction and 3238 high-quality pavement performance data for IRI prediction. Finally, three distinct prediction models are established, namely, the Fully Connected Neural Network (FCNN) model, the Long Short-Term Memory (LSTM) model, and the combined LSTM-Attention model. The study shows that the LSTM-Attention model performs significantly better than the FCNN and LSTM models, with an R2 coefficient of determination of 0.81 for PCI and 0.79 for IRI. The innovation of this paper is that the authors have introduced the Attention mechanism on the basic of LSTM model, which makes the fitting accuracy of the prediction model further improved.