Additional Input Features Research Articles

Sound recurrence analysis for acoustic scene classification

In everyday life, people experience different soundscapes in which natural sounds, animal noises, and man-made sounds blend together. Although there have been several studies on the importance of recurring sound patterns in music and language, the relevance of this phenomenon in natural soundscapes is still largely unexplored. In this article, we study the repetition patterns of harmonic and transient sound events as potential cues for acoustic scene classification (ASC). In the first part of our study, our aim is to identify acoustic scene classes that exhibit characteristic sound repetition patterns concerning harmonic and transient sounds. We propose three metrics to measure the overall prevalence of sound repetitions as well as their repetition periods and temporal stability. In the second part, we evaluate three strategies to incorporate self-similarity matrices as an additional input feature to a convolutional neural network architecture for ASC. We observe the characteristic repetition of transient sounds in recordings of “park” and “street traffic” as well as harmonic sound repetitions in acoustic scene classes related to public transportation. In the ASC experiments, hybrid network architectures, which combine spectrogram features and features from sound recurrence analysis, show increased accuracy for those classes with prominent sound repetition patterns. Our findings provide additional perspective on the distinctions among acoustic scenes previously primarily ascribed in the literature to their spectral features.

Multitask knowledge-primed neural network for predicting missing metadata and host phenotype based on human microbiome.

Microbial signatures in the human microbiome are closely associated with various human diseases, driving the development of machine learning models for microbiome-based disease prediction. Despite progress, challenges remain in enhancing prediction accuracy, generalizability, and interpretability. Confounding factors, such as host's gender, age, and body mass index, significantly influence the human microbiome, complicating microbiome-based predictions. To address these challenges, we developed MicroKPNN-MT, a unified model for predicting human phenotype based on microbiome data, as well as additional metadata like age and gender. This model builds on our earlier MicroKPNN framework, which incorporates prior knowledge of microbial species into neural networks to enhance prediction accuracy and interpretability. In MicroKPNN-MT, metadata, when available, serves as additional input features for prediction. Otherwise, the model predicts metadata from microbiome data using additional decoders. We applied MicroKPNN-MT to microbiome data collected in mBodyMap, covering healthy individuals and 25 different diseases, and demonstrated its potential as a predictive tool for multiple diseases, which at the same time provided predictions for the missing metadata. Our results showed that incorporating real or predicted metadata helped improve the accuracy of disease predictions, and more importantly, helped improve the generalizability of the predictive models. https://github.com/mgtools/MicroKPNN-MT.

Data-Driven Bi-Directional Lattice Property Customization and Optimization.

Customizing and optimizing lattice materials poses a challenge to designers. This study proposed a data-driven generative method to customize and optimize lattice material. The method utilizes subdivision modeling to parametrically describe lattice morphologies and skeletons. Next, the homogenization method is employed to analyze elastic moduli for collecting a dataset. Then, a two-tiered machine learning (ML) framework is proposed to predict the elastic modulus for a forward design. The first-tier model employs polynomial regression to estimate relative density, which serves as an additional input feature for the second-tier model. The prediction accuracy of the second-tier model is improved through the additional inputs. The forward and reverse design strategies offer a flexible and accurate means of tailoring lattice properties to meet specific performance requirements. Two case studies demonstrate the practical value of the framework: customizing a lattice material to achieve a desired elastic modulus and optimizing the mechanical performance of lattice materials under relative density constraints. The results show that the prediction accuracy of the elastic modulus using the two-tiered ML model achieved an error of less than 10% compared to finite element analysis, demonstrating the reliability of the proposed approach. Furthermore, the optimization design achieved up to a 25% improvement in mechanical performance compared to conventional lattice configurations under the same relative density constraints. These findings underscore the advantages of combining generative design, machine learning, and genetic algorithms to navigate complex design spaces and achieve enhanced material performance.

ANI/EFP: Modeling Long-Range Interactions in ANI Neural Network with Effective Fragment Potentials.

Deep learning Neural Networks (NN) have been developed in the field of molecular modeling for the purpose of circumventing the high computational cost of quantum-mechanical calculations while rivaling their accuracies. Although these networks have found great success, they generally lack the ability to accurately describe long-range interactions, which makes them unusable for extended molecular systems. Herein, we provide a method for partially retraining the deep learning general-use neural network ANI, in which the long-range interactions are represented via atomic electrostatic potentials. The electrostatic potentials, generated with polarizable effective fragment potentials (EFP), are used as an additional input feature for the network. This new ANI/EFP network can predict solute-solvent interaction energies on a trained data set with a kcal/mol accuracy. It also shows promise in predicting the interaction energies of a solute in solvent environments that have not been included in a training data set. The proposed protocol can be taken as an example and further developed, leading to highly accurate and transferable neural network potentials capable of handling long-range interactions and extended molecular systems.

Optimized gradient boosting models and reliability analysis for rock stiffness [formula omitted]

The Extreme gradient boosting algorithm XGBoost has been confirmed to be an accurate method for predicting rock stiffnesses and anisotropic parameters from basic input features such as rock porosity, density, vertical compression stress, pore pressure and burial depth (Nguyen-Sy, T., To, Q.D., Vu, M.N., Nguyen, T.D. and Nguyen, T.T., 2020. Study the elastic properties and the anisotropy of rocks using different machine learning methods. Geophysical Prospecting, 68(8), 2557–2578). This study has the following contributions: reducing the R2-error score (that is, 1-R2) by 35 %, RMSE by 21 % and MAE by 16 % comparing to the previous study by considering an advanced CatXG hybrid boosting model in combination with the optimizer Optuna for predicting C13 (the most difficult stiffness to accurately predict); 2-conduct a reliability analysis for the predicted stiffness C13 with respect to the randomness of the input features. We also discuss the use of C11 or C33 as additional input features for accurately predicting C13 as well as the prediction of the related anisotropic parameter δ. This significant improvement of predicted stiffness C13 is extremely important because it encourages petrophysical engineers to use machine learning for predicting the elastic stiffnesses of rocks.

Enhancing Remote Sensing Water Quality Inversion through Integration of Multisource Spatial Covariates: A Case Study of Hong Kong’s Coastal Nutrient Concentrations

The application of remote sensing technology for water quality monitoring has attracted much attention recently. Remote sensing inversion in coastal waters with complex hydrodynamics for non-optically active parameters such as total nitrogen (TN) and total phosphorus (TP) remains a challenge. Existing studies build the relationships between remote sensing spectral data and TN/TP directly or indirectly via the mediation of optically active parameters (e.g., total suspended solids). Such models are often prone to overfitting, performing well with the training set but underperforming with the testing set, even though both datasets are from the same region. Using the Hong Kong coastal region as a case study, we address this issue by incorporating spatial covariates such as hydrometeorological and locational variables as additional input features for machine learning-based inversion models. The proposed model effectively alleviates overfitting while maintaining a decent level of accuracy (R2 exceeding 0.7) during the training, validation and testing steps. The gap between model R2 values in training and testing sets is controlled within 7%. A bootstrap uncertainty analysis shows significantly improved model performance as compared to the model with only remote sensing inputs. We further employ the Shapely Additive Explanations (SHAP) analysis to explore each input’s contribution to the model prediction, verifying the important role of hydrometeorological and locational variables. Our results provide a new perspective for the development of remote sensing inversion models for TN and TP in similar coastal waters.

Multi-time scales prediction of aggregated schedulable capacity of electric vehicle fleets based on enhanced Prophet-LGBM algorithm

The fast-increasing applications of renewable energy and electric vehicles (EVs) pose significant challenges to the safe and economic operation of the power grid. However, through Virtual Power Plant (VPP) technology, EV fleets can be aggregated into large-scale EV generalized energy storage systems, providing scarce dispatchable flexible resources for the future power system. Rapid and accurate prediction of the Aggregated Schedulable Capacity of EV (EVASC) fleets is a crucial technology for VPP with Electric Vehicles (EV-VPP) to participate in various ancillary services of the power system. In this paper, orienting for various dispatch scenarios in the power system, a multi-time scale prediction model of the EVASC for EV-VPPs is presented based on the enhanced Prophet algorithm combining a light gradient boosting machine (LGBM) algorithm. The enhanced Prophet-LGBM algorithm is initially proposed here by incorporating the LGBM algorithm into the traditional Prophet algorithm in series and adding extra sequential features in its input, such as temperature, weather, and alike. In this way, the problem of overfitting and low capability in addressing additional input features in the traditional Prophet is overcome. The proposed enhanced Prophet-LGBM-based prediction method of the EVASC for EV-VPPs is validated using 1.8 million actual charging records of over 4000 charging piles for half a year at the provincial level. The 30-day simulation results of the day ahead and intraday predictions of the EVASC for EV-VPPs show that much higher performances in the prediction accuracy can be achieved, specifically in holidays.

Enhanced multi-horizon occupancy prediction in smart buildings using cascaded Bi-LSTM models with integrated features

Accurate occupancy prediction in smart buildings is crucial for optimizing energy management, improving occupant comfort, and effectively controlling building systems, particularly for short- and long-term horizons. Recently, deep learning-based occupancy prediction methods have gained considerable attention. However, the full potential of these methods remains under explored in terms of model architecture variations and prediction horizons. This study introduces cascaded LSTM and cascaded Bi-LSTM models for multi-horizon predictions from 10 minutes to 24 hours, integrating a modified activation function, additional input features, and optimized hyper-parameters using OPTUNA. Traditional performance metrics and various other analyses were conducted to compare the models. Both models performed well for short- and long-term predictions, with minimal differences in the results. Nevertheless, analysis focusing on non-zero data errors (accounting for approximately 11% of occupied periods) and occupancy-wise errors showed a significant performance gap between the two models. The cascaded Bi-LSTM model demonstrated consistent performance across various prediction horizons and occupancy variations, with accuracy approximately 10-15% higher than the cascaded LSTM model, highlighting its superior capability in capturing complex dataset dynamics through a bidirectional process. This study highlights the importance of additional input features, data feature analysis, and multi-perspective result analysis to select the most suitable model for occupancy prediction, validated with pre- and post-modeling feature importance analysis.

Application of physics-informed machine learning methods in buckling design of axially compressed cylindrical shells

A physics-informed artificial neural network (ANN) is developed for the buckling design of cylindrical shells under axial compression, and two strategies are applied to incorporate physical knowledge into the ANN model. One strategy is to introduce physics-informed features derived from the local reduced stiffness method (LRSM) as additional input features. The other strategy is to incorporate physical constraints based on elastic buckling theory into the loss function of ANN. The accuracy of the proposed model is verified using a buckling dataset of metal and non-metal cylindrical shells. Results demonstrate that the proposed physics-informed ANN model with four hidden layers achieves better predictive performance than pure data-driven random forest (RF), support vector machine (SVM) and ANN. Furthermore, the physics-informed model could predict moderately conservative buckling loads with a design factor of 1.25.

Convolutional Neural Network using Social Group Optimization for Electrocardiogram

Electrocardiogram (ECG) analysis plays a pivotal role in diagnosing various cardiac conditions, making it essential to develop accurate and robust classification models. This project proposes a novel approach to optimize ECG analysis by considering social group factors, leveraging the combined power of XG Boost classifier and Convolutional Neural Networks (CNNs). The methodology involves collecting a diverse dataset of ECG recordings, spanning various demographic groups. Preprocessing techniques are applied to standardize and clean the data, followed by feature extraction using CNNs to capture intricate patterns in the ECG signals. Subsequently, XG Boost classifier is employed to classify the ECG signals based on the extracted features, considering social group factors as additional input features. Keywords : Electrocardiogram (ECG), Convolutional Neural Network (CNN), XG Boost ,Social Group Optimization (SGO), Deep Learning ,Machine Learning , Signal Processing ,Classification ,Feature Extraction, Cardiac Diseases.

Application of the principles of systems analysis for the development of environment for integrated air pollution forecasting

Air pollution forecasting is a well-known scientific problem, but nitrogen dioxide is not often considered as a pollutant in the problem. Traffic, presented by vehicle emissions, is one of the main sources of air pollution by nitrogen dioxide. Some existing investigations take traffic into account as input features for forecasting models but traffic is presented by values measured during previous hours. Therefore, separate models are created and trained for every air pollution measurement station. Trained forecasting models are not used systemati-cally and models for forecasting other indicators are not applied. But air pollution over cur-rent hour is caused by traffic mostly at the same place over the same hour but not over previ-ous hours. Therefore, it is important to investigate how usage of forecasted values of traffic impacts on air pollution by nitrogen dioxide in comparison with usage of traffic values meas-ured over previous hours. This task is considered in the paper. The objective of the study is to investigate impact of system approach on accuracy of forecasting air pollution by nitrogen dioxide. Three types of forecasting models based on long short-term memory were created to investigate such an impact. These types include models with concentration of nitrogen dioxide over previous hours as input features and two types of models with additional input features presented by values of traffic over previous hours and by forecasted values of traffic over output hours correspondingly. The last type of models pre-sented application of systems approach, as it uses other models available in the integrated forecasting environment. Experimental investigation allowed to create and train models as well as to compare the results of the models on the datasets representing car traffic and air pollution in Madrid. Application of the approach based on the principles of systems analysis increases accuracy of forecasting air pollution by nitrogen dioxide. It enables to create inte-grated air pollution forecasting environment, which combines information technologies of traffic and air pollution forecasting.

Laying the experimental foundation for corrosion inhibitor discovery through machine learning

Creating durable, eco-friendly coatings for long-term corrosion protection requires innovative strategies to streamline design and development processes, conserve resources, and decrease maintenance costs. In this pursuit, machine learning emerges as a promising catalyst, despite the challenges presented by the scarcity of high-quality datasets in the field of corrosion inhibition research. To address this obstacle, we have created an extensive electrochemical library of around 80 inhibitor candidates. The electrochemical behaviour of inhibitor-exposed AA2024-T3 substrates was captured using linear polarisation resistance, electrochemical impedance spectroscopy, and potentiodynamic polarisation techniques at different exposure times to obtain the most comprehensive electrochemical picture of the corrosion inhibition over a 24-h period. The experimental results yield target parameters and additional input features that can be combined with computational descriptors to develop quantitative structure–property relationship (QSPR) models augmented by mechanistic input features.

A comparative analysis of linear regression, neural networks and random forest regression for predicting air ozone employing soft sensor models

The proposed methodology presents a comprehensive analysis of soft sensor modeling techniques for air ozone prediction. We compare the performance of three different modeling techniques: LR (linear regression), NN (neural networks), and RFR (random forest regression). Additionally, we evaluate the impact of different variable sets on prediction performance. Our findings indicate that neural network models, particularly the RNN (recurrent neural networks), outperform the other modeling techniques in terms of prediction accuracy. The proposed methodology evaluates the impact of different variable sets on prediction performance, finding that variable set E demonstrates exceptional performance and achieves the highest average prediction accuracy among various software sensor models. In comparing variable set E and A, B, C, D, it is observed that the inclusion of an additional input feature, PM10, in the latter sets does not improve overall performance, potentially due to multicollinearity between PM10 and PM2.5 variables. The proposed methodology provides valuable insights into soft sensor modeling for air ozone prediction.Among the 72 sensors, sensor NNR[Y]C outperforms all other evaluated sensors, demonstrating exceptional predictive performance with an impressive R2 of 0.8902, low RMSE of 24.91, and remarkable MAE of 19.16. With a prediction accuracy of 81.44%, sensor NNR[Y]C is reliable and suitable for various technological applications.

Physics inspired hybrid attention for SAR target recognition

There has been a recent emphasis on integrating physical models and deep neural networks (DNNs) for SAR target recognition, to improve performance and achieve a higher level of physical interpretability. The attributed scattering center (ASC) parameters garnered the most interest, being considered as additional input data or features for fusion in most methods. However, the performance greatly depends on the ASC optimization result, and the fusion strategy is not adaptable to different types of physical information. Meanwhile, the current evaluation scheme is inadequate to assess the model’s robustness and generalizability. Thus, we propose a physics inspired hybrid attention (PIHA) mechanism and the once-for-all (OFA) evaluation protocol to address the above issues. PIHA leverages the high-level semantics of physical information to activate and guide the feature group aware of local semantics of target, so as to re-weight the feature importance based on knowledge prior. It is flexible and generally applicable to various physical models, and can be integrated into arbitrary DNNs without modifying the original architecture. The experiments involve a rigorous assessment using the proposed OFA, which entails training and validating a model on either sufficient or limited data and evaluating on multiple test sets with different data distributions. Our method outperforms other state-of-the-art approaches in 12 test scenarios with same ASC parameters. Moreover, we analyze the working mechanism of PIHA and evaluate various PIHA enabled DNNs. The experiments also show PIHA is effective for different physical information. The source code together with the adopted physical information is available at https://github.com/XAI4SAR/PIHA.

NOx emissions prediction in diesel engines: a deep neural network approach.

The reduction of various nitrogen oxide (NOx) emissions from diesel engines is an important environmental issue due to their negative impact on air quality and public health. Selective catalytic reduction (SCR) has emerged as an effective technology to mitigate NOx emissions, but predicting the performance of SCR systems remains a challenge due to the complex chemistry involved. In this study, we propose using DNN models to predict NOx emission reductions in SCR systems. Four types of datasets were created; each consisted of five variables as inputs. We evaluated the models using experimental data collected from a diesel engine equipped with an SCR system. Our results indicated that the deep neural network (DNN) model produces precise estimates for exhaust gas temperature, NOx concentration, and De-NOx efficiency. Moreover, inclusion of additional input features, such as engine speed and temperature, improved the prediction accuracy of the DNN model. The mean absolute error (MAE) values for these parameters were 3.1°C, 3.04ppm, and 3.65%, respectively. Furthermore, the R-squared coefficient of determination values for the estimates were 0.912, 0.983, and 0.905, respectively. Overall, this study demonstrates the potential of using DNNs to accurately predict NOx emissions from diesel engines and provides insights into the impact of input features on the performance of the model.

Geometric Deep Learning Using Vascular Surface Meshes for Modality-Independent Unruptured Intracranial Aneurysm Detection.

Early detection of unruptured intracranial aneurysms (UIAs) enables better rupture risk and preventative treatment assessment. UIAs are usually diagnosed on Time-of-Flight Magnetic Resonance Angiographs (TOF-MRA) or contrast-enhanced Computed Tomography Angiographs (CTA). Various automatic voxel-based deep learning UIA detection methods have been developed, but these are limited to a single modality. We propose a modality-independent UIA detection method using a geometric deep learning model with high resolution surface meshes of brain vessels. A mesh convolutional neural network with ResU-Net style architecture was used. UIA detection performance was investigated with different input and pooling mesh resolutions, and including additional edge input features (shape index and curvedness). Both a higher resolution mesh (15,000 edges) and additional curvature edge features improved performance (average sensitivity: 65.6%, false positive count/image (FPC/image): 1.61). UIAs were detected in an independent TOF-MRA test set and a CTA test set with average sensitivity of 52.0% and 48.3% and average FPC/image of 1.04 and 1.05 respectively. We provide modality-independent UIA detection using a deep-learning vascular surface mesh model with comparable performance to state-of-the-art UIA detection methods.

Enhancing Runoff Simulation Using BTOP-LSTM Hybrid Model in the Shinano River Basin

Runoff simulation is an ongoing challenge in the field of hydrology. Process-based (PB) hydrological models often gain unsatisfactory simulation accuracy due to incomplete physical process representations. While the deep learning (DL) models demonstrate their capacity to grasp intricate hydrological response processes, they still face constraints pertaining to the representative training data and comprehensive hydrological observations. In order to provide unobservable hydrological variables from the PB model to the DL model, this study constructed hybrid models by feeding the output variables of the PB model (BTOP) into the DL model (LSTM) as additional input features. These variables underwent feature dimensionality reduction using the feature selection method (Pearson Correlation Coefficient, PCC) and the feature extraction method (Principal Component Analysis, PCA) before input into LSTM. The results showed that the standalone LSTM performed well across the basin, with NSE values all exceeding 0.70. The hybrid models enhanced the simulation performance of the standalone LSTM. The NSE values increased from 0.75 to nearly 0.80 in a sub-basin. Lastly, if the BTOP output is directly fed into LSTM without feature dimensionality reduction, the model’s accuracy significantly decreases due to noise interference. The NSE value decreased by 0.09 compared to the standalone LSTM in a sub-basin. The results demonstrated the effectiveness of PCC and PCA in removing redundant information within hydrological variables. These findings provide new insights into incorporating physical information into LSTM and constructing hybrid models.

Modeling orientational features via geometric algebra for 3D protein coordinates prediction

Protein structure prediction (PSP) is the prediction of the three‐dimensional (3D) folding of a protein (its tertiary structure) starting from its amino acid sequence (its primary structure). The state of the art in PSP is achieved by deep learning pipelines that require several input features extracted from amino acid sequences. It has been demonstrated that features that grasp the relative orientation of amino acids positively impact the prediction accuracy of the 3D coordinates of atoms in the protein backbone. In this paper, we demonstrate the relevance of geometric algebra (GA) in instantiating orientational features for PSP problems. We do so by proposing two novel GA‐based metrics which contain information on relative orientations of amino acid residues. We then employ these metrics as additional input features to a graph transformer (GT) architecture to aid the prediction of the 3D coordinates of a protein, and compare them to classical angle‐based metrics. We show how our GA features yield comparable results to angle maps in terms of accuracy of the predicted coordinates. This is despite being constructed from less initial information about the protein backbone. The features are also fewer and more informative and can be (i) closely associated to protein secondary structures and (ii) more easily predicted compared to angle maps. We hence deduce that GA can be employed as a tool to simplify the modeling of protein structures and pack orientational information in a more natural and meaningful way.

Vehicular Traffic Flow Prediction Model Deep Learning

Abstract: Efficient traffic flow prediction is crucial for effective traffic management and congestion reduction in urban areas. However, traditional statistical models often struggle to accurately capture the intricate dynamics of vehicular traffic flow, particularly under dynamic conditions. In this research project, we propose a novel approach that leverages deep learning techniques, specifically Long Short-Term Memory (LSTM) neural networks, AdaBoost, and gradient descent, to enhance the accuracy of traffic flow predictions .By harnessing historical traffic data, our model generates precise predictions for the next time step, empowering traffic managers to optimize signal timings and proactively reroute traffic. To boost the model's performance, we incorporate AdaBoost, which integrates LSTM predictions as additional input features. We evaluate the accuracy of our model using mean absolute error (MAE) and R2 score techniques, comparing the predicted traffic flow against the actual traffic flow .Experimental results demonstrate that our proposed model outperforms traditional statistical models, exhibiting lower MAE and higher R2 scores. This indicates its efficacy in accurately predicting traffic flow and presents a promising solution for traffic management and congestion reduction. Our research contributes to the advancement of traffic flow prediction models by offering a more reliable and accurate approach. Future work may explore the integration of real- time data streams and external factors, such as weather conditions and events, to further enhance prediction accuracy and effectively address dynamic traffic situations. By optimizing traffic management strategies, reducing congestion, and improving overall traffic flow efficiency, our proposed model holds significant potential for improving urban traffic conditions.

Physical metallurgy guided machine learning to predict hot deformation mechanism of stainless steel

This study proposes a novel approach to address the limitations of traditional experimental and mathematical statistical methods in predicting the deformation mechanism of materials. Specifically, a physical metallurgy-guided machine learning method is proposed to achieve fast and accurate prediction of the thermal deformation mechanisms of antibacterial stainless steels with varying Cu contents. First, the Arrhenius and the dynamic recrystallization model were developed utilizing experimental data for the preliminary prediction of the effect of Cu content on the thermal deformation behavior of antibacterial stainless steel. Then, the calculated intermediate parameters were incorporated into the original data set as additional input features to construct a physical metallurgy (PM) guided least squares support vector machine (LSSVM) prediction model, and the optimization of the prediction model is performed through utilization of a particle swarm optimization (PSO) algorithm. The results indicate that the addition of Cu content influences the rheological stress of antibacterial stainless steel at low temperatures and high strain rates. Furthermore, the increase of Cu content reduces the thermal deformation activation energy, inhibiting the dynamic recrystallization of antibacterial stainless steel, while an increase in deformation temperature or a decrease in strain rate promotes recrystallization. The proposed PM-PSO-LSSVM model is in good agreement with experiments, providing theoretical guidance for optimizing the composition and process parameters of antibacterial stainless steel.