Heterogeneous Model Research Articles

Crude Oil Markets Volatility Forecasting: A Novel Deep Learning Hybrid Model

ABSTRACTTo the national economy, increasing the forecasting accuracy of realised volatility (RV) on crude oil futures markets is of critical strategic importance. However, the RV of crude oil futures cannot be accurately predicted with a single model. For this study, we adopt a hybrid model which combines gated recurrent unit (GRU) and complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) to forecast the RV of crude oil futures. Moreover, back propagation neural networks (BP), Elman neural networks (Elman), support vector regression machine (SVR), autoregressive model (AR), heterogeneous autoregressive model (HAR), and their hybrid models with CEEMDAN are adopted as comparisons. In general, this article demonstrates the superiority of the CEEMDAN‐GRU model in RV forecasting from several aspects: for both evaluation criteria, CEEMDAN‐GRU achieves the highest RV forecasting accuracy in emerging and developed crude oil futures markets; furthermore, the empirical results are robust to alternative realised measures and training sets of different lengths.

HGNN−BRFE: Heterogeneous Graph Neural Network Model Based on Region Feature Extraction

With the strong capability of heterogeneous graphs in accurately modeling various types of nodes and their interactions, they have gradually become a research hotspot, promoting the rapid development of the field of heterogeneous graph neural networks (HGNNs). However, most existing HGNN models rely on meta−paths for feature extraction, which can only utilize part of the data from the graph for training and learning. This not only limits the data generalization ability of deep learning models but also affects the application effect of data−driven adaptive technologies. In response to this challenge, this study proposes a new model—heterogeneous graph neural network based on regional feature extraction (HGNN−BRFE). This model enhances performance through an “extraction−fusion” strategy in three key aspects: first, it efficiently extracts features of neighboring nodes of the same type according to specific regions; second, it effectively fuses information from different regions and hierarchical neighbors using attention mechanisms; third, it specially designs a process for feature extraction and fusion targeting heterogeneous type nodes, ensuring that the rich semantic and heterogeneity information within the heterogeneous graph is retained while maintaining the node’s own characteristics during the node embedding process to prevent the loss of its own features and potential over−smoothing issues. Experimental results show that HGNN−BRFE achieves a performance improvement of 1–3% over existing methods on classification tasks across multiple real−world datasets.

DNAR-05. VALIDATING THE ERK5 PROTAC OS11 AS A POTENTIAL NOVEL THERAPEUTIC INEX VIVO GLIOMA STEM CELL MODELS OF INTRATUMOURAL HETEROGENEITY AND RESIDUAL DISEASE

Abstract Glioblastoma is a grade IV brain tumour with poor survival rates of 15 months post-diagnosis, and 5-year survival rates of 10%. Limited progress over the past 40 years to improve the current treatment regimen of surgery and chemo-/radio-therapy are mainly due to tumour heterogeneity that drive resistance mechanisms. The persistence of glioma stem-like cells (GSCs) sub-populations, particularly within post-surgical residual tumour tissue. ERK5 is an emerging oncology drug target that we previously highlighted as a potential target for combined therapy with temozolomide (TMZ) in glioblastoma due to heightened ERK5 expression, which is associated with poor survival. This current project aims to expand on our previous findings by establishing whether targeting ERK5 can sensitise glioblastoma cells to TMZ using primary derived ex vivo GSC models of intratumor heterogeneity and residual disease. Primary GSC models were derived from patients at the Royal Hallamshire Hospital. The PROTAC OS11 (CRUK & University of Manchester) was used to degrade ERK5. DNA damage and cell survival were evaluated following treatment of TMZ alone and in combination with OS11. Consistent with our findings in established glioma cell lines, OS11 was able to robustly degrade ERK5 in a 3D GSC model for at least 72hrs. Combining OS11 and TMZ in GSCs led to a significant increase in DNA damage, with MGMT negative cells exhibiting a greater increase in DNA damage compared to MGMT positive cells. However, unlike our previous findings in established cell lines, this did not lead to reduced cell survival in combination with TMZ. These findings further support the preclinical use of ex vivo models that better represent intratumoural heterogeneity and post-surgical GCS niches. Overall, these data highlight the value of preclinical validation in more clinically relevant models to identify the best possible novel therapeutic approaches to pursue further towards clinical delivery.

STEM-01. SHARED ASTROCYTE-LIKE GLIOMA STEM CELLS DRIVE HETEROGENEITY AND IMMUNE DYNAMICS IN ADULT-TYPE DIFFUSE GLIOMAS

Abstract Adult-type diffuse gliomas present significant treatment challenges due to their complex cellular composition and evolving nature. Understanding this heterogeneity is critical for the development of effective, targeted therapies. We conducted single-cell and single-nucleus RNA sequencing on a unique cohort of 66 tumours (single surgery) and 54 paired samples (two surgeries). This comprehensive dataset of approximately 900,000 cells allows us to dissect conserved cellular landscapes and dynamic evolution across diverse glioma subtypes. Using a novel phylogeny-based approach, we identified an astrocyte-like glioma stem cell (GSC) as the root of diverse glioma lineages. This discovery transforms our understanding of glioma development, pinpointing the GSC as a potential therapeutic target. The GSC’s similarity to healthy adult neural stem cells provides clues as to the origin of glioma heterogeneity and the shared basis of intratumoural diversity. Our analysis uncovered a shared cellular architecture across astrocytomas, oligodendrogliomas, and glioblastomas (GBMs). This architecture encompasses seven recurring malignant cell states: neuro-, oligo-, neuro-oligo-lineage, cycling (G1/S, G2/M), hypoxia-associated, and a quiescent astrocyte-like GSC. Importantly, these cell state proportions vary significantly between glioma subtypes. Additionally, we demonstrate that GBMs display substantially higher T-cell and myeloid cell infiltration than IDH-mutant gliomas. Counterintuitively, this correlates with a poorer prognosis, implying profound T-cell dysfunction and intricate myeloid cell-tumour interactions. Longitudinal analysis further highlights the dynamic nature of glioma cell populations and the immune landscape. Subtype-specific T-cell and myeloid gene expression profiles were identified, including potential targets such as PARD6G (GBM) and CXCL13 (IDH-mutant-astrocytoma), laying the groundwork for tailored immunotherapies and prognostic biomarkers. This study presents a comprehensive model of glioma heterogeneity and evolution. The model reveals a shared astrocyte-like GSC that fuels cellular diversity, coupled with distinct, dynamic immune landscapes in different glioma subtypes. These findings hold extraordinary potential for developing transformative treatment strategies. Future research functionally validating the GSC and its impact on tumour dynamics will be essential for translating these insights into improved patient outcomes.

ViT‐based quantification of intratumoral heterogeneity for predicting the early recurrence in HCC following multiple ablation

AbstractObjectivesThis study aimed to develop a quantitative intratumoral heterogeneity (ITH) model for assessing the risk of early recurrence (ER) in pre‐treatment multimodal imaging for hepatocellular carcinoma (HCC) patients undergoing ablation treatments.MethodsThis multi‐centre study enrolled 633 HCC patients who underwent ultrasound‐guided local ablation between January 2015 and September 2022. Among them, 422, 85, 57 and 69 patients underwent radiofrequency ablation (RFA), microwave ablation (MWA), laser ablation (LA) and irreversible electroporation (IRE) ablation, respectively. Vision‐Transformer‐based quantitative ITH (ViT‐Q‐ITH) features were extracted from the US and MRI sequences. Multivariable logistic regression analysis was used to identify variables associated with ER. A combined model integrated clinic‐radiologic and ViT‐Q‐ITH scores. The prediction performance was evaluated concerning calibration, clinical usefulness and discrimination.ResultsThe final training cohort and internal validation cohort included 318 patients and 83 patients, respectively, who underwent RFA and MWA. The three external testing cohorts comprised of 106 patients treated with RFA, 57 patients treated with LA and 69 patients who underwent IRE ablation. The combined model showed excellent predictive performance for ER in the training (AUC: .99, 95% CI: .99–1.00), internal validation (AUC: .86, 95% CI: .78–.94), external testing (AUC: .83, 95% CI: .73–.92), LA (AUC: .84, 95% CI: .73–.95) and IRE (AUC: .82, 95% CI: .72–.93) cohorts, respectively. Decision curve analysis further affirmed the clinical utility of the combined model.ConclusionsThe multimodal‐based model, incorporating clinic‐radiologic factors and ITH features, demonstrated superior performance in predicting ER among early‐stage HCC patients undergoing different ablation modalities.

EXTH-33. COMBATING TUMOR HETEROGENEITY IN GLIOBLASTOMA WITH A NOVEL CAR-BASED PLATFORM

Abstract Tumor heterogeneity is a major hindrance impeding success of antigen targeting immunotherapies, such as chimeric antigen receptor (CAR) T cells, in solid tumors. CAR T cells engineered to target EGFRvIII, IL13Rα2, and HER2 have mostly failed clinically to treat glioblastoma, because these antigens are typically only expressed on a subset of a tumor cells within a patient. Thus, finding a surface target that more universally characterizes tumor cells will likely be critical if antigen-specific immunotherapies are to succeed in glioblastoma and other solid tumors. Recently, we uncovered a mechanism whereby CD8 T cells can directly kill tumor cells independently of tumor-expressed major histocompatibility complex (MHC)-I. We found this mechanism to be dependent on NKG2D expression on T cells and NKG2D ligand (NKG2DL) expression on tumor cells. Interestingly, NKG2DL expression was not limited to MHC-I-deficient tumor cells, nor was it limited to gliomas, but NKG2DL expression characterized a variety of solid tumor types. Therefore, NKG2DL is potentially an ideal target for immune-based therapy. Here, we developed a bi-specific fusion protein comprised of the extracellular domain of NKG2D linked to an anti-CD3scFv. Thus, T cells will be redirected to target NKG2DL-expressing tumor cells. We have termed this fusion protein Bi-specific Engagers without Antigen Requirements (BEAR). To ensure local, intratumoral delivery of BEAR, we engineered an EGFRvIII-targeting CAR T cell to secrete BEAR (VIII CAR Engineered with BEAR; VIII “CARE-BEAR”). VIII CARE-BEAR demonstrated potent tumor kill in both in vitro and in vivo tumor heterogeneity models. Moreover, we engineered an EGFRwt-targeting (D2C7) CARE-BEAR to exemplify the potential utility of CARE-BEAR beyond glioblastoma.

Graph-Based Methods for Forecasting Realized Covariances

Abstract We forecast the realized covariance matrix of asset returns in the U.S. equity market by exploiting the predictive information of graphs in volatility and correlation. Specifically, we augment the Heterogeneous Autoregressive model via neighborhood aggregation on these graphs. Our proposed method allows for the modeling of interdependence in volatility (also known as spillover effect) and correlation, while maintaining parsimony and interpretability. We explore various graph construction methods, including sector membership and graphical LASSO (for modeling volatility), and line graph (for modeling correlation). The results generally suggest that the augmented model incorporating graph information yields both statistically and economically significant improvements for out-of-sample performance over the traditional models. Such improvements remain significant over horizons up to 1 month ahead, but decay in time. The robustness tests demonstrate that the forecast improvements are obtained consistently over the different out-of-sample sub-periods and are insensitive to measurement errors of volatilities.

Dynamics of a single population model with memory and nonlinear boundary condition

In this paper, a heterogeneous single population model with memory effect and nonlinear boundary condition is investigated. By virtue of the implicit function theorem and Lyapunov-Schmidt reduction, spatially nonconstant positive steady state solutions appear from two trivial solutions, respectively. Through the distribution of the eigenvalues and bifurcation analysis, sufficient conditions for the occurrence of the Hopf bifurcation associated with one spatially nonconstant positive steady state are obtained. Results about the stability associated with the other one are also yielded. It is found that with the interaction of memory delay, spatial heterogeneity and nonlinear boundary condition, the Hopf bifurcation will happen in such diffusive model. To be specific, the memory delay could induce a single stability switch from stability to instability. As contrast to the diffusive model only with memory effect, the stability switch is independent of memory delay. The obtained results are applied to some specific models, from which we can find that the theoretical analysis is verified and the results enrich the existing ones.

A novel personalized homogenous finite element model to predict the pull-out strength of cancellous bone screws

BackgroundOrthopedic surgeries often involve the insertion of bone screws for various fixation systems. The risk of postoperative screw loosening is usually assessed through experimental or finite element (FE) evaluations of the screw pull-out strength. FE simulations are based on either personalized complex but accurate heterogeneous modeling or non-personalized simple but relatively less accurate homogeneous modeling. This study aimed to develop and validate a novel personalized computed tomography (CT)-based homogeneous FE simulation approach to predict the pull-out force of cancellous bone screws.MethodsTwenty FE simulations of L1-L5 vertebral screw pull-out tests were conducted, i.e., 10 heterogeneous and 10 homogenous models. Screws were inserted into the lower-middle region of vertebrae. In our novel homogeneous model, the region around approximately twice the diameter of the screw was used as a bone material reference volume. Subsequently, the overall material property of this region was homogeneously attributed to the entire vertebra, and pull-out simulations were conducted.ResultsThe mean error of the predicted pull-out forces by our novel homogenous simulations was ~ 7.9% with respect to our heterogeneous model. When solely the cancellous bone was involved during the pull-out process (i.e., for L1, L2, and L3 vertebral bodies whose cortical bone in the inferior region is thin), the novel homogenous model yielded small mean error of < 6.0%. This error, however, increased to ~ 11% when the screw got involved to the cortical bone (for L4 and L5 vertebrae whose cortical bone in the inferior region is thick).ConclusionThe proposed personalized CT-based homogenous model was highly accurate in estimating the pull-out force especially when only the cancellous bone was involved with the screw.

Synergistic development path of conventional and unconventional water in China: From the perspectives of output and substitution elasticities

Exploring the synergistic relationship between conventional water (CW) and unconventional water (UW) is a better way to alleviate water scarcity. However, research remains lacking on the complex relationship between CW and UW from a water resources subdivision perspective. By constructing a heterogeneous trans-log stochastic frontier model, this study estimated the technical efficiency, output elasticity, and substitution elasticity; further, the study innovatively explored five development pathways for CW and UW. The results are as follows. First, China's overall technical efficiency demonstrated a fluctuating upward trend over the study period, which was significantly boosted by higher levels of research and development investment and an increased number of granted patents. Second, compared with UW, CW made a more substantial contribution to output. The output elasticity of CW demonstrated a “low in the center, high on the periphery” pattern, with the lowest elasticity areas clustered in the middle and lower reaches of the Yellow River. Conversely, the output elasticity of UW followed a “high in the east and low in the west” pattern, which was similar to that of capital. Third, with 2012 as the turning point, CW and UW shifted from a substitution to a complementary relationship. Fourth, this study proposes alternative structural optimization paths under various scenarios, namely balanced development of CW and UW, CW-driven, UW-driven, coordinated development of CW and UW, and aided development of CW and UW.

Overcoming clay structure challenges in lithium recovery from boron waste using high‐temperature pressure acid leaching

AbstractBoron mines contain significant amounts of lithium along with boron. After boron is extracted, lithium remains in the waste, which has a carbonate‐hosted clay‐type structure, along with other impurities. The scarcity of lithium resources and the increasing need for lithium worldwide make such resources economically important. Although the best hydrometallurgical method for the recovery of lithium trapped within the clay‐structured mineral resources is roasting with chemicals to disrupt the clay structure and acid leaching, the process is quite difficult and costly due to the high energy and chemical addition requirements. To overcome this challenge, this study proposed a high‐temperature–pressure sulphuric acid leaching process to recover lithium from the boron waste. Under the optimized conditions (liquid/solid ratio: 10, acid concentration: 1 M, temperature: 150°C, and contact time: 120 min), 100% of lithium was leached. The leaching mechanism was determined through mineral characterization (X‐ray diffractometry [XRD], X‐ray fluorescence spectrophotometer [XRF], scanning electron microscopy–energy‐dispersive X‐ray spectroscopy [SEM–EDX], Mastersizer), and a shrinking core heterogeneous kinetics model. It was found that high‐temperature–pressure sulphuric acid leaching disrupted clay structure and promoted the leaching of lithium, the leaching kinetics fit the shrinking core heterogeneous kinetics model, and was controlled by a dual mechanism with ash diffusion and chemical reactions on the particle surface. The reaction rate constants increased with increasing temperature, and the activation energy was found to be 32.17 kJ/mol.

Two-tier stochastic frontier analysis: heterogeneous error distributions and model selection

Two-tier stochastic frontier analysis: heterogeneous error distributions and model selection

Secure multi-synchronization of heterogeneous dynamical networks with deception attacks via event-triggered impulsive control

This article investigates the secure multi-synchronization control problem of the heterogeneous dynamical networks (DNs) subject to deception attacks (DAs). First of all, a heterogeneous DNs model under DAs is built. The deception attackers are assumed to inject false information into the sensor-to-controller channels and the controller-to-actuator channels. Then, the centralized and distributed event-triggered impulsive controllers (ETIC) are designed respectively for DNs subject to DAs to achieve secure multi-synchronization. The synchronization error converges to a bounded set dependent on the heterogeneity of networks and the total attack energy. In addition, the heterogeneous DNs without DAs under ETIC can achieve multi-synchronization. Finally, a numerical example is given to verify the correctness of the theoretical results.

FedGen: Personalized federated learning with data generation for enhanced model customization and class imbalance

Federated learning has emerged as a prominent solution for the collaborative training of machine learning models without exchanging local data. However, existing approaches often impose rigid constraints on model heterogeneity, limiting the ability of clients to customize unique models and increasing the vulnerability of models to potential attacks. This paper presents FedGen, a novel personalized federated learning framework based on generative adversarial networks (GANs). FedGen shifts the focus from training task-specific models to generating data, especially for minority classes with imbalanced data. With FedGen, clients can gain knowledge from others by training generators, while maintaining a heterogeneous local model and avoiding sharing model information with other participants. Moreover, to address challenges arising from imbalanced data, we propose AT-GAN, a novel generative model incorporating pseudo augmentation and differentiable augmentation modules to foster healthy competition between the generator and discriminator. To evaluate the effectiveness of our approach, we conduct extensive experiments on real-world tabular datasets. The experimental results demonstrate that FedGen significantly enhances the performance of local models, achieving improvements of up to 11.92% in F1 score and up to 9.14% in MCC score compared to existing methods.

A Novel Heterogeneous Deformable Surface Model Based on Elasticity

A Novel Heterogeneous Deformable Surface Model Based on Elasticity

Stack-AVP: A Stacked Ensemble Predictor Based on Multi-view Information for Fast and Accurate Discovery of Antiviral Peptides

AVPs, or antiviral peptides, are short chains of amino acids capable of inhibiting viral replication, preventing viral entry, or disrupting viral membranes. They represent a promising area of research for developing new antiviral therapies due to their potential to target a broad spectrum of viruses, incorporating those resistant to traditional antiviral drugs. However, traditional experimental methods for identifying AVPs are often costly and labour-intensive. Thus far, multiple computational methods have been introduced for the in silico identification of AVPs, but these methods still have certain shortcomings. In this study, we propose a novel stacked ensemble learning framework, termed Stack-AVP, for fast and accurate AVP identification. In Stack-AVP, we investigated heterogeneous prediction models, which were trained with 12 commonly used machine learning algorithms coupled with a wide range of multiple feature encoding schemes. Subsequently, these prediction models were adopted to generate multi-view features providing class information and probability information. Finally, we applied our feature selection method to determine the best feature subset for the construction of the final stacked model. Comparative assessments on the independent test dataset revealed that Stack-AVP surpassed the performance of current state-of-the-art methods, with an accuracy of 0.930, MCC of 0.860, and AUC of 0.975. Furthermore, it was found that our multi-view features exhibited a crucial mechanism to improve the prediction performance of AVPs. To facilitate experimental scientists in performing high-throughput identification of AVPs, the prediction sever Stack-AVP is publicly accessible at https://pmlabqsar.pythonanywhere.com/Stack-AVP.

Collaborative Learning With Heterogeneous Local Models: A Rule-Based Knowledge Fusion Approach

Collaborative Learning With Heterogeneous Local Models: A Rule-Based Knowledge Fusion Approach

Heterogeneous graph representation learning via mutual information estimation for fraud detection

In the fraud detection, fraudsters frequently engage with numerous benign users to disguise their activities. Consequently, the fraud graph exhibits not only homogeneous connections between the fraudsters and the same labelled nodes, but also heterogeneous connections, where fraudsters interact with the legitimate nodes. Heterogeneous graph representation learning aims at extracting the structural and semantic information and embed it into the low-dimensional node representation. Recently, maximizing the mutual information between the local node embedding and the summary representation has achieved the promising results on node classification tasks. However, existing deep graph infomax methods still have the following limitations. Firstly, attribute information of nodes in the graph is not fully utilized for capturing the semantic relationships between nodes. Secondly, the local and global supervision signal are not simultaneously exploited for the node embedding learning. Thirdly, the multiplex heterogeneous relations among nodes are ignored. To address these issues, a heterogeneous graph representation learning model by mutual information estimation (MIE-HetGRL) is proposed in this paper to identify the fraudsters in the fraud review graph. Concretely, a high-order mutual information estimation is proposed to integrate the local and global mutual information as the supervision signal. Then we devise a semantic attention fusion module to aggregate the relation-aware node embeddings into a compact node representation. Finally, a joint contrastive learning is designed for facilitating the training and optimization of model. The experimental results show that our proposed model significantly outperforms state-of-the-art baselines for fraud detection.

Exploring the influence of block environmental characteristics on land surface temperature and its spatial heterogeneity for a high-density city

A fundamental understanding of the spatial change trends and driving mechanisms of land surface temperature (LST) under urbanization is a prerequisite for the development of effective strategies to mitigate the urban heat island effect. In this study, the built-up blocks of Shenzhen, a high-density city in China, were selected as the unit of analysis. Multi-source datasets were utilized to calculate a total of 44 environmental characteristic indicators, covering four categories. In order to comprehensively analyze the influence of each environmental feature indicator on LST and spatial heterogeneity, MLR, XGBoost and MGWR models were constructed. Furthermore, the nonlinear relationship between the variables was investigated using the SHAP method. The results demonstrated that the predictive efficacy of the MGWR and XGBoost models was markedly superior to that of the MLR model. The percentage cover of forest, the average elevation, NDVI, the frontal area index and the standard deviation of building height were identified as the primary determinants of the LST. These factors account for more than 52% to the explanation of the LST distribution. The effects of the majority of landscape pattern, building form and street view indicators on LST exhibited spatial heterogeneity. Furthermore, the indicators also showed nonlinear patterns and threshold effects on LST. The findings offer valuable insights for enhancing the urban thermal environment, particularly in high-density urban areas.

Air leakage mechanism and hole sealing technology in directional long-drilled perimeter rock-borehole composite fissures: for gas enrichment in varying coal-seam thickness change areas

The issue of gas enrichment in the tectonic zone of coal seams, which poses significant threats to coal mine safety and leads to gas disasters, is primarily addressed through borehole extraction. The quality of this extraction is notably influenced by air leakage and sealing technologies used during drilling. To tackle the problem of gas leakage in directional long boreholes within the floor, a composite fissure leakage model for the surrounding rock and borehole is developed. This model inverts the characteristics of coal seam thickness variations and abnormal gas enrichment through heterogeneous geostatistical modeling. It enables the quantitative characterization of the air leakage mechanism in directional long boreholes of the bottom plate and proposes an effective sealing process. Due to damage and disturbance, fissures and irregular plastic zones form around the excavated roadway, which are major contributors to air leakage. Over time, the air content in the coal and rock seams gradually increases while the gas content and concentration decrease continuously. Compared to the polyurethane "two plugs and one injection" sealing process, the multi-stage grouting and sealing process provides a more stable gas extraction concentration and flow rate in the sealed drill holes. This process effectively seals the fissure zones of the roadway, reduces air leakage from the boreholes, and improves gas extraction concentration.