Large-scale Models Research Articles

CONCN: a high-resolution, integrated surface water–groundwater ParFlow modeling platform of continental China

Abstract. Large-scale hydrologic modeling at the national scale is an increasingly important effort worldwide to tackle ecohydrologic issues induced by global water scarcity. In this study, a surface water–groundwater integrated hydrologic modeling platform was built using ParFlow, covering the entirety of continental China with a resolution of 30 arcsec. This model, CONCN 1.0, offers a full treatment of 3D variably saturated groundwater by solving Richards' equation, along with the shallow-water equation at the ground surface. The performance of CONCN 1.0 was rigorously evaluated using both global data products and observations. RSR values (the ratio of the root mean squared error to the standard deviation of observations) show satisfying performance with regard to streamflow, yet the streamflow is lower in the endorheic, Hai, and Liao rivers due to uncertainties in potential recharge. RSR values also indicate satisfying performance in terms of the water table depth of the CONCN model. This is an intermediate performance compared to two global groundwater models, highlighting the uncertainties that persist in current large-scale groundwater modeling. Our modeling work is also a comprehensive evaluation of the current workflow for continental-scale hydrologic modeling using ParFlow and could be a good starting point for modeling in other regions worldwide, even when using different modeling systems. More specifically, the vast arid and semi-arid regions in China with substantial sinks (i.e., the endpoints of endorheic rivers) and the large uncertainties in potential recharge pose challenges for the numerical solution and model performance, respectively. Incompatibilities between data and the model, such as the mismatch of spatial resolutions between models and products and the shorter, less frequent observation records, necessitate further refinement of the workflow to enable fast modeling. This work not only establishes the first integrated hydrologic modeling platform in China for efficient water resources management but will also benefit the improvement of next-generation models worldwide.

Conformal prediction for uncertainty quantification in dynamic biological systems.

Uncertainty quantification (UQ) is the process of systematically determining and characterizing the degree of confidence in computational model predictions. In systems biology, and particularly with dynamic models, UQ is critical due to the nonlinearities and parameter sensitivities that influence the behavior of complex biological systems. Addressing these issues through robust UQ enables a deeper understanding of system dynamics and more reliable extrapolation beyond observed conditions. Many state-of-the-art UQ approaches in this field are grounded in Bayesian statistical methods. While these frameworks naturally incorporate uncertainty quantification, they often require the specification of parameter distributions as priors and may impose parametric assumptions that do not always reflect biological reality. Additionally, Bayesian methods can be computationally expensive, posing significant challenges when dealing with large-scale models and seeking rapid, reliable uncertainty calibration. As an alternative, we propose using conformal predictions methods and introduce two novel algorithms designed for dynamic biological systems. These approaches can provide non-asymptotic guarantees, improving robustness and scalability across various applications, even when the predictive models are misspecified. Through several illustrative scenarios, we demonstrate that these conformal algorithms can serve as powerful complements-or even alternatives-to conventional Bayesian methods, delivering effective uncertainty quantification for predictive tasks in systems biology.

A Time-Segmented SAI-Krylov Subspace Approach for Large-Scale Transient Electromagnetic Forward Modeling

After nearly two decades of development, transient electromagnetic (TEM) 3D forward modeling technology has significantly improved both numerical precision and computational efficiency, primarily through advancements in mesh generation and the optimization of linear equation solvers. However, the dominant approach still relies on direct solvers, which require substantial memory and complicate the modeling of electromagnetic responses in large-scale models. This paper proposes a new method for solving large-scale TEM responses, building on previous studies. The TEM response is expressed as a matrix exponential function with an analytic initial field for a step-off source, which can be efficiently solved using the Shift-and-Invert Krylov (SAI-Krylov) subspace method. The Arnoldi algorithm is used to construct the orthogonal basis for the Krylov subspace, and the preconditioned conjugate gradient (PCG) method is applied to solve large-scale linear equations. The paper further explores how dividing the off-time and optimizing parameters for each time interval can enhance computational efficiency. The numerical results show that this parameter optimization strategy reduces the iteration count of the PCG method, improving efficiency by a factor of 5 compared to conventional iterative methods. Additionally, the proposed method outperforms direct solvers for large-scale model calculations. Conventional approaches require numerous matrix factorizations and thousands of back-substitutions, whereas the proposed method only solves about 300 linear equations. The accuracy of the approach is validated using 1D and 3D models, and the propagation characteristics of the TEM field are studied in large-scale models.

Simulation of Internal Environmental Conditions Within Rock Wool Insulation: Implications for Corrosion Under Insulation in Piping Systems

Rock wool is widely used in industrial piping systems for its excellent thermal insulation properties, but its porous structure allows water infiltration that can lead to corrosion under insulation (CUI) on metal pipe surfaces. In order to investigate how water infiltration into the insulated pipeline system creates a corrosive environment, a study on the flow behavior of fluids in porous media was conducted. Experiments were performed to measure the flow velocity and pressure drop along three principal directions—axial, radial, and circumferential. These measurements enabled the derivation of specific viscous and inertial resistance coefficients, which characterize the flow through the rock wool structure. The results indicated that the flow parameters of rock wool change over time and with repeated use, particularly after dry–wet cycles. The experimentally derived parameters were incorporated into both small-scale and large-scale three-dimensional computational fluid dynamics (CFD) models to simulate water transport within the rock wool insulation layer. Validation experiments performed on a real rock wool-insulated pipeline system confirmed the predictive accuracy of the CFD simulations in capturing water movement through the insulation. The large-scale model further analyzed the influence of inlet velocity, rock wool aging, and pipeline inclination on the development of environmental conditions for CUI.

Bearing characteristics of recycled concrete aggregate encased column composite ground

This paper investigates the bearing characteristics of a recycled concrete aggregate encased column (RCAEC) composite ground with the objective of better-utilizing construction waste. A large-scale model test was conducted, in which recycled concrete aggregate (RCA) was employed as the column material and geosynthetics were encased with varying lengths and layers, with the similarity scale factor assumed to be 10. The impact of various factors, including encasement length, encasement stiffness, length-to-diameter ratio (l/d), and optimal replacement ratio, on the load-bearing characteristics was examined. The findings indicate that the maximum bearing capacity of the composite foundation can be achieved when the encasement length of RCAEC is approximately six times the column diameter, the l/d is 6 ∼ 7, the stiffness of the encasement material is approximately 100 kN/m, and the area replacement rate is around 13%. Lateral compression tests revealed a strong correlation between crushing degree, column-bearing capacity, and aggregate gradation. Increasing pressure led to significant destruction of 10 ∼ 5 mm particles, while particles smaller than 2.5 mm increased, alongside a rise in relative density. For optimal column construction and bearing processes, it is crucial to optimize RCA gradation and limit column loads to below 14,000 kN to prevent excessive crushing of RCA.

TextNeX: Text Network of eXperts for Robust Text Classification—Case Study on Machine-Generated-Text Detection

Efficient and accurate text classification is essential for a wide range of natural language processing applications, including sentiment analysis, spam detection and machine-generated text identification. While recent advancements in transformer-based large language models have achieved remarkable performance, they often come with significant computational costs, limiting their applicability in resource-constrained environments. In this work, we propose TextNeX, a new ensemble model that leverages lightweight language models to achieve state-of-the-art performance while maintaining computational efficiency. The development process of TextNeX model follows a three-phase procedure: (i) Expansion: generation of a pool of diverse lightweight models via randomized model setups and variations of training data; (ii) Selection: application of a clustering-based heterogeneity-driven selection to retain the most complementary models and (iii) Ensemble optimization: optimization of the selected models’ contributions using sequential quadratic programming. Experimental evaluations on three challenging text classification datasets demonstrate that TextNeX outperforms existing state-of-the-art ensemble models in accuracy, robustness and computational effectiveness, offering a practical alternative to large-scale models for real-world applications.

Extending Courant-Friedrichs-Lewy stability condition by local eigenvalue perturbation

The explicit time-marching finite-difference (FD) scheme is widely used for simulating wave propagation; however, its computational efficiency is constrained by the Courant-Friedrichs-Lewy (CFL) stability condition, which limits the maximum allowable time step. The eigenvalue perturbation method effectively extends this condition, but it incurs prohibitive memory demand and computational cost. In contrast, the optimal spatial-filtering method requires no additional memory and achieves numerical accuracy equivalent to that of the eigenvalue perturbation method in homogeneous models. However, its performance deteriorates significantly in heterogeneous models, particularly those with high velocity contrast. To address the excessive memory demand and computational cost of the eigenvalue perturbation method, we develop a local eigenvalue perturbation method to extend the CFL stability condition. First, we construct an augmented explicit time-marching FD scheme for the scalar wave equation; then, we derive the corresponding eigenvalue inequality relation; next, we develop a localization strategy to reduce the memory demand and computational cost of the eigenvalue perturbation method; finally, we perform theoretical analyses and numerical experiments to demonstrate the numerical accuracy of this method. In 2D models, this method significantly reduces memory demand and computational cost by one and two orders of magnitude, respectively, compared to the eigenvalue perturbation method. Moreover, under the same numerical accuracy conditions, the maximum time step allowed by this method is approximately three times greater than that permitted by the optimal spatial-filtering method. This method enables the use of larger time steps beyond the CFL stability condition, showing promise for large-scale models, particularly those with high velocity contrast.

Assessing Coincidence of Satellite Acquisitions and Flood Events to Predict Suitability for Flood Map Synthesis

Flooding is a global problem that impacts people, communities, and governments every year. A better understanding of flooding in an area can enable an improved emergency response before a flood hits. Flood maps are a crucial tool to translate what, for most, is an abstract streamflow into a more understandable and actionable representation of who and what is at risk. Satellite-based flood maps are a useful tool that has potential global applications. We developed methods to determine areas that are suitable for generating satellite-based synthetic flood maps. For our processes, we used Forecasting Inundation Extents using REOF analysis (FIER), a data-driven method of synthesizing flood maps by correlating extracted spatial and temporal patterns from satellite imagery with historical hydrological variables. To overcome the limitation of only using places where gauges are installed, we used large-scale hydrological models, namely the National Water Model (NWM) and the GEOGLOWS Streamflow Model, to provide simulated retrospective streamflow data to train our model. We evaluated locations where both optical and radar imagery would be suitable for creating these models. The procedures we developed and the results that we obtained are potentially transferable to many satellite data sources and methods of model generation.

Poster Generation Using Stable Diffusion 3 and Autoencoders

Creating posters can feel like a hassle, especially if design isn't your strong suit. It often requires a lot of manual adjustments and artistic decisions, which can be overwhelming. That's where PosterGen steps in—a framework designed to simplify and streamline the process. Using a large-scale visual-textual model, PosterGen can automatically find background images that complement your text. It also employs auto encoders to arrange the text in a visually appealing and easy-to-read way. On top of that, it selects fonts and colors that fit the theme, ensuring everything flows seamlessly. What's even more impressive is that PosterGen learns with very little labeled data by using weakly- and self-supervised methods. In our experiments, we've seen that it not only makes poster creation easier but also outperforms some of the current tools and techniques available.

FMEA risk assessment method based on large-scale group decision-making in social networks: considering quantum interference and multiple identities

PurposeIn quantum theory, the interference between the opinions of different decision-makers in group decision-making creates an unobservable interference effect, known as quantum entanglement. To address the interference effects caused by experts’ multiple identities during the decision-making process, this paper proposes a failure mode and effects analysis (FMEA) method to analyze the interference effect on the priority of failure modes.Design/methodology/approachThe study constructs a large-scale group decision-making model in social networks to assess natural hazards’ risk. A social network is constructed to describe the relationships among experts in the FMEA group, and the PageRank algorithm is established to determine expert weights. We quantify the interference caused by multiple identities of experts using information entropy. We update risk factors and subgroup weights considering quantum interference. Grey relational analysis is employed to facilitate the prioritization of failure modes.FindingsThe results of this study show that the failure modes requiring priority attention in the Huludao rainstorm on August 20, 2024, include the lack of an effective meteorological monitoring system, the absence of mandatory drills and delay in returning to normal life.Originality/valueThis study evaluates risks from the perspective of quantum cognition and analyzes the interference effects of multiple identities on group decision-making. The priority of failure modes obtained by our methods is more in line with the actual decision-making situation.

Frozen Large-Scale Pretrained Vision-Language Models are the Effective Foundational Backbone for Multimodal Breast Cancer Prediction.

Breast cancer is a pervasive global health concern among women. Leveraging multimodal data from enterprise patient databases-including Picture Archiving and Communication Systems (PACS) and Electronic Health Records (EHRs)-holds promise for improving prediction. This study introduces a multimodal deep-learning model leveraging mammogram datasets to evaluate breast cancer prediction. Our approach integrates frozen large-scale pretrained vision-language models, showcasing superior performance and stability compared to traditional image-tabular models across two public breast cancer datasets. The model consistently outperforms conventional full fine-tuning methods by using frozen pretrained vision-language models alongside a lightweight trainable classifier. The observed improvements are significant. In the CBIS-DDSM dataset, the Area Under the Curve (AUC) increases from 0.867 to 0.902 during validation and from 0.803 to 0.830 for the official test set. Within the EMBED dataset, AUC improves from 0.780 to 0.805 during validation. In scenarios with limited data, using Breast Imaging-Reporting and Data System category three (BI-RADS 3) cases, AUC improves from 0.91 to 0.96 on the official CBIS-DDSM test set and from 0.79 to 0.83 on a challenging validation set. This study underscores the benefits of vision-language models in jointly training diverse image-clinical datasets from multiple healthcare institutions, effectively addressing challenges related to non-aligned tabular features. Combining training data enhances breast cancer prediction on the EMBED dataset, outperforming all other experiments. In summary, our research emphasizes the efficacy of frozen large-scale pretrained vision-language models in multimodal breast cancer prediction, offering superior performance and stability over conventional methods, reinforcing their potential for breast cancer prediction.

Mapping Brain Lesions to Conduction Delays: The Next Step for Personalized Brain Models in Multiple Sclerosis.

Multiple sclerosis (MS) is a clinically heterogeneous, multifactorial autoimmune disorder affecting the central nervous system. Structural damage to the myelin sheath, resulting in the consequent slowing of the conduction velocities, is a key pathophysiological mechanism. In fact, the conduction velocities are closely related to the degree of myelination, with thicker myelin sheaths associated to higher conduction velocities. However, how the intensity of the structural lesions of the myelin translates to slowing of nerve conduction delays is not known. In this work, we use large-scale brain models and Bayesian model inversion to estimate how myelin lesions translate to longer conduction delays across the damaged tracts. A cohort of 38 subjects (20 healthy and 18 with MS) underwent MEG recordings during an eyes-closed resting-state condition, along with MRI acquisitions and detailed white matter tractography analysis. We observed that MS patients consistently showed decreased power within the alpha frequency band (8-13 Hz) as compared to the healthy group. We also derived a lesion matrix indicating the percentage of lesions for each tract in every patient. Using large-scale brain modeling, the neural activity of each region was represented as a Stuart-Landau oscillator operating in a regime showing damped oscillations, and the regions were coupled according to subject-specific connectomes. We propose a linear formulation to the relationship between the conduction delays and the amount of structural damage in each white matter tract. Dependent upon the parameter , this function translates lesions into edge-specific conduction delays (leading to shifts in the power spectra). Using deep neural density estimators, we found that the estimation of showed a strong correlation with the alpha peak in MEG recordings. The most probable inferred for each subject is inversely proportional to the observed peaks, while power peaks themselves do not correlate with total lesion volume. Furthermore, the estimated parameters were predictive (cross-sectionally) of individual clinical disability. This study represents the initial exploration showcasing the location-specific impact of myelin lesions on conduction delays, thereby enhancing the customization of models for individuals with multiple sclerosis.

Exploration and development of a structured multi-level fusion in an ensemble-based large-scale meta-decision model

Exploration and development of a structured multi-level fusion in an ensemble-based large-scale meta-decision model

Assessing daily GRACE Data Assimilation during flood events of the Brahmaputra River Basin.

The integration of satellite-based observations into hydrological models contributes to achieving more precise simulations, thus supporting hazard mitigation and policy-making especially in poorly gauged basins. Sub-monthly Terrestrial Water Storage (TWS) observations derived from the Gravity Recovery and Climate Experiment (GRACE) mission have been shown to contain useful information for the prediction and monitoring of sub-monthly water storage anomalies such as floods. This study assesses, for the first time, the benefits and challenges of integrating sub-monthly TWS into a large-scale hydrological model during flood events. The experiment is carried out for the Brahmaputra River Basin and the integration is performed through the state-of-the-art of sequential Data Assimilation (DA) with the aim of improving model water storage estimates. The results indicate that the daily TWS DA, based on the Ensemble Kalman Filter (EnKF), successfully introduces the observed sub-monthly TWS variability into the model (differences below 10mm with daily GRACE TWS). The daily TWS DA spatially and vertically downscales storage updates with precise timing and distribution. Especially, it modifies the river storage compartment, where sub-monthly variations are expected during floods. In contrast, the updates of monthly TWS DA, implemented through both an EnKF and an Ensemble Kalman Smoother (EnKS), introduce undesired peaks in the TWS time series. Choosing an adequate model covariance localization is found to be crucial for daily TWS DA. Finally, the statistical characteristics of the daily TWS DA and the translation of water storage updates into river discharge are investigated, and recommendations for future developments are provided.

Learnable Prompting SAM-Induced Knowledge Distillation for Semi-Supervised Medical Image Segmentation.

The limited availability of labeled data has driven advancements in semi-supervised learning for medical image segmentation. Modern large-scale models tailored for general segmentation, such as the Segment Anything Model (SAM), have revealed robust generalization capabilities. However, applying these models directly to medical image segmentation still exposes performance degradation. In this paper, we propose a learnable prompting SAM-induced Knowledge distillation framework (KnowSAM) for semi-supervised medical image segmentation. Firstly, we propose a Multi-view Co-training (MC) strategy that employs two distinct sub-networks to employ a co-teaching paradigm, resulting in more robust outcomes. Secondly, we present a Learnable Prompt Strategy (LPS) to dynamically produce dense prompts and integrate an adapter to fine-tune SAM specifically for medical image segmentation tasks. Moreover, we propose SAM-induced Knowledge Distillation (SKD) to transfer useful knowledge from SAM to two sub-networks, enabling them to learn from SAM's predictions and alleviate the effects of incorrect pseudo-labels during training. Notably, the predictions generated by our subnets are used to produce mask prompts for SAM, facilitating effective inter-module information exchange. Extensive experimental results on various medical segmentation tasks demonstrate that our model outperforms the state-of-the-art semi-supervised segmentation approaches. Crucially, our SAM distillation framework can be seamlessly integrated into other semi-supervised segmentation methods to enhance performance. The code will be released upon acceptance of this manuscript at https://github.com/taozh2017/KnowSAM.

Short-range order based ultra fast large-scale modeling of high-entropy alloys

Short-range order based ultra fast large-scale modeling of high-entropy alloys

SPAST: Arbitrary style transfer with style priors via pre-trained large-scale model

SPAST: Arbitrary style transfer with style priors via pre-trained large-scale model

A Stereo Disparity Map Refinement Method Without Training Based on Monocular Segmentation and Surface Normal

Stereo disparity estimation is an essential component in computer vision and photogrammetry with many applications. However, there is a lack of real-world large datasets and large-scale models in the domain. Inspired by recent advances in the foundation model for image segmentation, we explore the RANSAC disparity refinement based on zero-shot monocular surface normal prediction and SAM segmentation masks, which combine stereo matching models and advanced monocular large-scale vision models. The disparity refinement problem is formulated as follows: extracting geometric structures based on SAM masks and surface normal prediction, building disparity map hypotheses of the geometric structures, and selecting the hypotheses-based weighted RANSAC method. We believe that after obtaining geometry structures, even if there is only a part of the correct disparity in the geometry structure, the entire correct geometry structure can be reconstructed based on the prior geometry structure. Our method can best optimize the results of traditional models such as SGM or deep learning models such as MC-CNN. The model obtains 15.48% D1-error without training on the US3D dataset and obtains 6.09% bad 2.0 error and 3.65% bad 4.0 error on the Middlebury dataset. The research helps to promote the development of scene and geometric structure understanding in stereo disparity estimation and the application of combining advanced large-scale monocular vision models with stereo matching methods.

Surface Flux Homogenization and its Impacts on Convection Across CONUS

Abstract In large scale Earth System Models (ESMs) used to study climate processes, surface heterogeneity that is sub-grid to the larger atmospheric grid is often represented by a number of land tiles, effectively providing a higher resolution land surface to a coarser resolution overlying atmosphere. ESMs, however, average the surface fluxes and other surface characteristics before they are communicated to the atmosphere, ignoring the affect that this variability can have on the atmosphere. In this study, we examine the impact of this flux averaging through 257 2-day summer WRF simulations over the Continental United States (CONUS) at 3km resolution, including runs where the surface fluxes and temperature are homogenized at 60 km prior to communication to the overlying atmosphere. Results show large increases (200mm and higher) in precipitation in moisture limited regions of CONUS, a persistent increase in precipitation bias when compared to observations, and a near universal increase in evaporative fraction. Changes are most significant where moist areas (i.e. water bodies) are averaged with dry areas as the feedback between atmospheric moisture concentrations and the land are weakened when that moisture flux is more spatially distributed through homogenization. Results also show a significant decline in mesoscale flow activity within the atmospheric boundary layer, which in energy limited regions may cause the simulated decreases in precipitation due to less frequent convective initiation. Overall, results indicate that flux averaging applied in large scale models can have unintended consequences by neglecting the heterogeneous imprint of the surface on the atmosphere.

Building an LLM from Scratch

In this work, the development of a basic large language model (LLM) has been presented, with a primary focus on the pre-training process and model architecture. A simplified transformer-based design has been implemented to demonstrate core LLM principles with the incorporation of reinforcement learning techniques. Key components such as tokenization, and training objectives have been discussed to provide a foundational understanding of LLM construction. Additionally, an overview of several established models—including GPT-2, LLaMA 3.1, and DeepSeek—has been provided to contextualize current advancements in the field. Through this comparative and explanatory approach, the essential building blocks of large-scale language models have been explored in a clear and accessible manner.