Large-scale Models Research Articles

Evaluating ChatGPT's effectiveness and tendencies in Japanese internal medicine.

ChatGPT, a large-scale language model, is a notable example of AI's potential in health care. However, its effectiveness in clinical settings, especially when compared to human physicians, is not fully understood. This study evaluates ChatGPT's capabilities and limitations in answering questions for Japanese internal medicine specialists, aiming to clarify its accuracy and tendencies in both correct and incorrect responses. We utilized ChatGPT's answers on four sets of self-training questions for internal medicine specialists in Japan from 2020 to 2023. We ran three trials for each set to evaluate its overall accuracy and performance on nonimage questions. Subsequently, we categorized the questions into two groups: those ChatGPT consistently answered correctly (Confirmed Correct Answer, CCA) and those it consistently answered incorrectly (Confirmed Incorrect Answer, CIA). For these groups, we calculated the average accuracy rates and 95% confidence intervals based on the actual performance of internal medicine physicians on each question and analyzed the statistical significance between the two groups. This process was then similarly applied to the subset of nonimage CCA and CIA questions. ChatGPT's overall accuracy rate was 59.05%, increasing to 65.76% for nonimage questions. 24.87% of the questions had answers that varied between correct and incorrect in the three trials. Despite surpassing the passing threshold for nonimage questions, ChatGPT's accuracy was lower than that of human specialists. There was a significant variance in accuracy between CCA and CIA groups, with ChatGPT mirroring human physician patterns in responding to different question types. This study underscores ChatGPT's potential utility and limitations in internal medicine. While effective in some aspects, its dependence on question type and context suggests that it should supplement, not replace, professional medical judgment. Further research is needed to integrate Artificial Intelligence tools like ChatGPT more effectively into specialized medical practices.

Speculative computing for AAFM solutions in large-scale product configurations.

Parallel computing is a current algorithmic approach to looking for efficient solutions; that is, to define a set of processes in charge of performing at the same time the same task. Advances in hardware permit the massification of accessibility to and applications of parallel computing. Nonetheless, some algorithms include steps that require or depend on the results of other steps that cannot be parallelized. Speculative computing allows parallelizing those tasks and reviewing different execution flows, which can involve executing invalid steps. Speculative computing solutions should reduce those invalid flows. Product configuration refers to selecting features from a set of available options respecting some configuration constraints; a not complex task for small configurations and models, but a complex one for large-scale scenarios. This article exemplifies a videogame product line feature model and a few configurations, valid and non-valid, respectively. Configuring products of large-scale feature models is a complex and time-demanding task requiring algorithmic solutions. Hence, parallel solutions are highly desired to assist the feature model product configuration tasks. Existing solutions follow a sequential computing approach and include steps that depend on others that cannot be parallelized at all, where the speculative computing approach is necessary. This article describes traditional sequential solutions for conflict detection and diagnosis, two relevant tasks in the automated analysis of feature models, and how to define their speculative parallel version, highlighting their computing improvements. Given the current parallel computing world, we remark on the advantages and current applicability of speculative computing for producing faster algorithmic solutions.

Shaking Tables Test on Seismic Responses of a Long-Span Rigid-Framed Bridge Considering Traveling Wave Effect and Soil–Structure Interaction

The traveling wave effect and soil–structure interaction have significant influence on the seismic response of large-span bridges with complex site conditions. In this paper, a 1/10 scaled-down large-span rigid-framed bridge model was designed and fabricated, and a shaking tables test considering the traveling wave effect and soil–structure interaction was carried out on a large-scale continuous rigid bridge model by a real-time substructure hybrid test technique. Influences of the traveling wave effect and soil–structure interaction on the seismic responses of the rigid-framed bridge specimen were systematically analyzed with experimental data. The test results showed that when the apparent wave speed was small, the traveling wave effect increased the seismic responses of the rigid-framed bridge. With the increase in apparent wave speed, the structural response under traveling wave excitation and uniform excitation was basically the same. The SSI effect lead to a great change in the seismic input peaks and spectral characters at the bottom of the pier, and increased the seismic responses of the rigid-framed bridge. When both traveling wave and the SSI effect were considered, there was a phase difference in the seismic excitation. The dynamic responses of a continuous rigid-framed bridge could not be simply obtained by superposition of the separate traveling wave effect or SSI effect. Meanwhile, the real-time substructure test method in this paper solved the problems that the traditional soil box experiment cannot be applied to the test of a large-scale model, the soil and bridge structure find it difficult to meet the unified similarity ratio, and the boundary conditions are difficult to simulate accurately.

Maximizing user experience with LLMOps-driven personalized recommendation systems

Maximizing user experience with LLMOps-driven personalized recommendation systems

Secondary organic aerosols derived from intermediate-volatility n-alkanes adopt low-viscous phase state

Abstract. Secondary organic aerosol (SOA) derived from n-alkanes, as emitted from vehicles and volatile chemical products, is a major component of anthropogenic particulate matter, yet the chemical composition and phase state are poorly understood and thus poorly constrained in aerosol models. Here we provide a comprehensive analysis of n-alkane SOA by explicit gas-phase chemistry modeling, machine learning, and laboratory experiments to show that n-alkane SOA adopts low-viscous semi-solid or liquid states. Our study underlines the complex interplay of molecular composition and SOA viscosity: n-alkane SOA with a higher carbon number mostly consists of less functionalized first-generation products with lower viscosity, while the SOA with a lower carbon number contains more functionalized multigenerational products with higher viscosity. This study opens up a new avenue for analysis of SOA processes, and the results indicate few kinetic limitations of mass accommodation in SOA formation, supporting the application of equilibrium partitioning for simulating n-alkane SOA formation in large-scale atmospheric models.

On the Model Update Strategies for Supervised Learning in AIOps Solutions

AIOps (Artificial Intelligence for IT Operations) solutions leverage the massive data produced during the operation of large-scale systems and machine learning models to assist software engineers in their system operations. As operation data produced in the field are constantly evolving due to factors such as the changing operational environment and user base, the models in AIOps solutions need to be constantly maintained after deployment. While prior works focus on innovative modeling techniques to improve the performance of AIOps models before releasing them into the field, when and how to update AIOps models remain an under-investigated topic. In this work, we performed a case study on three large-scale public operation data: two trace datasets from the cloud computing platforms of Google and Alibaba and one disk stats dataset from the BackBlaze cloud storage data center. We empirically assessed five different types of model update strategies for supervised learning regarding their performance, updating cost, and stability. We observed that active model update strategies (e.g., periodical retraining, concept drift guided retraining, time-based model ensembles, and online learning) achieve better and more stable performance than a stationary model. Particularly, applying sophisticated model update strategies (e.g., concept drift detection, time-based ensembles, and online learning) could provide better performance, efficiency, and stability than simply retraining AIOps models periodically. In addition, we observed that, although some update strategies (e.g., time-based ensemble and online learning) can save model training time, they significantly sacrifice model testing time, which could hinder their applications in AIOps solutions where the operation data arrive at high pace and volume and where immediate inferences are required. Our findings highlight that practitioners should consider the evolution of operation data and actively maintain AIOps models over time. Our observations can also guide researchers and practitioners in investigating more efficient and effective model update strategies that fit in the context of AIOps.

SLM-SLAM: A Visual SLAM System Based on Segmented Large-Scale Model in Dynamic Scenes and Zero-shot Conditions

Abstract In the real world, the presence of various dynamic objects can impact the localization accuracy of the majority of typical Visual Simultaneous Localization and Mapping (VSLAM) systems. Simultaneously, many dynamic VSLAM systems based on neural networks require pre-training for specific application scenarios. We present SLM-SLAM, the first VSLAM system that implements zero-shot processing of dynamic scenes. It achieves the capability to handle various dynamic objects without the necessity for pre-training, enabling straightforward adaptation to different application scenarios. Firstly, we designed an open-world semantic segmentation module based on a segmented large-scale model to acquire semantic information in the scene. Subsequently, we devised a label-based strategy for selecting feature points, jointly optimizing poses with the weighted labels provided by both semantic and geometric information. Finally, we refined the keyframe selection strategy of ORB-SLAM3 to prevent matching errors caused by an insufficient number of remaining static feature points in the scene. We conducted experiments on the TUM dataset, the KITTI dataset, and real world scenarios. The results indicate that in dynamic scenes, our SLM-SLAM significantly improves localization accuracy compared to ORB-SLAM3, and its performance is comparable to state-of-the-art dynamic VSLAM systems.

Short-term extreme value prediction for the structural responses of the IEA 15 MW offshore wind turbine under extreme environmental conditions

As offshore wind turbines enter deep seas, large-scale models ranging from 15 MW to 20 MW are becoming the mainstream choice for the future market. The dynamic responses of 15 MW wind turbines under extreme environmental conditions should be considered in the design process. With increasing structure sizes, the mooring systems and tower base structures play essential roles in maintaining position and supporting wind turbines under harsh environmental conditions. Accurate estimation of the extreme mooring loads and dynamic responses at the tower base is crucial, particularly for the ultimate limit state (ULS) analysis. The dynamic responses of the offshore wind turbine are calculated by Open Fast and validated with other numerical tools. For the safety of deep-sea structures, it is necessary to consider the extreme environmental conditions with a 100-year return period, which is beyond the current standards. An environmental contour-based approach is proposed to select the short-term extreme conditions, enabling approximate prediction of the extreme values of the dynamic responses during the preliminary design stage. The Gumbel method and the ACER method are used to predict the short-term extreme dynamic responses. Comparative evaluations against the relevant standards for floating wind turbines are conducted to assess structural safety.

Remote Sensing and Modeling of the Cryosphere in High Mountain Asia: A Multidisciplinary Review

Over the past decades, the cryosphere has changed significantly in High Mountain Asia (HMA), leading to multiple natural hazards such as rock–ice avalanches, glacier collapse, debris flows, landslides, and glacial lake outburst floods (GLOFs). Monitoring cryosphere change and evaluating its hydrological effects are essential for studying climate change, the hydrological cycle, water resource management, and natural disaster mitigation and prevention. However, knowledge gaps, data uncertainties, and other substantial challenges limit comprehensive research in climate–cryosphere–hydrology–hazard systems. To address this, we provide an up-to-date, comprehensive, multidisciplinary review of remote sensing techniques in cryosphere studies, demonstrating primary methodologies for delineating glaciers and measuring geodetic glacier mass balance change, glacier thickness, glacier motion or ice velocity, snow extent and water equivalent, frozen ground or frozen soil, lake ice, and glacier-related hazards. The principal results and data achievements are summarized, including URL links for available products and related data platforms. We then describe the main challenges for cryosphere monitoring using satellite-based datasets. Among these challenges, the most significant limitations in accurate data inversion from remotely sensed data are attributed to the high uncertainties and inconsistent estimations due to rough terrain, the various techniques employed, data variability across the same regions (e.g., glacier mass balance change, snow depth retrieval, and the active layer thickness of frozen ground), and poor-quality optical images due to cloudy weather. The paucity of ground observations and validations with few long-term, continuous datasets also limits the utilization of satellite-based cryosphere studies and large-scale hydrological models. Lastly, we address potential breakthroughs in future studies, i.e., (1) outlining debris-covered glacier margins explicitly involving glacier areas in rough mountain shadows, (2) developing highly accurate snow depth retrieval methods by establishing a microwave emission model of snowpack in mountainous regions, (3) advancing techniques for subsurface complex freeze–thaw process observations from space, (4) filling knowledge gaps on scattering mechanisms varying with surface features (e.g., lake ice thickness and varying snow features on lake ice), and (5) improving and cross-verifying the data retrieval accuracy by combining different remote sensing techniques and physical models using machine learning methods and assimilation of multiple high-temporal-resolution datasets from multiple platforms. This comprehensive, multidisciplinary review highlights cryospheric studies incorporating spaceborne observations and hydrological models from diversified techniques/methodologies (e.g., multi-spectral optical data with thermal bands, SAR, InSAR, passive microwave, and altimetry), providing a valuable reference for what scientists have achieved in cryosphere change research and its hydrological effects on the Third Pole.

Fine-tuning vision foundation model for crack segmentation in civil infrastructures

Large-scale foundation models have become the mainstream deep learning method, while in civil engineering, the scale of AI models is strictly limited. In this work, a vision foundation model is introduced for crack segmentation. Two parameter-efficient fine-tuning methods, adapter and low-rank adaptation, are adopted to fine-tune the foundation model in semantic segmentation: the Segment Anything Model (SAM). The fine-tuned CrackSAM shows excellent performance on different scenes and materials. To test the zero-shot performance of the proposed method, two unique datasets related to road and exterior wall cracks are collected, annotated and open-sourced, for a total of 810 images. Comparative experiments are conducted with twelve mature semantic segmentation models. On datasets with artificial noise and previously unseen datasets, the performance of CrackSAM far exceeds that of all state-of-the-art models. CrackSAM exhibits remarkable superiority, particularly under challenging conditions such as dim lighting, shadows, road markings, construction joints, and other interference factors. These cross-scenario results demonstrate the outstanding zero-shot capability of foundation models and provide new ideas for developing vision models in civil engineering.

Simultaneous stochastic optimisation of mining complexes: Integrating progressive reclamation and waste management with contextual bandits

ABSTRACT Managing environmental performance of waste dump facilities in mining complexes is an integral part of long-term production planning. Sustainable long-term production scheduling solutions are desired to mitigate risk and return the environment to a productive post-mining state. A simultaneous stochastic optimisation framework for long-term production scheduling in mining complexes is developed that integrates waste management and progressive reclamation. The waste dump placement schedule is jointly optimised with the extraction sequence, destination policy, and stockpiling decisions in a single stochastic mathematical programming framework. This includes the timing of progressive reclamation activities in parallel with production to enhance waste dump rehabilitation. Uncertainty related to the production of acid rock drainage is quantified by simulating geochemical properties of waste and managing the blending of uncertain waste properties within the optimisation framework. With respect to the framework for simultaneous stochastic optimisation, contextual bandits are explored to improve the metaheuristic solution approach and solve the corresponding large-scale optimisation model. The framework is tested in a multi-mine copper-gold mining complex leading to improved environmental performance. Risk of acid rock drainage is decreased by 52.5% in the waste dump facilities. Reclamation planning activities for meeting environmental requirements are scheduled prior to closure. The solution approach more effectively improves the objective function with contextual bandits leading to a 24% improvement in the study presented.

Iterative method for large-scale Timoshenko beam models assessed on commercial-grade paperboard

AbstractLarge-scale structural simulations based on micro-mechanical models of paper products require extensive numerical resources and time. In such models, the fibrous material is often represented by connected beams. Whereas previous micro-mechanical simulations have been restricted to smaller sample problems, large-scale micro-mechanical models are considered here. These large-scale simulations are possible on a non-specialized desktop computer with 128GB of RAM using an iterative method developed for network models and based on domain decomposition. Moreover, this method is parallelizable and is also well-suited for computational clusters. In this work, the proposed memory-efficient iterative method is numerically validated for linear systems resulting from large networks of Timoshenko beams. Tensile stiffness and out-of-plane bending stiffness are simulated and validated for various commercial-grade three-ply paperboards consisting of layers composed of two different types of paper fibers. The results of these simulations show that a linear network model produces results consistent with theory and published experimental data

Attention towards chemistry agnostic and explainable battery lifetime prediction

Predicting and monitoring battery life early and across chemistries is a significant challenge due to the plethora of degradation paths, form factors, and electrochemical testing protocols. Existing models typically translate poorly across different electrode, electrolyte, and additive materials, mostly require a fixed number of cycles, and are limited to a single discharge protocol. Here, an attention-based recurrent algorithm for neural analysis (ARCANA) architecture is developed and trained on an ultra-large, proprietary dataset from BASF and a large Li-ion dataset gathered from literature across the globe. ARCANA generalizes well across this diverse set of chemistries, electrolyte formulations, battery designs, and cycling protocols and thus allows for an extraction of data-driven knowledge of the degradation mechanisms. The model’s adaptability is further demonstrated through fine-tuning on Na-ion batteries. ARCANA advances the frontier of large-scale time series models in analytical chemistry beyond textual data and holds the potential to significantly accelerate discovery-oriented battery research endeavors.

Experimental study on the mechanical properties of FRP reinforced shotcrete lining of tunnel in extremely fractured surrounding rock

Tunnel excavation in the southwestern area of China encounters many difficulties caused by the large deformation of extremely fractured surrounding rock, which is the product of high plate tectonic stress. Improving the initial cracking strength and ductility of shotcrete layer is one of the most important means to reduce the frequency of support structure repair, which will improve tunnel construction efficiency and ensure personnel safety. In this case, this paper introduced fiber reinforced polymer (FRP) mesh to replace the steel mesh as reinforcement to optimize the support effect of shotcrete lining. Both large-scale model test and mechanical test are conducted to compare the reinforcement effect between FRP and steel mesh. Research results indicate that replacing steel mesh with FRP can increase the ultimate deformation capacity of tunnel lining by more than four times. The parametric analysis results indicate that increasing the lining thickness, the diameter of the FRP bars and the mesh density have different degrees of effect on improving the stiffness and strength of tunnel lining. The research results can provide reference for relevant projects.

Estimation of rock joint roughness using binocular stereo vision

To quickly and cost-effectively evaluate the joint roughness coefficient (JRC) of the rock joints, a technique based on binocular stereo vision was developed to reconstruct the three-dimensional (3D) model. After camera calibration, only two images of rock joints were needed to be captured for 3D reconstruction. Then, the generated point clouds were processed to extract the geometric information, mainly focusing on the JRC calculation. A comparison was made with the point clouds obtained from laser scanning to verify the proposed method, and the possibility of the method in large-scale modeling was also explored. The results show that the proposed technique for 3D reconstruction of rock joints can meet the basic accuracy requirements in laboratory work, and it has a good consistency in roughness estimation with laser scanning. Compared to traditional photogrammetry, this method significantly simplifies the working procedure to improve measurement efficiency and reduce the project cost.

Large-Scale Laboratory Experiments On The Wave Generation Due To The Collapse Of Partially And Fully Submerged Granular Columns

Landslides that occur in coastal environments can drive cascading consequences such as wave forces, flooding, and infrastructure damage to coastal communities. It can be difficult to classify these slides as subaerial or submarine, and the mechanics of wave generation associated with partially submerged failures are not well understood. Limited physical modelling has been conducted that encompasses both the triggering of granular landslides and subsequent waves associated with partially and fully submerged mass movements. To date, laboratory work investigating tsunamis generated by submarine landslides has focused on the wave formed in the direction of the mass movement (seaward direction) for rigid block experiments (eg. Rzadkiewicz, 1997) and deformable slide masses (e.g. Grilli et al., 2017, Takabatake, 2020, Bullard et al., 2023). From these experimental data sets, predictive relationships connecting slide acceleration, mass, and initial submergence depth to the amplitude of the wave formed have been presented for the seaward wave. Such relationships have not been presented for the landward directed wave, which propagates in the opposite direction of the submarine landslide motion. Further, not all landslides are easily classified as either subaerial or submarine. Consider the 2018 Anak Krakatoa landslide in which the sliding surface was estimated to be 100 m below sea level (Pakoksung et al., 2020), resulting in one third of the total collapse being submerged. In comparison to the end-member conditions of subaerial and submarine failures, the mechanics of wave generation associated with partially submerged failures is much less clear. Granular column collapse experiments provide an idealized experimental framework to explore momentum transfer processes and the resulting waves generated in partially submerged and fully submerged conditions. Work by Cabrera et al., (2020) made use of granular collapse experiments of partially to fully submerged columns to derive a continuous momentum-based function to estimate the maximum seaward wave amplitude based on the initial column submergence ratio (Hw/Ho). However, these experiments were conducted at a small-scale (Ho = 0.15 m) with a width of one particle (2.4 mm diameter). To address this research gap, a series of 22 large-scale granular collapse experiments were conducted by releasing columns of river stone (0.75 m and 0.50 m high) into a laboratory flume reservoir with water depths ranging up to 1.10 m to explore the wave generation and runup processes in both seaward and landward directions. The columns were released by a rapid pneumatically actuated vertical rising gate designed to enable the near instantaneous loss of support of the source volumes resulting in granular collapse. The failure mechanics were captured with high-speed cameras (Figure 1a,b) and wave amplitudes were measured using wave capacitance gauges (Figure 1c,d). This work also provides the first experimental data set of the landward propagating wave and runup associated with submerged granular collapse experiments. Overall, the seaward wave amplitudes measured in these highly-instrumented, large-scale physical models agree with empirical relationships developed in a previous study using smaller-scale models.

Limited effect of the confluence angle and tributary gradient on Alpine confluence morphodynamics under intense sediment loads

Abstract. Confluences are dynamic morphological nodes that are found in all river networks. In mountain regions, they are influenced by hydraulic and sedimentary processes that occur in steep channels during extreme events in small watersheds. Sediment transport in the tributary channel and aggradation in the confluence can be massive, potentially causing overbank flooding and sedimentation into adjacent settlement areas. Previous works dealing with confluences have mainly focused on lowland regions, and those that have focused on mountain areas have used sediment concentrations and channel gradients that are largely under-representative of mountain river conditions. The presented work contributes to filling this research gap with 45 experiments that use a large-scale physical model. Geometric model parameters, the applied grain size distribution, and the considered discharges represent the conditions at 135 confluences in South Tyrol (Italy) and Tyrol (Austria). The experimental program allowed for a comprehensive analysis of the effects of (i) the confluence angle, (ii) the tributary gradient, (iii) the channel discharges, and (iv) the tributary sediment concentration. In contrast to most research dealing with confluences, results indicate that, in the presence of an intense tributary sediment supply and a small tributary-to-main-channel discharge ratio (0.1), the confluence angle does not have a decisive effect on confluence morphology. Adjustments to the tributary channel gradient yielded the same results. A reoccurring range of depositional geomorphic units was observed in which a deposition cone transitioned to a bank-attached bar. The confluence morphology and tributary channel gradient rapidly adjusted, tending towards an equilibrium state to accommodate both water discharges and the sediment load from the tributary. Statistical analyses demonstrated that the confluence morphology was controlled by the combined channel discharge and the depositional or erosional extent was controlled by the sediment concentration. Applying conclusions drawn from lowland confluence dynamics could misrepresent depositional and erosional patterns and the related flood hazard at mountain river confluences.

Investigating Load Arches and the Uplift Capacity of Rock Anchors: A Numerical Approach

AbstractRock anchors, which can withstand substantial forces and are essential for securing the foundations of the built environment, can fail due to steel tensile failure, bond detachment, or rock mass fracturing through uplift. In this study, uplift failure of the rock mass surrounding rock anchors was simulated using both physical and numerical models. The physical models used a specially designed testing rig with predefined sizes and different joint layouts to simulate rock mass uplift failure. The numerical models were calibrated using the physical models and used to investigate scenarios involving increased complexity and greater depths. This paper presents the load distribution within rock masses with orthogonal joints when exposed to an anchor load. Extensive analysis of the numerical models revealed that symmetrical load arches were induced in rock masses that contained a joint set oriented parallel to the loading direction. However, joints that were obliquely oriented relative to the anchor axis induced asymmetrical load arching. The greatest displacement occurred in the direction parallel to the joint set with the smallest angle to the anchor loading axis. Large-scale models with zero or near-zero in situ horizontal stress revealed that the load capacity in a rock mass with continuous vertical joints was relatively low, especially when compared to rock masses with discontinuous vertical joints. It was observed that the uplift capacity of the rock mass around an anchor decreased if the load arches of adjacent anchors overlapped; consequently, a decrease in anchor spacing was found to correspond to a decrease in uplift capacity.

Reconstruction of long-term hydrologic change and typhoon-induced flood events over the entire island of Taiwan

Study regionThe island of Taiwan. Study focusThis study presents long-term and high-resolution modeling of flood occurrence, interdecadal patterns of river-floodplain dynamics, and analysis of flooding during two typhoon events—Nari and Morakot over Taiwan. The modeling system combines a hydrological model (HiGW-MAT) and a river hydrodynamics model (CaMa-Flood), simulating hydrologic-hydrodynamic processes at ∼ 5 km resolution with flood occurrences downscaled to ∼ 90 m. New hydrological insights for the regionAs the first investigation to conduct spatially comprehensive and temporally continuous modeling in Taiwan, this study presents important advances on the application of large-scale hydrological-hydrodynamic models in settings like that of Taiwan with important implications on flood prediction under climate change. The assessment of interdecadal changes in streamflow indicates no consistent trends over the past four decades; however, the variabilities in monthly-scale streamflow are significant and regionally diverse across Taiwan. Decadal changes in flood occurrence are also minimal at the island-scale, but the changes vary substantially across different regions and exhibit an increased variability over time. Furthermore, the simulated flood patterns in response to Typhoons Nari and Morakot suggest that the modeling framework can be used to reproduce the spatial dynamics and temporal progression of flooding under extreme events in relatively small regions like Taiwan.

Understanding pathways from implementation to sustainment: a longitudinal, mixed methods analysis of promising practices implemented in the Veterans Health Administration

BackgroundThe Veterans Health Administration (VHA) is the United States largest learning health system. The Diffusion of Excellence (DoE) program is a large-scale model of diffusion that identifies and diffuses evidence-informed practices across VHA. During the period of 2016-2021, 57 evidence-informed practices were implemented across 82 VHA facilities. This setting provides a unique opportunity to understand sustainment determinants and pathways. Our objective was to characterize the longitudinal pathways of practices as they transition from initial implementation to long-term sustainment at each facility.MethodsA longitudinal, mixed-methods evaluation of 82 VHA facilities. Eighty-two facility representatives, chosen by leadership as points-of-contact for 57 DoE practices, were eligible for post-implementation interviews and annual sustainment surveys. Primary outcomes (implementation, sustainment), and secondary outcomes (institutionalization, effectiveness, anticipated sustainment) at four time-points were collected. We performed descriptive statistics and directed content analysis using Hailemariam et al.’s factors influencing sustainment.ResultsAfter approximately five years post-implementation (e.g., 2021 sustainment outcomes), of the 82 facilities, about one-third fully sustained their practice compared to one-third that did not fully sustain their practice because it was in a “liminal” stage (neither sustained nor discontinued) or permanently discontinued. The remaining one-third of facilities had missing 2021 sustainment outcomes. A higher percentage of facilities (70%) had inconsistent primary outcomes (changing over time) compared to facilities (30%) with consistent primary outcomes (same over time). Thirty-four percent of facilities with sustained practices reported resilience since they overcame implementation and sustainment barriers. Facilities with sustained practices reported more positive secondary outcomes compared to those that did not sustain their practice. Key factors facilitating practice sustainment included: demonstrating practice effectiveness/benefit, sufficient organizational leadership, sufficient workforce, and adaptation/alignment with local context. Key factors hindering practice sustainment included: insufficient workforce, not able to maintain practice fidelity/integrity, critical incidents related to the COVID-19 pandemic, organizational leadership did not support sustainment of practice, and no ongoing support.ConclusionsWe identified diverse pathways from implementation to sustainment, and our data underscore that initial implementation outcomes may not determine long-term sustainment outcomes. This longitudinal evaluation contributes to understanding impacts of the DoE program, including return on investment, achieving learning health system goals, and insights into achieving high-quality healthcare in VHA.