Large-scale Implementation Research Articles

California's Recovery Incentives Program: Implementation strategies

IntroductionDue to the increasing role of psychostimulants in the US drug poisoning crisis, there is an increasing need to effectively implement evidence-based treatment for individuals with stimulant use disorder (StimUD). Contingency management is a behavioral strategy with robust evidence of support for the treatment of StimUD. In 2023, California initiated a large-scale effort to implement CM as a treatment for individuals with a stimulant use disorder (cocaine, methamphetamine, amphetamine) called the Recovery Incentives Program: California's Contingency Management Benefit. MethodsThe Recovery Incentives Program is being systematically implemented using the Becker et al. Science to Service Lab (SSL) implementation approach with several augmentations for this project. The SSL features three core components: didactic training, performance feedback, and external facilitation. We have augmented this approach with a readiness assessment process for sites prior to CM service launch, and an ongoing fidelity monitoring and feedback component post-launch. ResultsThe present paper is a preliminary report describing the use of this augmented SSL strategy for CM implementation in a large-scale implementation effort. Data are presented to describe the implementation activities during the first ten months of the Recovery Incentives Program. ConclusionThe California Recovery Incentives Program has been systematically implemented and appears to be receiving a positive response from treatment program staff and enrolled members. Future papers and evaluation reports will continue to document member response to the Program and report on the ongoing training and implementation process.

Symptom vs context: lessons learned from a large-scale implementation of the Cultural Formulation Interview.

Mental health services in multicultural societies require culturally sensitive approaches to reduce health disparities. The Cultural Formulation Interview (CFI) is thought to enhance shared decision making and to facilitate culturally and contextually informed treatment. There is, however, little known regarding its implementability in large-scale psychiatric services. The current paper reports on (a) efforts to implement the CFI in a large organization for mental health services in the Netherlands, and (b) two studies that evaluated this implementation process and identified barriers to CFI adoption in clinical practice. Implementation of the CFI was facilitated by developing an online course, an advanced training of "team coaches", (digital) resources, and integrating the CFI questions into the intake format. A preliminary evaluation revealed that the CFI was administered in only 13.2% of all intakes across the entire organization, with minimal utilization of training resources. Study 1 aimed to investigate how the CFI was perceived by clinicians and stakeholders. A survey of 150 clinicians found a great lack of familiarity with the CFI and its purpose. While 67% reported partial CFI use, 50% saw no added value, and 61% deemed it relevant only for ethnic minorities. Study 2 examined which patient and clinician variables were associated with adequate CFI use (i.e., correct documentation of the CFI information in initial intake reports). The sample consisted of 112 intakes of patients conducted by ten clinicians. Regression analysis showed a significant association between clinicians' cultural competences and adequate CFI use, meaning that more culturally competent clinicians tended to generate better cultural assessments using the CFI. In addition, the CFI information was documented more adequately among patients who were unemployed at the time of assessment. In conclusion, implementation of the CFI requires a fundamental rethinking of the entire intake assessment, shifting it from a symptom-oriented approach towards a context- and person-centered one. Future trainings may benefit from embedding the CFI within a broader cultural competency training, rather than solely focusing on the CFI, which is currently the common practice.

Modeling and optimization for arrays of water turbine OWC devices

The large-scale implementation of wave energy conversion requires the installation of parks of devices. We study the problem of optimizing park layout and control for wave energy converters of the oscillating water column type. As a test case, we consider a device with a Wells turbine working in water. First, a novel model based on a nonlinear ordinary differential equation is derived to describe the behavior of the water column and used to estimate the power matrix. Then, its linearization is derived in order to enable the fast simulation of large parks of devices. The choice of the hydrodynamic model allows obtaining the gradient of the power with respect to the positions through an adjoint approach, making it especially convenient for optimization. We consider in particular the case of interaction with the piles of a floating wind energy plant. The results from the developed computational framework allow us to draw interesting conclusions about park layout design. In particular, we observe that mutual interaction effects can be significant even in parks made up of devices of small size, and that wave reflection from the piles of an offshore structure can improve energy production.

Availability of Evidence for Predictive Machine Learning Algorithms in Primary Care

The aging and multimorbid population and health personnel shortages pose a substantial burden on primary health care. While predictive machine learning (ML) algorithms have the potential to address these challenges, concerns include transparency and insufficient reporting of model validation and effectiveness of the implementation in the clinical workflow. To systematically identify predictive ML algorithms implemented in primary care from peer-reviewed literature and US Food and Drug Administration (FDA) and Conformité Européene (CE) registration databases and to ascertain the public availability of evidence, including peer-reviewed literature, gray literature, and technical reports across the artificial intelligence (AI) life cycle. PubMed, Embase, Web of Science, Cochrane Library, Emcare, Academic Search Premier, IEEE Xplore, ACM Digital Library, MathSciNet, AAAI.org (Association for the Advancement of Artificial Intelligence), arXiv, Epistemonikos, PsycINFO, and Google Scholar were searched for studies published between January 2000 and July 2023, with search terms that were related to AI, primary care, and implementation. The search extended to CE-marked or FDA-approved predictive ML algorithms obtained from relevant registration databases. Three reviewers gathered subsequent evidence involving strategies such as product searches, exploration of references, manufacturer website visits, and direct inquiries to authors and product owners. The extent to which the evidence for each predictive ML algorithm aligned with the Dutch AI predictive algorithm (AIPA) guideline requirements was assessed per AI life cycle phase, producing evidence availability scores. The systematic search identified 43 predictive ML algorithms, of which 25 were commercially available and CE-marked or FDA-approved. The predictive ML algorithms spanned multiple clinical domains, but most (27 [63%]) focused on cardiovascular diseases and diabetes. Most (35 [81%]) were published within the past 5 years. The availability of evidence varied across different phases of the predictive ML algorithm life cycle, with evidence being reported the least for phase 1 (preparation) and phase 5 (impact assessment) (19% and 30%, respectively). Twelve (28%) predictive ML algorithms achieved approximately half of their maximum individual evidence availability score. Overall, predictive ML algorithms from peer-reviewed literature showed higher evidence availability compared with those from FDA-approved or CE-marked databases (45% vs 29%). The findings indicate an urgent need to improve the availability of evidence regarding the predictive ML algorithms' quality criteria. Adopting the Dutch AIPA guideline could facilitate transparent and consistent reporting of the quality criteria that could foster trust among end users and facilitating large-scale implementation.

Environmental savings from concrete reuse: examining the limitations and optimal practices for cutting thresholds of concrete building components for reuse

Abstract Concrete, a widely used construction material, presents both opportunities and challenges for promoting environmentally responsible practices. This research focuses on the concept of concrete reuse as a sustainable strategy, examining the limitations of cutting dimensions. To investigate the greenhouse gas (GHG) emissions associated with preparing concrete for reuse, a mathematical model was developed considering several factors, such as transportation emissions, energy used in cutting, and the effects of varying cutting performance and energy sources. By incorporating real-life data and connecting with industrial practices, this study establishes a screening standard to determine the minimum cutting size for concrete reuse. The results indicate that the ratio of transportation distances for the reuse case versus the new production of concrete case has an influence on the minimum cutting size for concrete reuse. Moreover, the cutting size itself significantly impacts the overall GHG emissions of the reuse strategy. We offer practical insights for industry stakeholders to optimize concrete reuse practices and effectively reduce GHG emissions. As further research explores cost-effectiveness and on-site case studies, the potential for large-scale implementation of concrete reuse has become increasingly promising. Thus, concrete reuse has emerged as a viable and effective approach to sustainable construction, provided that the minimum cutting dimensions are carefully considered.

Simulation Evaluation of a Large-Scale Implementation of Virtual-Phase Link–Based Model Predictive Control

Simulation Evaluation of a Large-Scale Implementation of Virtual-Phase Link–Based Model Predictive Control

Mitigating risk of credit reversal in nature-based climate solutions by optimally anticipating carbon release

Nature-based climate solutions supply carbon credits generated from net carbon drawdown in exchange for project funding, but their credibility is challenged by the inherent variability and impermanence of drawdown. By evaluating drawdown benefits from a social cost of carbon perspective, project developers can enhance credibility and estimate impermanence by conservatively anticipating drawdowns to be eventually released following a release schedule, issuing additional credits when actual release is less severe than anticipated. We demonstrate how we can use ex post observations of drawdowns to construct optimal release schedules that limit the risk of credit reversals (when net drawdown is negative). We simulate both theoretical and real-life projects to examine how this approach balances the trade-off between generating credits evaluated as more permanent and limiting reversal risk. We discuss how this approach incentivizes project performance and provides a pragmatic solution to challenges facing larger-scale implementation of nature-based climate solutions.

Enhancing Preoperative Outcome Prediction: A Comparative Retrospective Case-Control Study on Machine Learning versus the International Esodata Study Group Risk Model for Predicting 90-Day Mortality in Oncologic Esophagectomy.

Risk prediction prior to oncologic esophagectomy is crucial for assisting surgeons and patients in their joint informed decision making. Recently, a new risk prediction model for 90-day mortality after esophagectomy using the International Esodata Study Group (IESG) database was proposed, allowing for the preoperative assignment of patients into different risk categories. However, given the non-linear dependencies between patient- and tumor-related risk factors contributing to cumulative surgical risk, machine learning (ML) may evolve as a novel and more integrated approach for mortality prediction. We evaluated the IESG risk model and compared its performance to ML models. Multiple classifiers were trained and validated on 552 patients from two independent centers undergoing oncologic esophagectomies. The discrimination performance of each model was assessed utilizing the area under the receiver operating characteristics curve (AUROC), the area under the precision-recall curve (AUPRC), and the Matthews correlation coefficient (MCC). The 90-day mortality rate was 5.8%. We found that IESG categorization allowed for adequate group-based risk prediction. However, ML models provided better discrimination performance, reaching superior AUROCs (0.64 [0.63-0.65] vs. 0.44 [0.32-0.56]), AUPRCs (0.25 [0.24-0.27] vs. 0.11 [0.05-0.21]), and MCCs (0.27 ([0.25-0.28] vs. 0.15 [0.03-0.27]). Conclusively, ML shows promising potential to identify patients at risk prior to surgery, surpassing conventional statistics. Still, larger datasets are needed to achieve higher discrimination performances for large-scale clinical implementation in the future.

Microbial-Derived Carotenoids and Their Health Benefits

Natural carotenoids (CARs) such as β-carotene, astaxanthin, lutein, norbixin, bixin, capsanthin, lycopene, β-Apo-8-carotenal, canthaxanthin, β-apo-8-carotenal-ester, and zeaxanthin are being explored for possible applications in feed, food, cosmeceuticals, and nutraceuticals. Three primary areas of carotenoid research are emerging: (1) encapsulations for improved chemical and physical properties; (2) natural source carotenoid manufacturing; and (3) preclinical, epidemiological, and clinical studies of carotenoids’ potential health benefits. The recent advancements in research on the chemistry and antioxidant activity, marketing strategies, dietary sources, bioavailability, and bioaccessibility, extraction, dietary consumption, encapsulating techniques, and health advantages of carotenoids are all extensively discussed in this review. Carotenoids are pigments found naturally in most fruits and vegetables, algae, plants, and photosynthetic bacteria. Carotenoids cannot be synthesized by humans and must be consumed in the form of food or supplements. There are several roles for carotenoids in human health. Although individual carotenoids may function in different ways, their main action is to act as antioxidants. There are validated techniques for separating and purifying carotenoids, yet, industrial production requires the development of economically viable techniques for larger-scale implementation. Carotenoids have been shown to boost cognitive performance and cardiovascular health, as well as help prevent some types of cancer. Despite evidence for carotenoids’ health benefits, major population-based supplementation trials have yielded conflicting outcomes for several carotenoids. This review includes recent developments in carotenoid metabolism and nutritional and health advantages. It also offers an outlook on future directions in these areas.

Enhancing maternal health service utilization among highly vulnerable pregnant women through a human-centered design process: Study protocol for a quasi-experimental study in Oromia, Ethiopia

Background Disproportionate uptake of and access to maternal and child health services remain significant challenges across and within countries. Differing geographic, economic, environmental, and social factors contribute to varying degrees of vulnerabilities among individuals, which manifest as disparities in maternal and newborn health outcomes. Designing solutions according to need is vital to improve maternal and child health outcomes. In this paper, we describe our study protocol on evaluating the effectiveness of an intervention package developed through a human-centered design (HCD) process in improving maternal health service uptake among vulnerable pregnant women in rural areas of Ethiopia. Methods The study has two distinct phases. For both phases, solutions will be developed leveraging HCD approaches to improve institutional delivery and antenatal care uptake among vulnerable pregnant women. Final solutions will be implemented in collaboration with community health workers and health officers. A community-based, quasi-experimental, mixed-method study design is used to assess differences between intervention and control arms. A panel sample will be enrolled after screening for pregnancy and vulnerability level and surveyed at baseline and midline in Phase 1. In Phase 2, a post-only study will be conducted in the same intervention and control sites. Pregnant women will receive refined solutions for six months, which will be evaluated using end-line assessment. Qualitative interviews will be conducted among vulnerable pregnant women and healthcare providers. Conclusions and Implications Our sequential approach to evaluating initial solutions, which in turn will be used to inform the enhancement of solutions, will provide practical insights into how solutions are accepted among vulnerable women and how they can be better integrated into women’s lives and health systems. Our findings on program effectiveness and implementation research will be disseminated to the Ethiopian Ministry of Health and its partners to inform large-scale implementation at the national level.

Evaluating the efficacy of machine learning in calibrating low-cost sensors

Ambient air quality monitoring requires low-cost environmental sensor devices that are affordable and feasible for large-scale implementation. However, issues such as sensor drift, environmental sensitivity, and inter-sensor variability affect data accuracy and cannot be adequately addressed by traditional calibration methods. This paper summarizes the use of machine learning techniques for calibrating low-cost sensors. The literature review shows that machine learning models like Random Forest, Support Vector Regression, and Neural Networks significantly improve sensor accuracy and reliability. For instance, Random Forest models reduced the root mean squared error by 30% for PM2.5 measurements, while Neural Networks achieved an R value of 0.997 for methane sensors. Integrating machine learning with IoT and mobile technologies enhances real-time monitoring and spatial resolution. Identified gaps include the quality of training datasets, managing environmental variability, and improving model transferability across different contexts. Addressing these gaps through advanced models and real-time calibration methodologies will further enhance sensor performance, ensuring more precise and reliable environmental data.

The Role of Health Informatics Research: A Case of a Large-Scale Implementation in Norway.

Many see the role of health informatics research as informing the development and implementation of information technology in clinical practice. The aim of this study is to see if this role is realized in the ongoing implementation of a large-scale health information system in central Norway. By doing a document analysis of the planning documents for the implementation, we assess to what extend evidence from the scientific community is explicitly referenced and used in the implementation planning. We found that evidence available is not explicitly used, and that evidence required is not widely available.

Biochar Amendment in Green Roof Substrate: A Comprehensive Review of the Benefits, Performance, and Challenges

Green roofs (GRs) are a well-established green infrastructure (GI) strategy that have been extensively studied for decades to address a growing array of social and environmental challenges. Research efforts have been continuously made to contribute to the awareness of benefits of GRs and towards their widespread application. The substrate, which is one of the crucial layers of a GR system, plays a major role in the serviceability of GRs. Thus, several studies have been undertaken to alter the substrate characteristics by applying innovative substrate additives. Biochar, a carbon-rich material with a highly porous structure and large specific surface area, has been found advantageous in several areas such as agriculture, water filtration, environmental remediation, construction, and so on. However, the application of biochar in GRs has been insufficiently studied, partially because biochar amendment in GRs is a relatively recent innovation. Furthermore, a comprehensive review of the performance of biochar-amended GR substrates is lacking. This review paper aims to summarize the past performance of GRs enhanced with biochar by considering the various benefits that biochar offers. The results indicate that most of the reviewed studies observed increased retention of runoff and nutrients when utilizing biochar. Additionally, the capabilities of biochar in improving thermal insulation, plant performance, and microbial diversity, as well as its effectiveness in sequestrating carbon and controlling soil erosion, were mostly agreed upon. Notwithstanding, a definitive conclusion cannot yet be confidently made due to the limited research information from biochar–GR systems and the uneven research focus observed in the studies reviewed. The influence of biochar-related variables (including amendment rates, application methods, processed forms, and particle size) on the effectiveness of biochar was also discussed. Opportunities for future research were suggested to fill the research gaps and address challenges restricting the application of biochar in GRs. Detailed information from past research findings could serve as a foundation for further investigations into the large-scale implementation of biochar in GRs.

Achieving nitrite shunt using in-situ free ammonia enriched by natural zeolite: Pilot-scale mainstream anammox with flexible nitritation strategy

The mainstream partial nitritation/anammox (PN/A) process represents a significant innovation in decarbonizing municipal wastewater treatment. However, its implementation is considerably hampered by the challenge of stable nitrite supply. In this study, a pilot-scale PN/A system receiving real sewage (20 m3) was operated at room temperature for nearly one year. Remarkable PN performance with relatively high nitrite accumulation ratio of 75.04 ± 10.05 % was obtained via in-situ free ammonia (FA) strategy. The ammonium concentration enriched in the zeolite increased significantly by 548.8 times compared to that in the aqueous phase by ion exchange. This substantial increase robustly inhibited nitrite-oxidizing bacteria (NOB), resulting in high relative abundance ratio of ammonia-oxidizing bacteria (AOB) to NOB of 37.93 ± 12.61 in the zeolite biofilm, compared to 10.22 ± 1.67 in suspended floc sludge. The significant differences in FA concentrations between zeolite biofilm and suspended floc sludge resulted in distinct spatial distribution disparities of AOB and NOB, which were central to achieving stable nitrite accumulation without complex multiple selective pressures. Consequently, compliant effluent with total nitrogen of 10.91 ± 4.23 mg N/L was achieved at 10.4-31.1 °C without external carbon source addition. The biocarriers in the anammox process played a key role in enhancing functional genes and electron flow, supporting anammox-dominated nitrogen removal. This study presents a flexible and adaptable strategy for mainstream nitrite shunting, highlighting its potential for large-scale implementation of mainstream anammox treatment.

Isolation and characterization of a Halomonas species for non-axenic growth-associated production of bio-polyesters from sustainable feedstocks.

The urgent need for renewable replacements for synthetic materials may be addressed through microbial biotechnology. To simplify the large-scale implementation of such bio-processes, robust cell factories that can utilize sustainable and widely available feedstocks are pivotal. To this end, non-axenic growth-associated production could reduce operational costs and enhance biomass productivity, thereby improving commercial competitiveness. Another major cost factor is downstream processing, especially in the case of intracellular products, such as bio-polyesters. Simplified cell-lysis strategies could also further improve economic viability.

Implementing primary care behavioral health in Swedish primary care – study protocol for a pragmatic stepped wedge cluster trial

BackgroundMental health problems represent a large and growing public health concern. Primary care handles most of the patients with mental health problems, but there are many barriers to detection and treatment in this setting, causing under-recognition and under-treatment of patients. The service delivery model Primary Care Behavioral Health (PCBH) shows promise to manage mental health problems in primary care, but more research is needed regarding its effects on multiple levels.MethodsThis project investigates the effectiveness and implementation of a large-scale implementation of PCBH in Region Östergötland, Sweden. The aim is to generate new knowledge concerning the impact of a real-world implementation and use of PCBH in routine primary care. A Pragmatic Stepped-Wedge Cluster Trial will be used: 24 PCBH primary care centres in one region will be compared with 48 standard care centres in three other regions. The model will be implemented sequentially at the PCBH centres according to a staggered timetable. Results will be investigated at patient, staff and organization levels and various forms of data will be collected: (1) local and national registry data; (2) questionnaire data; (3) interview data; and (4) document data.DiscussionThis project investigates the effectiveness and implementation of PCBH in routine primary care. The project could result in improved mental health care for the included patients and contribute to the general good for a wider population who have mental health problems. The project’s study design will make it possible to assess many important effects of the PCBH service delivery model at different levels, providing evidence of the effectiveness (or not) of the PCBH model under routine conditions in primary care. The project has the potential to generate clinically meaningful results that can provide a basis for decisions concerning further implementation and use of the model and thus for future development of mental health care provision in primary care.Trial registrationNCT05633940, date of registration: 2021–04-21.

Shared Micro-mobility Market Disturbances: the Polish Bike Sharing Market in Light of Z-score Analysis

Urban transportation has changed dramatically in the recent years through the large scale implementation of shared micro-mobility services, especially bike sharing systems (BSS) and electric scooter sharing (ESS). The COVID-19 pandemic brought further changes and uncertainty to this turbulent business environment. In 2020, the major BSS operator in Poland filed for bankruptcy claiming that its problems arise from the COVID-19 pandemic. Market reports and media speculated that BSS business, despite being publicly financed, and considered to be a of part of public transportation system, could be unsustainable in face of the competition from ESS. We used Z‑score analysis to investigate if bike sharing systems operators’ problems began before or during the COVID-19 pandemic and large scale development of ESS. Our study focuses on the Nextbike company, which held a dominant stake in the Polish BSS market. It also covers two other major Polish operators and includes a German operator for comparative analysis. To complement the quantitative findings from z‑score analysis, we have also interviewed representatives of major stakeholders, which deepened our understanding of BSS problems. The results of our research indicate that although the market was affected by the COVID-19 pandemic and growing scooter-sharing competition, the problems of the largest BSS operator were specific to this company, and the entire market was not under the risk of failure.

Alternative payment models in the music streaming market: A comparative approach based on stream-level data

Music streaming platforms’ models for sharing revenues with content providers have been the subject of intense debate for nearly a decade. The dominating model involves pooling platform revenues and allocating these funds to songs based on a song's share of the total number of platform streams. Since this model has several controversial consequences, alternative models have been proposed. This paper uses a novel approach to assess the two most discussed models – the “user-centric” and the “artist-centric”. Our approach relies on a unique data set of 154,505 streaming platform users (890 million streams) and simulates how a large-scale implementation of these models may reallocate revenues across different songs and rightsholders. We disentangle the static effects of a transition to a “user-centric” or an “artist-centric” model across each of six different song characteristics. We then compare the results of the two models. We show that contrary to its objective, an artist-centric payment system does not significantly improve remuneration to professional artists while the user-centric payment system would generate more significant changes in revenue reallocation, mainly at the expense of Rap & Hip-hop songs, superstars and new releases. Finally, we analyze the positions of the various stakeholders with regard to each of them.

Behavioral risk profiling: Measuring loss aversion of individual investors

Loss aversion has been shown to be a key driver of people's investment decisions. Encouraged by regulators, financial institutions are seeking ways to integrate this behavioral factor into client risk classifications. A critical obstacle is the lack of a valid measurement method for loss aversion that can be straightforwardly incorporated into existing processes. This paper reports on two large-scale implementations of such a method within the risk-profiling application of an established financial institution. We elicit loss aversion for 1,040 employees and 3,740 clients, observing distributions that align with existing findings. Importantly, our results demonstrate that loss aversion is largely independent of the risk-return preferences commonly used for investor classification. Furthermore, the correlations we observe between these two preferences and individuals’ background characteristics align with previous research: loss aversion is strongly correlated with education—higher educated individuals exhibit greater loss aversion—whereas risk aversion is related to gender, age, and financial status—women, older individuals, and those less financially secure are more risk averse. These findings support the conjecture that risk and loss aversion are complementary in capturing investor intent.

Automatic detection of cardiac conditions from photos of electrocardiogram captured by smartphones

BackgroundResearchers have developed machine learning-based ECG diagnostic algorithms that match or even surpass cardiologist level of performance. However, most of them cannot be used in real-world, as older generation ECG...