Open Software Development Research Articles

Performance of an Open-Source Large Language Model in Extracting Information from Free-Text Radiology Reports.

Purpose To assess the performance of a local open-source large language model (LLM) in various information extraction tasks from real-life emergency brain MRI reports. Materials and Methods All consecutive emergency brain MRI reports written in 2022 from a French quaternary center were retrospectively reviewed. Two radiologists identified MRI scans that were performed in the emergency department for headaches. Four radiologists scored the reports' conclusions as either normal or abnormal. Abnormalities were labeled as either headache-causing or incidental. Vicuna (LMSYS Org), an open-source LLM, performed the same tasks. Vicuna's performance metrics were evaluated using the radiologists' consensus as the reference standard. Results Among the 2398 reports during the study period, radiologists identified 595 that included headaches in the indication (median age of patients, 35 years [IQR, 26-51 years]; 68% [403 of 595] women). A positive finding was reported in 227 of 595 (38%) cases, 136 of which could explain the headache. The LLM had a sensitivity of 98.0% (95% CI: 96.5, 99.0) and specificity of 99.3% (95% CI: 98.8, 99.7) for detecting the presence of headache in the clinical context, a sensitivity of 99.4% (95% CI: 98.3, 99.9) and specificity of 98.6% (95% CI: 92.2, 100.0) for the use of contrast medium injection, a sensitivity of 96.0% (95% CI: 92.5, 98.2) and specificity of 98.9% (95% CI: 97.2, 99.7) for study categorization as either normal or abnormal, and a sensitivity of 88.2% (95% CI: 81.6, 93.1) and specificity of 73% (95% CI: 62, 81) for causal inference between MRI findings and headache. Conclusion An open-source LLM was able to extract information from free-text radiology reports with excellent accuracy without requiring further training. Keywords: Large Language Model (LLM), Generative Pretrained Transformers (GPT), Open Source, Information Extraction, Report, Brain, MRI Supplemental material is available for this article. Published under a CC BY 4.0 license. See also the commentary by Akinci D'Antonoli and Bluethgen in this issue.

Revealing the supply-demand relationship of urban cultural ecosystem services: The combination of open-source spatial model and topic model

Cultural ecosystem services (CES) are of significant interest due to their effectiveness in assessing the impact of ecosystem degradation on human well-being. The evaluation of spatial coordination between the supply and demand of urban CES is crucial for guiding the sustainable development of urban areas. However, current research and survey methods are time-consuming and laborious, lacking in full integration of beneficiary opinions, and insufficient in examining multiple CES in an integrated manner. In this study, we have developed a novel framework that integrates an investigation-driven approach with a natural language processing topic model to assess the supply and demand of CES and their spatial correspondence. Using the main urban area of Xi'an as a case study, we analyzed the spatial coherence of supply and demand including aesthetic, cultural, educational, recreational, and spiritual services. Our findings demonstrate that the fusion of multiple models can effectively evaluate the CES supply from a natural perspective of nature and the demand for CES from a beneficiary perspective. High-supply areas for CES are concentrated in the center region of the study area and in proximity to roads and rivers. Aesthetic service supply was higher than that of other service types, followed by recreational and spiritual services supply. Demand is notably high in the city's hinterland, with substantial interest in recreational and aesthetic services. We observed mismatches between supply and demand across service categories ranging from 33.58% to 67.64%, with supply exceeding demand in most regions. Therefore, tailored measures should be implemented to address the supply and demand mismatch of CES based on local conditions. Our research introduces a new technical tool for analyzing the supply-demand matching relationship of CES and establishes a theoretical foundation for sustainable urban management.

Leveraging computer vision towards high-efficiency autonomous industrial facilities

AbstractManufacturers face two opposing challenges: the escalating demand for customized products and the pressure to reduce delivery lead times. To address these expectations, manufacturers must refine their processes, to achieve highly efficient and autonomous operations. Current manufacturing equipment deployed in several facilities, while reliable and produces quality products, often lacks the ability to utilize advancements from newer technologies. Since replacing legacy equipment may be financially infeasible for many manufacturers, implementing digital transformation practices and technologies can overcome the stated deficiencies and offer cost-affordable initiatives to improve operations, increase productivity, and reduce costs. This paper explores the implementation of computer vision, as a cutting-edge, cost-effective, open-source digital transformation technology in manufacturing facilities. As a rapidly advancing technology, computer vision has the potential to transform manufacturing operations in general, and quality control in particular. The study integrates a digital twin application at the endpoint of an assembly line, effectively performing the role of a quality officer by utilizing state-of-the-art computer vision algorithms to validate end-product assembly orientation. The proposed digital twin, featuring a novel object recognition approach, efficiently classifies objects, identifies and segments errors in assembly, and schedules the paths through the data pipeline to the corresponding robot for autonomous correction. This minimizes the need for human interaction and reduces disruptions to manufacturing operations.

Low-Cost, Open-Source, Experimental Setup Communication Platform for Emergencies, Based on SD-WAN Technology

The rapid advancement of communication technologies underscores the urgent need for robust and adaptable emergency communication systems (ECSs), particularly crucial during crises and natural disasters. Although network-based ECSs have been extensively studied, integrating open-source technologies, such as software-defined wide area networks (SD-WAN) with private long-term evolution (LTE) base stations, is a relatively unexplored domain. This study endeavors to fill this gap by introducing an experimental ECS platform that utilizes a hybrid network, incorporating a VoIP network to enhance open-source and on-premises communications in targeted areas. Our hypothesis posits that a hybrid network architecture, combining SD-WAN and private LTE, can substantially improve the reliability and efficiency of ECSs. Our findings, supported by the open-source OMNeT++ simulator, illuminate the enhanced communication reliability of the network. Moreover, the proposed platform, characterized by autonomous wireless 4G/LTE base stations and an Asterisk VoIP server, demonstrates improved quality of service (QoS) and quality of experience (QoE), with minimal data loss. This research not only has immediate practical applications but also bears significant implications for the development of cost-effective, open-source communication networks, optimized for emergencies, critical infrastructure, and remote areas.

@theSource: Welcome.

This inaugural article sets the stage and scope for a new department in IEEE Computer Graphics and Applications: @theSource. In this department, we set out to address the questions, "How have open source projects and open Standards driven graphics innovations and applications?" and "What can we learn from them?" Thus, we are broadly concerned with how open communities and ecosystems have (and are) impacting computer graphics. The intent is to highlight: open source software (such as architectures, engines, frameworks, libraries, services); open Standards and open source data and models; and applications as well as the impacts of open graphics technologies. We also consider historical and summative reviews on the cultural and economic aspects of open source and open Standards graphics ecosystems, such as visualization and mixed reality.

Emission savings through the COP26 declaration of deforestation could come at the expense of non-forest land conversion

The majority of signatories to The United Nations Climate Change Conference in 2021 (COP26) made a declaration to end deforestation by 2030. Here, we quantify future changes in land use and associated CO2 emissions to examine the impact of ending deforestation by 2030 on global land dynamics and emissions using an open-source land-use model. We show that if the COP26 declaration to end deforestation is fully implemented globally, about 167 Mha of deforestation could be avoided until 2050, compared to a baseline scenario which does not have extended forest protection. However, avoided deforestation and associated emissions come at the cost of strongly increased conversion of unprotected non-forested land to agricultural land, while land-use intensification in most regions is similar compared to a baseline scenario. Global initiatives are needed to facilitate a common dialogue on addressing the possible carbon emissions and non-forest leakage effects due to the expedited loss of non-forested land under a policy aimed at halting deforestation by 2030.

A modular simulation framework for organ allocation

BackgroundWe describe and validate a new simulation framework addressing important limitations of the Simulated Allocation Models (SAMs) long used to project population effects of transplant policy changes. MethodsWe developed the Computational Open-source Model for Evaluating Transplantation (COMET), an agent-based model simulating interactions of individual donors and candidates over time to project population outcomes. COMET functionality is organized into interacting modules. Donors and candidates are synthetically generated using data-driven probability models which are adaptable to account for ongoing or hypothetical donor/candidate population trends and evolving disease management. To validate the first implementation of COMET, COMET-Lung, we attempted to reproduce lung transplant outcomes for U.S. adults from 2018-2019 and in the six months following adoption of the Composite Allocation Score (CAS) for lung transplant. ResultsSimulated (median [Interquartile Range, IQR]) versus observed outcomes for 2018-2019 were: 0.162 [0.157, 0.167] versus 0.170 waitlist deaths per waitlist year; 1.25 [1.23, 1.28] versus 1.26 transplants per waitlist year; 0.115 [0.112, 0.118] versus 0.113 post-transplant deaths per patient year; 202 [102, 377] versus 165 nautical miles travel distance. The model accurately predicted the observed precipitous decrease in transplants received by type O lung candidates in the six months following CAS implementation. ConclusionsCOMET-Lung closely reproduced most observed outcomes. The use of synthetic populations in the COMET framework paves the way for examining possible transplant policy and clinical practice changes in populations reflecting realistic future states. Its flexible, modular nature can accelerate development of features to address specific research or policy questions across multiple organs.

Prompt engineering: The next big skill in rheumatology research.

Large language models (LLMs) like GPT-4 and Claude are catalyzing transformation across medical research including rheumatology. This review examines their applications, highlighting the pivotal role of prompt engineering in effectively guiding LLMs. Key aspects explored include literature synthesis, data analysis, manuscript drafting, coding assistance, privacy considerations, and generative artificial intelligence integrations. While LLMs accelerate workflows, reliance without apt prompting jeopardizes accuracy. By methodically constructing prompts and gauging model outputs, researchers can maximize relevance and utility. Locally run open-source models also offer data privacy protections. As LLMs permeate rheumatology research, developing expertise in strategic prompting and assessing model limitations is critical. With proper oversight, LLMs markedly boost scholarly productivity.

Research on the Recognition and Application of Montreal Forced Aligner for Singing Audio

This paper discusses the feasibility of obtaining phoneme-aligned time segments for singing audio using the Montreal Forced Aligner (MFA) tool. Initially, the recognition effectiveness of singing audio data is tested using the open-source MFA model. Subsequently, samples with high recognition accuracy are manually annotated and used to train the MFA model. Finally, the recognition effectiveness of the open-source MFA model and the MFA model trained on singing audio data is observed. It is found that the performance of the trained MFA model is significantly improved. This also confirms the feasibility of MFA in recognizing singing audio.

Local large language models for privacy-preserving accelerated review of historic echocardiogram reports.

The study developed framework that leverages an open-source Large Language Model (LLM) to enable clinicians to ask plain-language questions about a patient's entire echocardiogram report history. This approach is intended to streamline the extraction of clinical insights from multiple echocardiogram reports, particularly in patients with complex cardiac diseases, thereby enhancing both patient care and research efficiency. Data from over 10 years were collected, comprising echocardiogram reports from patients with more than 10 echocardiograms on file at the Mount Sinai Health System. These reports were converted into a single document per patient for analysis, broken down into snippets and relevant snippets were retrieved using text similarity measures. The LLaMA-2 70B model was employed for analyzing the text using a specially crafted prompt. The model's performance was evaluated against ground-truth answers created by faculty cardiologists. The study analyzed 432 reports from 37 patients for a total of 100 question-answer pairs. The LLM correctly answered 90% questions, with accuracies of 83% for temporality, 93% for severity assessment, 84% for intervention identification, and 100% for diagnosis retrieval. Errors mainly stemmed from the LLM's inherent limitations, such as misinterpreting numbers or hallucinations. The study demonstrates the feasibility and effectiveness of using a local, open-source LLM for querying and interpreting echocardiogram report data. This approach offers a significant improvement over traditional keyword-based searches, enabling more contextually relevant and semantically accurate responses; in turn showing promise in enhancing clinical decision-making and research by facilitating more efficient access to complex patient data.

Numerical Model Test of Spilled Oil Transport Near the Korean Coasts Using Various Input Parametric Models

Oil spills pose significant threats to marine ecosystems, human health, socioeconomic aspects, and coastal communities. Accurate real-time predictions of oil slick transport along coastlines are paramount for quick preparedness and response efforts. This study used an open-source OpenOil numerical model to simulate the fate and trajectories of oil slicks released during the 2007 Hebei Spirit accident along the Korean coasts. Six combinations of input parameters, derived from a five-day met-ocean dataset incorporating various hydrodynamic, meteorological, and wave models, were investigated to determine the input variables that lead to the most reasonable results. The predictive performance of each combination was evaluated quantitatively by comparing the dimensions and matching rates between the simulated and observed oil slicks extracted from synthetic aperture radar (SAR) data on the ocean surface. The results show that the combination incorporating the Hybrid Coordinate Ocean Model (HYCOM) for hydrodynamic parameters exhibited more substantial agreement with the observed spill areas than Copernicus Marine Environment Monitoring Service (CMEMS), yielding up to 88% and 53% similarity, respectively, during a more than four-day oil transportation near Taean coasts. This study underscores the importance of integrating high-resolution met-ocean models into oil spill modeling efforts to enhance the predictive accuracy regarding oil spill dynamics and weathering processes.

Harmonizing financial futures: AI enhancement and methodical fusion in banking risk oversight

The paper will deal with the link between finance and artificial intelligence (AI) in the context of banking, including how AI was introduced from simple tasks to more complex ones involving risk management and fraud detection. The phenomenon displays use of AI by banks as exemplified by First Financial Bank, which was able to open new accounts through intelligent digital assistants. The article focuses on the question of ethical AI that aims to create trust and compliance. This article has explored the position that big language models (LLMs) hold in enhancing virtual assistants in banking and the significance of open source models, as they contribute to collaboration and innovation in the financial industry.

Harnessing Open Innovation for Translating Global Languages into Indian Lanuages

Understanding foreign languages can be challenging for individuals living in India's diverse linguistic landscapes. We propose a new technology that utilizes machine translation to address this issue, specifically focusing on speech recognition and synthesis. It aims to convert online video resources into Indian languages by integrating open-source technologies like text-to-speech (TTS), speech-to-text (STT) systems, and FFmpeg library to separate or augment audio and video. We used the whisper model, the application that can read up to 60 different Languages in the form of audio as input, and it transcripts the audio into text with segments of sentences based on timestamps. The sentence-based transcription generated by whisper is then translated into the desired language using Google Cloud translate_v2. Later, Each timestamp was individually converted into audio using the Google Cloud text-to-speech service, ensuring the audio fits inside the length of its respective timestamp. The individual audio segments are then augmented to generate the final audio in the desired language. Finally, the audio is attached to the original video, ensuring video-audio synchronization. The accuracy of the translation was verified by comparing the naturalness of the audio with general spoken language standards. This application benefits visually impaired individuals and those who cannot read text, providing them with a means to acquire knowledge in their native languages.

Microservice Architecture in E-commerce Platforms using Docker and Kubernetes

software development and deployment by increasing modularity, scalability, and flexibility. This technique divides apps into smaller, independent services that communicate over HTTP APIs. Docker plays an important role in containerizing these services, guaranteeing consistent operation across several settings. Docker containers are lightweight, use the host kernel, and start quicker than typical virtual machines, making them perfect for microservices. Kubernetes, Google's open-source technology, enhances microservices by automating container deployment, scaling, and administration across host clusters. This combination creates a stable environment for delivering microservices, allowing for continuous integration and deployment (CI/CD). This article investigates the synergistic usage of microservices, Docker, and Kubernetes, showing their combined efficiency in developing, deploying, and scaling applications across many environments. Keywords— microservice, architecture, Monolithic Architecture, Kubernetes, Docker

Development and evaluation of two open-source nnU-Net models for automatic segmentation of lung tumors on PET and CT images with and without respiratory motion compensation

ObjectivesIn lung cancer, one of the main limitations for the optimal integration of the biological and anatomical information derived from Positron Emission Tomography (PET) and Computed Tomography (CT) is the time and expertise required for the evaluation of the different respiratory phases. In this study, we present two open-source models able to automatically segment lung tumors on PET and CT, with and without motion compensation.Materials and methodsThis study involved time-bin gated (4D) and non-gated (3D) PET/CT images from two prospective lung cancer cohorts (Trials 108237 and 108472) and one retrospective. For model construction, the ground truth (GT) was defined by consensus of two experts, and the nnU-Net with 5-fold cross-validation was applied to 560 4D-images for PET and 100 3D-images for CT. The test sets included 270 4D- images and 19 3D-images for PET and 80 4D-images and 27 3D-images for CT, recruited at 10 different centres.ResultsIn the performance evaluation with the multicentre test sets, the Dice Similarity Coefficients (DSC) obtained for our PET model were DSC(4D-PET) = 0.74 ± 0.06, improving 19% relative to the DSC between experts and DSC(3D-PET) = 0.82 ± 0.11. The performance for CT was DSC(4D-CT) = 0.61 ± 0.28 and DSC(3D-CT) = 0.63 ± 0.34, improving 4% and 15% relative to DSC between experts.ConclusionsPerformance evaluation demonstrated that the automatic segmentation models have the potential to achieve accuracy comparable to manual segmentation and thus hold promise for clinical application. The resulting models can be freely downloaded and employed to support the integration of 3D- or 4D- PET/CT and to facilitate the evaluation of its impact on lung cancer clinical practice.Clinical relevance statementWe provide two open-source nnU-Net models for the automatic segmentation of lung tumors on PET/CT to facilitate the optimal integration of biological and anatomical information in clinical practice. The models have superior performance compared to the variability observed in manual segmentations by the different experts for images with and without motion compensation, allowing to take advantage in the clinical practice of the more accurate and robust 4D-quantification.Key PointsLung tumor segmentation on PET/CT imaging is limited by respiratory motion and manual delineation is time consuming and suffer from inter- and intra-variability.Our segmentation models had superior performance compared to the manual segmentations by different experts.Automating PET image segmentation allows for easier clinical implementation of biological information.

SPACE+ DATA DASHBOARD: EMPOWERING INSTITUTIONS AND CITIZENS WITH SEAMLESS SPACE DATA ACCESS

Abstract. This paper presents the Space+ Data Dashboard (SDD), a robust platform developed by the Philippine Space Agency (PhilSA), designed to revolutionize space data access for institutions and citizens. Built using open-source technologies, the SDD seamlessly manages spatial and non-spatial datasets. The SDD is composed of three core components: backend, frontend, and database. The backend uses PostgreSQL with PostGIS for data storage and indexing, Geoserver for vector files, and the Open Data Cube for raster files. Django handles other data types and serves API requests. The front-end uses TerriaMap, a customizable web front-end built on TerriaJS to provide public access and geo-visualization, and ReactJS for user interface and interaction. The SDD is envisioned to fulfill PhilSA’s mandate to empower government institutions and the Filipino people using space technologies. It democratizes data access to facilitate the use of space + geospatial data in disaster management, urban planning, environmental monitoring, and informed decision-making. The dashboard also offers direct downloads of various space-derived datasets. In championing data accessibility, PhilSA ensures data-driven decision-making, advancing both national space initiatives and the United Nations' Sustainable Development Goals.

Wflow_sbm v0.7.3, a spatially distributed hydrological model: from global data to local applications

Abstract. The wflow_sbm hydrological model, recently released by Deltares, as part of the Wflow.jl (v0.7.3) modelling framework, is being used to better understand and potentially address multiple operational and water resource planning challenges from a catchment scale to national scale to continental and global scale. Wflow.jl is a free and open-source distributed hydrological modelling framework written in the Julia programming language. The development of wflow_sbm, the model structure, equations and functionalities are described in detail, including example applications of wflow_sbm. The wflow_sbm model aims to strike a balance between low-resolution, low-complexity and high-resolution, high-complexity hydrological models. Most wflow_sbm parameters are based on physical characteristics or processes, and at the same time wflow_sbm has a runtime performance well suited for large-scale high-resolution model applications. Wflow_sbm models can be set a priori for any catchment with the Python tool HydroMT-Wflow based on globally available datasets and through the use of point-scale (pedo)transfer functions and suitable upscaling rules and generally result in a satisfactory (0.4 ≥ Kling–Gupta efficiency (KGE) < 0.7) to good (KGE ≥ 0.7) performance for discharge a priori (without further tuning). Wflow_sbm includes relevant hydrological processes such as glacier and snow processes, evapotranspiration processes, unsaturated zone dynamics, (shallow) groundwater, and surface flow routing including lakes and reservoirs. Further planned developments include improvements on the computational efficiency and flexibility of the routing scheme, implementation of a water demand and allocation module for water resource modelling, the addition of a deep groundwater concept, and computational efficiency improvements through for example distributed computing and graphics processing unit (GPU) acceleration.

Is silicon worth it? Modelling degradation in composite silicon–graphite lithium-ion battery electrodes

The addition of silicon into graphite lithium-ion battery anodes has the potential to increase cell energy density. However, understanding the complex degradation behaviour in these composite systems remains a research challenge. Here, we developed a coupled electrochemical–mechanical model of a composite silicon/graphite electrode, including stress-driven crack formation and solid electrolyte interphase layer growth for each material, validated with experimental degradation data from an LG M50T cell. The model reveals self-limiting loss of silicon due to decreasing stress in the silicon as the silicon activity shifts to a lower state-of-charge. Higher C-rates can lead to lower degradation due to lower phase utilisation as voltage cut-offs are reached earlier. Increasing silicon content can reduce the stress in the silicon by distributing reaction current density over more material. Using this model, we explored whether the extra capacity from silicon is generally ‘worth’ the faster degradation compared to graphite-only electrodes. The model shows if you use the silicon, you lose it, as the higher initial capacity is rapidly lost with regular high depth-of-discharge events. However, silicon does have value if it enables full graphite utilisation without range anxiety; if high depth-of-discharge events are minimised then graphite’s superior longevity can be utilised while exploiting silicon’s high specific capacity. The model is integrated into PyBaMM (an open-source physics-based modelling platform); providing the research community and industry with the capability to reproduce our results and further explore the dynamic lifetime behaviour of composite electrodes.

VeriGen: A Large Language Model for Verilog Code Generation

In this study, we explore the capability of Large Language Models (LLMs) to automate hardware design by automatically completing partial Verilog code, a common language for designing and modeling digital systems. We fine-tune pre-existing LLMs on Verilog datasets compiled from GitHub and Verilog textbooks. We evaluate the functional correctness of the generated Verilog code using a specially designed test suite, featuring a custom problem set and testing benches. Here, our fine-tuned open-source CodeGen-16B model outperforms the commercial state-of-the-art GPT-3.5-turbo model with a 1.1% overall increase. Upon testing with a more diverse and complex problem set, we find that the fine-tuned model shows competitive performance against state-of-the-art gpt-3.5-turbo, excelling in certain scenarios. Notably, it demonstrates a 41% improvement in generating syntactically correct Verilog code across various problem categories compared to its pre-trained counterpart, highlighting the potential of smaller, in-house LLMs in hardware design automation. We release our training/evaluation scripts and LLM checkpoints as open-source contributions.

UAV based comprehensive modelling approach for flood hazard assessment and mitigation planning

The assessment of flood hazard in semi-arid region under the scare data situation is a challenging task under the changing climatic situation. Therefore, the present research is carry out for Rel river catchment, Gujrat, India as case study and presented the comprehensive approach to identify the possible flood extend and flood mitigation measures for better flood assessment. The UAV based high resolution (3.6 cm × 3.6 cm) DEM is generated for most flood vulnerable area (i.e. Dhanera city), whereas scare precipitation data under ungauged condition is generated using GEE and National Aeronautics and Space Administration (NASA) Global Precipitation Measurement Mission (GPM) system. The HEC-HMS based hydrologic model is utilized for simulation of stream flow under the observed data, however, the 2D HEC-RAS based open-source model is used for flood assessment in 2D under extreme flood, dam break and levees condition. The flood decision making maps i.e water surface elevation, flood inundation, flood arrival time, and flood velocity are prepared for flood event 2017. The six flood hazard classes were selected from H1 (Low) to H6 (High) categories. The hazard maps of flood for different scenario i.e., 2D hydrodynamic analysis, 2D dam breach, 2D analysis with extended levees, and Dam break analysis with extended levees are derived. The model was validated with observed river gauge (water level) and discharge data at Dhanera City of flood 2017. The developed UAV based comprehensive modelling approach for flood hazard and mitigation planning provides the systematic approach for flood hazard assessment and would be adopted as a system framework for flood risk assessment for any similar case worldwide.