Software Validation Research Articles

4D-DSA for Assessment of the Angioarchitecture and Grading of Cranial Dural AVF.

Time-resolved 3D rotational angiography (4D-DSA) has been used to demonstrate details of the angioarchitecture of AVM, whereas it has rarely been used to describe features of dural AVF. In this exploratory study, we analyzed dural AVFs with a novel 4D software prototype, developed and provided by Siemens, to determine whether identification of the location of the fistulous point, grading, and treatment planning were feasible. 4D-DSA volumes were calculated from existing 3D rotational angiography data sets of patients with dural AVFs. The 4D-DSA volumes were displayed in a virtual DSA mode and MPR or MIP in 3 orthogonal planes and compared with 2D-DSA by 2 experienced neuroradiologists. Fusions with unenhanced CT or MR images were used to improve visualization of adjacent anatomic structures. Comparison with 2D-DSA showed that evaluation of the fistulous point and grading according to the classification of Borden, Cognard, or Barrow was feasible in 26 of 27 cases. In 8 of 27 cases, 4D-DSA was considered advantageous for determining the fistulous point and the course of the draining vein in the dural AVF with cortical venous drainage, especially in the frontoethmoidal and frontoparietal regions. In 6 cases, the display of angioarchitecture was considered inferior to that of 2D-DSA due to motion artifacts, suboptimal selection of the injected vessel, and lack of temporal resolution. Detailed analysis of dural AVFs according to the standardized display of 4D-DSA volumes was feasible and helpful in understanding the angioarchitecture in selected cases. Further improvement and validation of the 4D software should solidify the complementary role of 4D-DSA to conventional 2D-DSA series.

Development and validation of an artificial intelligence software for periodontal bone loss in panoramic imaging

AbstractThis retrospective study is aimed at developing a web‐based artificial intelligence (AI) software (DiagnoCat) for periodontal bone loss detection on panoramic radiographs and evaluating the model's performance by comparing it with clinicians' results. Separate models are trained for tooth and periodontal bone loss detection. The first model's objective was to detect teeth, segmenting their masks, and to define their numbering and developed with Mask R‐CNN using pretrained ResNet‐101 as a backbone. The second model was based on Cascade R‐CNN architecture and used for bone loss prediction. Around 100 radiographs are evaluated by three clinicians regarding tooth identification and periodontal bone loss, separately. Ground truth is determined by the consensus and model's performance is evaluated with kappa, precision, recall, and F‐score statistics. For tooth conditions, the overall F‐score, accuracy, and Cohen's kappa coefficients were found to be 0.948, 0.977, and 0.933 for the binary, and 0.992, 0.988, and 0.961 for the multiclass results. For bone loss detection, the overall F‐score, accuracy, and Cohen's kappa coefficients were found to be 0.985, 0.980, and 0.956 for the binary, and 0.996, 0.993, and 0.974 for the multiclass results. The results of this study suggest that the use of a web‐based AI software (DiagnoCat) can be beneficial in detecting periodontal bone loss on panoramic radiographs.

Verification and validation of embedded software in an automotive context: a systematic literature review

In automotive context, the embedded software Verification and Validation (V&V) is always a critical step for each project that involves testing solutions for new function, system optimization and compliance with legal requirements. However, automotive software V&V is laborious and time-consuming. Activities such as planning workshops at test tracks and public roads and functionality and durability tests require significant effort and robust coordination. The rigorous management and storage of test results are also a challenge. This review consolidates the state of the art on automotive software V&V to realize the most common standards in the industry and understand current testing concepts. The consolidated knowledge will help in the future development of a flexible V&V framework for embedded automotive software. A Systematic Literature Review (SLR) was performed by searching four digital libraries from 2011 to March 2022. Sixty-two papers were selected, which indicated that the automotive software V&V process is usually based on the ISO 26262 standard and that the software development life cycle V-model is the most common test platform in the automotive domain. Automotive software for a specific domain has been developed to cover a wide variety of vehicles. Variables from specific regions or countries can influence the entire V&V process for automotive software, such as differences in homologation requirements, infrastructure, driver behavior, customer desires, and electromagnetic force interference. The SLR identified specific characteristics of automotive software and regional factors that can affect the V&V process, as well as significant considerations to ensure correct decision-making, resource allocation, and support of team members.

Validation of a deep learning-based software for automated analysis of T2 mapping in cardiac magnetic resonance imaging.

The reliability and diagnostic performance of deep learning (DL)-based automated T2 measurements on T2 map of 3.0-T cardiac magnetic resonance imaging (MRI) using multi-institutional datasets have not been investigated. We aimed to evaluate the performance of a DL-based software for measuring automated T2 values from 3.0-T cardiac MRI obtained at two centers. Eighty-three subjects were retrospectively enrolled from two centers (42 healthy subjects and 41 patients with myocarditis) to validate a commercial DL-based software that was trained to segment the left ventricular myocardium and measure T2 values on T2 mapping sequences. Manual reference T2 values by two experienced radiologists and those calculated by the DL-based software were obtained. The segmentation performance of the DL-based software and the non-inferiority of automated T2 values were assessed compared with the manual reference standard per segment level. The software's performance in detecting elevated T2 values was assessed by calculating the sensitivity, specificity, and accuracy per segment. The average Dice similarity coefficient for segmentation of myocardium on T2 maps was 0.844. The automated T2 values were non-inferior to the manual reference T2 values on a per-segment analysis (45.35 vs. 44.32 ms). The DL-based software exhibited good performance (sensitivity: 83.6-92.8%; specificity: 82.5-92.0%; accuracy: 82.7-92.2%) in detecting elevated T2 values. The DL-based software for automated T2 map analysis yields non-inferior measurements at the per-segment level and good performance for detecting myocardial segments with elevated T2 values compared with manual analysis.

Validation of an image source method room acoustics software in a highly reverberant room with a directional impulsive noise source

Exposure to impulsive noise sources is common during routine operations military contexts. This type of noise exposure is prevalent in training procedures carried out with small caliber firearms in both indoor and outdoor settings. Typically, hearing protection is required to mitigate hearing damage risk to military personnel. Consequently, proper prescription of hearing protection required bioenvironmental engineers and industrial hygienists to travel and collect data in situ which can be time consuming and expensive. As such, the researchers at the 711th Human Performance Wing of the United States Air Force Research Lab have developed a software named the Shooting Range Impulse Noise Calculator (ShRINC). The ShRINC software provides the capability to simulate high fidelity acoustic levels within indoor and outdoor shooting ranges with directional impulsive noise sources. This is accomplished by leveraging principles from Image Source Room (ISM) based room acoustics and combining them with weapon source models with complicated directivity patterns. In this work, validation will be completed by comparing results collected in a highly reverberant facility with the M4A1 firearm operating blank rounds. An investigation into the proper order for the ISM solver will be analyzed and sound exposure levels between the simulation and measured data will be compared.

Design and analysis of coil spring suspension system using N4518 grade flux levitation for motor bikes

Riding comfort is of paramount importance when designing a suspension system considering vehicle dynamics & stability control. Generally, two-wheelers use springs, hydraulic and pneumatic-type damping mechanisms. The present work was a neoteric magnetic suspension system for two-wheelers that use both magnetic and spring cushioning effects to provide comfort to the rider. In this work, two Neodymium ferromagnetic materials were fixed inside the shock absorber cylinders facing same pole each other. The repulsive magnetic flux they produce when come closer due to shocking load was utilized as media for dampening the shock. The combination of permanent magnet with coiled spring has been utilized to enhance the cushioning effect against the shock transmitted due to acceleration on an uneven road condition. The designed suspension system was more efficient over other type of suspension systems, absorbs a greater number of shocks with high accuracy and had no leakage problem unlike in hydraulic and pneumatic system. The design was validated using 3D modelling and Ansys simulation. The beneficial software validation made the design tentative to work efficiently with less maintenance cost.

Solid-liquid rotary kilns: An experimental and CFD-DEM study

Experiments for the validation of unresolved CFD-DEM software for solid–liquid flows are often expensive, time consuming and generally provide little insight into the local particles dynamics. Additionally, several DEM parameters such as the particle surface properties are often obtained from experiments in air and used for all CFD-DEM simulations even when the fluid is a liquid. We design and perform a simple and time efficient gas–solid and liquid–solid rotary kiln benchmark for the purpose of creating a validation case for unresolved CFD-DEM software which we use to validate the unresolved CFD-DEM model of the open-source software Lethe. This case, which contains dense solid–solid contacts and strong solid–fluid forces gives insight on the importance of proper calibration of DEM surface properties in solid–liquid flows as well as on the importance of the lift force. Furthermore, it is highly sensitive to the accuracy of the CFD discretization.

GUI-Based Data Visualisation Tool for Option C IPMVP using Jupyter Notebook

IPMVP is an acronym for International Performance Measurement and Verification Protocol, used for calculating energy savings by taking some external factors into account to give an equal base for a linear comparison between two periods. However, determining the significant independent variable for routine adjustment causes inconvenient repeated action of regression analysis modeling, which leads to the exhaustive calculation and human error risk. These exasperating actions led to the creation of this project, which aims to provide a user-friendly, insightful, and accurate energy-saving calculation program tool. Thus, the main objective of this project is to develop an energy-saving computational tool for option C IPMVP using web-based data analytic tools to improve efficiency and accuracy in each project. This paper used a web-based mathematical computation, Jupyter Notebook, to determine the significant independent variable for routine adjustment of the adjusted baseline. The program tool was validated using V-model software verification and validation methodology based on IEEE 1012 standards. The study can be concluded that the exhaustive calculation to determine the energy saving based on IPMVP standards can be significantly reduced with precision values at 50%, 80%, 90%, and 95% confidence levels using the proposed program which is equipped with insightful graphs and a user-friendly interface.

Validation of HiG-Flow Software for Simulating Two-Phase Flows with a 3D Geometric Volume of Fluid Algorithm

This study reports the development of a numerical method to simulate two-phase flows of Newtonian fluids that are incompressible, immiscible, and isothermal. The interface in the simulation is located and reconstructed using the geometric volume of fluid (VOF) method. The implementation of the piecewise-linear interface calculation (PLIC) scheme of the VOF method is performed to solve the three-dimensional (3D) interface transport during the dynamics of two-phase flows. In this method, the interface is approximated by a line segment in each interfacial cell. The balance of forces at the interface is accounted for using the continuum interfacial force (CSF) model. To solve the Navier–Stokes equations, meshless finite difference schemes from the HiG-Flow computational fluid dynamics software are employed. The 3D PLIC-VOF HiG-Flow algorithm is used to simulate several benchmark two-phase flows for the purpose of validating the numerical implementation. First, the performance of the PLIC implementation is evaluated by conducting two standard advection numerical tests: the 3D shearing flow test and the 3D deforming field test. Good agreement is obtained for the 3D interface shape using both the 3D PLIC-VOF HiG-Flow algorithm and those found in the scientific literature, specifically, the piecewise-constant flux surface calculation, the volume of fluid method implemented in OpenFOAM, and the high-order finite-element software FEEL. In addition, the absolute error of the volume tracking advection calculation obtained by our 3D PLIC-VOF HiG-Flow algorithm is found to be smaller than the one found in the scientific literature for both the 3D shearing and 3D deforming flow tests. The volume fraction conservation absolute errors obtained using our algorithm are 4.48×10−5 and 9.41×10−6 for both shearing and deforming flow tests, respectively, being two orders lower than the results presented in the scientific literature at the same level of mesh refinement. Lastly, the 3D bubble rising problem is simulated for different fluid densities (ρ1/ρ2=10 and ρ1/ρ2=1000) and viscosity ratios (μ1/μ2=10 and μ1/μ2=100). Again, good agreement is obtained for the 3D interface shape using both the newly implemented algorithm and OpenFOAM, DROPS, and NaSt3D software. The 3D PLIC-VOF HiG-Flow algorithm predicted a stable ellipsoidal droplet shape for ρ1/ρ2=10 and μ1/μ2=10, and a stable cap shape for ρ1/ρ2=1000 and μ1/μ2=100. The bubble’s rise velocity and evolution of the bubble’s center of mass are also computed with the 3D PLIC-VOF HiG-Flow algorithm and found to be in agreement with those software. The rise velocity of the droplet for both the ellipsoidal and cap flow regime’s is found, in the initial stages of the simulation, to gradually increase from its initial value of zero to a maximum magnitude; then, the steady-state velocity of the droplet decreases, being more accentuated for the cap regime.

PO93: Validation of 3rd Party 2nd Check Software for HDR

PO93: Validation of 3rd Party 2nd Check Software for HDR

P127 Validation of HLA typing reagents and analysis software: Can we do better to prevent erroneous typing results?

P127 Validation of HLA typing reagents and analysis software: Can we do better to prevent erroneous typing results?

A Tool for Semi-Automated Extraction of Cotton Gin Energy Consumption from Power Data

The gin stand power is measurable using common tools; however, such tools typically do not detect active ginning. Detecting active ginning is important when trying to separate out the energy going to the moving parts of the gin stand (i.e., the baseline energy) versus the active energy doing work to remove the cotton fibers from the seed. Studies have shown that the gin stand is the second largest consumer of electricity in the ginning operation, while electricity accounts for nearly 17% of the average expense per bale. If active energy differences exist between cotton cultivars, there may be room to optimize and lower these expenses. The goal of the current work is to provide a method (and software tool) to analyze typical power logger data, and extract periods of active ginning, along with the energy consumed and ginning times, in a semi-automated way. The new method presented allows multiple periods of active ginning in a single file, and can separate the total energy into the active and baseline components. Other metrics of interest that the software calculates include the ginning time, and average power. Software validation using a simulated test signal showed that a 2%-or-lower error is possible with a noisy signal.

Design and Validation of Software for the Training and Automatic Evaluation of Music Intonation on Non-Fixed Pitch Instruments for Novice Students

Music education, whether professional or amateur, includes learning musical instruments. Intonation is a critical factor in their training. The main objective of this research work is the design and validation of online educational software for the real-time training and evaluation of intonation in non-fixed pitch musical instruments, such as fretted string instruments (violin, viola, and cello) and brass instruments (trumpet, horn, and trombone). This software is intended to create a practice artefact for novice music students. A design sciences research methodology is adopted to achieve a product tested for functionality and usability. Novice students carry out the validation phase through a study consisting of previous practice with the software and the administration of a questionnaire with open-ended items grouped in technical-didactic, emotional, and overall dimensions, plus two additional questions. The results show evidence that the software was well received, confirming previous studies on the design and validation of educational music education software.

A new software for automated counting of glistenings in intraocular lenses in vivo.

To assess the performance of a bespoke software for automated counting of intraocular lens (IOL) glistenings in slit-lamp images. IOL glistenings from slit-lamp-derived digital images were counted manually and automatically by the bespoke software. The images of one randomly selected eye from each of 34 participants were used as a training set to determine the threshold setting that gave the best agreement between manual and automatic grading. A second set of 63 images, selected using randomised stratified sampling from 290 images, were used for software validation. The images were obtained using a previously described protocol. Software-derived automated glistenings counts were compared to manual counts produced by three ophthalmologists. A threshold value of 140 was determined that minimised the total deviation in the number of glistenings for the 34 images in the training set. Using this threshold value, only slight agreement was found between automated software counts and manual expert counts for the validating set of 63 images (κ=0.104, 95%CI, 0.040-0.168). Ten images (15.9%) had glistenings counts that agreed between the software and manual counting. There were 49 images (77.8%) where the software overestimated the number of glistenings. The low levels of agreement show between an initial release of software used to automatically count glistenings in in vivo slit-lamp images and manual counting indicates that this is a non-trivial application. Iterative improvement involving a dialogue between software developers and experienced ophthalmologists is required to optimise agreement. The results suggest that validation of software is necessary for studies involving semi-automatic evaluation of glistenings.

DockSurf: A Molecular Modeling Software for the Prediction of Protein/Surface Adhesion.

The elucidation of structural interfaces between proteins and inorganic surfaces is a crucial aspect of bionanotechnology development. Despite its significance, the interfacial structures between proteins and metallic surfaces are yet to be fully understood, and the lack of experimental investigation has impeded the development of many devices. To overcome this limitation, we suggest considering the generation of protein/surface structures as a molecular docking problem with a homogenous plan as the target. To this extent, we propose a new software, DockSurf, which aims to quickly propose reliable protein/surface structures. Our approach considers the conformational exploration with Euler's angles, which provide a cartography instead of a unique structure. Interaction energies were derived from quantum mechanics computations for a set of small molecules that describe protein atom types and implemented in a Derjaguin, Landau, Verwey, and Overbeek potential for the consideration of large systems such as proteins. The validation of DockSurf software was conducted with molecular dynamics for corona proteins with gold surfaces and provided enthusiastic results. This software is implemented in the RPBS platform to facilitate widespread access to the scientific community.

Three-Dimensional Preoperative Planning Software for Hip Resurfacing Arthroplasty.

Three-dimensional planning of hip arthroplasty is associated with better visualisation of anatomical landmarks and enhanced mapping for preoperative implant sizing, which can lead to a decrease in surgical time and complications. Despite the advantages of hip resurfacing arthroplasty (HRA), it is considered a technically challenging procedure and associated with inaccurate implant placement. This study aimed to examine the validity, reliability, and usability of preoperative 3D Hip Planner software for HRA. Fifty random cases of various hip osteoarthritis severity were planned twice by two junior trainees using the 3D Hip Planner within a one-month interval. Outcome measures included femoral/cup implant size, stem-shaft angle, and cup inclination angle, and were assessed by comparing outcomes from 2D and 3D planning. An adapted unified theory of acceptance and use of technology (UTAUT) survey was used for software usability. Bland-Altman plots between 3D and 2D planning for stem-shaft and inclination angles showed mean differences of 0.7 and -0.6, respectively (r = 0.93, p < 0.001). Stem-shaft and inclination angles showed inter-rater reliability biases of around -2° and 3°, respectively. Chi-square and Pearson's correlation for femoral implant size showed a significant association between the two assessors (r = 0.91, p < 0.001). The 3D test-retest coefficient of repeatability for stem-shaft and inclination angles were around ±2° and ±3°, respectively, with a strong significant association for femoral implant size (r = 0.98, p < 0.001). Survey analyses showed that 70-90% agreed that 3D planning improved expectancy in four domains. 3D hip planner appears to be valid and reliable in preoperative HRA and shows significant potential in optimising the quality and accuracy of surgical planning.

Validation of AI-based software for objectification of conjunctival provocation test.

Provocation tests are widely used in allergology to objectively reveal patients' sensitivity to specific allergens. The objective quantification of an allergic reaction is a crucial characteristic of these tests. Because of the absence of objective quantitative measurements, the conjunctival provocation test (CPT) is a less frequently used method despite its sensitivity and simplicity. We developed a new artificial intelligence (AI)-based method, called AllergoEye, for quantitative evaluation of conjunctival allergic reactions and validated it in a clinical study. AllergoEye was implemented as a 2-component system. The first component is based on an Android smartphone camera for screening and imaging the patient's eye, and the second is personal computer-based for image analysis and quantification. For the validation of AllergoEye, an open-label, prospective, monocentric study was carried out on 41 patients. Standardized CPT was performed with sequential titration of grass allergens in 4 dilutions, with the reaction evaluated by subjective/qualitative symptom scores and by quantitative AllergoEye scores. AllergoEye demonstrated high sensitivity (98%) and specificity (90%) as compared with human estimation of allergic reaction. Tuning cutoff thresholds allowed us to increase the specificity of AllergoEye to 97%, at which point the correlation between detected sensitivity to allergen and specific IgE carrier-polymer system class becomes obvious. Strikingly, such correlation was not found with sensitivity to allergen detected on the basis of subjective and qualitative symptom scores. The clinical validation demonstrated that AllergoEye is a sensitive and efficient instrument for objective measurement of allergic reactions in CPT for clinical studies as well as for routine therapy control.

The Support of MISRA C++ Analyzer for Reliability of Embedded Systems

Cyber-Physical Systems (CPS) are increasingly used in many complex applications, such as autonomous delivery drones, the automotive CPS design, power grid control systems, and medical robotics. However, existing programming languages lack certain design patterns for CPS designs, including temporal semantics and concurrency models. Future research directions may involve programming language extensions to support CPS designs. On the other hand, JSF++, MISRA, and MISRA C++ are providing specifications intended to increase the reliability of safety-critical systems. This article also describes the development of rule checkers based on the MISRA C++ specification using the Clang open-source tool, which allows for the annotation of code and the easy extension of the MISRA C++ specification to other programming languages and systems. This is potentially useful for future CPS language research extensions to work with reliability software specifications using the Clang tool. Experiments were performed using key C++ benchmarks to validate our method in comparison with the well-known Coverity commercial tool. We illustrate key rules related to class, inheritance, template, overloading, and exception handling. Open-source benchmarks that violate the rules detected by our checkers are also illustrated. A random graph generator is further used to generate diamond case with multiple inheritance testdata for our software validations. The experimental results demonstrate that our method can provide information that is more detailed than that obtained using Coverity for nine open-source C++ benchmarks. Since the Clang tool is widely used, it will further allow developers to annotate their own extensions.

Current Status and Key Issues of Data Management in Tertiary Hospitals: A Case Study of Seoul National University Hospital.

In the era of the Fourth Industrial Revolution, where an ecosystem is being developed to enhance the quality of healthcare services by applying information and communication technologies, systematic and sustainable data management is essential for medical institutions. In this study, we assessed the data management status and emerging concerns of three medical institutions, while also examining future directions for seamless data management. To evaluate the data management status, we examined data types, capacities, infrastructure, backup methods, and related organizations. We also discussed challenges, such as resource and infrastructure issues, problems related to government regulations, and considerations for future data management. Hospitals are grappling with the increasing data storage space and a shortage of management personnel due to costs and project termination, which necessitates countermeasures and support. Data management regulations on the destruction or maintenance of medical records are needed, and institutional consideration for secondary utilization such as long-term treatment or research is required. Government-level guidelines for facilitating hospital data sharing and mobile patient services should be developed. Additionally, hospital executives at the organizational level need to make efforts to facilitate the clinical validation of artificial intelligence software. This analysis of the current status and emerging issues of data management reveals potential solutions and sets the stage for future organizational and policy directions. If medical big data is systematically managed, accumulated over time, and strategically monetized, it has the potential to create new value.

Is It Possible to Diagnose Surgical Uterine Prolapse With Transperineal Ultrasound? Multicenter Validation of Diagnostic Software.

To validate an ultrasound software that uses transperineal ultrasound to diagnose uterine prolapse (UP). Multicenter, observational and prospective study with 155 patients that had indications for surgical intervention for dysfunctional pelvic floor pathology. Each patient underwent an examination with Pozzi tenaculum forceps was performed in the operating room with the patient anesthetized, followed by surgical correction of stages II-IV UP. Transperineal ultrasound was used to assess the difference in the pubis-uterine fundus measurement. With a multivariate logistic regression binary model (with the measurement ultrasound at rest, the Valsalva maneuver and age) using nonautomated methods to predict UP. With the purpose of evaluating the model, a table with coordinates of the receiver operating characteristic (ROC) curve, after which sensitivity and specificity were assessed. A total of 153 patients were included (73 with a diagnosis of surgical UP). It was obtained from the AUC (0.89) of the probabilities predicted by the model (95% confidence interval, 0.84-0.95; P < .0005). Based on the ROC curve for the model, obtaining a sensitivity of 91.8% and a specificity of 72.7%, values that were superior to those for the clinical exam for surgical UP (sensitivity: 80.8%; specificity: 71.3%). We validated software that uses transperineal ultrasound of the pelvic floor and patient age to generate a more reliable diagnosis of surgical UP than that obtained from clinical examinations.