Automatic Validation Research Articles

A workflow reproducibility scale for automatic validation of biological interpretation results.

Reproducibility of data analysis workflow is a key issue in the field of bioinformatics. Recent computing technologies, such as virtualization, have made it possible to reproduce workflow execution with ease. However, the reproducibility of results is not well discussed; that is, there is no standard way to verify whether the biological interpretation of reproduced results is the same. Therefore, it still remains a challenge to automatically evaluate the reproducibility of results. We propose a new metric, a reproducibility scale of workflow execution results, to evaluate the reproducibility of results. This metric is based on the idea of evaluating the reproducibility of results using biological feature values (e.g., number of reads, mapping rate, and variant frequency) representing their biological interpretation. We also implemented a prototype system that automatically evaluates the reproducibility of results using the proposed metric. To demonstrate our approach, we conducted an experiment using workflows used by researchers in real research projects and the use cases that are frequently encountered in the field of bioinformatics. Our approach enables automatic evaluation of the reproducibility of results using a fine-grained scale. By introducing our approach, it is possible to evolve from a binary view of whether the results are superficially identical or not to a more graduated view. We believe that our approach will contribute to more informed discussion on reproducibility in bioinformatics.

Automatic validation and analysis of predictive models by means of big data and data science

Validation is an essential procedure in the development of a predictive model in several engineering fields. In addition, recent data analysis techniques and the increasing availability of data have the potential to provide a deeper understanding of experimental data and simulation models. This work proposes a systematic, objective, and automatic methodology to validate and analyze experiments and models from a high-level perspective. The proposed methodology exploits the opportunities offered by the ‘data ecosystem’ concept, combining data and model evaluation and providing an integrated set of techniques to produce synthetic but comprehensive insights about the experiment and the predictive model. The methodology focuses on data assessment of the experiments used in the process, the use of a trend similarity comparison index to measure the model performance, and data science techniques to systematically extract models’ behavior insight by analyzing a large number of validation results and linking them to the experiment characteristics.The automated proposed approach follows the generality principle and can be extended to different application domains in which predictive models are validated against big data in the chemical engineering domain. As a case study, the proposed methodology is applied with hundreds of experimental datasets to evaluate a kinetic model that describes the pyrolysis and combustion of hydrocarbons.

Algorithms for Automatic Data Validation and Performance Assessment of MOX Gas Sensor Data Using Time Series Analysis

The following work presents algorithms for semi-automatic validation, feature extraction and ranking of time series measurements acquired from MOX gas sensors. Semi-automatic measurement validation is accomplished by extending established curve similarity algorithms with a slope-based signature calculation. Furthermore, a feature-based ranking metric is introduced. It allows for individual prioritization of each feature and can be used to find the best performing sensors regarding multiple research questions. Finally, the functionality of the algorithms, as well as the developed software suite, are demonstrated with an exemplary scenario, illustrating how to find the most power-efficient MOX gas sensor in a data set collected during an extensive screening consisting of 16,320 measurements, all taken with different sensors at various temperatures and analytes.

Automatic 3D mitral valve leaflet segmentation and validation of quantitative measurement

3D transesophageal echocardiography (TEE) is widely used in the diagnosis of mitral valve disease and is also well suited for guiding cardiac interventions. The aim of this work is to achieve patient-specific 3D TEE mitral valve leaflet segmentation without any user interaction and to assess the feasibility of 3D quantitative measurements on automatic segmentation model. We suggested a novel pre-training strategy to better implement automatic segmentation. The strategy refers to classify the diastolic and systolic states of the mitral valve through a 3D convolutional neural network architecture, and then use the pretrained weights obtained from the classification task to initialize the parameters of the 3D segmentation deep learning framework. To determine the accuracy of geometric parameters of segmentation model, the measurements of the segmentation model were compared with those obtained by the clinical software. Statistical analysis was performed by using Intraclass Correlation Coefficient and Bland–Altman method. Fourteen 3D volumes were used to evaluate the segmentation performance. The results show a Dice Similarity Coefficient (DSC) of 0.877±0.027 and an Average Surface Distance (ASD) of 0.925±0.392 mm. Twenty-eight 3D volumes were used for the quantitative measurement. The statistical results show that the mitral annular parameters have a good agreement between segmentation model and clinical software except for the annular height. We developed a fully automatic methodology to segment the mitral valve leaflet from 3D TEE and demonstrated the feasibility of improving segmentation performance with the proposed pre-training strategy. The automatic segmentation model was proved to be reliable for performing quantitative measurements of mitral valve annulus dimensions. The results indicate that the precision of the automatic segmentation methodology could pave the way for application in quantification, modeling and surgical planning tools.

Blockchain-Based Offline Auditing for the Cloud in Vehicular Networks

The rapid growth of various vehicular apps such as automotive navigation and in-car entertainment has brought the explosion of vehicular data. Such a growth has given rise to a huge challenge of maintaining the quality of cloud storage services for the whole period of storage in vehicular networks. As a result, poor quality of services easily causes data corruption problems and thereby threats vehicular data integrity. Blockchain, a tamper-proofing technique, is considered a promising approach for mitigating data integrity risks in cloud storage. However, existing blockchain-based schemes for auditing long-term cloud data integrity suffer from poor communication performance in a vehicular network. In this study, a blockchain-based offline auditing scheme for cloud storage in the vehicular network is proposed to improve auditing performance. Inspired by the data structure of blockchain, we design an evidence chain to achieve offline auditing, which allows the cloud to spontaneously generate data integrity evidence without communicating with auditors during the evidence generation phase. Furthermore, we extend our scheme to support public and automatic validation based on the smart contract. We prove the security of the proposed scheme under the random oracle model and further provide the performance evaluation by comparing with the state-of-the-art approaches.

Evaluation of medical assays by multiple-choice questions: Example of 10-years’ experience of a North African Country

Construct: Authors examined the use of multiple choice question tests in assessment of undergraduate students, and made a docimological evaluation of the quality of the tests. Background: Moroccan universities are gradually integrating multiple-choice questions as a method of evaluation. At the Faculty of Medicine of Marrakech-Morocco, this type of test is used to assess medical students’ knowledge, in addition to easier large-scale assessments and automatic correction, test reliability and validity are major qualities of multiple-choice questions, their important disadvantage is that they are difficult and time-consuming to construct. The aim this study was to report the authors experience in assessment of undergraduate students and to make a docimological evaluation of the quality of the tests.Approach: we conducted an observational study on tests used for the assessment of semiology and pathology tests in urology and nephrology, during 10 years (2007 to 2017), we performed the analysis of the different criteria for writing an assay. Results: six-hundred and forty questions were analyzed in the pathology modules, 56,2% of questions addressed level I of Bloom's modified taxonomy, level II (28,8%), level III (15%)

SST Anywhere—A Portable Solution for Wide Field Low Earth Orbit Surveillance

The low-Earth orbit (LEO) is filled with active satellites, but also with space debris, which need constant observation. The orbiting objects may be affected by collisions or by atmospheric drag, and therefore they can change their orbit or even fall to the ground, a process known as reentry. The low altitude of these objects (below 2000 km, usually even below 1000 km) means that at given time they can be observed from a limited range of locations on the Earths’ surface, and therefore having multiple, easy to set up observation stations can be extremely useful. This paper presents a portable hardware solution for on-demand wide-field surveillance of the LEO region, the image processing algorithms for detecting the satellite streaks and for joining these streaks into tracklets, and the solution for astrometrical reduction and generating the result file for each tracklet. An automatic validation solution that is able to automatically identify the detected satellites and compute the measurement angular errors is also presented. The acquisition and processing system is built with commercially available items of low and moderate costs and is capable of on-site acquisition and real-time processing of images. The acquired images are processed by background subtraction, analysis of the difference between frames, extraction of elongated objects corresponding to the satellite streaks, and forming trajectories (tracklets) from consecutive detections. The pixel coordinates of the tracklets are converted to angular coordinates using the tools from Astrometry.net, subsequently filtered for improving the accuracy. The results are validated by using daily updated orbital parameters (TLEs), which are used to predict the angular positions that are subsequently matched with the detection results.

Machine‐readable universal data format for bidirectional reflectance distribution function and BiRDview—An open‐source web‐based application for viewing and comparing bidirectional reflectance data

AbstractModern studies of bidirectional reflectance distribution function (BRDF) and its applications using data and machine‐driven science require formatting of BRDF data according to Findable, accessible, interoperable and reusable (FAIR) data principles. As a solution a FAIR universal BRDF file‐format based on Java Script Object Notation (JSON) is proposed. JSON principles as well as file structure are explained and examples are given. Automatic validation of universal BRDF file format is realized with the help of JSON schema. Furthermore, the source code and accompanying documentation are presented in dedicated supporting material files. It is expected that after its wide adoption, the proposed BRDF file format will enhance collaboration between different research groups and benefit machine‐driven science. The uptake is facilitated by introducing a BiRDview—a modern open‐source web‐based application for BRDF visualization.

Systemically Improving Data Validity of In-Hospital Patients with Central Catheterization by Using Nursing Records

<p>自2015年1月起，本院依據P4P（Hospital Quality Performance Measurement Index，醫院品質績效量測指標系統與落實品質改善計畫）及QIP（Quality Improvement Program，持續性監測指標），將中心導管人日數分母由人工收集方式改由護囑系統自動化收集，目的在於建立結構化的資料探勘模式，智慧解決指標數據來源的正確性及穩定度，並落實改善影響指標監測的人為因素，以確保指標管理可維護醫療品質與病人安全之功能。經由1.確認指標收案定義及標準、2.整合中心導管注射管路撈取系統、3.運用防呆設計以避免重複計算、4.提供各單位中心導管查詢報表及5.每月自動拋轉數據及專人稽核驗證等方法介入，達成中心導管住院人日數收集正確性。透過指標資訊化，可達節時優化、身體評估零時差及避免管路登錄遺漏，且能減碳及優化空間。未來可將此系統推廣至各單位、導入圖形化管理，以利臨床單位監測運用，成為智慧改善品質的重要執行依據。</p> <p> </p><p>Data on the number of in-hospital patient with central catheterization, which is defined using P4P (Hospital Quality Performance Measurement Index) and QIP (Quality Improvement Program) indicators, were extracted from the nursing records of E-Da Hospital for the period from January 2015 to the present. The main study purposes were to construct a data mining model, ensure the accuracy and stability of data sources, and implement improving human factors affecting indicator monitoring in the interest of consolidating the functions of indicator management to maintain healthcare quality and patient safety. Therefore, we uses five steps to increase the accuracy of the data collected on in-hospital patients with central catheterization. The five steps were as follows: 1. Confirm the definitions and standards applied to the indicators; 2. standardize the data collection method for the studied patients; 3. apply a foolproof mechanism to avoid repeating calculations; 4. provide subquery reports of central catheters to each station; and 5. perform real-time automatic data collection and validation. Mining data directly from nursing records is time efficient, allows for real-time physical evaluations, and simplifies the sampling process; furthermore, the use of IT-based indicators reduces the amount of missing data and the use of paper documents, thereby reducing emissions and creating more space. Through this study, we discovered that we should promote our automation system to all units and utilize graphic tools so it can be used to perform daily monitoring at all units and intelligently improve healthcare quality.</p> <p> </p>

Beep4Me: Automatic Ticket Validation to Support Fare Clearing and Service Planning.

An integrated transport service fare system, supported by an agreement for ticket revenue sharing among service providers, is an essential component to improve the experience of the users who can find single tickets for the integrated transport services they look for. A challenge is to find a model to share the revenue which all providers agree on. A solution is to adopt data-driven approaches where user-generated data are collected to extract information on the extent each transport service was used. This is consistently used. However, it suffers from incomplete data, as not all users always validate their ticket when checking out or when switching lines. We studied all technologies available to support automatic ticket validation in order to record when the users access and exit each service line. The contributions of this work are the following: we give an in-depth description of the inner workings of this novel approach describing how we take advantage of each technology; we present the developed solution (Beep4Me), which adds new functionalities to an existing mobile ticketing platform; and we describe our testing framework, which includes most cases users might encounter during a trip. Our results demonstrate how it is possible to collect key data related to validations which can be used first for clearing purposes and then for network planning/fleet optimization.

Scalable reaction network modeling with automatic validation of consistency in Event-B

Constructing a large biological model is a difficult, error-prone process. Small errors in writing a part of the model cascade to the system level and their sources are difficult to trace back. In this paper we extend a recent approach based on Event-B, a state-based formal method with refinement as its central ingredient, allowing us to validate for model consistency step-by-step in an automated way. We demonstrate this approach on a model of the heat shock response in eukaryotes and its scalability on a model of the mathsf {ErbB} signaling pathway. All consistency properties of the model were proved automatically with computer support.

Automatic Validation and Analysis of Predictive Models By Means of Big Data and Data Science

Automatic Validation and Analysis of Predictive Models By Means of Big Data and Data Science

Automatic Optimize-time Validation for Binary Optimizers

Automatic Optimize-time Validation for Binary Optimizers

Flow pattern identification of liquid-liquid (oil and water) in vertical pipelines using machine learning techniques

Given the importance of process control in the petrochemical industry, there is a need to determine the behavior of the fluids inside the pipes. In this work a methodology is developed for the identification of flow patterns in vertical pipes with diameters between 0.01 m and 0.10 m, from the implementation of artificial intelligence techniques, for a liquid combination of two phases composed of oil with viscosity in the range of 792 Kg/m3 to 1823 Kg/m3 and water at room temperature. The predictive models generated in the structuring of the methodology were trained with 70% of data based on viscosity parameters, pipe diameter, volume fraction and surface velocities of the working fluids stored in a database. The remaining information, equivalent to 30% of the total, was used to develop the automatic model validation. The flow patterns identified by the intelligent system for oil and water flow, without considering the predominant substance, are churning, dispersed, very fine dispersion, transition flow, intermittent, and annular

Automatic simulation-based design and validation of robotic gripper fingers

The design of robotic gripper fingers is a complex process and often requires significant effort and time. This paper investigates a method to automatically generate new iterations of the gripper finger design as well as to validate its performance in a simulation environment. A Computer-Aided Design (CAD) software platform and a physics-based simulation framework are deployed to work in tandem to redesign and validate an initial gripper finger design aiming at reducing the overall time and cost required for physical validation. The proposed approach is validated in a real robotic case scenario, performing a series of pick and place tasks.

Böhmen interimperial. Die böhmische Jurisprudenz als Drehscheibe der zentraleuropäischen Wissenszirkulation

My article uses Bohemian jurisprudence between the 17th and the 20th centuries to showcase the interpretive benefits of a relational history of Habsburg Central Europe that both moves beyond artificial national divisions and sacrosanct epochal thresholds, 1918 in particular. Since the 17th century, Bohemia’s position at the interstices between the Holy Roman Empire and the Habsburg lands made its jurisprudence a centre of sprawling innovation: Prague jurist Franz X. Neumann disputed the Holy Roman Empire’s suzerainty over Bohemia and hence dismantled claims about the automatic validity of Roman law for the kingdom of St. Wenceslaus. This position at the fringes of the Holy Roman Empire turned Bohemia into a hotbed of natural law and into the chief fulcrum of Habsburg private law codification under Maria Theresia (1750s). The Bohemian ingredient also played a crucial role in Leo Thun‐Hohenstein’s reform of the Habsburg Monarchy’s educational system in the 1850s: A Bohemian noble patriot, Thun‐Hohenstein sought to extirpate natural law, which he viewed as the mainspring of Revolution and centralisation – the two menaces that subverted Bohemia’s statehood within the Monarchy. Thun’s hostility to pan‐imperial public law and to social contract theories destroyed the validity grounds of Habsburg legal culture but reinforced the primacy of private law. Thereby Thun produced a legal positivism that depended on the categories of Pandectist private law and permeated ostensibly sharply separated “national” communities of lawyers, as I show with reference to Czech and Germanspeaking jurists in Bohemia, the schools of Jiří Pražák and Joseph Ulbrich in particular. My paper moves beyond 1918 to flesh out the Monarchy’s legal and administrative legacy that cuts against the grain of programmatic De‐Austrification (Odrakouštění), and discusses the longevity of the Monarchy’s 1867 December constitution in post‐1918 Czechoslovakia. Swallowing their pride, German nationalists of different hues retracted their pre‐1918 criticisms of the constitution and rediscovered it as a weapon against the Versailles system.

Ensemble-learning approach for the classification of Levels Of Geometry (LOG) of building elements

The provision of geometric and semantic information is among the most fundamental tasks in BIM-based building design. As the design is constantly developing along with the design phases, there is a need for a formalism to define its maturity and detailing. In practice, the concept of Level of Development (LOD) is used to specify what information must be available at which time. Such information is contractually binding and crucial for different kinds of evaluations. Numerous commercial and open-source BIM tools currently support the automatic validation of semantic information. However, the automatic validation of the modeled geometry for fulfilling the expected detailing requirements is a complex and still unsolved task. In current practice, domain experts evaluate the models manually based on their experience. Hence, this paper presents a framework for formally analyzing and automatically checking the Level Of Geometry (LOG) of building information models. The proposed framework first focuses on generating a LOG dataset according to the popular LOD specifications. Afterwards, multiple geometric features representing the elements’ complexity are extracted. Finally, two tree-based ensemble models are trained on the extracted features and compared according to their accuracy in classifying building elements with the correct LOG. Measuring the modeling time showed a 1.88–2.80-fold increase between subsequent LOGs, with an 8–15-fold increase for LOG 400 compared to LOG 200. The results of classifying the LOG indicated that the combination of 16 features can represent the LOG complexity. They also indicated that the trained ensemble models are capable of classifying building elements with an accuracy between 83% and 85%.

EMPAIA App Interface: An open and vendor-neutral interface for AI applications in pathology

Background and objectiveArtificial intelligence (AI) apps hold great potential to make pathological diagnoses more accurate and time efficient. Widespread use of AI in pathology is hampered by interface incompatibilities between pathology software. We studied the existing interfaces in order to develop the EMPAIA App Interface, an open standard for the integration of pathology AI apps. MethodsThe EMPAIA App Interface relies on widely-used web communication protocols and containerization. It consists of three parts: A standardized format to describe the semantics of an app, a mechanism to deploy and execute apps in computing environments, and a web API through which apps can exchange data with a host application. ResultsFive commercial AI app manufacturers successfully adapted their products to the EMPAIA App Interface and helped improve it with their feedback. Open source tools facilitate the adoption of the interface by providing reusable data access and scheduling functionality and enabling automatic validation of app compliance. ConclusionsExisting AI apps and pathology software can be adapted to the EMPAIA App Interface with little effort. It is a viable alternative to the proprietary interfaces of current software. If enough vendors join in, the EMPAIA App Interface can help to advance the use of AI in pathology.

Leveraging non-lattice subgraphs for suggestion of new concepts for SNOMED CT.

Missing hierarchical is-a relations and missing concepts are common quality issues in biomedical ontologies. Non-lattice subgraphs have been extensively studied for automatically identifying missing is-a relations in biomedical ontologies like SNOMED CT. However, little is known about non-lattice subgraphs' capability to uncover new or missing concepts in biomedical ontologies. In this work, we investigate a lexical-based intersection approach based on non-lattice subgraphs to identify potential missing concepts in SNOMED CT. We first construct lexical features of concepts using their fully specified names. Then we generate hierarchically unrelated concept pairs in non-lattice subgraphs as the candidates to derive new concepts. For each candidate pair of concepts, we conduct an order-preserving intersection based on the two concepts' lexical features, with the intersection result serving as the potential new concept name suggested. We further perform automatic validation through terminologies in the Unified Medical Language System (UMLS) and literature in PubMed. Applying this approach to the March 2021 release of SNOMED CT US Edition, we obtained 7,702 potential missing concepts, among which 1,288 were validated through UMLS and 1,309 were validated through PubMed. The results showed that non-lattice subgraphs have the potential to facilitate suggestion of new concepts for SNOMED CT.

Automatic analysis and validation of digitized speech markers in Lewy body spectrum diseases with Alzheimer’s disease co‐pathology

AbstractBackgroundLewy Body Spectrum Disorders (LBSD) are hallmarked by α‐synucleinopathy. Up to 50% of LBSD have co‐occurring Alzheimer’s Disease (AD) at autopsy and this is associated with earlier dementia onset, greater cognitive impairments, and reduced survival. In vivo identification of AD co‐pathology is currently achievable with invasive cerebrospinal fluid (CSF) AD biomarkers or costly molecular PET imaging. There remains a need for non‐invasive, inexpensive, and validated markers that can serve as screening tools for this purpose. We examined automated speech markers of a short semi‐structured natural speech sample in LBSD with and without neurobiologically‐validated AD co‐pathology.MethodWe analyzed digitized speech of Cookie Theft picture descriptions from 62 LBSD patients and 44 healthy controls (HC). AD co‐pathology was determined by neuropathological (25%) or CSF based diagnosis (75%): n=18 LBSD+AD (age= 73.7y (7.4), 67% Parkinson Disease (PD) /PD with Dementia (PDD), 33% Dementia with Lewy Bodies (DLB)); n=44 LBSD‐AD (age= 67.8y (6.5), 61% PD/PDD, 39% DLB). Speech samples were automatically parsed into voiced and silent segments and duration measures were extracted. Transcripts were processed for parts‐of‐speech usage and lexical‐semantic features. We conducted group comparisons of speech measures, examined their effects on determining pathologic group, and assessed relations with MRI cortical thickness.ResultLBSD+AD and LBSD‐AD groups did not differ in disease duration, motor disease severity (UPDRS Part III), nor neuropsychological performance (overall and across individual cognitive domains). Both LBSD+AD and LBSD‐AD spoke in shorter speech segments and longer and more frequent pauses; and speaking rate was slower compared to HC. Compared to LBSD‐AD, speakers with LBSD+AD used shorter words (by phoneme count) acquired at a younger age, and produced more conjunctions per 100 words. The features showed excellent model fit for determining pathologic group (McFadden R2 = 0.23), but speech features did not distinguish between PDD and DLB. Reduced word age of acquisition related to atrophy in left and right temporal regions in LBSD+AD only.ConclusionAutomated speech measures distinguished biologically‐confirmed LBSD+AD from LBSD‐AD regardless of clinical phenotype. Findings highlight the potential use of automated digitized speech analytics as a valid screening tool for AD co‐pathology in LBSD.