Evaluation Script Research Articles

An open source corpus and automatic tool for section identification in Spanish health records

Background:Electronic Clinical Narratives (ECNs) store valuable individual’s health information. However, there are few available open-source data. Besides, ECNs can be structurally heterogeneous, ranging from documents with explicit section headings or titles to unstructured notes. This lack of structure complicates building automatic systems and their evaluation. Objective:The aim of the present work is to provide the scientific community with a Spanish open-source dataset to build and evaluate automatic section identification systems. Together with this dataset, the purpose is to design and implement a suitable evaluation measure and a fine-tuned language model adapted to the task. Materials and Methods:A corpus of unstructured clinical records, in this case progress notes written in Spanish, was annotated with seven major section types. Existing metrics for the presented task were thoroughly assessed and, based on the most suitable one, we defined a new B2 metric better tailored given the task. Results:The annotated corpus, as well as the designed new evaluation script and a baseline model are freely available for the community. This model reaches an average B2 score of 71.3 on our open source dataset and an average B2 of 67.0 in data scarcity scenarios where the target corpus and its structure differs from the dataset used for training the LM. Conclusion:Although section identification in unstructured clinical narratives is challenging, this work shows that it is possible to build competitive automatic systems when both data and the right evaluation metrics are available. The annotated data, the implemented evaluation scripts, and the section identification Language Model are open-sourced hoping that this contribution will foster the building of more and better systems.

Flexible Hybrid Table Recognition and Semantic Interpretation System

Extracting information from documents containing quantitative data in tabular format is an important but still unsolved task due to the heterogeneity of document layouts. This work aims to take a step toward developing a solution to this problem. This paper proposes a flexible, hybrid table extraction system consisting of a deep learning-based table detection module, a heuristic-based structure recognition method, and a graph-based semantic interpretation component. The proposed system is modular and supports the most frequent table layouts. Moreover, it handles both the documents in image format and PDF files with embedded text. The proposed system outperforms the baseline method and achieves results on par with state-of-the-art approaches on the challenging benchmarks from ICDAR 2013 and ICDAR 2019 table interpretation competitions. Moreover, we correct an issue with the evaluation script used in the latter competition and report extended results of the proposed method in comparison with a leading commercial product. Finally, our table extraction system achieves a high F_1 score in the scenario where raw documents are given as input and the targeted information is contained in a subset of table columns. The presented system achieves results competitive with leading methods in the field. It has already been evaluated on general-purpose data and biomedical benchmarks. We intend to continuously improve our approach and process data from other domains, e.g., financial documents. To support future research on information extraction from documents, we make the evaluation scripts and results from our experiments publicly available at https://github.com/mnamysl/tabrec-sncs.

Pengembangan media komik dengan metode picture and picture untuk meningkatkan keterampilan literasi matematika kelas IV

AbstractIn Indonesia, mathematical literacy in International scope is still not satisfying. In order to increase mathematical literacy ability, it can be done by applying some variation of learning methods supported by the utilization of learning media. The goal of this study is to develop and examine the validity, the practicality, and the effectiveness of comic media by picture and picture method to increase mathematical literacy ability of grade 6th students. This R&D study used the ADDIE development model combined with steps in making comics by Sadiman. Those steps were analysis (analyzing the needs, goal and main material), design (synopsis, first step), development (prototype product, evaluation, revision, and last script), implementation (trials), and evaluation (final product). The validity of the product could be seen from expert judgment by using material, media, and learning expert validation instrument. The practicality of the product was done by the students’ response questionnaire instrument to the media. The effectiveness of the product was seen from one class trial by one group pre-test post-test, and the result was processed using Paired-Samples T-Test check with the help of SPSS 20.0. The result of this study was the comics media to increase grade 6th students’ mathematical literacy ability. Comics media proved to be valid from material and media aspects. Comics media proved to be practice from students’ response questionnaire. Comics media also proved to be effective based on the result of the pre-test, post-test and Paired-Samples T-Test check.Keywords: Comics, Mathematical Literacy, Picture and Picture

Using the HPTLC-bioluminescence bacteria assay for the determination of acute toxicities in marine sediments and its eligibility as a monitoring assessment tool

For an integrated ecological risk assessment of marine sediment contamination, the determination of target-compound concentrations by e.g. mass spectrometric methods is not sufficient to explain sediment toxicity. Due to the presence of a multitude of environmental contaminants in this complex matrix causing a mixed toxicity, the identification and assessment of main toxicants is a challenge. One approach in identifying main toxicants is the application of effect-directed analysis (EDA). In this study, an EDA approach was developed using high performance thin layer chromatography (HPTLC) coupled to bioluminescence bacteria detection with Aliivibrio fischeri for the determination of marine sediments acute toxicity. In a first attempt, the HPTLC separation was optimized with a fast, two-step gradient to separate main hydrophobic organic contaminant (HOC) classes found in marine sediments. An easy-to-use evaluation script for the resulting bioluminescence inhibition images was programed using R. The developed method was applied to sediment extracts of two different sample sets: (i) Fourteen marine sediment samples from the German Bight representing a wide range of contaminant loads and sediment properties and (ii) sediment samples from a core representing temporal trends of contamination. Results from the HPTLC-bioluminescence bacteria assay were compared to HOC concentrations determined by GC-MS/MS. A correlation of the determined inhibition Γ-values for the PAH inhibition zone to PAH concentrations showed a very good agreement (R2 = 0.91). The results of this study were used to evaluate the suitability of the EDA approach to be used as an assessment tool for marine sediments.

Harmonic and pulsed eddy current testing methods as tools for surface and subsurface defect evaluation

Abstract In most publications eddy current testing (ECT) methods are said to be suitable for surface defects up to several millimeters. This paper aims to evaluate the detection capabilities of eddy current testing based on two examples of modern approach. One of them is a harmonic eddy current method with use of wavelet transformation and neural network. The second method is pulsed eddy current testing (PECT) with use of evaluation script based on cross-correlation of measured data with the set of known reference signals. Two samples with five artificial defects each were investigated from the near and far side. Geometric parameters of every defect were then estimated. Results show that both methods are limited by the skin effect. However, the PECT with use of cross-correlation exhibits detection capabilities of subsurface defects in significant depth. On the other hand, harmonic eddy current testing shows lower errors in dimensions estimation for set of surface defects.

Validation of CT image-based software for three-dimensional measurement of acetabular coverage profile.

Plain radiography, 2-dimensional (2D) magnetic resonance imaging (MRI), and computed tomography (CT) do not precisely display morphology and acetabular coverage in developmental dysplasia of the hip or pincer-type femoroacetabular impingement. Pelvic position and pelvic tilt affect assessment of the acetabular parameters, leading to misinterpretation. We tested a 3-dimensional (3D) CT evaluation script to calculate the crossover sign (COS), acetabular coverage and morphology. To test the method, we constructed a phantom pelvic model, in which the acetabulum was mounted at different coverages of the femoral head, and simulated a COS and the acetabular morphology. Additionally we examined the reliability and objectivity of this method in ten patients with CT scans of the pelvis for conditions unrelated to hip disorders. We obtained an average accuracy of the 3D CT evaluation script of -0.37∘ (range -3.84 to 3.88; SD ± 1.43) for morphology, and 0.002% (range -7.28% to 6.90%; SD ± 1.60%) for coverage of the femoral head. Significant correlation between the expected and calculated COS (p= 0.01) was found. Our 3D CT evaluation script permits precise evaluation of the acetabular coverage profile, the presence or absence of a COS and acetabular morphology, independent of patient positioning or pelvic tilt.

Methods for segmentation of rheumatoid arthritis bone erosions in high-resolution peripheral quantitative computed tomography (HR-pQCT).

The comparison between different techniques to quantify the 3-dimensional size of inflammatory bone erosions in rheumatoid arthritis(RA) patients. Anti-cyclic citrullinated peptide antibody(ACPA) positive RA patients received high-resolution peripheral quantitative computed tomography (HR-pQCT) scans of the metacarpophalangeal joints (MCP). Erosions were measured by three different segmentation techniques: (1) manual method with calculation by half-ellipsoid formula, (2) semi-automated modified Evaluation Script for Erosions (mESE), and (3) semi-automated Medical Image Analysis Framework (MIAF) software. Bland & Altman plots were used to describe agreement between methods. Furthermore, shape of erosions was classified as regular or irregular and then compared to the sphericity obtained by MIAF. A total of 76 erosions from 65 RA patients (46 females/19 males), median age 57 years, median disease duration 6.1 years and median disease activity score 28 of 2.8 units were analyzed. While mESE and MIAF showed good agreement in the measurement of erosion size, the manual method with calculation by half-ellipsoid formula underestimated erosions size, particularly with larger erosions. Accurate segmentation is particularly important in larger erosions, which are irregularly shaped. In all three segmentation techniques irregular erosions were larger in size than regular erosions (MIAF: 19.7 vs. 3.4mm3; mESE: 15.5 vs. 2.3mm3; manual = 7.2 vs. 1.52mm3; all p < 0.001). In accordance, sphericity of erosions measured by MIAF significantly decreased with their size (p < 0.001). MIAF and mESE allow segmentation of inflammatory bone erosions in RA patients with excellent inter reader reliability. They allow calculating erosion volume independent of erosion shape and therefore provide an attractive tool to quantify structural damage in individual joints of RA patients.

Estudo epidemiológico sobre acidente vascular encefálico em uma clínica escola de fisioterapia

Objetivo identificar o perfil epidemiológico dos pacientes com sequelas de Acidente vascular encefálico que procuraram atendimento na Clínica Escola de Fisioterapia da UNICENTRO, no município de Guarapuava/PR. Métodos Coleta de dados dos prontuários arquivados pelo serviço de Fisioterapia no período de 2003 a 2012. Foi utilizado um roteiro de avaliação, elaborado pelas pesquisadoras. Resultados Os resultados evidenciaram que 54,25% dos indivíduos eram do sexo masculino, 56,60% eram casados e idade média foi de 62,37±14,48 anos para ambos os gêneros. Como fatores de risco, a hiperlipidemia, a hipertensão arterial sistêmica e o diabetes foram os mais evidentes, com valores de 90,57%, 76,42% e 25,94%, respectivamente. A maioria (53,18%) dos pacientes apresentou grau de dependência total ou parcial para realização das atividades de vida diária. Conclusão É preciso intensificar as ações de prevenção em saúde para que os fatores de risco sejam minimizados, visando a redução na incidência do AVE.

Multimedia material about velopharynx and primary palatoplasty for orientation of caregivers of children with cleft lip and palate.

The objective of this study was to describe the process of elaboration and evaluation of multimedia material for caregivers about velopharynx, speech, and primary palatoplasty in babies with cleft lip and palate. The elaboration of the material involved an interdisciplinary relationship between the fields of Speech Language Pathology and Audiology, Dentistry and Arts. The definition and execution of the following activities were based on the principles of art education involving the following: characterization of audience, characterization of content, identification and elaboration of illustrations, characterization of educational approach, elaboration of text and narratives, definition of audiovisual sequence, and video preparation. The material was evaluated with the participation of 41 caregivers of patients with cleft lip and palate involving the comparison between acquired knowledge using an evaluation script applied before and after presenting the material. An increase was observed in correct responses regarding the role of velopharynx and the importance of primary palatoplasty for speech. The multimedia was effective in optimizing the knowledge of caregivers, suggesting the importance of such material during orientation.

Collaborating All the Way to the Top

The classes I teach have a predictive modeling component. As a student, having participated in blind protein structure prediction competitions (CASP, http://predictioncenter.org) and data mining competitions like KDD Cup, I have implemented this form of competitions in my bioinformatics and data mining classes. This semester I extended this idea to a different class (Parallel Computing). Specifically, as part of an assignment (or final project) students have to train a predictive models to distinguish a specific class of proteins called "solenoids" using the available protein sequence information. As part of this competition, the truth-values are hidden from the students and they have to make a prediction (guess) and submit their results to the instructor. The instructor then evaluates the results using the truth-values and provides a ranking of the class students based on the predictive performance. The concepts introduced in class allow the students to build base line predictive models, but to improve performance, students have to research, think critically and come up with innovative solutions. In my past two implementations of this project, I have used an in-house evaluation script and requested participants to send me solutions via a simple web server. Both times, the assignment was run for a 4-week period. I have also used technologies like Kaggle to setup this competition. In the future, I would like to implement the competition for the duration of the semester. Students would be taught a concept, and they would implement the same towards improving their predictive models and engineer better solutions as new, advanced concepts are taught.I am also developing a model that allows students to achieve these projects in a collaborative fashion by enabling resources like Wikis and other tools. As such, this session will be an introduction to the tools used and how they could be adapted to general purpose classes

Integrated framework for monitoring groundwater pollution using a geographical information system and multivariate analysis

Appropriate assessment of groundwater is important to ensure sustainable and safe use of this natural resource. However, evaluating overall groundwater quality is difficult due to the spatial variability of multiple contaminants. This research proposes a geographical information system (GIS)-based groundwater quality pollution mapping technique, which synthesizes different available water quality data, normalized with the World Health Organization (WHO) standards. The normalized difference index (NDI) is used to perform the normalization process. This study utilizes a multi-criteria evaluation (MCE) script (MATLAB 10.0), developed to assign weights to each of the analysed water quality parameters. The consistency of judgments of weight assignment is further analysed using the consistency ratio (CR) and consistency index (CI) techniques. The Shuttle Radar Topography Mission (SRTM) C-band radar and Landsat TM satellite image data are used to derive a digital elevation model (DEM) and land-use/land-cover map of the area. A new sensitivity analysis method is introduced to estimate the responsible factors associated with the proposed groundwater pollution zone model (GPZM). Multivariate analysis methods, such as factor analysis (FA), cluster analysis (CA) and principal component analysis (PCA), are used to uncover the latent structure of the data, to understand the correlations across hierarchical levels, and for dimensionality reduction, respectively. Editor D. Koutsoyiannis; Associate editor Chong-yu Xu Citation Srivastava, P. K., Han, D., Gupta, M., and Mukherjee, S., 2012. Integrated framework for monitoring groundwater pollution using a geographical information system and multivariate analysis. Hydrological Sciences Journal, 57 (7), 1453–1472.

A study of machine-learning-based approaches to extract clinical entities and their assertions from discharge summaries

The authors' goal was to develop and evaluate machine-learning-based approaches to extracting clinical entities-including medical problems, tests, and treatments, as well as their asserted status-from hospital discharge summaries written using natural language. This project was part of the 2010 Center of Informatics for Integrating Biology and the Bedside/Veterans Affairs (VA) natural-language-processing challenge. The authors implemented a machine-learning-based named entity recognition system for clinical text and systematically evaluated the contributions of different types of features and ML algorithms, using a training corpus of 349 annotated notes. Based on the results from training data, the authors developed a novel hybrid clinical entity extraction system, which integrated heuristic rule-based modules with the ML-base named entity recognition module. The authors applied the hybrid system to the concept extraction and assertion classification tasks in the challenge and evaluated its performance using a test data set with 477 annotated notes. Standard measures including precision, recall, and F-measure were calculated using the evaluation script provided by the Center of Informatics for Integrating Biology and the Bedside/VA challenge organizers. The overall performance for all three types of clinical entities and all six types of assertions across 477 annotated notes were considered as the primary metric in the challenge. Systematic evaluation on the training set showed that Conditional Random Fields outperformed Support Vector Machines, and semantic information from existing natural-language-processing systems largely improved performance, although contributions from different types of features varied. The authors' hybrid entity extraction system achieved a maximum overall F-score of 0.8391 for concept extraction (ranked second) and 0.9313 for assertion classification (ranked fourth, but not statistically different than the first three systems) on the test data set in the challenge.