Open-source Data Science Research Articles

Development of an Algorithm to Evaluate the Quality of Geolocated Addresses in Urban Areas

The spatial and semantic data of geographic addresses are extremely important for citizens, governments, and companies. The addresses can georeference environmental, economic, security, health, and demographic parameters in urban areas. Additionally, address components can be used by users to locate any point of interest (POI) with location-based systems (LBSs). For this reason, errors in address data can affect the geographic location of events, map representations, and spatial analyses. Thus, this paper presents the development of an algorithm for evaluating the quality of semantic and geographic information in any geospatial address dataset. The reference datasets are accessible using open data platforms or spatial data infrastructure (SDI) and volunteered geographic information (VGI), and both have been compared with commercial datasets using geocoding web services. Address quality analysis was developed using several open-source data science code libraries combined with spatial databases and geographic information systems. In addition, the quality of geographic addresses was evaluated by carrying out normalized tests in accordance with International Geospatial Standards (ISO 19157). Finally, this methodology assesses the quality of authorized and VGI address datasets that can be used for geocoding any relevant information in specific urban areas.

Fatigue Damage estimation in vehicle from minimal instrumentation thanks to a mixed Engineering / Data Science approach

Monitoring damage and its potential causes in lab-tested structures requires extensive instrumentation that cannot be feasibly replicated in production assets. Since strain gauge instrumentation at a large scale is impractical, other proxy measurements for damage such as pressure, temperature or acceleration can be monitored and damage can be inferred from them. There is considerable difficulty in understanding not only which events lead to significant damage accumulation, but also in suitably and economically instrumenting assets in order to capture these data.We propose a general framework for exploring data which is inspired from the field of data science. Using a combination of engineering calculation packages and open-source data science tools, we show how engineers can further their understanding of the problem domain. We will present two case studies, each involving a different automotive component that is extensively instrumented with strain gauges, accelerometers and other sensors. We evaluate absolute damage from the strain data and, by using engineering indicators, data reduction, visualisations, correlations, we show how a minimal instrumentation subset can be identified for the purpose of damage approximation.

Enabling Collaborative Data Science Development with the Ballet Framework

While the open-source software development model has led to successful large-scale collaborations in building software systems, data science projects are frequently developed by individuals or small teams. We describe challenges to scaling data science collaborations and present a conceptual framework and ML programming model to address them. We instantiate these ideas in Ballet, the first lightweight framework for collaborative, open-source data science through a focus on feature engineering, and an accompanying cloud-based development environment. Using our framework, collaborators incrementally propose feature definitions to a repository which are each subjected to software and ML performance validation and can be automatically merged into an executable feature engineering pipeline. We leverage Ballet to conduct a case study analysis of an income prediction problem with 27 collaborators, and discuss implications for future designers of collaborative projects.

Hands-on training about overfitting.

Overfitting is one of the critical problems in developing models by machine learning. With machine learning becoming an essential technology in computational biology, we must include training about overfitting in all courses that introduce this technology to students and practitioners. We here propose a hands-on training for overfitting that is suitable for introductory level courses and can be carried out on its own or embedded within any data science course. We use workflow-based design of machine learning pipelines, experimentation-based teaching, and hands-on approach that focuses on concepts rather than underlying mathematics. We here detail the data analysis workflows we use in training and motivate them from the viewpoint of teaching goals. Our proposed approach relies on Orange, an open-source data science toolbox that combines data visualization and machine learning, and that is tailored for education in machine learning and explorative data analysis.

GENERATOR Breast DataMart-The Novel Breast Cancer Data Discovery System for Research and Monitoring: Preliminary Results and Future Perspectives.

Background: Artificial Intelligence (AI) is increasingly used for process management in daily life. In the medical field AI is becoming part of computerized systems to manage information and encourage the generation of evidence. Here we present the development of the application of AI to IT systems present in the hospital, for the creation of a DataMart for the management of clinical and research processes in the field of breast cancer. Materials and methods: A multidisciplinary team of radiation oncologists, epidemiologists, medical oncologists, breast surgeons, data scientists, and data management experts worked together to identify relevant data and sources located inside the hospital system. Combinations of open-source data science packages and industry solutions were used to design the target framework. To validate the DataMart directly on real-life cases, the working team defined tumoral pathology and clinical purposes of proof of concepts (PoCs). Results: Data were classified into “Not organized, not ‘ontologized’ data”, “Organized, not ‘ontologized’ data”, and “Organized and ‘ontologized’ data”. Archives of real-world data (RWD) identified were platform based on ontology, hospital data warehouse, PDF documents, and electronic reports. Data extraction was performed by direct connection with structured data or text-mining technology. Two PoCs were performed, by which waiting time interval for radiotherapy and performance index of breast unit were tested and resulted available. Conclusions: GENERATOR Breast DataMart was created for supporting breast cancer pathways of care. An AI-based process automatically extracts data from different sources and uses them for generating trend studies and clinical evidence. Further studies and more proof of concepts are needed to exploit all the potentials of this system.

Technical Perspective:

Entity matching (EM) has been a long-standing challenge in data management. Most current EM works focus only on developing matching algorithms. We argue that far more efforts should be devoted to building EM systems. We discuss the limitations of current EM systems, then describe Magellan, a new kind of EM system. Magellan is novel in four important aspects. (1) It provides how-to guides that tell users what to do in each EM scenario, step by step. (2) It provides tools to help users execute these steps; the tools seek to cover the entire EM pipeline, not just blocking and matching as current EM systems do. (3) Tools are built into the Python open-source data science ecosystem, allowing Magellan to borrow a rich set of capabilities in data cleaning, IE, visualization, learning, etc. (4) Magellan provides a powerful scripting environment to facilitate interactive experimentation and quick "patching" of the system. We describe research challenges and present extensive experiments that show the promise of the Magellan approach.

Technical Perspective:

Entity matching (EM) has been a long-standing challenge in data management. Most current EM works focus only on developing matching algorithms. We argue that far more efforts should be devoted to building EM systems. We discuss the limitations of current EM systems, then describe Magellan, a new kind of EM system. Magellan is novel in four important aspects. (1) It provides how-to guides that tell users what to do in each EM scenario, step by step. (2) It provides tools to help users execute these steps; the tools seek to cover the entire EM pipeline, not just blocking and matching as current EM systems do. (3) Tools are built into the Python open-source data science ecosystem, allowing Magellan to borrow a rich set of capabilities in data cleaning, IE, visualization, learning, etc. (4) Magellan provides a powerful scripting environment to facilitate interactive experimentation and quick "patching" of the system. We describe research challenges and present extensive experiments that show the promise of the Magellan approach.