Schema Mapping Research Articles

Overview on Data Ingestion and Schema Matching

This overview traced the evolution of data management, transitioning from traditional ETL processes to addressing contemporary challenges in Big Data, with a particular emphasis on data ingestion and schema matching. It explored the classification of data ingestion into batch, real-time, and hybrid processing, underscoring the challenges associated with data quality and heterogeneity. Central to the discussion was the role of schema mapping in data alignment, proving indispensable for linking diverse data sources. Recent advancements, notably the adoption of machine learning techniques, were significantly reshaping the landscape. The paper also addressed current challenges, including the integration of new technologies and the necessity for effective schema matching solutions, highlighting the continuously evolving nature of schema matching in the context of Big Data

ANALYSIS OF MODERN DATABASES

In the current digital landscape, handling the vast and varied data requires customized processing and storage methods. Within a system, data within the same domain often uses different models, underscoring the importance of multi-model databases that can accommodate diverse data structures. These databases typically differentiate between primary and secondary models, necessitating clear data schema mappings. While the traditional relational data model persists as a well-studied and widely used approach, graph data models are gaining traction in areas like social networks, recommendation systems, and transportation networks. This article offers insights into the pros and cons of graph databases, providing a clearer understanding of their role in modern data management. Keywords: Graph databases, Database architecture, Graph-based data management, Multi-model DBMS, Relational databases.

A heterogeneous 3-D stacked PIM accelerator for GCN-based recommender systems

AbstractModern recommendation systems integrate graph convolution neural networks (GCN) for enhancing embedding representation. Compared with widely deployed neural network-based models, the extra message propagation layer of GCN-based recommendation is featured with extensive computations and irregular memory access. However, architecture designs for prevailing deep neural network recommendation models assume simple pooling in the embedding layer. ReRAM-based GCN accelerators are specialized for graph-related operations. However, they are designed for general graphs, while GCN-based recommendation models mainly operate on the user-item graph. In this paper, we proposed a resistive random accessed memory (ReRAM) based processing-in-memory (PIM) accelerator, ReGCNR, for GCN-based recommendation. ReGCNR is featured with three key innovations. First, we exploit the 3-dimensional (3-D) stacked heterogeneous ReRAM to fit with the large-size embedding table and user-item graph. Then, we propose a joint degree mapping schema that maximizes the efficiency of the execution pipeline. After that, ReGCNR assembles a well-coordinated pipeline and hardware scheduling design to boost overall system performance. Results show that ReGCNR outperforms GPU by 69.83$$\times$$ × and 56.67$$\times$$ × in terms of average speedup and energy saving, respectively. In addition, ReGCNR outperforms state-of-the-art ReRAM-based solutions by 11.13$$\times$$ × speedups and 7.22$$\times$$ × energy savings on average.

Bedrock Geologic Database of the Indiana Portions of the Princeton, Evansville, Mount Vernon, and West Frankfort 30- x 60-minute Quadrangles

Located in the eastern Illinois Basin, the Princeton 1:100,000-scale quadrangle comprises portions of Knox, Gibson, Pike, Dubois, Posey, Vanderburgh, Warrick, and Spencer Counties in Indiana as well as small portions of Edwards, Gallatin, Wabash, and White Counties in Illinois. The mapped area is situated in the rolling topography of the Wabash Lowland and dominated by Pennsylvanian-age coal-bearing strata. Pennsylvanian siliciclastics in the Princeton quadrangle host economically important coal seams. The most widespread coal beds occur in the Carbondale Group, but the Raccoon Creek Group hosts coal of superior quality (low sulfur, low ash) in Indiana. Surface and underground mining of these coals have been extensive, with expansive impacts on the landscape and subsequent reclamation. Bedrock outcrops are obscured mainly by glacial sediments and extensive fluvial deposits from the Wabash River that formed deep channels (paleo and active) filled with outwash and alluvium. The geologic map of Gray and others (1970), the Compendium of Paleozoic Rock-Unit Stratigraphy in Indiana (1986), and Indiana Geologic Names Information System (IGNIS, https://legacy.igws.indiana.edu/ignis/) are the fundamental compilation of previous geological knowledge from this area. Additional data come from previous 1:24,000-scale mapping through the National Cooperative Mapping Program STATEMAP component. However, most information for the Bedrock Geologic Database of the Indiana Portions of the Princeton, Evansville, Mount Vernon, and West Frankfort 30- x 60-minute Quadrangles and, more broadly, the western corridor, is derived from Indiana-scale coal seam mapping conducted between 2011 and 2017 based on the National Coal Resources Data System (NCRDS), Indiana Coal Quality Database (ICQD), and geophysical logs from oil and gas wells archived in the Petroleum Database Management System (PDMS). The majority of these data were previously presented in Esri-based Indiana Coal Atlas StoryMaps for the entire western corridor. This geologic mapping project was funded in part through the STATEMAP Program supported by the U.S. Geological Survey under Cooperative Agreement No. G22AC00424. The Bedrock Geologic Database of the Indiana Portions of the Princeton, Evansville, Mount Vernon, and West Frankfort 30- x 60-minute Quadrangles is a composite geodata set that conforms to "GeMS (Geologic Map Schema)--a standard format for the digital publication of geologic maps," available at http://ngmdb.usgs.gov/Info/standards/GeMS/. Metadata records associated with each element within the geodata set contain more detailed descriptions of their purposes, constituent entities, and attributes. An OPEN shapefile version of the data set is also available. It consists of shapefiles, DBF files, and delimited text files and retains all information in the native geodatabase, but some programming will likely be necessary to assemble these components into usable formats.

Analysis of the Path of Improving English Listening and Speaking Ability Based on Big Data Technology

Abstract The training of college students’ English listening and speaking skills needs to be individualized scientifically and strategically in order to get a qualitative improvement. In this paper, we first analyze the correlation between students’ listening and speaking skill levels by using the improved maximum information coefficient (MIC) with the aid of an online platform. The students’ listening ability is assessed by the plain Bayesian algorithm, trained by the neural network to reduce the error, and the similarity between the feature vector and each pattern is calculated so that the element that obtains the highest value is used as the candidate mapping. Next, the schema mapping of data distribution (SMDD) method is combined to obtain the best matching schema. Finally, the effectiveness of English listening and speaking training with the Smart Teaching Platform was investigated through controlled experiments. The results show that in the post-test speech scores, the experimental group’s increase in speech scores was 2.322, while the control group’s increase was only 0.652. The effectiveness of the English listening and speaking ability training strategy is evident due to the obvious difference. This study proposes rational suggestions for improving college students’ English listening and speaking abilities.

Geologic Map of the Bedford 30- x 60-minute Quadrangle, Indiana

The Geologic Map of the Bedford 30- x 60-minute Quadrangle shows the Mississippian and lower Pennsylvanian bedrock units and the overlying Pleistocene and Holocene deposits distributed over eight counties. The mapped area includes five physiographic divisions of the Southern Hills and Lowlands Region in south-central Indiana. Bedrock units and unconsolidated deposits were characterized by outcrop field observations, and new and archived borehole data. Bedrock data was consolidated into a single mapping database, which was then processed using gridding and contouring tools in ArcGIS Pro, resulting in rock formation contact lines, mappable at 1:100,000 scale. Additional information used to delineate contacts for unconsolidated deposits was based upon Indiana Department of Natural Resources water well database and soils data from the Soil Survey Geographic (SSURGO) database. Light detection and ranging (LiDAR) digital elevation models were also used to identify landforms and refine contacts between surficial units. The mapped bedrock units, from east to west, range in age from the Borden Group (lower Mississippian) to the Raccoon Creek Group (Lower Pennsylvanian) with the strike oriented generally from northwest to southeast. Quaternary deposits are found in river and stream valleys and filling the Wabash and Scottsburg lowlands with glacial deposits. Quaternary sediments were mostly deposited during the last two major advances of the Laurentide Ice Sheet into south-central Indiana. Brief summaries of all mapped bedrock units and unconsolidated deposits are listed on the map sheet, with more detailed descriptions in the accompanying pamphlet. Bedrock survey drill hole data points are also displayed on the map with corresponding links to expanded drill hole information for each of the survey drill holes listed in the pamphlet. In addition to the map and pamphlet, a composite spatial data set that conforms to the standardized database schema known as GeMS (Geologic Map Schema) is also available for download from this site. Both ESRI-licensed geodatabase and open shapefile versions of the spatial data set are available for download. Metadata records associated with each element within the spatial data set contain detailed descriptions of their purpose, constituent entities, and attributes. This geologic map was funded in part through the STATEMAP program supported by the U.S. Geological Survey under Cooperative Agreement No. G21AC10646.

Quaternary Geology of the Washington-Jasper Area

"Quaternary Geology of the Washington-Jasper Area" was created to present basic quaternary geologic information that contributes to the characterization of potential aggregate resources, characterization of bedrock aquifer systems, and analysis of the overlying predominantly glacial deposits. This map is based on data compiled from project field observations and project exploratory borings, complemented with IGWS cartographic (e.g., Gray, 1989) and archive borehole data, select Indiana Department of Natural Resources water well logs, and United States Department of Agriculture soil survey maps. Expert knowledge of fluvial, glacifluvial and ice-marginal environments was used to draw contacts between units. This geologic map was funded in part through the STATEMAP program supported by the U.S. Geological Survey under Cooperative Agreement No. G21AC10646. "Quaternary Geology of the Washington-Jasper Area" is a composite geodata set that conforms to "GeMS (Geologic Map Schema)--a standard format for the digital publication of geologic maps," available at http://ngmdb.usgs.gov/Info/standards/GeMS/. Metadata records associated with each element within the geodata set contain more detailed descriptions of their purposes, constituent entities, and attributes. An OPEN shapefile version of the data set is also available. It consists of shapefiles, DBF files, and delimited text files and retains all information in the native geodatabase, but some programming will likely be necessary to assemble these components into usable formats.

Bedrock Geologic Database of the Vincennes 30- x 60-Minute Quadrangle, Indiana

Located in the eastern Illinois Basin, the Vincennes 1:100,000-scale quadrangle comprises portions of Daviess, Knox, Sullivan, and Greene Counties in Indiana. It is situated in the rolling topography of the Wabash Lowland and dominated by Pennsylvanian-age coal-bearing strata. Pennsylvanian siliciclastics in the Vincennes quadrangle host economically significant coal seams. The most widespread coal beds occur in the Carbondale Group, but the Raccoon Creek Group hosts superior coal quality (low sulfur, low ash) in Indiana. Both surface and underground mining of these coals have been extensive, with expansive impacts on the landscape and subsequent reclamation. Bedrock outcrop is largely obscured by multiple cycles of glacial sediments, and extensive fluvial activity of the Wabash River has formed deep channels (paleo and active) filled with outwash and alluvium. The geological map of Gray and others (1970) is the fundamental compilation of previous geological knowledge from this area. Additional geological information for the Vincennes quadrangle comes from previous 1:24k-scale mapping through the STATEMAP program. Specifically, Fraser and others (1998) conducted bedrock mapping in the 7.5-minute Wheatland and Washington quadrangles, and Fraser and Zlotin (1997) did some mapping in the 7.5-minute Montgomery quadrangle. Conolly and others (1996) performed a coal availability study in the 7.5-minute Wheatland quadrangle. However, the most information for the 1:100k Vincennes quadrangle and, more broadly, the western corridor, comes from the Indiana-scale coal seam mapping conducted between 2011 and 2017 based on the Indiana Coal Stratigraphic Database (NCRDS) and Indiana Coal Quality Database (ICQD), and geophysical logs from petroleum wells archived in the Petroleum Database Management System (PDMS). These data were previously presented in Esri-based Indiana Coal Atlas StoryMaps for the entire western corridor (Drobniak and others, 2017a, 2017b, 2017c, 2017d). This geologic map was funded in part through the STATEMAP Program supported by the U.S. Geological Survey under Cooperative Agreement No. G21AC10646. The Bedrock Geologic Database of the Vincennes 30- x 60- Minute Quadrangle is a composite geodata set that conforms to "GeMS (Geologic Map Schema)--a standard format for the digital publication of geologic maps," available at http://ngmdb.usgs.gov/Info/standards/GeMS/. Metadata records associated with each element within the geodata set contain more detailed descriptions of their purposes, constituent entities, and attributes. An OPEN shapefile version of the data set is also available. It consists of shapefiles, DBF files, and delimited text files and retains all information in the native geodatabase, but some programming will likely be necessary to assemble these components into usable formats.

2021 GIS Data for Bedrock Geologic Map of the Indianapolis 30 x 60 Minute Quadrangle, Central Indiana

The 2021 GIS Data for Bedrock Geologic Map of the Indianapolis 30 x 60 Minute Quadrangle, Central Indiana is an Esri File Geodatabase that contains two feature data sets and four geodatabase tables detailing the bedrock geology of the Indianapolis 30 x 60 Minute Quadrangle in central Indiana. This data set conforms to "GeMS (Geologic Map Schema)--a standard format for the digital publication of geologic maps." For more information on GeMS, please refer to the supplemental information within this metadata. This was a conversion from paper map to GeMS-compliant GIS data. (Hasenmueller, W. A., and James, C. L., 2003, Bedrock geologic map of the Indianapolis 30 X 60 minute quadrangle, central Indiana: Indiana Geological Survey Open-File Study 03-07) The Bedrock Geologic Map of the Indianapolis 30 x 60 Minute Quadrangle, Central Indiana was created to present basic bedrock geologic information that contributes to the characterization of potential aggregate resources, characterization of bedrock aquifer systems, and analysis of the overlying predominantly glacial deposits. This map is based on data obtained from several thousand records including petroleum well drillers' logs, geophysical logs, water well drillers' logs, descriptions of cores recovered by the Indiana Geological and Water Survey, seismic refraction records collected by the Indiana Geological and Water Survey, natural exposures in and near the map area, and exposures in active and abandoned quarries. This database is, in large part, the result of a cooperative mapping agreement between the U.S. Geological Survey (USGS) and the Indiana Geological and Water Survey through the STATEMAP program of the USGS. The Bedrock Geologic Map of the Indianapolis 30 x 60 Minute Quadrangle, Central Indiana is a composite geodata set that conforms to "GeMS (Geologic Map Schema)--a standard format for the digital publication of geologic maps," available at http://ngmdb.usgs.gov/Info/standards/GeMS/. Metadata records associated with each element within the geodata set contain more detailed descriptions of their purposes, constituent entities, and attributes. An OPEN shapefile version of the data set is also available. It consists of shapefiles, DBF files, and delimited text files and retains all information in the native geodatabase, but some programming will likely be necessary to assemble these components into usable formats.

Parker: Data fusion through consistent repairs using edit rules under partial keys

Data integration is the problem of consolidating information provided by multiple sources. After schema mapping and duplicate detection have been dealt with, the problem consists in fusing disparate data that comes from the sources. In this work, we treat this problem as a data cleaning problem where consistency within and between sources is modelled by respectively edit rules and a partial key. The combination of these constraints leads to the definition of edit rules under a partial key (EPKs). We show that we can adapt the well-known set cover method for edit rules to the setting of EPKs and this yields an efficient algorithm to find minimal-cost repairs. These repairs then provide solutions for the initial data fusion problem. The methodology is implemented in a repair engine called Parker. Empirical results show that Parker is several orders of magnitude faster than state-of-the-art repair tools. At the same time, the quality of the repairs in terms of F1-score ranges from comparable to better compared to these tools.

Biomedical heterogeneous data categorization and schema mapping toward data integration.

Data integration is a well-motivated problem in the clinical data science domain. Availability of patient data, reference clinical cases, and datasets for research have the potential to advance the healthcare industry. However, the unstructured (text, audio, or video data) and heterogeneous nature of the data, the variety of data standards and formats, and patient privacy constraint make data interoperability and integration a challenge. The clinical text is further categorized into different semantic groups and may be stored in different files and formats. Even the same organization may store cases in different data structures, making data integration more challenging. With such inherent complexity, domain experts and domain knowledge are often necessary to perform data integration. However, expert human labor is time and cost prohibitive. To overcome the variability in the structure, format, and content of the different data sources, we map the text into common categories and compute similarity within those. In this paper, we present a method to categorize and merge clinical data by considering the underlying semantics behind the cases and use reference information about the cases to perform data integration. Evaluation shows that we were able to merge 88% of clinical data from five different sources.

Fairness-aware Data Integration

Machine learning can be applied in applications that take decisions that impact people’s lives. Such techniques have the potential to make decision making more objective, but there also is a risk that the decisions can discriminate against certain groups as a result of bias in the underlying data. Reducing bias, or promoting fairness, has been a focus of significant investigation in machine learning, for example, based on pre-processing the training data, changing the learning algorithm, or post-processing the results of the learning. However, prior to these activities, data integration discovers and integrates the data that is used for training, and data integration processes have the potential to produce data that leads to biased conclusions. In this article, we propose an approach that generates schema mappings in ways that take into account: (i) properties that are intrinsic to mapping results that may give rise to bias in analyses; and (ii) bias observed in classifiers trained on the results of different sets of mappings. The approach explores a space of different ways of integrating the data, using a Tabu search algorithm, guided by bias-aware objective functions that represent different types of bias.The resulting approach is evaluated using Adult Census and German Credit datasets to explore the extent to which and the circumstances in which the approach can increase the fairness of the results of the data integration process.

Conjunctive Queries: Unique Characterizations and Exact Learnability

We answer the question of which conjunctive queries are uniquely characterized by polynomially many positive and negative examples and how to construct such examples efficiently. As a consequence, we obtain a new efficient exact learning algorithm for a class of conjunctive queries. At the core of our contributions lie two new polynomial-time algorithms for constructing frontiers in the homomorphism lattice of finite structures. We also discuss implications for the unique characterizability and learnability of schema mappings and of description logic concepts.

Metadata Integration Framework for Data Integration of Socio-Cultural Anthropology Digital Repositories: A Case Study of Princess Maha Chakri Sirindhorn Anthropology Centre

Data integration is one of the most challenging tasks for digital collections whose data are stored across various repositories. Data integration across digital repositories has several challenges. First, data heterogeneity in terms of data schema and data values usually occurs across diverse data sources. Second, heterogeneity in data representation and semantic issues are among the problems. The same data may appear in different repositories with varied data representations, i.e., metadata schema. Recent research has focused on matching several related metadata schemas. In this paper, a metadata integration framework is proposed to support digital repositories in socio-cultural anthropology at the Princess Maha Chakri Sirindhorn Anthropology Centre (SAC), Thailand. The proposed framework is defined based on the Metadata Lifecycle Model (MLM). It utilizes non-procedural schema mappings to express data relationships in diverse schemas. A case study of metadata integration over the SAC digital repositories was conducted to validate the framework. The SAC common metadata schema was designed to support data mapping across 13 digital repositories. The SAC “One Search” system was developed to exemplify the system implementation of the framework. Evaluation results showed that the proposed metadata integration framework can support domain experts in socio-cultural anthropology in unified searching across the repositories.

Semantic Integration of Heterogeneous Data Sources Using Ontology-Based Domain Knowledge Modeling for Early Detection of COVID-19.

The enormous outbreak of biomedical knowledge, the aim of reducing computation and processing costs and the widespread availability of internet connection have created a profuse amount of electronic data. Such data are stored across the globe in various data sources that are semantically, structurally and syntactically different. This decentralized nature of biomedical data has made it difficult to obtain a unified view of the data. Data integration plays a crucial role in enhancing access to heterogeneous data making the retrieval easier and faster. A variety of ontology, machine learning, deep learning and fuzzy logic-based solutions are being developed for heterogeneous data integration. The proposed model concentrates on the automatic ontology-based data integration method that can be effectively deployed and used in the healthcare domain. The proposed model is divided into three phases. The first phase includes the automatic mapping of data and generation of local ontology across heterogeneous data sources, the second phase combines the local ontology models developed in the first phase to create a root global schema mapping and the third phase queries diverse databases to retrieve semantically analogous records. The model is created based on the medical records, chest X-ray details and COVID-19 symptom questionnaire data of various patients distributed across three data sources (SQL, mongodb and excel). Based on the data, the patients who have moderate/higher risk of developing serious illness from COVID-19 are retrieved.

Designing a professional e-portfolio to promote 3D printing-based entrepreneurial thinking as a case study: A technical writing project

The purpose of this project is to design a professional e-portfolio in the form of an interactive website that could be potentially accessed by budding entrepreneurs looking to get information to start a 3D-printing based business locally in the Aizu/Fukushima area. 3D printing here is used as a case study to understand how such technical content could be positioned most optimally in a professional context and with maximum impact. This paper outlines a process that leads towards the final design of the website and will focus on the technical writing, design, and usability principles. Adoption of such principles is expected to make the website more interactive and usable in the target situation. This website is geared towards helping anyone who wants to start a new 3D printing business. In this research, we will limit our consideration to help people living in Aizu to revitalise the Aizu area for the 3D printing business. The content in the e-portfolio includes types of 3D printers, 3D printing technologies, sharing of 3D models, with the web design based on various types of concept mapping schemas, faceted search, SWOT analysis, etc. From this information, users can search for and browse what they want towards customising their preferences. Users can also contact the administrator for business consultations and website requests for start-ups. This is a work-in-progress paper, as the design of the e-portfolio has been left at a preliminary level, with more focus on content placement and organisation, but without any usability testing.

Bedrock Geologic Map of the Northern Half of the Bedford 30- X 60-Minute Quadrangle, Indiana

The Bedrock Geologic Map of the Northern Half of the Bedford 30- X 60-Minute Quadrangle is an Esri File Geodatabase that contains six feature data sets, five geodatabase tables, and two raster data sets detailing the bedrock geology of the northern half of the Bedford 30- X 60-minute quadrangle in Indiana. This data set conforms to "GeMS (Geologic Map Schema)--a standard format for the digital publication of geologic maps." For more information on GeMS please refer to the supplemental information within this metadata.

Schema mapping generation in the wild

Schema mappings enable declarative and executable specification of transformations between different schematic representations of application concepts. Most work on mapping generation has assumed that the source and target schemas are well defined, e.g., with declared keys and foreign keys, and that the mapping generation processes exist to support the data engineer in the labour-intensive process of producing a high-quality integration. However, organizations increasingly have access to numerous independently produced datasets, e.g., in a data lake, with a requirement to produce rapid, best-effort integrations, without extensive manual effort. As a result, there is a need to generate mappings in settings without declared relationships, and thus on the basis of inferred profiling data, and over large numbers of sources. Our contributions include a dynamic programming algorithm for exploring the space of potential mappings, and techniques for propagating profiling data through mappings, so that the fitness of candidate mappings can be estimated. The paper also describes how the resulting mappings can be used to populate single and multi-relation target schemas. Experimental results show the effectiveness and scalability of the approach in a variety of synthetic and real-world scenarios.

Systematic Comparison of Data Models used in Mapping Knowledge Organization Systems

AbstractThis study explores six projects in mapping Knowledge Organization Systems (schemas, taxonomies, ontologies), and examines the data models used in these projects. We discuss (1) the two most common data models presented in these projects, namely tabular model, and tree model; (2) the implicit and explicit information shown in the data models; and (3) the information gains and losses of each model. We found that though the tree model maintains the hierarchical structure of the KOS while a tabular model flattens out the structure, both models can potentially obscure the information of the original KOS. This makes it problematic if we want to reconstruct a mapping project based on the outcomes manifested in either model. Our findings contribute to the Knowledge Organization literature by showing how choices of data models can impact the results in mapping schemas, taxonomies, and ontologies.

The visualization of 3D nuclear radiation dose field

The nuclear energy is widely used as a new energy source due to the increasing depletion of fossil fuels. In the development of nuclear energy and technology, people are paying more and more attention to the issue of nuclear radiation protection. As a result, advanced computer technology has become a new way to solve nuclear radiation safety issues. This paper proposes to utilize CUDA-based ray-casting volume rendering to build an interactive nuclear radiation visualization system to solve the nuclear radiation safety problems, such as the judgment of radiation dose leakage and the planning of construction path. The system includes three modules, which are the overall dose rate distribution display of multiple power plants, the dose view of a specific equipment in power plants, and the dynamic evolution of nuclear radiation leakage. The analysis and user feedbacks of the experimental results show that this interactive visualization system can present the nuclear dose distribution of power plant in real time and from different granularities. This system utilizes the color mapping schema that as the dose density increases, the color changes from green to red, which is in line with human visual perception. Meanwhile, the dynamic radiation leakage scenario view is assistant to the design of nuclear radiation protection.