Database Techniques Research Articles

Geospatial Operations of Discrete Global Grid Systems—a Comparison with Traditional GIS

As the foundation of the next-generation Digital Earth, Discrete Global Grid Systems (DGGS) have demonstrated both theoretical and practical development, with a variety of state-of-the-art implementations proposed. These emerging DGGS platforms or libraries support preliminary operations such as quantization, cell-level navigation, and conversion between cell addresses and geographical coordinates, while leaving the other more complicated functions unexplored. This paper discusses the functional operations in a DGGS environment, including the essential operations defined by the Open Geospatial Consortium (OGC) Abstract Specification, and the extended operations potentially supported by DGGS. The extended operations are discussed in comparison to the traditional GIS, from the aspects of database techniques, data pre-processing and manipulation, spatial analysis and data interpretation, data computation, and data visualization. It was found that with the OGC-required operations and pre-processing operations as the baseline of development, some function algorithms can facilitate the algorithm development of other analytical functions. Several future research directions regarding the data modeling uncertainties, extended analytic algorithm development, and database and computation technologies are presented. This paper provides a comparison between DGGS and traditional GIS operations and can serve as a reference for future DGGS operation development.

Data Mining and Machine Learning Techniques for Aerodynamic Databases: Introduction, Methodology and Potential Benefits

Machine learning and data mining techniques are nowadays being used in many business sectors to exploit the data in order to detect trends, discover certain features and patters, or even predict the future. However, in the field of aerodynamics, the application of these techniques is still in the initial stages. This paper focuses on exploring the benefits that machine learning and data mining techniques can offer to aerodynamicists in order to extract knowledge from the CFD data and to make quick predictions of aerodynamic coefficients. For this purpose, three aerodynamic databases (NACA0012 airfoil, RAE2822 airfoil and 3D DPW wing) have been used and results show that machine-learning and data-mining techniques have a huge potential also in this field.

Detailed Study of Clustering Technique In Data Mining with Principle of Data Mining

Clustering technique in data mining is a main approach to deal with the data an extraction of useful patterns and knowledge from it. Clustering is involved in the datamining process. Datamining is the way of pulling out the knowledge, information, useful patterns and a reliable data from a huge gigantic amount of raw data as per the needs of the targeted sector. In technical aspects the Data Mining is a way of finding out the useful patterns from the raw data by using the suitable techniques of statistics, Machine learning, and Database techniques. Data mining target two major aspects of extraction of meaning full pattern data for concern of large-scale for better understanding of shapes and profitable patterns of data which impacts globally and the other is small-scale which deals with the lesser impact on the global scale. This paper give a brief overview of Clustering technique under the Data mining process their features and functionality. Majorly concentrate on Clustering technique and their algorithms with the pro’s & con’s and understand the need of clustering and its importance in Data mining process. The Data mining principle is also explained briefly just to build a base to understand the techniques and their importance which has to be discussed

Database techniques for resilient network monitoring and inspection

Network connection logs have long been recognized as integral to proper network security, maintenance, and performance management. This paper provides a development of distributed systems and write optimized databases: However, even a somewhat sizable network will generate large amounts of logs at very high rates. This paper explains why many storage methods are insufficient for providing real-time analysis on sizable datasets and examines database techniques attempt to address this challenge. We argue that sufficient methods include distributing storage, computation, and write optimized datastructures (WOD). Diventi, a project developed by Sandia National Laboratories, is here used to evaluate the potential of WODs to manage large datasets of network connection logs. It can ingest billions of connection logs at rates over 100,000 events per second while allowing most queries to complete in under one second. Storage and computation distribution are then evaluated using Elastic-search, an open-source distributed search and analytics engine. Then, to provide an example application of these databases, we develop a simple analytic which collects statistical information and classifies IP addresses based upon behavior. Finally, we examine the results of running the proposed analytic in real-time upon broconn (now Zeek) flow data collected by Diventi at IEEE/ACM Supercomputing 2019.

NEMA: Automatic Integration of Large Network Management Databases

Network management, whether for malfunction analysis, failure prediction, performance monitoring and improvement, generally involves large amounts of data from different sources. To effectively integrate and manage these sources, automatically finding semantic matches among their schemas or ontologies is crucial. Existing approaches on database matching mainly fall into two categories. One focuses on the schema-level matching based on schema properties such as field names, data types, constraints and schema structures. Network management databases contain massive tables (e.g., network products, incidents, security alert and logs) from different departments and groups with nonuniform field names and schema characteristics. It is not reliable to match them by those schema properties. The other category is based on the instance-level matching using general string similarity techniques, which are not applicable for the matching of large network management databases. In this article, we develop a matching technique for large NEtwork MAnagement databases (NEMA) deploying instance-level matching for effective data integration and connection. We design matching metrics and scores for both numerical and non-numerical fields and propose algorithms for matching these fields. The effectiveness and efficiency of NEMA are evaluated by conducting experiments based on ground truth field pairs in large network management databases. Our measurement on large databases with 1,458 fields, each of which contains over 10 million records, reveals that NEMA can achieve accuracy of 95%. We further compare with several other existing algorithms, and show that NEMA outperforms them by 7%-15% in numerical matching and achieves the best trade-off for non-numerical matching.

A time-field search method for AE source location with a regional velocity model

A time-field search method was proposed for AE event source location, combining the grid search method and the database technique. Theoretical process was given, and laboratory hydraulic fracture stimulation experiments under true-triaxial confining stresses were performed to prove the practicability of the method. A regional velocity model with variable wave velocities and directionality was required to fit observations. The location results showed that time-field location method obtains the global optimal source location solution with a relatively high efficiency and accuracy.

Orca-SR

Reconstructing a high dimensional unknown signal, using lower dimensional observations is a challenging problem, known as signal reconstruction problem (SRP), with diverse applications including network traffic engineering, medical image reconstruction, and astronomy. Recently the database community has shown significant advancements in solving the SRP problem efficiently, effectively, and in scale by leveraging database techniques such as similarity joins. In this demo, we demonstrate Orca-SR that highlights the benefits of signal reconstruction in scale by demonstrating real-time network traffic flow analysis on large networks that were not possible before. Orca-SR is a web application that enables a user to generate network flow and load the network for interactive analysis of the impact of different traffic patterns on signal reconstruction.

Electronics and Its Worldwide Research

The contributions of researchers at a global level in the journal Electronics in the period 2012–2020 are analyzed. The objective of this work is to establish a global vision of the issues published in the Electronic magazine and their importance, advances and developments that have been particularly relevant for subsequent research. The magazine has 15 thematic sections and a general one, with the programming of 385 special issues for 2020–2021. Using the Scopus database and bibliometric techniques, 2310 documents are obtained and distributed in 14 thematic communities. The communities that contribute to the greatest number of works are Power Electronics (20.13%), Embedded Computer Systems (13.59%) and Internet of Things and Machine Learning Systems (8.11%). A study of the publications by authors, affiliations, countries as well as the H index was undertaken. The 7561 authors analyzed are distributed in 87 countries, with China being the country of the majority (2407 authors), followed by South Korea (763 authors). The H-index of most authors (75.89%) ranges from 0 to 9, where the authors with the highest H-Index are from the United States, Denmark, Italy and India. The main publication format is the article (92.16%) and the review (5.84%). The magazine publishes topics in continuous development that will be further investigated and published in the near future in fields as varied as the transport sector, energy systems, the development of new broadband semiconductors, new modulation and control techniques, and more.

Database Meets Artificial Intelligence: A Survey

Database and Artificial Intelligence (AI) can benefit from each other. On one hand, AI can make database more intelligent (AI4DB). For example, traditional empirical database optimization techniques (e.g., cost estimation, join order selection, knob tuning, index and view selection) cannot meet the high-performance requirement for large-scale database instances, various applications and diversified users, especially on the cloud. Fortunately, learning-based techniques can alleviate this problem. On the other hand, database techniques can optimize AI models (DB4AI). For example, AI is hard to deploy in real applications, because it requires developers to write complex codes and train complicated models. Database techniques can be used to reduce the complexity of using AI models, accelerate AI algorithms and provide AI capability inside databases. Thus both DB4AI and AI4DB have been extensively studied recently. In this article, we review existing studies on AI4DB and DB4AI. For AI4DB, we review the techniques on learning-based configuration tuning, optimizer, index/view advisor, and security. For DB4AI, we review AI-oriented declarative language, AI-oriented data governance, training acceleration, and inference acceleration. Finally, we provide research challenges and future directions.

Simulation and health monitoring of a pressure regulating station

Pressure regulating stations (PRS) are critical assets of gas distribution networks. Therefore, it is important that they are closely monitored, and potential faults are identified and rectified before the supply of gas is adversely affected. In this work, we have developed a rigorous framework for continuous health monitoring of the PRS. We first develop a mathematical model that can accurately mimic the behavior of the PRS. The physical model for a PRS improves on existing works in the literature by incorporating a real gas law through a metamodel for the real gas equation using artificial neural network. We have highlighted various potential health issues in a PRS and proposed several automated strategies to monitor them. The proposed framework uses both model-based as well as data-based techniques depending on the nature of available operational data. We have illustrated the potential of our framework using a case study on a PRS with 1 year of simulated data.

Application of knowledge discovery in database (KDD) techniques in cost overrun of construction projects

Currently, cost overrun is a global challenge to completing construction projects successfully. To overcome this problem, earlier studies investigated factors of cost overrun. Knowledge Discovery in Data (KDD) and data mining techniques have been implemented effectively in various research areas to extract novel and valuable knowledge from historical data but have only recently been implemented in the construction industry. The aim of this research is to develop a model that predicts project cost overrun using a suitable data mining technique and cost overrun factors as predictors. A review of the literature identified twelve factors that can be easily measured and analyzed in construction projects. A case study was performed to validate the model with an actual data set of executed projects. The resulting model is simple, interpretable, and relatively accurate (60.87%), and it uses three steps of data mining – clustering, feature selection, and classification. These steps improve model performance.

Indexing Object Database using HC Tree

One of the recent applications of object technology is in the area of databases. One of the stumbling blocks in the commercial development and deployment of object databases is the lack of an efficient indexing technique. The properties of object databases make the task of development of an indexing technique all the more difficult. This paper discusses the development of an indexing technique for object databases. A new indexing technique based on a new structure, HC-Tree has been proposed. Performance analysis has been conducted, and experimental and analytical results indicate that the HC-Tree is an efficient indexing structure for object databases. The performance of the HC-Tree has also been compared with that of the other popular existing techniques - CH-Tree, H-Tree and hcC-Tree.

Condition prediction for oil-immersed cellulose insulation in field transformer using fitting fingerprint database

The results of frequency domain spectroscopy (FDS) will be affected by moisture which often leads to an unreliable condition prediction of oil-immersed cellulose insulation in transformers. In view of this, a solution is proposed in this study by adopting a fitting fingerprint database (FFD) technique, which can be utilized for condition prediction of aging and moisture content. In the current work, the relevant fingerprints that could characterize the insulation aging condition and moisture content are extracted from the FDS curves and the DC conductivity of the transformer oil. The FFD is then constructed by fitting fingerprints. The accuracy of the reported FFD technique is verified from tests on oil-immersed pressboard prepared in the lab and a transformer from the field. In this respect, the present contribution highlights FFD might be used as a powerful tool for condition prediction of oil-immersed cellulose insulation in transformers.

Privacy-preserving frequent itemset mining in vertically partitioned database using symmetric homomorphic encryption scheme

Frequent itemset mining (FIM) and association rule mining (ARM) are two popular and widely used mining techniques for transactional databases with wide range of applications such as medical diagnosis, market basket analysis, protein sequences, census data, etc. In privacy-preserving frequent itemset mining (PPFIM) over vertically partitioned database, data owners wish to learn frequent itemsets from their concatenated dataset without disclosing their sensitive information. A collusion resistant scheme has been proposed using Du-Atallah's SBDP protocol for multiparty multi-vector scenario which works under two party two-vector case. Also, a scheme is designed by generalising the Du-Atallah's scheme for more than two parties. However it fails in privacy requirements when more than two parties/vectors are considered. In this paper, we give a critique on this approach and focus on efficient and more secure methods for privacy preserving FIM in vertically partitioned databases. To ensure privacy of raw data of different data owners having heterogeneous attributes, we proposed a secure-sum algorithm which uses symmetric homomorphic encryption scheme as a sub-part and a semihonest trusted third party (STTP) environment for calculating the support count of itemsets in privacy preserving manner.

Privacy-preserving frequent itemset mining in vertically partitioned database using symmetric homomorphic encryption scheme

Frequent itemset mining (FIM) and association rule mining (ARM) are two popular and widely used mining techniques for transactional databases with wide range of applications such as medical diagnosis, market basket analysis, protein sequences, census data, etc. In privacy-preserving frequent itemset mining (PPFIM) over vertically partitioned database, data owners wish to learn frequent itemsets from their concatenated dataset without disclosing their sensitive information. A collusion resistant scheme has been proposed using Du-Atallah's SBDP protocol for multiparty multi-vector scenario which works under two party two-vector case. Also, a scheme is designed by generalising the Du-Atallah's scheme for more than two parties. However it fails in privacy requirements when more than two parties/vectors are considered. In this paper, we give a critique on this approach and focus on efficient and more secure methods for privacy preserving FIM in vertically partitioned databases. To ensure privacy of raw data of different data owners having heterogeneous attributes, we proposed a secure-sum algorithm which uses symmetric homomorphic encryption scheme as a sub-part and a semihonest trusted third party (STTP) environment for calculating the support count of itemsets in privacy preserving manner.

Analysis for the Evaluation and Security Management of a Database in a Public Organization to Mitigate Cyber Attacks

In data handling, security and privacy are essential for database administrators and users that own their information. Information about security models or architectures was analyzed for databases that minimize Cyber Attacks. The problem is that security models deployed in databases in public organizations suffer from computer attacks due to vulnerabilities in their security management systems. Data providers often publish the information for research purposes, publishing compromises on the privacy of the data of users registered in the entities. There are many security techniques for databases based on data encryption processes in transactions, however, the techniques found compromise information. The development of the document used the deductive method and exploratory research technique that allows us to study the information in the articles presented. In this document we have proposed a Security Model Assessment Prototype for Databases, a Prototype of Security Management Architecture in Blockchain for a Database, a Database Security Algorithm, a Logical Structure of the Management System in Blockchain, and a Prototype to mitigate cyberattacks. The use of a hybrid blockchain model provides the system with the optimal security that public organizations in relation to test results performed with a 99.50% security effectiveness. The simulations presented in this document were conducted based on in-depth studies and the data were randomly placed with a range according to the study results based on real-world situations.

Flavonoid Database Based on Indonesian Foods.

The prevalence of NCDs (Non-communicable Diseases) was increasing in Indonesia. The antioxidants have been known as a potential nutrient that prevent and reduce the risk of the NCDs. Flavonoid is the most abundant antioxidant in plant-source diet and commonly consumed in Indonesia. In other side, there was not a flavonoid database based on various Indonesian foods. The aim of this study was to build a flavonoid database based on Indonesian foods. The selected food items used food items of Indonesian Total Diet Survey 2014 (SDT 2014). We did not determine the flavonoids contents in the laboratories, we compiled its content by using the United States Department of Agriculture Database and related refferences. The estimation for calculating flavonoid content used USDA Database technique. The total flavonoid content for each food items were summarized into six flavonoid subclasses (anthocyanidine, flavan-3-ols, flavanone, flavone, flavonols, dan isoflavone). The total flavonoid content was presented by mg/100 gram of each food items. There were 13 food groups and 1,203 food items in this database. There were 654 food items had flavonoid content. The highest mean of flavonoid content was legumes (45.36 mg/100 g). The vegetables and products were the food group with the most foods containing flavonoid (120 food items). There were 654 food items in Indonesian foods contained flavonoid. The highest mean of flavonoid content were legumes. This findings can provide valuable information for Indonesian people to increase dietary flavonoid intake.

Summary of the 12th IAEA technical meeting on control, data acquisition and remote participation for fusion research

This report summarizes the contributions presented at the 12th IAEA technical meeting on control, data acquisition and remote participation for fusion research, held in Daejeon, Republic of Korea, 13–17 May 2019. The meeting brought together more than 70 experts from nuclear fusion research sites worldwide. The main topics of the meeting were plasma control; machine control, monitoring, safety and remote manipulation; data acquisition and signal processing; database techniques for information storage and retrieval; advanced computing and massive data analysis; remote participation and virtual laboratory; and fast network technology and its application.

Efficient Signal Reconstruction for a Broad Range of Applications

The signal reconstruction problem (SRP) is an important optimization problem where the objective is to identify a solution to an under-determined system of linear equations AX = b that is closest to a given prior. It has a substantial number of applications in diverse areas including network traffic engineering, medical image reconstruction, acoustics, astronomy and many more. Most common approaches for solving SRP do not scale to large problem sizes. In this paper, we propose a dual formulation of this problem and show how adapting database techniques developed for scalable similarity joins provides a significant speedup when the A matrix is sparse and binary. Extensive experiments on real-world and synthetic data show that our approach produces a significant speedup of up to 20x over competing approaches.

Performance Analysis of Duplicate Record Detection Techniques

In this paper, a comprehensive performance analysis of duplicate data detection techniques for relational databases has been performed. The research focuses on traditional SQL based and modern bloom filter techniques to find and eliminate records which already exist in the database while performing bulk insertion operation (i.e. bulk insertion involved in the loading phase of the Extract, Transform, and Load (ETL) process and data synchronization in multisite database synchronization). The comprehensive performance analysis was performed on several data sizes using SQL, bloom filter, and parallel bloom filter. The results show that the parallel bloom filter is highly suitable for duplicate detection in the database.