Massive Databases Research Articles

Deep Neural Networks for Creating Reliable PmP Database With a Case Study in Southern California

AbstractRecent progresses in artificial intelligence and machine learning make it possible to automatically identify seismic phases from exponentially growing seismic data. Despite some exciting successes in automatic picking of the first P‐ and S‐wave arrivals, auto‐identification of later seismic phases such as the Moho‐reflected PmP waves remains a significant challenge in matching the performance of experienced analysts. The main difficulty of machine‐identifying PmP waves is that the identifiable PmP waves are rare, making the problem of identifying the PmP waves from a massive seismic database inherently unbalanced. In this work, by utilizing a high‐quality PmP data set (10,192 manual picks) in southern California, we develop PmPNet, a deep‐neural‐network‐based algorithm to automatically identify PmP waves efficiently; by doing so, we accelerate the process of identifying the PmP waves. PmPNet applies similar techniques in the machine learning community to address the unbalancement of PmP datasets. The architecture of PmPNet is a residual neural network (ResNet)‐autoencoder with additional predictor block, where encoder, decoder, and predictor are equipped with ResNet connection. We conduct systematic research with field data, concluding that PmPNet can efficiently achieve high precision and high recall simultaneously to automatically identify PmP waves from a massive seismic database. Applying the pre‐trained PmPNet to the seismic database from January 1990 to December 1999 in southern California, we obtain nearly twice more PmP picks than the original PmP data set, providing valuable data for other studies such as mapping the topography of the Moho discontinuity and imaging the lower crust structures of southern California.

Florida public hurricane loss model: Software system for insurance loss projection

AbstractA six‐month‐long Atlantic hurricane season impacts Florida residents every year and can result in devastating consequences, including loss of life, property damage, and business interruptions. Hurricane risk assessment and loss prediction are critical to various uses such as determining homeowner insurance premiums, regulating these premiums, conducting scenario analysis, conducting stress tests for companies, disaster management, and evaluating the benefits of disaster mitigation techniques. This article describes the Florida Public Hurricane Loss Model (FPHLM): a large‐scale catastrophe model with massive databases and analytics tools for business and government decision‐making. We will discuss the design and implementation of each component in FPHLM and explain the tools and techniques utilized to tackle challenges in data availability, data analytics, and the interface between the data, analytical techniques, and computing. Results are shown to validate the software system's effectiveness and reliability and illustrate some of the system's use cases.

Recurrent neural network variants based model for Cassini-Huygens spacecraft trajectory modifications recognition

Over the last 13.7 years period of the Cassini mission, amendments to the spacecraft’s flight path were needed. This research is being carried out as there is a limited number of studies that use a temporal discrimination analysis to handle raw data. More complex inspection and analysis of the collected broad trajectory dataset is necessary to classify orbital events in the signal travel period (approximately 88 minutes on the Earth-Cassini travel channel length). This paper provides an innovative, in-depth learning method to identify offline modifications in the Cassini spacecraft trajectory. The models are based on variants of Recurrent Neural Networks (RNNs: Gated Recurrent Unit (GRU)/ Long Short-Term Memory (LSTM)/ Bidirectional Long Short-Term Memory (BiLSTM)) to derive valuable data and learn the inner data structure of the time sequence, along with the penetration of long-term and short-term phase-dependencies of the RNNs layers. To validate our models, we used a variety of statistical approaches in our analysis. A considerable number of tests have been carried out, and the findings obtained have shown that the GRU and LSTM give a substantial boost to increasing the efficiency of the detection mechanism. The proposed model would consolidate potential exploration in outer space exploration to accommodate massive databases, search for correlations, and recognize complex events and outliers with an accuracy that exceeds 99 %. This method can be utilized for similar detection processes within the future outer space expedition. The results show that binary classifications of Matthews Correlation Coefficient (MCC) are more accurate than F_1 score.

A simplified suction-based approach to determine potential vertical movement of natural subgrade soil

The prediction of vertical movement in expansive soil is important primarily for the purpose of designing foundations and other horizontal structural elements such as pavements. Most of the existing approaches to predict vertical movement are conservative and over-predict soil movement and are based on field measurements from localised soil data sets. Therefore, a suction-based approach is adopted in this study, which has the advantage of considering the effects of both moisture energy (suction) and mechanical stresses that surround the soil. A simplified Excel spreadsheet is programmed to facilitate estimation of potential vertical movement (PVM) in subgrade soils using physical properties of the soil along with local climatic indices. The soil database from the Natural Resource Conservation Service (NRCS) is used to develop prediction models of suction compressibility index (γh) and diffusivity coefficient (α) and is subsequently used in the spreadsheet. A contour map of the maximum PVM values for the entire continental USA is generated using the geographic information system platform and the estimated results from the spreadsheet. Finally, measured international roughness index values collected from the massive long-term pavement performance database are used to validate the calculated PVM values.

Research on data resource management of biomass energy engineering based on data mining

In order to improve the information management level of biomass energy industry, it is necessary to build and improve the data resource management platform of biomass energy engineering, so as to grasp the detailed operation status and production situation of biomass energy engineering in real time. In this paper, the data mining concept and technology based on Hadoop are applied to the construction of data resource management platform of biomass energy engineering, which solves the difficult problems of mass data collection, storage, processing and analysis of biomass energy. The bottleneck of Apriori algorithm in processing big data lies in the high time complexity. Aiming at this shortcoming of Apriori, this paper combines Apriori algorithm with Hadoop framework, and combine Apriori algorithm with Hadoop framework, and test the data mining of Apriori algorithm after combination. Results The test realized the optimal design of storage mode of massive database, and the performance of Apriori algorithm was improved by 40% compared with traditional Apriori algorithm.

Method for the Identification of Potentially Bioactive Argon Binding Sites in Protein Families.

Argon belongs to the group of chemically inert noble gases, which display a remarkable spectrum of clinically useful biological properties. In an attempt to better understand noble gases, notably argon's mechanism of action, we mined a massive noble gas modeling database which lists all possible noble gas binding sites in the proteins from the Protein Data Bank. We developed a method of analysis to identify among all predicted noble gas binding sites the potentially relevant ones within protein families which are likely to be modulated by Ar. Our method consists in determining within structurally aligned proteins the conserved binding sites whose shape, localization, hydrophobicity, and binding energies are to be further examined. This method was applied to the analysis of two protein families where crystallographic noble gas binding sites have been experimentally determined. Our findings indicate that among the most conserved binding sites, either the most hydrophobic one and/or the site which has the best binding energy corresponds to the crystallographic noble gas binding sites with the best occupancies, therefore the best affinity for the gas. This method will allow us to predict relevant noble gas binding sites that have potential pharmacological interest and thus potential Ar targets that will be prioritized for further studies including in vitro validation.

Knowledge management of product design: A requirements-oriented knowledge management framework based on Kansei engineering and knowledge map

For the knowledge management of product design, knowledge innovation is the foundation and motivation for the independent innovation and enhancing the core competitiveness. Most of the product knowledge exists in the brain of designers. How to obtain the required knowledge accurately in massive knowledge database becomes the key to knowledge innovation. However, the design knowledge based on consumer’s requirement has not been extensively studied. There is no consensus on the reasonable and effective implementation of the knowledge management framework to select the optimum design knowledge based on the consumer’s requirements. In this study, to efficiently realize the knowledge acquisition and knowledge selection, a requirements-oriented knowledge management model is established, with the advantage of Kansei engineering in knowledge acquisition and multi-objective decision-making in knowledge selection. Finally, the outdoor leisure chairs design is used as a case study to explain the implementation of the knowledge management framework. To reveal the advantages of the framework, it was compared with other frameworks. The results show that the proposed knowledge management framework is more efficient and provided a method of designers to acquire design knowledge based on the consumer’s requirements.

A Retrieval Method for Spatiotemporal Information of Chorography Based on Deep Learning

Background: On the retrieval of spatiotemporal information of chorography (STIC), one of the most important topics is how to quickly pinpoint the desired STIC text out of the massive chorography databases. Domestically, there are not diverse means to retrieve the spatiotemporal information from chorography database. Emerging techniques like data mining, artificial intelligence (AI), and natural language processing (NLP) should be introduced into the informatization of chorography. Objective: This study intends to devise an information retrieval method for STIC based on deep learning, and fully demonstrates its feasibility. Methods: Firstly, the authors explained the flow for retrieving and analyzing the data features of STIC texts, and established a deep hash model for STIC texts. Next, the data matching flow was defined for STIC texts, the learned hash code was adopted as the memory address of STIC texts, and the hash Hamming distance of the text information was computed through linear search, thereby completing the task of STIC retrieval. Results: Our STIC text feature extraction model learned better STIC text features than the contrastive method. It learned many hash features, and differentiated between different information well, when there were many hash bits. Conclusion: In addition, our hash algorithm achieved the best retrieval accuracy among various methods. Finally, the hash features acquired by our algorithm can accelerate the retrieval speed of STIC texts. These experimental results demonstrate the effectiveness of the proposed model and algorithm.

Fraud and Deceit in Medical Research

Fraud and Deceit in Medical Research

Three‐dimensional human activity recognition by forming a movement polygon using posture skeletal data from depth sensor

Human activity recognition in real time is a challenging task. Recently, a plethora of studies has been proposed using deep learning architectures. The implementation of these architectures requires the high computing power of the machine and a massive database. However, handcrafted features-based machine learning models need less computing power and very accurate where features are effectively extracted. In this study, we propose a handcrafted model based on three-dimensional sequential skeleton data. The human body skeleton movement over a frame is computed through joint positions in a frame. The joints of these skeletal frames are projected into two-dimensional space, forming a “movement polygon.” These polygons are further transformed into a one-dimensional space by computing amplitudes at different angles from the centroid of polygons. The feature vector is formed by the sampling of these amplitudes at different angles. The performance of the algorithm is evaluated using a support vector machine on four public datasets: MSR Action3D, Berkeley MHAD, TST Fall Detection, and NTU-RGB+D, and the highest accuracies achieved on these datasets are 94.13%, 93.34%, 95.7%, and 86.8%, respectively. These accuracies are compared with similar state-of-the-art and show superior performance.

Apache Spark and MLlib-Based Intrusion Detection System or How the Big Data Technologies Can Secure the Data

Since the turn of the millennium, the volume of data has increased significantly in both industries and scientific institutions. The processing of these volumes and variety of data we are dealing with are unlikely to be accomplished with conventional software solutions. Thus, new technologies belonging to the big data processing area, able to distribute and process data in a scalable way, are integrated into classical Business Intelligence (BI) systems or replace them. Furthermore, we can benefit from big data technologies to gain knowledge about security, which can be obtained from massive databases. The paper presents a security-relevant data analysis based on the big data analytics engine Apache Spark. A prototype intrusion detection system is developed aimed at detecting data anomalies through machine learning by using the k-means algorithm for clustering analysis implemented in Sparks MLlib. The extraction of features to detect anomalies is currently challenging because the problem of detecting anomalies is not actively and exhaustively monitored. The detection of abnormal data can be effectuated by using relevant data that are already in companies’ and scientific organizations’ possession. Their interpretation and further processing in a continuous manner can sufficiently contribute to anomaly and intrusion detection.

Research on Fast Face Retrieval Optimization Algorithm Based on Fuzzy Clustering

Aiming at the problem of long retrieval time for massive face image databases under a given threshold, a fast retrieval algorithm for massive face images based on fuzzy clustering is proposed. The algorithm builds a deep convolutional neural network model. The model can be used to extract features from face photos to obtain a high-dimensional vector to represent the high-level semantic features of face photos. On this basis, the fuzzy clustering algorithm is used to perform fuzzy clustering on the feature vectors of the face database to construct a retrieval pedigree map. When the threshold is passed in for database retrieval of the target face photos, the pedigree map can be quickly retrieved. Experiments on the LFW face dataset and self-collected face dataset show that the model is better than the commonly used K-means model in face recognition accuracy, clustering effect, and retrieval speed and has certain commercial value.

Overview of the development of GIS in the education process in the Republic of North Macedonia

The paper presents an overview of the development of Geographic Information Systems in the educational system in the Republic of North Macedonia. It presents the period of development of geography and geographic education during the last century, while providing a more general review of the beginnings and the continuing course of development of GIS at the Institute of Geography at the Faculty of Natural Mathematics in Skopje, and across the public institutions in the Republic of North Macedonia overall. The goal is to understand the cultural overview of the events related to the development of GIS, the exchange of experiences and benefits from the organization of the educational processes of the undergraduate and postgraduate studies at the Institute of Geography at the Faculty of Natural Mathematics in Skopje and the contribution and guidance by the GIS-trained (graduate and Master degree) personnel in the development of GIS techniques and technologies in the application and design activities in the Republic of North Macedonia. The scientific contribution is multi-purpose in terms of advancing GIS techniques and technologies in various geographical and other disciplines, by recognizing the potentials that GIS has with regard to the work with massive databases and by obtaining more relevant analyses, automated process of creating various graphic and thematic cartographic database displays by using a content-based retrieval system.

The role of massive demographic databases in intractable illnesses: Denomics for dementia.

Despite intensive research, effective treatments for many common and devastating diseases are lacking. For example, huge efforts and billions of dollars have been invested in Alzheimer's disease (AD), which affects over 50 million people worldwide. However, there is still no effective drug that can slow or cure AD. This relates, in part, to the absence of an animal model or cellular system that incorporates all the relevant features of the disease. Therefore, large scale studies on human populations and tissues will be key to better understanding dementia and developing methods to prevent or treat it. This is especially difficult because the dementia phenotype can result from many different processes and is likely to be affected by multiple personal and environmental variables. We hypothesize that analyzing massive volumes of demographic data that are currently available and combining this with genomic, proteomic, and metabolomic profiles of AD patients and their families, new insights into pathophysiology and treatment of AD may arise. While this requires much coordination and cooperation among large institutions, the potential for advancement would be life-changing for millions of people. In many ways this represents the next step in the information revolution started by the Human Genome Project.

An Approach Based on Deep Learning for Tree Species Classification in LiDAR Data Acquired in Mixed Forest

This letter proposes a novel method based on Deep Learning (DL) to forest species classification in airborne Light Detection and Ranging (LiDAR) data. Differently from the state-of- the-art approaches, the proposed method: (1) does not assume any prior knowledge either on the forest to be classified or on the sensor used to acquire the LiDAR data, and (2) can be applied to heterogeneous forest characterized by mixed species. First, the 3D point cloud of each individual tree is decomposed into 8 angular sectors to generate a multi-slices representation of the vertical structure of the tree. This representation models the foliage, the stem and the branches of the tree crown as well as depicts the internal and external crown properties. Then, a Multi-View CNN (MVCNN) DL automatically extracts features used to discriminate the different tree species. This network is pre-trained on the massive ImageNet database, thus guaranteeing fast convergence with a relatively small number of ground reference data. Experiments were carried out on high density airborne LiDAR data collected over a multi-layer multi-age forest characterized by four conifers and three broadleaf species. The proposed method outperformed the state-of-the-art approaches increasing the Overall Accuracy (OA) up to 16% and 18.9% compared to a DL and a shallow tree species classification methods, respectively. When applied to coniferous or broadlaef forests, the proposed method showed an increase of OA 10.1% and 15.9% (for conifers), and 9.5% and 21.6% (for broadleafs) compared to the DL and shallow methods, respectively.

An Efficient Deep Learning-based Content-based Image Retrieval Framework

The use of massive image databases has increased drastically over the few years due to evolution of multimedia technology. Image retrieval has become one of the vital tools in image processing applications. Content-Based Image Retrieval (CBIR) has been widely used in varied applications. But, the results produced by the usage of a single image feature are not satisfactory. So, multiple image features are used very often for attaining better results. But, fast and effective searching for relevant images from a database becomes a challenging task. In the previous existing system, the CBIR has used the combined feature extraction technique using color auto-correlogram, Rotation-Invariant Uniform Local Binary Patterns (RULBP) and local energy. However, the existing system does not provide significant results in terms of recall and precision. Also, the computational complexity is higher for the existing CBIR systems. In order to handle the above mentioned issues, the Gray Level Co-occurrence Matrix (GLCM) with Deep Learning based Enhanced Convolution Neural Network (DLECNN) is proposed in this work. The proposed system framework includes noise reduction using histogram equalization, feature extraction using GLCM, similarity matching computation using Hierarchal and Fuzzy c- Means (HFCM) algorithm and the image retrieval using DLECNN algorithm. The histogram equalization has been used for computing the image enhancement. This enhanced image has a uniform histogram. Then, the GLCM method has been used to extract the features such as shape, texture, colour, annotations and keywords. The HFCM similarity measure is used for computing the query image vector's similarity index with every database images. For enhancing the performance of this image retrieval approach, the DLECNN algorithm is proposed to retrieve more accurate features of the image. The proposed GLCM+DLECNN algorithm provides better results associated with high accuracy, precision, recall, f-measure and lesser complexity. From the experimental results, it is clearly observed that the proposed system provides efficient image retrieval for the given query image.

A dynamic knowledge base and its data compression with homomorphism

In reality there are always a large number of complex massive databases. The notion of homomorphism may be a mathematical tool for studying data compression in knowledge bases. This paper investigates a knowledge base in dynamic environments and its data compression with homomorphism, where “dynamic” refers to the fact that the involved information systems need to be updated with time due to the inflow of new information. First, the relationships among knowledge bases, information systems and relation information systems are illustrated. Next, the idea of non-incremental algorithm for data compression with homomorphism and the concept of dynamic knowledge base are introduced. Two incremental algorithms for data compression with homomorphism in dynamic knowledge bases are presented. Finally, an experimental analysis is employed to demonstrate the applications of the non-incremental algorithm and the incremental algorithms for data compression when calculating the knowledge reduction of dynamic knowledge bases.

An Optimized K-means with Density and Distance-Based Clustering Algorithm for Multidimensional Spatial Databases

From massive and complex spatial database, the useful information and knowledge are extracted using spatial data mining. To analyze the complexity, efficient clustering algorithm for spatial database has been used in this area of research. The geographic areas containing spatial points are discovered using clustering methods in many applications. With spatial attributes, the spatial clustering problem have been designed using many approaches, but non-overlapping constraints are not considered. Most existing data mining algorithms suffer in high dimensions. With non-overlapping named as Non Overlapping Constraint based Optimized K-Means with Density and Distance-based Clustering (NOC-OKMDDC),a multidimensional optimization clustering is designed to solve this problem by the proposed system and the clusters with diverse shapes and densities in spatial databases are fast found. Proposed method consists of three main phases. Using weighted convolutional Neural Networks(Weighted CNN), attributes are reduced from the multidimensional dataset in this first phase. A partition-based algorithm (K-means) used by Optimized K-Means with Density and Distance-based Clustering (OKMDD) and several relatively small spherical or ball-shaped sub clusters are made by Clustering the dataset in this second phase. The optimal sub cluster count is performed with the help of Adaptive Adjustment Factor based Glowworm Swarm Optimization algorithm (AAFGSO). Then the proposed system designed an Enhanced Penalized Spatial Distance (EPSD) Measure to satisfy the non-overlapping condition. According to the spatial attribute values, the spatial distance between two points are well adjusted to achieving the EPSD. In third phase, to merge sub clusters the proposed system utilizes the Density based clustering with relative distance scheme. In terms of adjusted rand index, rand index, mirkins index and huberts index, better performance is achieved by proposed system when compared to the existing system which is shown by experimental result.

A case study of ACV variables discovered in the Zwicky Transient Facility survey

Context. Magnetic chemically peculiar (mCP) stars exhibit complex atmospheres that allow the investigation of the interplay of atomic diffusion, magnetic fields, and stellar rotation. A non-uniform surface distribution of chemical elements and the non-alignment of the rotational and magnetic axes result in the variability of several observables. Photometrically variable mCP stars are referred to as α2 Canum Venaticorum (ACV) variables. Aims. The present work presents a case study of known variables from the Zwicky Transient Facility (ZTF) survey, with the aim of investigating the survey’s suitability for the detection and study of new ACV variables. Methods. Using suitable selection criteria based on the known characteristics of ACV variables, candidate ACV stars were selected from the ZTF Catalog of Periodic Variable Stars. All light curves were inspected in detail to select the most promising candidates. Where available, low-resolution spectra from the Large Sky Area Multi-Object Fiber Spectroscopic Telescope (LAMOST) were employed to classify the stars on the MK system and confirm their status as mCP stars. Results. We have identified 86 new promising ACV star candidates. Fifteen of these stars have LAMOST spectra available that, in all cases, confirm them as classical mCP stars, which highlights the viability of our approach. We present astrophysical parameters for all sample stars that can be sorted into four subgroups characterized by distinct light curve shapes. Anti-phase variations in different photometric passbands, in particular, are a unique characteristic of a subset of ACV stars readily usable for their identification. The availability of data in three different passbands (g, r, and i) is a major advantage of the ZTF survey. Conclusions. On the basis of our experience with other photometric surveys and the analysis of light curves, we conclude that the ZTF is well suited to the search for, and the analysis of, ACV variables that are not considered in the available ZTF variable star catalogs. Further work will focus on the development and refinement of a search algorithm to correctly identify these stars in ZTF data and, subsequently, in massive photometric time series databases in general.

Towards an Architecture of a Multi-purpose, User-Extendable Reference Human Brain Atlas

Human brain atlas development is predominantly research-oriented and the use of atlases in clinical practice is limited. Here I introduce a new definition of a reference human brain atlas that serves education, research and clinical applications, and is extendable by its user. Subsequently, an architecture of a multi-purpose, user-extendable reference human brain atlas is proposed and its implementation discussed. The human brain atlas is defined as a vehicle to gather, present, use, share, and discover knowledge about the human brain with highly organized content, tools enabling a wide range of its applications, massive and heterogeneous knowledge database, and means for content and knowledge growing by its users. The proposed architecture determines major components of the atlas, their mutual relationships, and functional roles. It contains four functional units, core cerebral models, knowledge database, research and clinical data input and conversion, and toolkit (supporting processing, content extension, atlas individualization, navigation, exploration, and display), all united by a user interface. Each unit is described in terms of its function, component modules and sub-modules, data handling, and implementation aspects. This novel architecture supports brain knowledge gathering, presentation, use, sharing, and discovery and is broadly applicable and useful in student- and educator-oriented neuroeducation for knowledge presentation and communication, research for knowledge acquisition, aggregation and discovery, and clinical applications in decision making support for prevention, diagnosis, treatment, monitoring, and prediction. It establishes a backbone for designing and developing new, multi-purpose and user-extendable brain atlas platforms, serving as a potential standard across labs, hospitals, and medical schools.