Big Database Research Articles

Clustering method of unbalanced large data density based on dynamic grid

In order to effectively ensure the clustering quality of unbalanced big data density, improve the clustering accuracy of unbalanced big data density and shorten the clustering time of unbalanced big data density, an unbalanced big data density clustering method based on dynamic grid is proposed. This paper analyzes the definition and characteristics of dynamic grid, and expounds the clustering method based on density and dynamic grid. Build a dynamic grid of unbalanced big data through unbalanced big data database. Consider a single spatial object, divide the unbalanced big data dynamic grid, and calculate the cell density of the unbalanced big data dynamic grid. Dynamic grid technology is adopted to realize unbalanced large data density clustering. Experimental results show that the proposed algorithm has good clustering effect, which can effectively improve the clustering accuracy of unbalanced big data and shorten the clustering time of unbalanced big data density.

Features of the development of electronic services in the digital society

The digital revolution is the impact of information and communication technologies on all the levels of society. Society is witnessing the simultaneous development of big databases, artificial intelligence, blockchain, robotics, database science and other rapidly developing technologies. They influence and reinforce each other in all areas of our lives (from food systems to utility systems, energy supply, education, health care and social security). This determines the direction of economic development: the formation of electronic government, electronic business and electronic commerce. It is not just about technology, it is about people and the planet in general. E-Government, in particular, brings significant and long-term changes to the way people live and interact with each other, the environment and public services. The use of e-Government has great potential not only for increasing the efficiency of public services, but also from the viewpoint of increasing the transparency of organizational and business processes and for holding people accountable. Implementation and automation of economic processes with the help of information and communication technologies in real time is a necessary condition for the creation and development of electronic services, which allows reducing operational costs and the cost of production services. With the use of technologies, the concept of “electronic services” (increasing efficiency, simplifying communication between the public and the state) is widely used. Furthermore, electronic services are considered to beone of the most effective tools ensuring transparency and preventing corruption. The process of spreading electronic government in our country is no longer a technological innovation for the population and is being transformed into an accessible tool for obtaining public services, which contributes to increasing the transparency of public authorities. For a deeper penetration of electronic government technology, it is necessary to carry out systematic and purposeful work at the level of regional authorities, which will contribute to the involvement of the population in dialogue with them and the formation of the integration stage of electronic government. Keywords: digital economy, information society, electronic government, electronic services, trends in the development of electronic services.

SUPERCONSISTENCY OF TESTS IN HIGH DIMENSIONS

To assess whether there is some signal in a big database, aggregate tests for the global null hypothesis of no effect are routinely applied in practice before more specialized analysis is carried out. Although a plethora of aggregate tests is available, each test has its strengths but also its blind spots. In a Gaussian sequence model, we study whether it is possible to obtain a test with substantially better consistency properties than the likelihood ratio (LR; i.e., Euclidean norm-based) test. We establish an impossibility result, showing that in the high-dimensional framework we consider, the set of alternatives for which a test may improve upon the LR test (i.e., its superconsistency points) is always asymptotically negligible in a relative volume sense.

Big databases and their contribution to dental research.

Big databases and their contribution to dental research.

International Registries: U.K. National Joint Registry, Nordic Registries, and Australian Orthopaedic Association National Joint Replacement Registry (AOANJRR).

National-level joint arthroplasty registries were among the first large orthopaedic surgery databases and represent some of the longest-running and most influential big databases in our profession. Nordic registries were among the first registries and were followed by excellent registries in the United Kingdom and Australia. In this article, we describe each of these registries and highlight the data elements collected, the data points that can be obtained by linking the national arthroplasty registries to other national registries or databases, the completeness of data, and the strengths and weaknesses of each database. Each of these registries publishes an annual report that is available online, and each also can do more detailed analysis of certain aspects of its data for special studies.When evaluating and interpreting data from national joint registries, the user should be aware that, despite the power of huge numbers, there remain numerous limitations to the observational data. Strong selection biases exist with regard to which patients are chosen for which procedure. Surgeons of different skill levels may use one technique or implant differentially compared with another. The end points that registries collect differ widely: for example, some report only hip dislocations leading to revision rather than all dislocations. Registries in countries in which there are long wait times for revision surgery may report artificially low revision rates compared with countries in which revision surgery is more easily accessible. Despite these limitations, registries have the merit of reporting the actual results of huge numbers of surgical procedures performed across the spectrum of hospitals, surgeons, and patients, making the data extremely powerful for identifying trends, identifying early signs of problems related to certain implants or implant classes, and identifying associations between variables that can be further studied to determine if the effect is causal.

Crop Yield Prediction in Precision Agriculture

Predicting crop yields is one of the most challenging tasks in agriculture. It plays an essential role in decision making at global, regional, and field levels. Soil, meteorological, environmental, and crop parameters are used to predict crop yield. A wide variety of decision support models are used to extract significant crop features for prediction. In precision agriculture, monitoring (sensing technologies), management information systems, variable rate technologies, and responses to inter- and intravariability in cropping systems are all important. The benefits of precision agriculture involve increasing crop yield and crop quality, while reducing the environmental impact. Simulations of crop yield help to understand the cumulative effects of water and nutrient deficiencies, pests, diseases, and other field conditions during the growing season. Farm and in situ observations (Internet of Things databases from sensors) together with existing databases provide the opportunity to both predict yields using “simpler” statistical methods or decision support systems that are already used as an extension, and also enable the potential use of artificial intelligence. In contrast, big data databases created using precision management tools and data collection capabilities are able to handle many parameters indefinitely in time and space, i.e., they can be used for the analysis of meteorology, technology, and soils, including characterizing different plant species.

Artificial Intelligence tool for fire safety design (IFETool): Demonstration in large open spaces

Fire modelling is a common practice in building fire safety analysis, but it is costly. This work develops an AI software, Intelligent Fire Engineering Tool (IFETool), to speed up the fire safety analysis and quickly identify design limits. A big numerical atrium-fire database is firstly formed by considering key building and fire parameters. Then, a deep learning model is trained to predict the evolution of tenable smoke visibility, temperature and CO concentration with an accuracy of 97%. The tenability descending profile is further processed to assess the available safe egress time (ASET) and the fire safety of the atriums that have complex roof shapes and slab extensions. This AI design software is able to make a quick assessment of the proposed atrium fire engineering design and give valuable suggestions for potential improvement. Finally, the operation guidelines of IFETool are provided for common design tasks of atrium fire safety.

FSOLAP: A fuzzy logic-based spatial OLAP framework for effective predictive analytics

Nowadays, with the rise in sensor technology, the amount of spatial and temporal data increases day by day. Fast, effective, and accurate analysis and prediction of collected data have become more essential than ever. Spatial Online Analytical Processing (SOLAP) emerged to perform data mining on spatial and temporal data that naturally contains the hierarchical structure used in many complex applications. In addition, uncertainty and fuzziness are inherently essential elements of data in many complex data applications, particularly in spatial–temporal database applications. In this study, FSOLAP is proposed as a new fuzzy SOLAP-based framework to compose the benefits of fuzzy logic and SOLAP concepts and is extended with inference capability to the framework to support predictive analytics. The predictive accuracy and resource utilization performance of FSOLAP are compared using real data with some well-known machine learning techniques such as Support Vector Machine, Random Forest, and Fuzzy Random Forest. The extensive experimental results show that the FSOLAP framework for the predictive analytics of various spatiotemporal events in big meteorological databases is considerably more accurate and scalable than using conventional machine learning techniques.

Age-dependent association of discharge heart-failure medications with clinical outcomes in a super-aged society

AimsReal-world evidence about heart failure (HF) medical care for elderly patients is limited. The aim of this study was to clarify the age-dependent associations of HF medications at discharge with clinical outcomes in a super-aged society using a real-world big database. Methods and resultsPatients hospitalized with a first episode of acute HF in 2013–2019 were identified in a nationwide claims database in Japan and were analyzed based on the HF medications at discharge. Hazard ratios (HRs) and 95 % confidence intervals (95 % CIs) for clinical outcomes were calculated after adjustment. Among 325,468 patients from 4351 hospitals, 130,230 (40.0 %) were aged ≥ 85 years. All-cause mortality and HF readmission rate within 1 year were 14 % and 23 %, respectively. β-Blockers and angiotensin-converting-enzyme inhibitors or angiotensin II receptor blockers (ACEI/ARB) uses were associated with lower mortality (HR [95 %CI]: 0.84 [0.83,0.86] and 0.73 [0.71,0.74], respectively) and HF readmission (0.98 [0.97,0.99] and 0.89 [0.88,0.90], respectively). The associations were attenuated as the age increased and, in β-blockers use, the favorable associations with mortality and HF readmission were not observed in patients with ≥ 95 years and ≥ 80 years, respectively. In contrast, mineralocorticoid receptor antagonists (MRA) use was associated only with lower HF readmission (0.83 [0.82,0.84]), which was not attenuated with increased age. ConclusionsThe favorable associations of β-blockers and ACEI/ARB uses with mortality and HF readmission were age-dependently attenuated in patients across a broad spectrum of HF in a super-aged society. In contrast, positive impact of MRA use on HF readmission was not attenuated with aging.

A REVIEW OF DIRECT SHEAR AND INCLINED PLANE TESTS RESULTS FOR DIFFERENT INTERFACES IN LANDFILL CAPPING

One of the crucial aspects in design of a landfill capping is the interface behavior between the different layers of the cover system, from levelling layer above waste up to the topsoil. Design guidelines and international codes require a geotechnical stability analysis to be performed along every interface. The critical interface is the one which gives the minimum shear resistance, in terms of friction angle and adhesion. Evaluation of the correct values to be used is then essential. Shear resistance at the interface between different geosynthetics or between a geosynthetic and a soil can be measured through laboratory tests. Testing methods are EN ISO 12957-1 and ASTM D5321 (for direct shear test) and EN ISO 12957-2 (for inclined plane). The paper briefly describes direct shear and inclined plane testing methods and enhances pros and cons. In the last 25 years the authors have coordinated a great number of the above tests with different types of geosynthetics and soils. The main results of these tests are reported in the paper, summarizing the values obtained with contact interface between different products belonging to the same families. The purpose of this work is to validate the already big database of interface strength measured with direct shear tests and to evaluate the differences with the results obtained for the different types of tests. This can give to designers the chance to have a critical approach toward the most suitable testing method to be used according to the specific needs of a project.

Evaluation of Geo-SPEBH algorithm based on Bandwidth for Big Data retrieval in Cloud Computing

The fast increase in volume and speed of information created by mobile devices, along with the availability of web-based applications, has considerably contributed to the massive collection of data. Approximate Nearest Neighbor (ANN) is essential in big size databases for comparison search to offer the nearest neighbor of a given query in the field of computer vision and pattern recognition. Many hashing algorithms have been developed to improve data management and retrieval accuracy in huge databases. However, none of these algorithms took bandwidth into consideration, which is a significant aspect in information retrieval and pattern recognition. As a result, our work created a Geo-SPEBH algorithm to solve this basic gap. The paper then assesses the performance of the Geo-SPEBH algorithm in terms of bandwidth in a distributed computing environment. Geo-performance SPEBH's was compared to existing state-of-the-art approaches using a network analyzer called Wire shark. The simulation results reveal that during retrieval, the same kb/sec of data is carried from source to destination and from destination to user. When the coding length is 8bit, the findings show that 0.091kb/sec of data is required to transport data from source to destination. Each algorithm with the same bit code requires the same amount of bandwidth to convey data.

Analysis of the Role of Design-Driven Innovation in the Interaction Design of Image Indexing Software under the Background of the Internet of Things

More and more medical images are being produced quickly by modern imaging devices that are connected to the Internet of Things (IoT). For the retrieval of necessary images from a conventional big volume database, the continuous scanning of all image data in an IoT network appears to be ineffective and computationally expensive. Effective image retrieval from a big image database necessitates the use of an effective and scalable image indexing approach. Existing image indexing methods have significant drawbacks, including lower efficiency, constrained scalability, increased computing demands, and longer processing times. To index the medical images obtained by IoT sensors, we suggested a novel, effective Content-Based Cascaded Gabor Wavelet Algorithm (CBCGWA) in this study. Further in this study, the medical images from computed tomography (CT) are employed for medical indexing Totally, the dataset included 168 individually annotated square patches in a subset of 115 high-resolution CT (High-Resolution CT) slices used in this study. An adaptive median filter is used for preprocessing the CT medical images once they are acquired from IoT sensor nodes connected to medical imaging systems. The Gaussian Adaptive Attention Network is then used to cluster together images with comparable attributes (GAAN). The suggested method is used to index the images after image clustering. The cloud database is where the indexed images are eventually maintained. The comparison of the recommended indexing approach to the existing indexing strategies revealed that it is better in terms of processing time, power, and indexing efficiency.

A deep learning algorithm with smart-sized training data for transient thermal performance prediction

There is an impulsion to introduce machine-learning algorithms into the thermo-fluids areas. Machine learning modeling, especially in high-precision regression problems, relies on large amounts of randomly-selected training data. Such randomness has little effect on final results because a big database (millions or billions of data points) can swallow such randomness. However, training data in the heat transfer area is not so huge and in fact it should be kept as small as possible because data acquisition is not convenient. Therefore, training data need to be shrunk without sacrificing prediction accuracy. Based on our previous work, we herein detail the selection process for a smart-sized training dataset. The smart-sized training dataset was slimmed by 50% with improved prediction accuracy. The prediction accuracy for the worst condition was still above 90%, an increase of 3.5% on previous results. This is a great improvement on our previous work because the training data scale was substantially reduced, demonstrating better results with half of the effort. The developed small-sample modeling using the deep learning algorithm provides a powerful information-processing tool to augment the understanding of the nonlinear systems in thermo-fluids mechanics, and to even transform current lines of thermo-fluids-related and more extensive applications.

Identifying Drug Targets of Oral Squamous Cell Carcinoma through a Systems Biology Method and Genome-Wide Microarray Data for Drug Discovery by Deep Learning and Drug Design Specifications.

In this study, we provide a systems biology method to investigate the carcinogenic mechanism of oral squamous cell carcinoma (OSCC) in order to identify some important biomarkers as drug targets. Further, a systematic drug discovery method with a deep neural network (DNN)-based drug–target interaction (DTI) model and drug design specifications is proposed to design a potential multiple-molecule drug for the medical treatment of OSCC before clinical trials. First, we use big database mining to construct the candidate genome-wide genetic and epigenetic network (GWGEN) including a protein–protein interaction network (PPIN) and a gene regulatory network (GRN) for OSCC and non-OSCC. In the next step, real GWGENs are identified for OSCC and non-OSCC by system identification and system order detection methods based on the OSCC and non-OSCC microarray data, respectively. Then, the principal network projection (PNP) method was used to extract core GWGENs of OSCC and non-OSCC from real GWGENs of OSCC and non-OSCC, respectively. Afterward, core signaling pathways were constructed through the annotation of KEGG pathways, and then the carcinogenic mechanism of OSCC was investigated by comparing the core signal pathways and their downstream abnormal cellular functions of OSCC and non-OSCC. Consequently, HES1, TCF, NF-κB and SP1 are identified as significant biomarkers of OSCC. In order to discover multiple molecular drugs for these significant biomarkers (drug targets) of the carcinogenic mechanism of OSCC, we trained a DNN-based drug–target interaction (DTI) model by DTI databases to predict candidate drugs for these significant biomarkers. Finally, drug design specifications such as adequate drug regulation ability, low toxicity and high sensitivity are employed to filter out the appropriate molecular drugs metformin, gefitinib and gallic-acid to combine as a potential multiple-molecule drug for the therapeutic treatment of OSCC.

Systems Medicine Design based on Systems Biology Approaches and Deep Neural Network for Gastric Cancer.

Gastric cancer (GC) is the third leading cause of cancer death in the world. It is associated with the stimulation of microenvironment, aberrant epigenetic modification, and chronic inflammation. However, few researches discuss the GC molecular progression mechanisms from the perspective of the system level. In this study, we proposed a systems medicine design procedure to identify essential biomarkers and find corresponding drugs for GC. At first, we did big database mining to construct candidate protein-protein interaction network (PPIN) and candidate gene regulation network (GRN). Second, by leveraging the next-generation sequencing (NGS) data, we performed system modeling and applied system identification and model selection to obtain real genome-wide genetic and epigenetic networks (GWGENs). To make the real GWGENs easy to analyze, the principal network projection method was used to extract the core signaling pathways denoted by KEGG pathways. Subsequently, based on the identified biomarkers, we trained a deep neural network of drug-target interaction (DeepDTI) with supervised learning and filtered our candidate drugs considering drug regulation ability and drug sensitivity. With the proposed systematic strategy, we not only shed the light on the progression of GC but also suggested potential multiple-molecule drugs efficiently.

Movie Recommendation System Using TF-IDF Vectorization and Cosine Similarity

Abstract: The internet has widened the horizons of numerous areas to engage and share relevant information in recent years. As it is said, everything has its advantages and disadvantages, thus with the increase in the field comes data saturation and data extraction difficulties. The suggestion system is critical in overcoming this challenge. Its purpose is to improve the user's experience by providing quick and comprehensible suggestions. Because of its ability to provide improved amusement, a movie suggestion is vital in our personal interaction. Users might be recommended a collection of movies depending on their interests or the appeal of the films. A recommendation system is being used to make suggestions for things to buy or see. They comb through a big database of information to lead people to the things that can suit their demands. A recommender system, also known as a recommendation engine or platform, is a type of data filtering system that attempts to forecast a user's "rating" or "preference" for an item. They're mostly employed for business purposes. This project outlines a method for providing users with generic options based on film popularity and/or theme.

Predicting real-time fire heat release rate by flame images and deep learning

The heat release rate (HRR) is the most critical parameter in characterizing the fire behavior and thermal effects of a burning item. However, traditional fire calorimetry methods are not applicable due to the lack of equipment in most fire scenarios. This work explores the real-time fire heat release rate prediction by using fire scene images and deep learning algorithms. A big database of 112 fire tests from the NIST Fire Calorimetry Database is formed, and 69,662 fire scene images labeled by their transient heat release rate are adopted to train the deep learning model. The fire tests conducted in the lab environment and the real fire events are used to validate and demonstrate the reliability of the trained model. Results show that regardless of the fire sources, background, light conditions, and camera settings, the proposed AI-image fire calorimetry method can well identify the transient fire heat release rate using only fire scene images. This work demonstrates that the deep learning algorithms can provide an alternative method to measure the fire HRR when traditional calorimetric methods cannot be used, which shows great potential in smart firefighting applications.

Evaluating the performance of Bayesian and frequentist approaches for longitudinal modeling: application to Alzheimer’s disease

Linear mixed effects (LME) modelling under both frequentist and Bayesian frameworks can be used to study longitudinal trajectories. We studied the performance of both frameworks on different dataset configurations using hippocampal volumes from longitudinal MRI data across groups—healthy controls (HC), mild cognitive impairment (MCI) and Alzheimer’s disease (AD) patients, including subjects that converted from MCI to AD. We started from a big database of 1250 subjects from the Alzheimer’s disease neuroimaging initiative (ADNI), and we created different reduced datasets simulating real-life situations using a random-removal permutation-based approach. The number of subjects needed to differentiate groups and to detect conversion to AD was 147 and 115 respectively. The Bayesian approach allowed estimating the LME model even with very sparse databases, with high number of missing points, which was not possible with the frequentist approach. Our results indicate that the frequentist approach is computationally simpler, but it fails in modelling data with high number of missing values.

Correlated RNN Framework to Quickly Generate Molecules with Desired Properties for Energetic Materials in the Low Data Regime.

Motivated by the challenging of deep learning on the low data regime and the urgent demand for intelligent design on highly energetic materials, we explore a correlated deep learning framework, which consists of three recurrent neural networks (RNNs) correlated by the transfer learning strategy, to efficiently generate new energetic molecules with a high detonation velocity in the case of very limited data available. To avoid the dependence on the external big data set, data augmentation by fragment shuffling of 303 energetic compounds is utilized to produce 500,000 molecules to pretrain RNN, through which the model can learn sufficient structure knowledge. Then the pretrained RNN is fine-tuned by focusing on the 303 energetic compounds to generate 7153 molecules similar to the energetic compounds. In order to more reliably screen the molecules with a high detonation velocity, the SMILE enumeration augmentation coupled with the pretrained knowledge is utilized to build an RNN-based prediction model, through which R2 is boosted from 0.4446 to 0.9572. The comparable performance with the transfer learning strategy based on an existing big database (ChEMBL) to produce the energetic molecules and drug-like ones further supports the effectiveness and generality of our strategy in the low data regime. High-precision quantum mechanics calculations further confirm that 35 new molecules present a higher detonation velocity and lower synthetic accessibility than the classic explosive RDX, along with good thermal stability. In particular, three new molecules are comparable to caged CL-20 in the detonation velocity. All the source codes and the data set are freely available at https://github.com/wangchenghuidream/RNNMGM.

A Study of Performance and Comparison of NoSQL Databases: MongoDB, Cassandra, and Redis Using YCSB

Background/Objectives: Relational databases are a commonly utilized technology that allows for the storage, administration, and retrieval of various data schemas. However, for certain big databases, executing queries can become a time-consuming and inefficient procedure. Furthermore, storing enormous volumes of data necessitates servers with greater capacity and scalability. Relational databases have limits when it comes to dealing with scalability for big amounts of data. On the other hand, non-relational database systems, often known as NoSQL, were created to better fulfill the demands of key-value storing of enormous volumes of records. However, there are several NoSQL options, and the majority have not yet been extensively compared. The goal of this research is to examine different NoSQL databases and evaluate their performance in terms of typical data storage and retrieval. Methods: In this study, we use the YCSB tool to measure the performance of three NoSQL databases: MongoDB, Cassandra, and Redis. We test six different workloads with 100000, 250000, 500000, 750000, and 1000000 operations. Our test was designed with five different operations, i.e., 100000, 250000, 500000, 750000, and 1000000, with six different workloads to see which database is most suitable for applications which use a large amount of data to process. Findings: MongoDB is a superior performing NoSQL database among Cassandra and Redis. The numerous optimizations used by the designers of NoSQL solutions to improve performance, such as good cache memory operation, have a direct impact on the execution time. In all workloads except workload D, MongoDB has significantly reduced latency across all operation counts. Novelty: We also measure the average latency of different workload scenarios that include a mix of read, write, and update activities. Keywords: NoSQL; YCSB; big data; cloud computing; MongoDB; Cassandra; Redis