NoSQL MongoDB Research Articles

PENERAPAN METODE SIMPLE ADDITIVE WEIGHTING UNTUK MEMILIH JURUSAN PADA KAMPUS NEGERI

Students in grade twelfth high school must be able to determine their individual student interests to be able to continue further studies at university. But there are still many students who have not been able to determine their specialization even though they have sufficient understanding and knowledge of learning during high school. This study aims to apply simple additive weighting and calculation of the tightness capacity of each specialization major on a public campus in making a Decision Support System in determining student majors in the Angkasa High School environment. By paying attention to the weight and each criterion for the specialization major, then calculate with the tight capacity of the specialization department on each state campus so that the results issued by the calculation have provided decision support for students to determine specialization and state campuses more precisely. The decision support system uses the Kotlin programming language and Android-based NoSQL MongoDB database. The results of the manual calculation of the Simple Additive Weighting and Opportunity method with a decision support system reach 99% according.

Smart Contract Authentication assisted GraphMap-Based HL7 FHIR architecture for interoperable e-healthcare system

The exponential growth in the global population and significant advancements in healthcare broadened the scope of intervention for e-Healthcare through decentralized data access and information exchange, making complex clinical decisions. e-Healthcare can perform several functionalities, including EHR communication, telemedicine, and complex clinical decision systems (CCDS), but large-scale users still find it challenging to maintain interoperability, stability, and scalability. Accommodating an extensive array of stakeholders, which includes patients, doctors, hospitals, and laboratories, demands interoperability to serve scalable services. FHIR frameworks have played a vital role in e-Healthcare designs. Most of the existing HL7-FHIR frameworks have used REST-API using HTTP-query for CRUD tasks that impose numerous rules and constraints, making the process more complex and time-consuming, violating the quality-of-service (QoS) standards on different levels. This paper develops a novel, robust Smart-Contract Authentication Assisted HL7-FHIR framework toward an interoperable e-Healthcare solution. Unlike classical REST API-based FHIR, our proposed method applies a Graph-mapping concept that transforms each resource variable into an equivalent Graph-Mapped Data Structure (GMS), which is subsequently stored in the NoSQL MongoDB database, reducing computational costs and time to meet QoS demands. The proposed model employs three key components, GMS-driven HL7 FHIR Gateway Model, Smart Contract Authentication and Client Model. The Smart Contract function helped verify and authenticate users to ensure privacy and secure EHR exchange. The assessment of the performance of the proposed model reveals a significant reduction in computational time with optimal resource utilization making it a significant and viable option to better the real-world e-Healthcare mechanisms.

BIG DATA ANALYTICS: INTEGRATION, IMPORT SUBSTITUTION, OPERATIVENESS, RELIABILITY AND AVAILABILITY

Big data analytics faces the complexities of dealing with big, unstructured, and rapidly changing data that traditional methods of managing are faltering. The exponential growth of the internet and the digital economy has led to a huge increase in demand for data storage and data analysis. This new type of information, often referred to as unstructured data or big data, includes documents, images, audio, video, and social media content, rather than the traditional structured data found in databases. Big Data Analytics aims to extract valuable insights from these vast amounts of information. The article explores diverse technologies, such as Apache Flume, Pig, Hive and ZooKeeper, along with the open-source MongoDB NoSQL database, which collectively make up the big data analytics system. This system is used to predict volumes, gain ideas, take proactive actions and make effective strategic decisions. The article delves into the implementation, use and impact of Big Data Analytics on the value of an enterprise, taking advantage of algorithms designed for large data sets such as Hadoop and MapReduce. In addition, he provides illustrative examples of big data processing, specifically leveraging the integration of Hadoop and Matlab to solve handwritten number recognition tasks with a neural network trained using the MNIST dataset. Moreover, the article briefly touches on software import substitution in the context of the Russian Railways company, noting the use of Postgres Pro and Greenplum DBMS, and the loginom analytical platform. The article also provides a brief description of approaches to predictive assessment of the operation, reliability and availability of Big Data Processing Systems.

A Distributed Big Data Analytics Architecture for Vehicle Sensor Data.

The unceasingly increasing needs for data acquisition, storage and analysis in transportation systems have led to the adoption of new technologies and methods in order to provide efficient and reliable solutions. Both highways and vehicles, nowadays, host a vast variety of sensors collecting different types of highly fluctuating data such as speed, acceleration, direction, and so on. From the vast volume and variety of these data emerges the need for the employment of big data techniques and analytics in the context of state-of-the-art intelligent transportation systems (ITS). Moreover, the scalability needs of fleet and traffic management systems point to the direction of designing and deploying distributed architecture solutions that can be expanded in order to avoid technological and/or technical entrapments. Based on the needs and gaps detected in the literature as well as the available technologies for data gathering, storage and analysis for ITS, the aim of this study is to provide a distributed architecture platform to address these deficiencies. The architectural design of the system proposed, engages big data frameworks and tools (e.g., NoSQL Mongo DB, Apache Hadoop, etc.) as well as analytics tools (e.g., Apache Spark). The main contribution of this study is the introduction of a holistic platform that can be used for the needs of the ITS domain offering continuous collection, storage and data analysis capabilities. To achieve that, different modules of state-of-the-art methods and tools were utilized and combined in a unified platform that supports the entire cycle of data acquisition, storage and analysis in a single point. This leads to a complete solution for ITS applications which lifts the limitations imposed in legacy and current systems by the vast amounts of rapidly changing data, while offering a reliable system for acquisition, storage as well as timely analysis and reporting capabilities of these data.

Smart Weather Data Management Based on Artificial Intelligence and Big Data Analytics for Precision Agriculture

Smart management of weather data is an essential step toward implementing sustainability and precision in agriculture. It represents an important input for numerous tasks, such as crop growth, development, yield, and irrigation scheduling, to name a few. Advances in technology allow collecting this weather data from heterogeneous sources with high temporal resolution and at low cost. Generating and using these data in their raw form makes no sense, and therefore implementing adequate infrastructure and tools is necessary. For that purpose, this paper presents a smart weather data management system evaluated using data from a meteorological station installed in our study area covering the period from 2013 to 2020 at a half-hourly scale. The proposed system makes use of state-of-the-art statistical methods, machine learning, and deep learning models to derive actionable insights from these raw data. The general architecture is made up of four layers: data acquisition, data storage, data processing, and application layers. The data sources include real-time sensors, IoT devices, reanalysis data, and raw files. The data are then checked for errors and missing values using a proposed method based on ERA5-Land reanalysis data and deep learning. The resulting coefficient of determination (R2) and Root Mean Squared Error (RMSE) for this method were 0.96 and 0.04, respectively, for the scaled air temperature estimate. The MongoDB NoSQL database is used for storage thanks to its ability to deal with real-world big data. The system offers various services such as (i) weather time series forecasts, (ii) visualization and analysis of meteorological data, and (iii) the use of machine learning to estimate the reference evapotranspiration (ET0) needed for efficient irrigation. To this, the platform uses the XGBoost model to achieve the precision of the Penman–Monteith method while using a limited number of meteorological variables (air temperature and global solar radiation). Results for this approach give R2 = 0.97 and RMSE = 0.07. This system represents the first incremental step toward implementing smart and sustainable agriculture in Morocco.

Metadata-driven Data Migration from Object-relational Database to NoSQL Document-oriented Database

The object-relational databases (ORDB) are powerful for managing complex data, but they suffer from problems of scalability and managing large-scale data. Therefore, the importance of the migration of ORDB to NoSQL derives from the fact that the large volume of data can be handled in the best way with high scalability and availability. This paper reports our metadata-driven approach for the migration of the ORDB to document-oriented NoSQL database. Our data migration approach involves three major stages: a preprocessing stage, to extract the data and the schema's components, a processing stage, to provide the data transformation, and a post-processing stage, to store the migrated data as BSON documents. The approach maintains the benefits of Oracle ORDB in NoSQL MongoDB by supporting integrity constraint checking. To validate our approach, we developed OR2DOD (Object Relational to Document-Oriented Databases) system, and the experimental results confirm the effectiveness of our proposal.

AUTO SCALING DATABASE SERVICE WITH MICRO KUBERNETES CLUSTER

Data storage media, or what is often referred to as a database is something that is quite vital for technological developments. As the amount of data increases, it allows database services to experience downtime. For this reason, it is necessary to build an infrastructure that can replicate itself, so that it will avoid downtime. This infrastructure can be built using a container orchestration tool called Kubernetes which has high availability and autoscaler features, so it can replicate and guarantee service availability, to avoid downtime. This research builds a MongoDB NoSQL database service. This service is built using micro Kubernetes clusters from several different data centers. This service also implements a horizontal pod autoscaler feature that is capable of replicating pods, to increase high availability and avoid downtime. The autoscaling process will be tested by providing a load request for the service. Testing is done several times on each server. This study will compare the MongoDB service that was built monolithically with a micro Kubernetes cluster, and with HPA features and without HPA features by paying attention to several things. Based on Response Time, Response Code per Seconds, and CPU Usage, the results obtained are that the service built on a micro Kubernetes cluster with HPA features is the best, with a constant response time value below 100 ms, Response Code per Seconds reaches 500 threads per second. seconds, and CPU Usage in the range of 30 – 55%.

Real-time internet of medical things framework for early detection of Covid-19.

The Covid-19 pandemic is a deadly epidemic and continues to affect all world. This situation dragged the countries into a global crisis and caused the collapse of some health systems. Therefore, many technologies are needed to slow down the spread of the Covid-19 epidemic and produce solutions. In this context, some developments have been made with artificial intelligence, machine learning and deep learning support systems in order to alleviate the burden on the health system. In this study, a new Internet of Medical Things (IoMT) framework is proposed for the detection and early prevention of Covid-19 infection. In the proposed IoMT framework, a Covid-19 scenario consisting of various numbers of sensors is created in the Riverbed Modeler simulation software. The health data produced in this scenario are analyzed in real time with Apache Spark technology, and disease prediction is made. In order to provide more accurate results for Covid-19 disease prediction, Random Forest and Gradient Boosted Tree (GBT) Ensemble Learning classifiers, which are formed by Decision Tree classifiers, are compared for the performance evaluation. In addition, throughput, end-to-end delay results and Apache Spark data processing performance of heterogeneous nodes with different priorities are analyzed in the Covid-19 scenario. The MongoDB NoSQL database is used in the IoMT framework to store big health data produced in real time and use it in subsequent processes. The proposed IoMT framework experimental results show that the GBTs classifier has the best performance with 95.70% training, 95.30% test accuracy and 0.970 area under the curve (AUC) values. Moreover, the promising real-time performances of wireless body area network (WBAN) simulation scenario and Apache Spark show that they can be used for the early detection of Covid-19 disease.

A Comparative Study of MongoDB and Document-Based MySQL for Big Data Application Data Management

In the context of the heavy demands of Big Data, software developers have also begun to consider NoSQL data storage solutions. One of the important criteria when choosing a NoSQL database for an application is its performance in terms of speed of data accessing and processing, including response times to the most important CRUD operations (CREATE, READ, UPDATE, DELETE). In this paper, the behavior of two of the major document-based NoSQL databases, MongoDB and document-based MySQL, was analyzed in terms of the complexity and performance of CRUD operations, especially in query operations. The main objective of the paper is to make a comparative analysis of the impact that each specific database has on application performance when realizing CRUD requests. To perform this analysis, a case-study application was developed using the two document-based MongoDB and MySQL databases, which aim to model and streamline the activity of service providers that use a lot of data. The results obtained demonstrate the performance of both databases for different volumes of data; based on these, a detailed analysis and several conclusions were presented to support a decision for choosing an appropriate solution that could be used in a big-data application.

Діагностична система аналізу log-файлів із віддалених вузлів обробки даних

To monitor and maintain the proper functioning of distributed systems, there are used log events to further analyze and identify the circumstances and causes of deviations from normal operation. Tools to achieve this goal should primarily provide: 1) rapid search and processing of log files in the process of analysis and identification of failure causes; 2) generation of offers for solving the identified problem. This article presents the results of the analysis of the exist-ing tools for logging and log file processing and substantiates the feasibility of the development of the proposed diagnostic system. The general structure of the developed diagnostic system and its functional model are described. In the paper, there are considered the functions of the main software modules: a parser that processes a log file and collects parts of messages into a single log object; a diagnostic module that processes information about an error in a log file ob-ject; and a module for accumulating and using knowledge about the known node malfunctions, ways to fix them and problems that were not previously identified. The choice of MongoDB NoSQL for saving the log file data and knowledge base storage DBMS is substantiated. A uni-fied structure for log file objects has been developed for the MongoDB environment. Diagnos-tics system control – interoperability of modules and their interaction with the database and knowledge base in the MongoDB environment – is provided by the main program LogHelper which allows the user to perform different queries on log objects, monitor their results, inspect known and not previously identified errors, work with unknown malfunctions, manually set their statuses and find a solution from an existing set of solutions. The results of the conducted research show that the developed diagnostic system is highly effective as a maintenance solu-tion for remote data processing nodes and it supports rapid identification of the malfunction and generation of proposed solutions.

SMAP FRAMEWORK: A NOSQL SOLUTION FOR MANAGING A JSON KNOWLEDGE BASE OF SMART POLITICAL DOCUMENTS

This paper is devoted to developing a MongoDB framework for managing a JSON Knowledge base of “Smart Political Documents” (SPD). For this purpose, we designed a full knowledge base in a JSON Repository that runs under a MongoDB NoSQL server. The study of the experimentally collected data from the United Nations site has shown that our SPD knowledge base will contain various information using different formats (text, image, audio, video, etc.). The representation of these data requires both hierarchical and relational models. In this respect, we are proposing a NoSQL data model based on both JSON data and relational representation. JSON collections can be related to one another to complete information by special JSON properties. We have also defined a JSON Query Language for SMAP, SPQL (Smart Politics Query Language), which provided a powerful environment for integrating these disparate forms of data. In this paper, we will so describe the repository architecture, a Speech Labeling Technique used to associate information to spoken discourses, and finally, the whole SMArt Political documents (SMAP) framework that is based on a 3-tiers architecture and a Big Data environment with automatic deductions from the analysis of these data. The new system's purpose is to offer the end-user an intelligent way to quickly find the desired documents with all semantic links between them—something impossible in the system's initial state. The actual novelty of the system is represented mainly by the combination of the NoSQL feature with a relational model and an intelligent linked data system.

Comparative Case Study Difference Between Azure Cloud SQL and Mongo Atlas MongoDB NoSQL Database

Azure SQL and Atlas Mongodb NoSQL(Azure instance) databases are the most popular, systematic process to database solutions. Which Azure SQL database is also referred to as RDBMS (Relational Database Management Systems). The data are structured into tables or associations. The Atlas Mongodb NoSQL database is called a non-relational database management systems. The data are included in unstructured tables or associations. In this research, evaluate both the Azure SQL and Atlas Mongodb NoSQL databases. During the experiment compare the loading time, response time, and retrieval time of both Azure SQL and Atlas Mongodb NoSQL databases, and justify which one is fast, efficient and better performance.

The Integration of the MDA Approach in Document-Oriented NoSQL Databases, the case of Mongo DB

Today with the growth of the internet, the use of social networks, mobile telephony, connected and communicating objects. The data has become so big, hence the need to exploit that data has become primordial. In practice, a very large number of companies specializing in the health sector, the banking and financial sector, insurance, manufacturing industry, etc… are based on traditional databases which are often well organized of customer data, machine data, etc ... but in most cases, very large volumes of data from these databases, and the speed with which they must be analyzed to meet the business needs of the company are real challenges. This article aims to respond to a problem of generating NoSQL MongoDB databases by applying an approach based on model-driven engineering (Model Driven Architecture Approach). We provide Model to Model (using the QVT model transformation language), and Model to Code transformations (using the code generator, Acceleo). We also propose vertical and horizontal transformations to demonstrate the validity of our approach on NoSQL MongoDB databases. We have studied in this article the PSM transformations towards the implementation. PIM to PSM transformations are the subject of another work.

Integrated Document-based Electronic Health Records Persistence Framework

Electronic health record systems work beyond just recording patients` health data. They have multiple secondary functionalities, such as data reporting and clinical decision support. As each of these systems` workloads has contradictory different needs, managing a multipurpose electronic health record is a challenge. This paper proposes a unified healthcare data framework that can simplify health information system infrastructure. It investigates the suitability of the document-based NoSQL persistence mechanism, storing electronic health records data as a design choice for managing varied complexity ad hoc queries used in operational business intelligence. The performance of the most popular two document-based NoSQL back-ends, Couchbase Server and MongoDB, is compared according to the size of the database and query execution time. Results showed that while MongoDB can execute simple single-document queries nearly in milliseconds. It does not provide satisfactory response time for unplanned complex queries spanning multiple documents. By utilizing its analytics services and multi-dimensional scaling architecture, Couchbase Server multi-node cluster outperforms the response times of MongoDB for both simple and complex healthcare data access patterns. The primary advantage of the proposed tightly coupled EHRs processing framework is its flexibility to manage workload according to changing requirements.

Prototyping a Disciplinary Information Space for a New Smart Learning Management System Based on IMS-LD and NoSQL

<strong> This work is mainly based on the prototyping of a disciplinary information space for a new LMS. Our work is first to think about the conditions for creating a real LMS between learners and teachers. We have chosen to base ourselves on the hybridization between four learning theories as the basis for teaching this LMS, namely traditional pedagogy, behaviorism, cognitivism and social constructivism. These pedagogies have already proven themselves, mainly in classroom learning situations. The LMS consists of different activity spaces for teaching and learning activities. We limit ourselves in this work to the prototyping of disciplinary information space for a new LMS based on IMS-LD, NoSQL MongoDB database, and Symfony framework.</strong>

Efficient QoS-Aware Spatial Join Processing for Scalable NoSQL Storage Frameworks

Current cloud-enabled NoSQL database frameworks support flexible and scalable storage of huge amounts of data arriving through various and often heterogeneous channels. However, they do not natively provide optimised processing of spatial data, thus making it more difficult to perform accurate data analytics needed in many smart city application scenarios. To improve the performance of spatial data computation in the NoSQL MongoDB storage framework, this article proposes a novel data partitioning method based on dimensionality reduction. The underlying key idea is to reduce a spatial data representation from multi to single dimensionality, by still maintaining its geometrical meaning and by employing a specific geo-encoding scheme, i.e., a geohash string. In particular, the geohash string is used as a sharding key in order to store geometrically-nearby objects into the same chunks (and consequently into the same shard). In addition, as a distinctive feature, we have extended the MongoDB framework with a custom spatial QoS-aware optimizer that exploits our novel partitioning scheme to support two, typically expensive, types of spatial queries with QoS guarantees. Those queries are containment (and consequently top-N) and proximity. The paper also contributes to the existing literature with extensive experimental results about the performance of both our partitioning method and query optimizer; the reported results show that our solutions outperform baselines by orders of magnitude.

A database for developing machine learning based disruption predictors

Machine learning based disruption prediction methods have exhibited good prediction performance with higher success rate, lower false alarm rate and earlier warning time than physical based methods. One important thing pushed recent advances in machine learning field is high-quality training data. So a database with rich set of accurate disruption related information is crucial to the development of a high performance disruption predictor. In order to develop machine learning based disruption predictors fast and iteratively on J-TEXT, a database dedicated for machine learning disruption prediction has been built. This database provides not only disruption related labels, interfaces for querying the data based on user proved filter and auto generation of training and test data, but also interface for benchmark the predictors. Its modular design allows us to plug-in various shot analysis modules which process diagnostic signals and generate different labels automatically. These modules can be scheduled and run parallel on a cluster which will speed up the shot analysis process. The generated labels are inserted into a MongoDB NoSQL database for later querying. But one major hurdle for machine learning disruption prediction is they perform un-acceptably poor on devices other than they are trained on. It requires data from many different tokamaks to possibly develop a cross machine predictor, so this database also has a data import module which reads diagnostic data from different data sources such as MDSplus and store them as HDF5 files with unified data structure on a parallel file system. It is easy to import data from different machines and provide a unified data access interface.

Integration of an information system for the management of the computer inventory of the Information and Communication Technologies Division of the Technological University of Tabasco

This article follows the project entitled "Control I / O accessories, computer equipment and telecommunications laboratories of the Division of Information Technology, radiofrequency (RFID) at the Technological University of Tabasco" financed with PROMEP, the project in question consists of two parts: hardware; involving RFID technology and software in which you have a web app, mobile app with access to a NoSQL database. Referencing the integration of front- end and back-end of the web app and the trend in data storage NoSQL MongoDB specific model.

Mortadelo: Automatic generation of NoSQL stores from platform-independent data models

In the last decade, several NoSQL systems have emerged as a response to the scalability problems manifested by classical relational databases when used in Big Data contexts. These NoSQL systems appeared first as physical-level solutions, initially lacking any design methodologies. After this initial batch of systems, several design methodologies for NoSQL have been recently created. Nevertheless, most of these methodologies target just one NoSQL paradigm. In addition, as each methodology uses a different conceptual modeling approach, NoSQL database designers would need to remake conceptual models as they switch from one NoSQL paradigm to another. Moreover, most of these design processes provide just a set of design heuristics and guidelines that database designers need to apply manually, which can be a time-consuming and error-prone process. To overcome these limitations, this article presents Mortadelo, a model-driven NoSQL database design process where, from a high-level conceptual model, independent of any specific NoSQL paradigm, an implementation for a concrete NoSQL database system can be automatically generated. Moreover, this database generation process can be customized, so that some design trade-offs can be managed differently according to each context needs. We evaluated Mortadelo’s capabilities by generating database implementations for several typical NoSQL case studies. In these cases, Mortadelo was able to generate implementations for the Cassandra and MongoDB NoSQL systems from the same conceptual data model. These implementations were similar to the ones generated by design methodologies specifically developed for a single paradigm. Therefore, design quality is not sacrificed by our approach in favor of generality.

An Archetype Query Language interpreter into MongoDB: Managing NoSQL standardized Electronic Health Record extracts systems.

The fast development of today's healthcare and the need to extract new medical knowledge from exponentially-growing volumes of standardized Electronic Health Records data, as required by studies in Precision Medicine, brings up a challenge that may probably only be addressed using NoSQL DBMSs, due to the non-optimal performance of traditional relational DBMSs on standardized data; and these database systems operated by semantic archetype-based query languages, because of the expected generalized extension of standardized EHR systems. An AQL into MongoDB interpreter has been developed to its first version. It translates system-independent AQL queries posed on ISO/EN 13606 standardized EHR extracts into the NoSQL MongoDB query language. The new interpreter has the advantages of both the archetype-based system-independent AQL queries and the dual-model-based standardized EHR extracts stored on document-centric NoSQL DBMSs, such as MongoDB. AQL queries are independent of applications, programming languages and system environments due to the use of the dual model, but EHR extracts featuring this model are best persisted on document-based NoSQL databases. Consequently, the interpreter allows us to query standardized EHR extracts semantically, and also affording optimal performance.