Dynamic Data Management Research Articles

Random Data Distribution for Efficient Parallel Point Cloud Processing

Abstract. Current point cloud data management systems and formats are heavily specialized and targeted solely towards visualization purposes and fail to address the diverse needs of progressive point cloud workflows like for example semantic segmentation using machine learning. We therefore propose a distributed data infrastructure for dynamic point cloud data management that can support interactive real-time visualization at scale while simultaneously serving as a platform for analytical tasks. By introducing random data distribution, we show that simple query fragmentation and efficient and effective parallelism at scale are possible. At the same time, arbitrary queries in space and time can be efficiently run over the infrastructure including query semantics which returns only a random sample of the query results or preferred points based on an importance dimension calculated, for example, from a local point density information as commonly done in point cloud visualization. To cope with the unknown amount of user-specific attributes and to support even multiple ways of deciding the importance of a given point (ground point removal, coverage of space, random subset) the system is designed to support all of them transparently as multidimensional range queries backed by spatial indices.

Hyper: A High-Performance and Memory-Efficient Learned Index via Hybrid Construction

Learned indexes use machine learning techniques to improve index construction. However, they often face a fundamental trade-off between performance and memory consumption, especially in dynamic environments with frequent insert and delete operations. This trade-off stems from the construction approaches used in learned indexes: The top-down approach increases performance at the cost of significant memory overhead, while the bottom-up approach focuses on memory efficiency but introduces performance issues due to prediction errors. % A unified solution that simultaneously optimizes performance and memory consumption in dynamic data management scenarios is therefore highly desirable. We propose Hyper, a highly efficient learned index with a novel two-phase hybrid construction approach. Our approach combines bottom-up construction for leaf nodes with top-down construction for inner nodes to achieve an optimal balance between performance and memory consumption. Hyper effectively handles concurrent writes and structure adjustments without sacrificing query performance. We evaluated Hyper on both simple and complex real-world datasets and compared it to seven state-of-the-art learned indexes and several traditional data structures for dynamic workloads. The evaluation results show that Hyper achieves a remarkable performance boost of up to 3.75× with significantly reduced index memory consumption of up to 1610× in the single-thread evaluation. In high concurrency scenarios, Hyper even achieves improvements up to 5.73×, 3.72×, and 3.99× in read-only, read-write, and write-only workloads.

Enhancing water quality prediction for fluctuating missing data scenarios: A dynamic Bayesian network-based processing system to monitor cyanobacteria proliferation

Tackling the impact of missing data in water management is crucial to ensure the reliability of scientific research that informs decision-making processes in public health. The goal of this study is to ascertain the root causes associated with cyanobacteria proliferation under major missing data scenarios. For this purpose, a dynamic missing data management methodology is proposed using Bayesian Machine Learning for accurate surface water quality prediction of a river from Limia basin (Spain). The methodology used entails a sequence of analytical steps, starting with data pre-processing, followed by the selection of a reliable dynamic Bayesian missing value prediction system, leading finally to a supervised analysis of the behavioral patterns exhibited by cyanobacteria. For that, a total of 2,118,844 data points were used, with 205,316 (9.69 %) missing values identified. The machine learning testing showed the iterative structural expectation maximization (SEM) as the best performing algorithm, above the dynamic imputation (DI) and entropy-based dynamic imputation methods (EBDI), enhancing in some cases the accuracy of imputations by approximately 50 % in R2, RMSE, NRMSE, and logarithmic loss values. These findings can impact how data on water quality is being processed and studied, thus, opening the door for more reliable water management strategies that better inform public health decisions.

Leveraging Scratchpad Memory in a Hierarchical Architecture for Multicore

This paper proposes a novel architecture for multi-core processors, tailored for high-performance parallel computing. The architecture is founded on the innovative notion that complex problems can be decomposed into three relatively independent sub-problems: data processing, data management, and data communication. It features a grid of small, programmable processing units intricately connected to their three neighbouring units, forming a physically scalable and fractal environment. With flexibility, modularity, and scalability as focal points, this architecture aims to address the anticipated real-time signal processing demands in future telecommunication and multimedia systems. One notable aspect of the proposed architecture is its direct support for object-oriented features at the hardware level. Adopting a hybrid approach with Scratchpad Memory (SPM) combined with Cache in the on-chip memory hierarchy enhances performance and adaptability for sophisticated multi-core applications. The study highlights SPM management and introduces a dynamic data management framework. Unlike traditional SPM allocation methods relying on compiler or profiling knowledge, the proposed approach leverages random sampling and probability theory to predict hot-access data during runtime. This dynamic memory access pattern guides SPM allocation, ensuring optimal utilization of SPM's advantages in access speed and energy consumption, complemented by hardware support from DataUnit. This paper presents a paradigm shift in multi-core processor architecture, offering advanced features to meet the evolving requirements of parallel computing, making it well-suited for future telecommunication and multimedia systems. This paper introduces a revolutionary multicore processor architecture designed for high-performance parallel computing. The foundation of the proposed architecture lies in the innovative concept that complex problems can be decomposed into three relatively independent sub-problems: data processing, data management, and data communication. The architecture features a grid of small, programmable processing units, each intricately connected to its three neighboring units, forming a physically scalable and fractal environment. With a focus on flexibility, modularity, and scalability, this architecture addresses the real-time signal processing demands anticipated in future telecommunication and multimedia systems. One distinctive aspect of the proposed architecture is its direct support for object-oriented features at the hardware level. The on-chip memory hierarchy adopts a hybrid approach with Scratchpad Memory (SPM) combined with Cache, enhancing performance and adaptability for sophisticated multi-core applications. The study emphasizes SPM management and introduces a dynamic data management framework. In contrast to traditional SPM allocation methods relying on compiler or profiling knowledge, the proposed approach leverages random sampling and probability theory to predict hot-access data during runtime. This dynamic memory access pattern guides SPM allocation, ensuring optimal utilization of SPM's advantages in access speed and energy consumption, complemented by hardware support from DataUnit. This paper presents a paradigm shift in multicore processor architecture, offering advanced features to meet the evolving requirements of parallel computing, making it well-suited for future telecommunication and multimedia systems

A Geospatial Decision Support System for Supporting the Assessment of Land Degradation in Europe

Nowadays, Land Degradation Neutrality (LDN) is on the political agenda as one of the main objectives in order to respond to the increasing degradation processes affecting soils and territories. Nevertheless, proper implementation of environmental policies is very difficult due to a lack of the operational, reliable and easily usable tools necessary to support political decisions when identifying problems, defining the causes of degradation and helping to find possible solutions. It is within this framework that this paper attempts to demonstrate a new valuable web-based operational LDN tool as a component of an already running Spatial Decision Support System (S-DSS) developed on a Geospatial Cyberinfrastructure (GCI). The tool could be offered to EU administrative units (e.g., municipalities) so that they may better evaluate the state and the impact of land degradation in their territories. The S-DSS supports the acquisition, management and processing of both static and dynamic data, together with data visualization and on-the-fly computing, in order to perform modelling, all of which is potentially accessible via the Web. The land degradation data utilized to develop the LDN tool refer to the SDG 15.3.1 indicator and were obtained from a platform named Trends.Earth, designed to monitor land change by using earth observations, and post-processed to correct some of the major artefacts relating to urban areas. The tool is designed to support land planning and management by producing data, statistics, reports and maps for any EU area of interest. The tool will be demonstrated through a short selection of practical case studies, where data, tables and stats are provided to challenge land degradation at different spatial extents. Currently, there are WEBGIS systems to visualize land degradation maps but—to our knowledge—this is the first S-DSS tool enabling customized LDN reporting at any NUTS (nomenclature of territorial units for statistics) level for the entire EU territory.

Research on the Visualization of Railway Signal Operation and Maintenance Based on BIM + GIS

To adapt to the “fine” and “extensive” management characteristics of railway signal equipment operation and maintenance, achieving real-time and interactive monitoring of signal equipment operation status, and developing an integrated approach to equipment operation and maintenance, this paper takes a comprehensive management perspective. To create a lightweight BIM model, the Garland folding algorithm is utilized to simplify the IFC file format. Building on this approach, the data are divided based on building component division standards to obtain separate files containing geometric information and semantic attributes. The geometric information files are converted to a 3D Tiles format, combining BIM semantic attributes with semantic attribute files through an intermediate format. Dynamic data management is achieved by setting the octree space index structure in combination with a view-frustum culling algorithm. Then, the 3D Tiles target file is imported into the Cesium platform, and Node.js is used to achieve three-dimensional visualization of railway signal operation and maintenance. The proposed method is verified using an inbound signal as an example to assess its feasibility. The results demonstrate the potential of the proposed method to achieve stable integration between BIM equipment full lifecycle maintenance and GIS geographical space display. Railway signal equipment is endowed with comprehensive one-click information query functions for equipment positioning and spatial analysis, improving the efficiency and scientific decision-making level of equipment operation and maintenance.

Technologies applied to information control in organizations: A review

In the contemporary global market, organizations face intense competition requiring the deployment of information technology (IT) for effective decision-making and to maintain market presence. This review article emphasizes the crucial role of information technologies in organizational decision-making processes. It outlines the evolution of data management tools, especially in the context of dynamic and contingent data management, and explores the adoption and implementation of technologies for information control within organizations. By analyzing various methodologies and results from existing research, the article illustrates the increasing reliance on IT to enhance organizational processes, decision-making, and overall competitiveness. Furthermore, it delves into the systematic review approach to understand the significance and application of information management technologies in organizational contexts, offering insights for entrepreneurs and academics. The review concludes with the assertion that proper information management through technology is vital for organizations to remain competitive in the ever-changing market landscape.

Dynamic Student Data Management Using Resource Optimization Technology in Higher Education Platforms

This study combines intelligent resource optimization technology to build a dynamic student data management model and suggests a fuzzy hierarchical network representation model based on isomorphism and homogeneity in order to increase the effectiveness of dynamic student data management in colleges and universities. The important semantics of nodes in the network are also captured in this study using fuzzy k-kernel decomposition as a technique for multigranularity partitioning. Based on the SIR model, FHNE compares the production of the sequence to the process of information transmission in the random walk stage, which increases the node sequence’s accuracy. According to the research, the dynamic student data management system that is used in the higher education platform that is suggested in this study can significantly increase the effectiveness of managing student data.

The role of sustainability knowledge-action platforms in advancing multi-stakeholder engagement on sustainability

Abstract Within the last decade, online sustainability knowledge-action platforms have proliferated. We surveyed 198 sustainability-oriented sites and conducted a review of 41 knowledge-action platforms, which we define as digital tools that advance sustainability through organized activities and knowledge dissemination. We analyzed platform structure and functionality through a systematic coding process based on key issues identified in three bodies of literature: (a) the emergence of digital platforms, (b) the localization of the sustainable development goals (SDGs), and (c) the importance of multi-level governance to sustainability action. While online collaborative tools offer an array of resources, our analysis indicates that they struggle to provide context-sensitivity and higher-level analysis of the trade-offs and synergies between sustainability actions. SDG localization adds another layer of complexity where multi-level governance, actor, and institutional priorities may generate tensions as well as opportunities for intra- and cross-sectoral alignment. On the basis of our analysis, we advocate for the development of integrative open-source and dynamic global online data management tools that would enable the monitoring of progress and facilitate peer-to-peer exchange of ideas and experience among local government, community, and business stakeholders. We argue that by showcasing and exemplifying local actions, an integrative platform that leverages existing content from multiple extant platforms through effective data interoperability can provide additional functionality and significantly empower local actors to accelerate local to global actions, while also complex system change.

Interlinked and integrated research support services

This poster introduces the efficient set of research support services available in University of Jyväskylä’s (JYU) Open Science Centre (OSC). The set includes several essential services for researchers to help them execute an open and responsible research process. The continuous improvement of research support services has led to an interlinked, integrated and harmonious set of technical solutions, processes and workflows which meet the needs of researchers and help them implement open science and FAIR policies in practice. They also save time and enable better working processes for the researchers but also for the experts conducting these services. The poster describes two main services which are interlinked and which both aim to make processes and workflows effortless. One of the main services is when metadata from research information system Converis is utilized using API in the self-archiving process of research articles to the institutional repository. OSC’s experts record the publication metadata on behalf of researchers and the same metadata of the publications is used in the self-archiving process. Also, data harvesting is used to get notification of new publications of researchers affiliated to JYU. Often the researchers don’t even have to inform OSC when their new publication is out. This effortless process combines two different systems (CRIS and institutional repository) and connects workflows regarding them. The poster also introduces a dynamic research data management tool which enables a fluent data management process. The goal of the development of this tool has been to free the researchers of having to learn the use of several different systems. The tool helps the researchers to track the handling process of their data and guides through all the necessary phases of the data management process. Again, the data once recorded is being utilized in other systems and in other phases of the whole process. Both the tool and the data management process it enables are integrated to other systems which are needed for example in storing and sharing the research data.

Digital Twin based built environment asset management services development

The Digital Twin (DT) concept emerged recently in the Architecture, Engineering, Constructions and Operations domain and the interest for digital built asset management service implementations is increasing. However, most of the applications are developed according to custom approaches, preventing the generalisation and modularisation. This article presents a methodological framework for DT-based applications development. DT-based applications support effective data-centric processes in the management of critical assets. The proposed approach is based on a flexible and federated data model, enabling dynamic data management. The federated data model provides an interface and connection among different knowledge domains. The service requirement definition provides the input specification for the application development, accomplished through the modelling of three fundamental components: data, processes, and algorithms. These components are assembled through Application Programming Interfaces (APIs), supporting the DT-based services. The proposed approach is applied to the West Cambridge DT research facility and tested in the building energy sub-metering strategy. The developed approach allows flexible and interoperable data-centric applications development, contributing to the generalisation and scalability of DT-based built asset management applications. This aims at providing a consistent approach that can be employed in different domains, where the DT technologies play a central role in service innovation.

Fully Dynamic Data Management in Cloud Storage Systems with Secure Proof of Retrievability

For reliable storage services, we need a way not only to monitor the state of stored data but also to recover the original data when some data loss is discovered. To solve the problem, a novel technique called HAIL has been proposed. Unfortunately, HAIL cannot support dynamic data which is changed according to users' modification queries. There are many applications where dynamic data are used. So, we need a way to support dynamic data in cloud services to use cloud storage system for various applications. In this paper, we propose a new technique that can support the use of dynamic data in cloud storage systems. For dynamic data update, we design a new data chunk generation strategy which guarantee efficient data insertion, deletion, and modification. Our technique requires O(1) operations for each data update when existing techniques require O(n) operations where n is the size of data.

Flexible privacy‐preserving machine learning: When searchable encryption meets homomorphic encryption

Now various privacy-preserving techniques have been combined with machine learning to ensure training data security. However, during the training, users were unable to select data and labels adaptively in the server, which made the trained models challenging in meeting user needs. Repeated model training not only wastes server resources, but also reduces query efficiency. In this paper, we combine two privacy-preserving technologies, searchable encryption and homomorphic encryption, and propose a highly flexible machine training framework. Homomorphic encryption technology is used to encrypt data, and symmetric searchable encryption technology is used to generate blind indexes and trapdoors. The server can perform ciphertext data search through blind query trapdoors and indexes, and can use the searched homomorphic ciphertext for model training. This framework significantly improves the flexibility of training and the fit of the model, and also ensures the security of dynamic data management. The server locally uses the header of the query trap and the storage address of the ciphertext model to build an automatically updatable ciphertext model table. On the basis of ensuring the effectiveness of the model, it greatly improves the efficiency of obtaining the model, effectively prevents redundant training when different users send the same request, and reasonably allocates resources in the server.

Devising a traffic control method for unmanned aerial vehicles with the use of gNB-IOT in 5G

UAVs or drones as an alternative solution to providing high-quality Internet service in difficult terrain are environmentally friendly and do not consume electricity during the day as is the case with communication towers. But the developers of the network face difficulties in the drone communication system associated with the need to take into consideration unpredictable weather conditions and terrain, as well as the short life of the drone's batteries. Therefore, the object of this study is the process of managing UAV traffic through the use of gNB-IoT in 5G. The possibility of using a mobile UAV repeater during traffic management using radio resources (RR), radio access network (RAN), the infrastructure with broadcasting tools and dynamic connection using MU-MIMO modulation is shown. The use of these tools makes it possible to connect the drone to the wired base network from the provider and then restore the radio frequency signal and broadcast to another coverage area where this subscriber does not have network coverage, use the channel quality indicator (CQI) representation as a QoE function. Undoubtedly, traffic management is the process of obtaining information about traffic control from one endpoint to another, which confirms the reliability and management of data transmission. Meanwhile, drone traffic management can be used to reduce time delays and remove network interference by relying on Internet of Things programs that use NB-5G technology. The UAV's traffic management improvement process uses a proposed algorithm to generate dynamic flow data management to enhance traffic processing of flow control in the IoT

A geospatial decision support system for ecotourism: a case study in the Campania region of Italy

Nowadays, the business of ecotourism and rural tourism is a very important activity for many inland areas, enabling them to both produce important income and sustain the rural economy. Moreover, sustainable tourism is included in the UN 17th Sustainable Development Goal, which is to be achieved by 2030 (SDG 8.9 and SDG 12). Recent progress in digital tourism promises to deliver important changes to this activity, but most of this progress is addressed at well-known tourist destinations, so not dealing with the challenge of inland, rural and ecotourism. It is within this framework that this paper attempts to demonstrate that a new type of geoSpatial Decision Support System (S-DSS), developed on a Geospatial Cyberinfrastructure (GCI) and with a substantial interdisciplinary core, could provide a valuable web-based operational tool which may be offered to both ecotourism and rural tourism end-users, planners and policy makers, so that they might better plan and manage this type of sustainable tourism. The S-DSS platform has also been designed to encourage use by the multi-user community (farmers, tourism enterprises, associations and public bodies). The methodology is linked to the creation of a GCI platform (www.landsupport.eu) that supports the acquisition, management, processing and analysis of both static (e.g. soil, geology) and dynamic data (e.g. environmental and daily climatic data), together with data visualization and computer on-the-fly applications, in order to perform modelling, all of which is potentially accessible via the Web. The S-DSS tool known as EcoSmarTour is demonstrated through a case study of the Campania region (South Italy) and, by connecting database and modelling, it aims to deliver a large amount of information that will improve knowledge of the territory, manage scenario analysis, produce maps and evaluate potential ecotourism footpaths or areas of interest, thus enabling the provision of better information on the entire ecotourism sector. The tool will also be demonstrated through reference to a short selection of additional use cases from elsewhere in Europe. Most importantly, the approach adopted is highly transferable because it relies on very general algorithms that can be easily applied wherever the necessary data are available.

Implementation of a data control framework to ensure confidentiality, integrity, and availability of high-quality real-world data (RWD) in the NeuroTransData (NTD) registry.

ObjectiveTo implement a dynamic data management and control framework that meets the multiple demands of high data quality, rigorous information technology security, and flexibility to continuously incorporate new methodology for a large disease registry.Materials and MethodsGuided by relevant sections of the COBIT framework and ISO 27001 standard, we created a data control framework supporting high-quality real-world data (RWD) studies in multiple disease areas. We first mapped and described the entire data journey and identified potential risks for data loss or inconsistencies. Based on this map, we implemented a control framework adhering to best practices and tested its effectiveness through an analysis of random data samples. An internal strategy board was set up to regularly identify and implement potential improvements.ResultsWe herein describe the implementation of a data management and control framework for multiple sclerosis, one disease area in the NeuroTransData (NTD) registry that exemplifies the dynamic needs for high-quality RWD analysis. Regular manual and automated analysis of random data samples at multiple checkpoints guided the development and implementation of the framework and continue to ensure timely identification of potential threats to data accuracy.Discussion and conclusionsHigh-quality RWD, especially those derived from long-term disease registries, are of increasing importance from regulatory and reimbursement perspectives, requiring owners to provide data of comparable quality to clinical trials. The framework presented herein responds to the call for transparency in real-world analyses and allows doctors and patients to experience an immediate benefit of the collected data for individualized optimal care.

A geospatial database management system for the collection of medicinal plants.

Medicinal plants are increasingly used, both for medical applications and personal healthcare. However, existing herbal database systems for plant retrieval offer only basic information and do not support real-time analysis of the spatial aspects of plantations and distribution sites. Moreover, data records are usually static and not publicly available as they rely on costly proprietary software packages. To address these shortcomings, including limiting the time needed for collection and data processing, a novel medicinal plants geospatial database management system is proposed. The system allows localization of plant sites and data presentation on an interactive heat map displaying spatial information of plants selected by the user within a specific radius from the user's location, including automatic presentation of an itinerary giving the optimal route between user location and plant destinations selected. The approach relies on dynamic and role-based data management, an interactive map that includes graphics and integrated geospatial analyses thanks to cross-platform, geographical a JavaScript library and Google API. Both spatial data and attributes are available in real time. The system would support effective collaboration, among herb farmers, government agencies, private investors, healthcare professionals and the general public with regard to various aspects of medicinal plants and their applications.

RETRACTED ARTICLE: Lightweight security scheme for data management in E-commerce platform using dynamic data management using blockchain model

RETRACTED ARTICLE: Lightweight security scheme for data management in E-commerce platform using dynamic data management using blockchain model

System technologies for the formation of a data control contour for the personal account of an applicant based on the 1C:Enterprise 8.3 platform

The article considers the actual information technologies of the organization of a component distributed system for the distance enrollment of applicants within the framework of the 1C:Enterprise 8.3 platform. The concept of a modified component architecture of interaction with a web contour is proposed, which implementing the module of dynamic designing of web contour interface, as well as the automated functionality of data management directly on the standard 1C software products. The system principles for the organization of the contour of the website “Personal account of an applicant” have been formulated, they allow to integrate the management of the web contour into the circulation software solutions both for higher education (1C:University PROF) and for secondary vocational education (1C:College PROF) in terms of mechanisms for the dynamic designing of interfaces and the contour of data processing of site users. The implemented contour of designing site interfaces allows you to dynamically change the components of the site’s web forms, as well as to define the details displayed in the web user interface. A description of the mechanisms of dynamic interaction of the interfaces of the “Personal account of an applicant” site with the dynamic data management contour within the framework of the 1C:Enterprise 8.3 platform is given. The information components of the interaction contour of the site interfaces have been determined. The elements of the site control contour and their purpose have been specified. A conceptual universal scheme for the development of a web based data moderating contour has been formulated, and technologies for interaction with an internal accounting system for automating an admission campaign have been determined.

Network Variability Impacts Mobile Application Performance

Internet connectivity became a mandatory on mobile device in recent days. Due to memory constrain and dynamic data management developers were going for the web application or hybrid application development approach for the morden mobile applications. Web applications completely depend on the internet and hybrid applications partially depend on internet. For both cases the network connectivity is mandatory. This paper aim to compare the different speed of the network and based on that what is the impact of the mobile application performance has been analysed.