Automated Data Management Research Articles

Redefining Enterprise Data Management with AI-Powered Automation

In today's rapidly evolving digital landscape, the volume of enterprise data has surged exponentially, posing significant challenges in effective data management. Traditional data management techniques are becoming increasingly inadequate to handle the complexity and scale of modern enterprise data. This paper presents an innovative approach to revolutionize enterprise data management through AI-powered automation, a solution that enhances accuracy, efficiency, and decision-making processes within organizations. By leveraging advanced artificial intelligence technologies, such as machine learning, natural language processing, and predictive analytics, our proposed system aims to streamline data processing, ensure data quality, and provide real-time insights. This paper will discuss the limitations of existing data management systems, illustrate the novel methodologies integrated within our AI-driven framework, and demonstrate the system's efficacy through empirical results. The transformative potential of AI in automating data management processes not only addresses current challenges but also sets a foundation for future advancements in the field. As enterprises strive to maintain a competitive edge, the adoption of AI-powered automation for data management is not merely an option but a necessity for sustaining growth and innovation.

Graveyard Management with Geographical Information System and Operational Business Analytics for Negros Occidental

Managing cemeteries and memorial gardens is challenging due to manual processes, leading to inefficiencies and errors. This study introduced a geographical information system (GIS) with operational business analytics to address these challenges. As an initial deployment, this system covered only the cemeteries and memorial gardens in Negros Occidental (Western Visayas, Philippines). In addition, this study streamlined cemetery operations, improved service delivery, automated data management and analysis, and optimized gravesite maintenance. The system utilizes GIS technology to map the cemetery and visualize data, providing a complete overview of burial grounds. By leveraging operational data, the system enabled data-driven decision-making, optimized resource allocation, and predictive capabilities. The information system component allows authorized personnel to access and update burial records, automate billing processes, and generate reports. It eventually reduced the administrative tasks and ensured accurate and up-to-date information. In addition, the system has a tracking feature that simplifies ownership transfers. Furthermore, the system assisted in optimizing gravesite maintenance by providing tools to track and schedule necessary upkeep tasks. It ensured that gravesites were well-maintained, creating a respectful and pleasant environment for visitors. To enhance the visitor experience, the system incorporated a user-friendly interface. Hence, visitorscanr gravesites, leave tributes, and receive notifications using the system, creating a more personalized and meaningful experience for individuals paying their respects to their loved ones.

Method and the device of actualization of controlling effect loading of knot of the electric power system

Subject of research: is the technique of identification СCL in electric power system knots.
 Purpose of research: consists in working out of a technique of monitoring of changes КREL in electric power system knot.
 Methods of research: imitating modelling of electric modes, digital processing of signals, method of natural experiment.
 Main results of research: the technique of monitoring КREL of knot of an electric power system of digital processing of the information based on a method has been as a result developed, having author's specific features, also has criteria of suitability of data and the device realising a technique is developed. Experimental approbation of work of the device analyzing data from means of a telemetering, an operatively-measuring complex or automated data management system is spent. Device check has shown, that on an indication range 0,75 1,2Uном the error of identification КREL has made 0,2 % that speaks about high reliability of an offered technique of actualisation КREL.

Metadata-independent classification of MRI sequences using convolutional neural networks: Successful application to prostate MRI

PurposeThe ever-increasing volume of medical imaging data and interest in Big Data research brings challenges to data organization, categorization, and retrieval. Although the radiological value chain is almost entirely digital, data structuring has been widely performed pragmatically, but with insufficient naming and metadata standards for the stringent needs of image analysis. To enable automated data management independent of naming and metadata, this study focused on developing a convolutional neural network (CNN) that classifies medical images based solely on voxel data. MethodA 3D CNN (3D-ResNet18) was trained using a dataset of 31,602 prostate MRI volumes with 10 different sequence types of 1243 patients. A five-fold cross-validation approach with patient-based splits was chosen for training and testing. Training was repeated with a gradual reduction in training data assessing classification accuracies to determine the minimum training data required for sufficient performance. The trained model and developed method were tested on three external datasets. ResultsThe model achieved an overall accuracy of 99.88 % ± 0.13 % in classifying typical prostate MRI sequence types. When being trained with approximately 10 % of the original cohort (112 patients), the CNN still achieved an accuracy of 97.43 % ± 2.10 %. In external testing the model achieved sensitivities of > 90 % for 10/15 tested sequence types. ConclusionsThe herein developed CNN enabled automatic and reliable sequence identification in prostate MRI. Ultimately, such CNN models for voxel-based sequence identification could substantially enhance the management of medical imaging data, improve workflow efficiency and data quality, and allow for robust clinical AI workflows.

Efficient and Reliable Data Management for Biomedical Applications.

This chapter discusses the challenges and requirements of modern Research Data Management (RDM), particularly for biomedical applications in the context of high-performance computing (HPC). The FAIR data principles (Findable, Accessible, Interoperable, Reusable) are of special importance. Data formats, publication platforms, annotation schemata, automated data management and staging, the data infrastructure in HPC centers, file transfer and staging methods in HPC, and the EUDAT components are discussed. Tools and approaches for automated data movement and replication in cross-center workflows are explained, as well as the development of ontologies for structuring and quality-checking of metadata in computational biomedicine. The CompBioMed project is used as a real-world example of implementing these principles and tools in practice. The LEXIS project has built a workflow-execution and data management platform that follows the paradigm of HPC-Cloud convergence for demanding Big Data applications. It is used for orchestrating workflows with YORC, utilizing the data documentation initiative (DDI) and distributed computing resources (DCI). The platform is accessed by a user-friendly LEXIS portal for workflow and data management, making HPC and Cloud Computing significantly more accessible. Checkpointing, duplicate runs, and spare images of the data are used to create resilient workflows. The CompBioMed project is completing the implementation of such a workflow, using data replication and brokering, which will enable urgent computing on exascale platforms.

УПРАВЛІННЯ НЕСТРУКТУРОВАНИМИ ДАНИМИ ПРИ ДОСЛІДЖЕННІ ВПЛИВУ ЗАСОБІВ МАСОВОЇ ІНФОРМАЦІЇ НА СОЦІАЛЬНУ БЕЗПЕКУ ЛЮДИНИ В УМОВАХ ГІБРИДНОЇ ВІЙНИ

Russia’s war against Ukraine significantly affects the state of Ukraine's national security and the social security of every Ukrainian. Its negative consequences are both direct and indirect in nature, which is a real social threat. The degree (level) of protection of the vital socio-economic interests of a person (person), his rights, freedoms and values from internal and external, real, and potential threats is an important element of assessing the state of the state's national security. In the conditions when Russia's military and informational aggression is directed against Ukraine, the study of issues of media influence, both from the inside and from the outside, on public opinion, moods among different categories of citizens, ethnic communities, their manifestations, and expression of will regarding political and economic transformations in the country is important and relevant. An important element of the analysis of the state of human social security in the conditions of a hybrid war is the study and assessment of the structure and dynamics of the development of public opinion in certain regions of the country. One of the most serious ways of assessing public opinion and forecasting its changes is, along with sociological research, monitoring the influence of mass media on the population of the region. Traditionally, such influence is assessed by monitoring local mass media and evaluating them according to a few formal qualities, such as audience coverage, ownership structure, political connections of owners, signs of commitment to one or another political position, etc. Such an analysis allows with a certain degree of accuracy to record the presence of various centers of influence on public opinion, however, it is completely insufficient to assess the real influence of these centers on public opinion itself, since it does not allow to assess exactly how this or that information influenced its consumers. When processing information flows from mass media, social networks, e-mail, etc., there are constant problems of processing large volumes of data containing textual information, which, although related to location, is not presented in traditional structured spatial formats, and does not explicitly contain geographic coordinates. Instead, its connection to geography is hidden in unstructured or semi-structured formats based on plain text. Unstructured data is usually considered to be information that does not have a predefined data structure or is not organized in a fixed order. Unstructured data, as a rule, is presented in the form of text, which can contain data such as dates, numbers, and facts, which immediately leads to difficulties in analysis, especially in the case of using traditional programs designed to work with structured data. Unstructured data is usually stored in human-readable formats, although such formats make automatic data management difficult. In connection with the rapid pace of progress and development of geoinformation systems, the specialists of the company Esri (USA) comprehensively approached the problem of the combined use of unstructured data and geoinformation systems. The Locate XT module was created to solve this problem. The new ArcGIS LocateXT add-on is designed to search, extract, and map data from plain text, including geographic coordinates in various formats, place names, and other location-related information. Geoinformation technology LocateXT is an effective tool for research and analysis of the impact of mass media in the conditions of a hybrid war between Russia and Ukraine on human social security, efficiency, completeness and quality of information and spatial support of management processes. It helps speed up decision-making procedures based on quality management of unstructured data obtained from disparate sources. Modern geoinformation tools in the decision-making support system provide an opportunity to systematize unstructured data, determine their geospatial reference, and subsequently conduct analysis and form the obtained results in a form convenient for perception.

Now is the time to conquer the last mile to full data automation

Innovative new technologies and a desire to update the operating model are paving the way for financial services firms to automate data management across their organisation. This is something that historically has been impossible due to the proliferation of unstructured financial data that legacy technology is unable to handle. The resulting mass adoption of spreadsheets as a workaround created a ‘last mile’ of data processes that firms have been unable to automate. This paper explores the current technology landscape in financial services, the problems this has created for data management and the benefits for firms of using new tools to conquer the ‘last mile’ of automation.

Hardware and software implementation of POCT1-A for integration of point of care testing in research

Point of care testing (POCT) is increasingly utilized in clinical medicine. Small, portable testing devices can now deliver reliable and accurate diagnostic results during a patient encounter. With these increases in POCT, the issue of data and results management quickly emerges. Results need to be cataloged accurately and efficiently while the providers/support staff are simultaneously managing patient encounters. The integration of electronic medical records (EMR) as data repositories requires that point of care testing data imports automatically into the EMR. POCT1-A was developed as a standard communication language for POCT device manufacturers to streamline automatic data import integration. While all modern POCT devices are built with this connectivity, the systems that provide the integration layer are often proprietary and require a fee for service. In the research environment, there is not enough throughput to justify the practical investment in these data management architectures. Moreover, researcher needs are different and unique compared to data management systems for clinicians. To meet this need, we developed a novel hardware and software connectivity solution using commercially available components to automate data management from a point-of-care blood biochemical analyzer during a critical care study in the preclinical research environment.

Online system expedites requests for accommodation

An automated data management system at Texas Woman's University has simplified the process of requesting and implementing accommodations for students and faculty. Crystal Hill, Director of Disability Services for Students, discussed the transition to an online system.

Development of the automated data management system for radiation monitoring of forests

Since the system of radiation control in forestry was formed even before the mass introduction of information technologies, data processing in radiation monitoring retains the approaches conditioned by the traditional, paper-based document management. The aim of this project is to develop a unified automated system for collection, control and analysis of data from radiation monitoring of forests in the Russian Federation. The system is based on the use of the unified software ‘SpectraLineBG’ for spectrometric devices, on the creation of a cloud data storage server, and the development of a software environment for storing and processing of the obtained material. The system allows to check the accuracy of all the obtained results of measurements and to make their subsequent correction. The described method of data collection is universal for spectrometric studies and can be used not only in forestry, but also in any organization involved in radiation control of the environment. Using the accumulated material, the system allows to analyze the distribution and the dynamics of radionuclides in space and time, to calculate the coefficients of the transition of radionuclides from soil to vegetation for one or more species in different areas.

Human Motion Data Retrieval Based on Staged Dynamic Time Deformation Optimization Algorithm

In recent years, with the rapid development of computer storage capabilities and network transmission capabilities, users can easily share their own video and image information on social networking sites, and the amount of multimedia data on the network is rapidly increasing. With the continuous increase of the amount of data in the network, the establishment of effective automated data management methods and search methods has become an increasingly urgent need. This paper proposes a retrieval method of human motion data based on motion capture in index space. By extracting key frames from the original motion to perform horizontal dimensionality reduction and defining features based on Laban motion analysis, the motion segment is subjected to vertical feature dimensionality reduction. After extracting features from the input motion segment, motion matching is performed on the index space. This paper designs the optimization method of the phased dynamic time deformation algorithm in time efficiency and analyzes the optimization method of the phased dynamic time deformation algorithm in time complexity. Considering the time efficiency redundancy, this paper optimizes the time complexity of the phased dynamic time deformation method. This improves the time efficiency of the staged dynamic time warping algorithm, making it suitable for larger-scale human motion data problems. Experiments show that the method in this paper has the advantage of speed, is more in line with the semantics of human motion, and can meet the retrieval requirements of human motion databases.

A New Long-Term Marine Biodiversity Monitoring Program for the Knowledge and Management in Marine Protected Areas of the Mexican Caribbean

In the Mexican Caribbean, 15 marine protected areas (MPAs) have been established for managing and protecting marine ecosystems. These MPAs receive high anthropogenic pressure from coastal development, tourism, and fishing, all in synergy with climate change. To contribute to the MPAs’ effectiveness, it is necessary to provide a long-term observation system of the condition of marine ecosystems and species. Our study proposes the establishment of a new marine biodiversity monitoring program (MBMP) focusing on three MPAs of the Mexican Caribbean. Five conservation objects (COs) were defined (coral reefs, seagrass beds, mangroves, marine turtles, and sharks-rays) for their ecological relevance and the pressures they are facing. Coral reef, seagrass and mangroves have multiple biological, biogeochemical and physical interactions. Marine turtles are listed as endangered species, and the status of their populations is unknown in the marine area of the MPAs. Elasmobranchs play a key role as top and medium predators, and their populations have been poorly studied. Indicators were proposed for monitoring each CO. As a technological innovation, all information obtained from the MBMP will be uploaded to the Coastal Marine Information and Analysis System (SIMAR), a public, user-friendly and interactive web platform that allows for automatic data management and processing.

The Business Case for Automating Data Management

Back in 2016, IDC forecast a ten-fold increase on business data processed globally by 2025. Combine this with increasing regulatory compliance and demand for quality data, then it is perhaps not a surprise to find that many businesses are feeling somewhat over-whelmed by the enormity of the data management and governance task they face. But, don’t despair, there is a compelling business case, both in terms of efficiency gains and tangible financial savings in adopting a strategy for centralising information governance and automating the management of your data.
 This paper will explore the emerging problem domain of data and look at the business case for use of automated data management tools to better prepare your organisation for the tsunami of data coming its way.

Macroscopic mid-FTIR mapping and clustering-based automated data-reduction: An advanced diagnostic tool for in situ investigations of artworks

The present study describes a multivariate strategy that can be used for automatic on-site processing of reflection mode macro FTIR mapping (MA-rFTIR) data obtained during investigation of artworks. The chemometric strategy is based on the integration of principal component analysis (PCA) with a clustering approach in the space subtended by the three lowest-order principal components and allows to automatically identify the regions of interest (ROIs) of the area scanned and to extract the average FTIR spectra related to each ROI. Thanks to the automatic data management, in-field HSI (hyperspectral imaging)-based analyses may be performed even by staff lacking specific advanced chemometric expertise, as it is sometimes the case for conservation scientists or conservators with a scientific background. MA-rFTIR was only recently introduced in the conservation field and, in this work the technique was employed to characterize the surface of metallic artefacts. The analytical protocol was employed as part of a rapid procedure to evaluate the conservation state and the performance of cleaning methods on bronze objects. Both activities are commonly part of restoration campaigns of bronzes and require an on-site analytical procedure for efficient and effective diagnosis. The performance of the method was first evaluated on aged standard samples (bronzes with a layer of green basic copper hydroxysulphate, treated with different organic coatings) and then scrutinized in situ on areas of the 16th century Neptune fountain statue (Piazza del Nettuno, Bologna, Italy) by Gianbologna.

METHOD FOR PLANNING NON-DETERMINED OPERATION PROCESSES OF RAILWAY TECHNICAL SYSTEM PARK

Purpose. The article is aimed to improve the automated systems operation of the railway technical system parks and switch D.C. electric motors (EMs), taking into account all uncertainties. Methodology. Solution of the problem was obtained through the development of the model and the method for optimal planning for the EMs set operation. The method is based on the information technology with the possibility to assess the parameters of the current and the predicted state of EMs based on their individual models. The models are built both for individual EMs and for the specified groups. The factors of non-determinism in the model are calculated based on the Hurst index. The task of planning is solved as calculating the optimal sequence of the EM facilities services, which provides a minimum of the total expected operating costs. Findings. The analysis of the main known models, the automated technologies and the systems of EM (ASEM) park operation on the basis of the remote monitoring was done in the research. Based on the practice of the EM park maintenance the new category of the analysis objects was proposed – the service group (SG). The new procedure for the processes classification was developed based on using the Hurst index to improve the reliability of EM and SG individual models forecasting. The technological and the economic model for planning the EM parks operation was created. The article presents the results of the developed automated data management system based on the improved model for the operation planning of the D.C. EM parks. The optimal planning model ensures the minimization of the expected operating costs for the EMs operation, due to the selection of the EM groups service queue. The specialized procedure is used to classify non-deterministic EM remote monitoring data during planning, which allows increasing the accuracy of forecasting the object state parameters. Origilnality . The article describes development of the mathematical model and the information technology for the remote monitoring of the railway technical systems park operation, the railway switch EMs based on the formation of EM and SG individual models, as well as on the evaluation of their current and predicted states, taking into account random factors. The proposed model of the optimal planning as the possibility to choose the SG service queue differs by the group maintenance of the EM facilities, as well as application of the specialized procedure for classifying EM monitoring data. Practical value . The practical value of the results is determined by the provision of the new opportunities for the group optimal planning of the EM service based on the criterion of the minimum expected costs. The procedure for the monitoring data classification of the operational processes makes it possible to increase the reliability of the forecasting antipersistent time sequences results. It also provides an interpretation of the observational data classification results based on the need for practical usage.

Development and Preliminary Feasibility of an Automated Hypertension Self-Management System

BackgroundUncontrolled hypertension constitutes a significant challenge throughout the world. Blood pressure measurement by patients is informative for both patients and providers but is rarely performed systematically, thereby reducing its utility. Mobile phones can be used to efficiently prompt individuals to measure blood pressure and automate data management while avoiding technology barriers to widespread adoption. Presented is the design and pilot test results of MyBP, an automated texting intervention to support blood pressure self-monitoring and patient self-management. MethodsThree sequential phases are described: (1) stakeholders’ needs assessment, (2) preliminary design pilot (n = 10), and (3) a 6-week pilot of the redesigned comprehensive program with hypertensive patients (n = 43) recruited from 3 clinical sites (Emergency Department, Primary Care, Hypertension Center). Outcomes of interest included participant adherence, perceived importance of blood pressure monitoring, and healthy behavior change. ResultsMedian adherence to MyBP prompts over 6 weeks was 79% (72% emergency department, 84% primary care, and 96% hypertension center, H[2] = 5.56, P = .06). Adherence did not vary by age, sex, education, or baseline use of texting but was lowest among patients recruited from the emergency department (χ[2]2 = 6.66, P = .04). In the exit survey, MyBP was associated with increased importance of blood pressure self-monitoring and particularly motivated primary care and emergency department groups to improve dietary habits, increase daily physical activity, and focus on stress reduction. The majority of participants (88%) indicated interest in using the program for 6 months. ConclusionsAutomated mobile-phone–based blood pressure self-monitoring using MyBP is feasible, acceptable, and scalable, and may improve self-management and support clinical care.

MAPPI-DAT: data management and analysis for protein-protein interaction data from the high-throughput MAPPIT cell microarray platform.

SummaryProtein-protein interaction (PPI) studies have dramatically expanded our knowledge about cellular behaviour and development in different conditions. A multitude of high-throughput PPI techniques have been developed to achieve proteome-scale coverage for PPI studies, including the microarray based Mammalian Protein-Protein Interaction Trap (MAPPIT) system. Because such high-throughput techniques typically report thousands of interactions, managing and analysing the large amounts of acquired data is a challenge. We have therefore built the MAPPIT cell microArray Protein Protein Interaction-Data management & Analysis Tool (MAPPI-DAT) as an automated data management and analysis tool for MAPPIT cell microarray experiments. MAPPI-DAT stores the experimental data and metadata in a systematic and structured way, automates data analysis and interpretation, and enables the meta-analysis of MAPPIT cell microarray data across all stored experiments.Availability and ImplementationMAPPI-DAT is developed in Python, using R for data analysis and MySQL as data management system. MAPPI-DAT is cross-platform and can be ran on Microsoft Windows, Linux and OS X/macOS. The source code and a Microsoft Windows executable are freely available under the permissive Apache2 open source license at https://github.com/compomics/MAPPI-DAT.Supplementary information Supplementary data are available at Bioinformatics online.

Global‐view coefficients: a data management solution for parallel quantum Monte Carlo applications

SummaryQuantum Monte Carlo (QMC) applications perform simulation with respect to an initial state of the quantum mechanical system, which is often captured by using a cubic B‐spline basis. This representation is stored as a read‐only table of coefficients and accesses to the table are generated at random as part of the Monte Carlo simulation. Current QMC applications, such as QWalk and QMCPACK, replicate this table at every process or node, which limits scalability because increasing the number of processors does not enable larger systems to be run. We present a partitioned global address space approach to transparently managing this data using Global Arrays in a manner that allows the memory of multiple nodes to be aggregated. We develop an automated data management system that significantly reduces communication overheads, enabling new capabilities for QMC codes. Experimental results with QWalk and QMCPACK demonstrate the effectiveness of the data management system. Copyright © 2016 John Wiley & Sons, Ltd.

A secure, scalable and versatile multi-layer client–server architecture for remote intelligent data processing

In recent years, the need for data collection and analysis is growing in many scientific disciplines. This is consequently causing an increase of research in automated data management and data mining to create reliable methods for data analysis. To deal with the need for smart environments and big computational resources, some previous works proposed to address the problem by moving on remote processing, with the aim of sharing supercomputer resources, algorithms and costs. Following this trend, in this work we propose an architecture for advanced remote data processing in a secure, smart and versatile client–server environment that is capable of integrating pre-existing local software. In order to assess the feasibility of our proposal, we developed a case study in the context of an image-based medical diagnostic environment. Our tests demonstrated that the proposed architecture has several benefits: increase of the system throughput, easy upgradability, maintainability and scalability. Moreover, for the scenario we have considered, the system showed a very low transmission overhead which settles on about 2.5 % for the widespread 10/100 mbps. Security has been achieved using client–server certificates and up-to-date standards.

Automatic task mapping and heterogeneity-aware fault tolerance: The benefits for runtime optimization and application development

The best mapping of a task to one or more processing units in a heterogeneous system depends on multiple variables. Several approaches based on runtime systems have been proposed that determine the best mapping under given circumstances automatically. Some of them also consider dynamic events like varying problem sizes or resource competition that may change the best mapping during application runtime but only a few even consider that task execution may fail. While aging or overheating are well-known causes for sudden faults, the ongoing miniaturization and the growing complexity of heterogeneous computing are expected to create further threats for successful application execution. However, if properly incorporated, heterogeneous systems also offer the opportunity to recover from different types of faults in hardware as well as in software. In this work, we propose a combination of both topics, dynamic performance-oriented task mapping and dependability, to leverage this opportunity. As we will show, this combination not only enables tolerating faults in hardware and software with minor assistance of the developer, it also provides benefits for application development itself and for application performance in case of faults due to a new metric and automatic data management.