Data Processing Performance Research Articles

Enhancing Data Processing Efficiency : The Synergy of Edge Computing and Hybrid Cloud Storage

This comprehensive article explores the transformative integration of edge computing and hybrid cloud storage, a technological convergence that is reshaping data processing architectures in the era of exponential data growth. The research delves into the fundamental principles of edge computing and hybrid cloud storage, examining their synergistic relationship in addressing the limitations of traditional centralized cloud computing. By bringing computational resources closer to data sources, this integrated approach significantly reduces latency, enhances processing efficiency by up to 50%, and improves overall system reliability. The article presents detailed case studies in autonomous driving and smart city infrastructure, showcasing real-world applications and benefits. It critically analyzes the challenges inherent in this integration, including security concerns in decentralized architectures, data consistency issues, and cost implications. Furthermore, the article explores future directions, discussing emerging technologies such as AI-powered edge devices, evolving hybrid cloud solutions, and the potential for further optimization. This research provides valuable insights for organizations and researchers navigating the complex landscape of distributed computing, offering a roadmap for leveraging edge computing and hybrid cloud storage to achieve unprecedented levels of performance, scalability, and flexibility in data management and processing.

Design of a triggerless readout electronics system for CDEX-50

Direct detection of light-mass dark matter is a frontier topic in international physics research. The reduction of system threshold is important in order to improve the sensitivity of light-mass dark matter detection. The scientific goal of the China Dark Matter Experiment (CDEX) at Jinping Underground Laboratory in China is to detect WIMPs utilizing a high-purity germanium array detector. CDEX-10 achieved the most sensitive results in the 4 to 5 GeV/c 2 range. CDEX-50 aims to achieve an energy threshold of 100 eV, which significantly increases data bandwidth and complicates the implementation of noise reduction algorithms, thereby posing challenges to the readout electronics system. In this paper, a triggerless readout electronics system based on FPGA-GPU is designed for CDEX-50, which can achieve full energy range detection from 100 eV to 10 MeV. A verification prototype of a triggerless electronics system utilizing a Broad Energy Germanium (BEGe) detector has been developed to test the performance of high-bandwidth transmission and data processing. The results demonstrate that the bit error rate for the high-speed transmission link of the triggerless readout electronics system is below 1015. Furthermore, the FPGA-GPU transmission bandwidth, utilizing P2P DMA, achieves 100.2 Gbps, and the mean filter implemented on the GPU is capable of processing a 64 Gbps data stream in real-time. These results provide foundation for the design of the triggerless readout electronics system for CDEX-50.

Dynamic Performance and Power Optimization with Heterogeneous Processing-in-Memory for AI Applications on Edge Devices

The rapid advancement of artificial intelligence (AI) technology, combined with the widespread proliferation of Internet of Things (IoT) devices, has significantly expanded the scope of AI applications, from data centers to edge devices. Running AI applications on edge devices requires a careful balance between data processing performance and energy efficiency. This challenge becomes even more critical when the computational load of applications dynamically changes over time, making it difficult to maintain optimal performance and energy efficiency simultaneously. To address these challenges, we propose a novel processing-in-memory (PIM) technology that dynamically optimizes performance and power consumption in response to real-time workload variations in AI applications. Our proposed solution consists of a new PIM architecture and an operational algorithm designed to maximize its effectiveness. The PIM architecture follows a well-established structure known for effectively handling data-centric tasks in AI applications. However, unlike conventional designs, it features a heterogeneous configuration of high-performance PIM (HP-PIM) modules and low-power PIM (LP-PIM) modules. This enables the system to dynamically adjust data processing based on varying computational load, optimizing energy efficiency according to the application’s workload demands. In addition, we present a data placement optimization algorithm to fully leverage the potential of the heterogeneous PIM architecture. This algorithm predicts changes in application workloads and optimally allocates data to the HP-PIM and LP-PIM modules, improving energy efficiency. To validate and evaluate the proposed technology, we implemented the PIM architecture and developed an embedded processor that integrates this architecture. We performed FPGA prototyping of the processor, and functional verification was successfully completed. Experimental results from running applications with varying workload demands on the prototype PIM processor demonstrate that the proposed technology achieves up to 29.54% energy savings.

Mapping global research in the field of health biotechnology for the period 1990–2023. A bibliometric analysis

Purpose The purpose of this study focused on a global longitudinal bibliometric mapping of research in the field of health biotechnology between 1990 and 2023 to determine who is leading this field of knowledge and to estimate the sub-disciplines that are emerging and project those that will prevail in the future. Design/methodology/approach The study identified the most relevant countries, institutions and researchers, as well as the type of scientific collaborations. The applied steps applied in the study were the following: identification and selection of keyword terms by a panel of experts; design and application of an algorithm to identify these selected keywords in titles, abstracts and keywords using Web of Science terms to contrast them; performance of JCR data processing during 2023 using R, Python and VOSviewer. Findings Among the most relevant conclusions of the study are the following exponential growth has been observed in the study period; new branches of knowledge have emerged in which the subjects have been acquiring their own autonomous capabilities; and R&D in this field is still concentrated in a small group of core countries, and the trend is for it to remain so due to the capacity needs required. Originality/value This contribution seeks to systematize the existing scientific knowledge in the field of biotechnology, specifically in the area of health, using the technique of scientific mapping based on a logical model of indicators that aims to determine potential thematic ramifications.

Micromagnetic analysis of magnetic vortex dynamics for reservoir computing

Reservoir computing (RC) has generated significant interest for its ability to reduce computational costs compared to traditional neural networks. The performance of the RC element is quantified by its memory capacity (MC) and prediction capability. In this study, we utilize micromagnetic simulations to investigate a magnetic vortex based on a permalloy ferromagnetic layer and its dynamics in RC. The nonlinear dynamics of the vortex core (VC), driven by continuous oscillating magnetic fields and binary digit data as spin-polarized current pulses, are analyzed. The highest MC observed is 4.1, corresponding to the nonlinear VC dynamics. Additionally, the prediction capability is evaluated using the Nonlinear Auto-Regressive Moving Average 2 task, demonstrating a normalized mean squared error of 0.0241 highlighting the time-series data prediction performance of the vortex as a reservoir.

GreatBlue: a 55-Pound Vertical-Takeoff-and-Landing Fixed-Wing sUAS for Science; Systems, Communication, Simulation, Data Processing, Payloads, Package Delivery, and Mission Flight Performance

As small, uncrewed systems (sUAS) grow in popularity and in number, larger and larger drone aircraft will become more common–up to the FAA limit of 55 pound gross take-off weight (GTOW) and beyond. Due to their larger payload capabilities, longer flight time, and better safety systems, autonomous systems that maximize CFR 14 Part 107 flight drone operations regulations will become more common, especially for operations such as imagery or other data collection which scale well with longer flight times and larger flight areas. In this new paper, a unique all-electric 55-pound VTOL transition fixed-wing sUAS specifically engineered for scientific data collection named “GreatBlue” is presented, along with systems, communications, scientific payload, data collection and processing, package delivery payload, ground control station, and mission simulation system. Able to fly for up to 2.5 hours while collecting multispectral remotely-sensed imagery, the unique GreatBlue system is shown, along with a package delivery flight example, flight data from two scientific data collection flights over California almond fields and a Utah Reservoir are shown including flight plan vs. as-flown.

Fire Blackbox Control for Public Marketing

This paper presents the system for Fire Blackbox control in Public Marketing. When information is required following a fire incident, investigators or police might use the smart black box system to collect information. The black box system is used to continuously record, temperature, smoke, and other data. The information is then stored in a storage device. We are able to examine, track, and evaluate fire situations. In this paper, we propose a novel concept termed "Fire Black Box Control" (FBBC) tailored specifically for public marketing spaces. FBBC integrates cutting-edge sensing technologies, data recording, and data analytics. We present a comprehensive framework for implementing FBBC, encompassing sensor deployment strategies, data processing techniques, decision-making algorithms, and performance evaluation methodologies. Through extensive simulations and analysis, we demonstrate the effectiveness of the proposed FBC approach in mitigating fire risks and ensuring regulatory compliance in public marketing spaces

Utilizing Python for Scalable Data Processing in Cloud Environments

In the age of big data and cloud computing, enterprises need to effectively analyze enormous datasets to get meaningful insights and stay ahead. Python, a popular programming language, is a strong tool for cloud-scale data processing. This research study examines Python's integration with cloud platforms and its effects on performance and efficiency in scalable data processing. The study introduces scalable data processing and cloud computing. It then discusses Python's ecosystem, including Dask, Apache Spark with PySpark, TensorFlow, and PyTorch for data processing and machine learning. The study also examines Python's interoperability with cloud services like AWS, Google Cloud Platform, and Microsoft Azure in data input, transformation, and analysis. Many case studies and real-world applications demonstrate how Python has been used in banking, healthcare, and e-commerce. Python is useful for managing massive amounts of data, streamlining processing processes, and scaling cloud applications, as shown in the case studies. The report also analyzes Python-based cloud systems' performance indicators and cost consequences, revealing best practices and possible issues. The article explores Python's involvement in cloud computing trends and technology. Serverless architectures, Docker and Kubernetes, and Python interaction with cloud-native tools and services are examples. These patterns show how data processing is changing and how Python is improving to meet current data needs. This study concludes that Python is a reliable and scalable cloud data processing option. The language's strengths, alignment with cloud technologies, and practical applications in many areas are covered in detail. The results indicate that Python's versatility and cloud scalability provide a robust foundation for handling and analyzing massive datasets, enabling better decision-making and innovation across fields.

Modern approaches to data storage: comparison of relational and cloud data warehouses using etl and elt methods

The paper analyses various aspects of the use of relational and cloud data warehouses as well as methods of integrating ETL and ELT data. A comparative analysis of these approaches, their advantages and disadvantages are provided. A central relational data warehouse is proposed that provides a single version of truth (SVOT), which allows standardising and structuring data, avoiding differences and providing the access to the same information for all users of an organisation. It is analysed the methodological approaches to implementing a data warehouse: top-down, bottom-up, and from middle. It is described cloud data warehouses that use cloud technologies to provide scalability, availability and fault tolerance, which is important for the companies with huge amounts of data. The advantages and disadvantages of ETL and ELT are analysed: ETL transforms data before it is loaded into the warehouse, which makes it easier to maintain data confidentiality. ELT performs transformation after loading, which allows for more flexible data processing directly in the warehouse. In the article, we deal with the approaches to implementing a data warehouse: top-down is suitable for strategic planning, bottom-up allows for faster results, and the middle approach combines both methods to achieve optimal efficiency. We considered cloud data storage: compared to relational storage, cloud storage is more flexible, scalable and efficient, providing speed and reducing infrastructure costs. It is described cloud storage architectures: massive parallel processing, hybrid architectures, lambda architectures, and multi-structured architectures. They provide high performance and flexibility in data processing. It is described data storage technologies: Data Lake, Polyglot Persistence, Apache Iceberg, Apache Parquet, and columnar databases that provide efficient storage and processing of large amounts of data

Sign Language Recognition for Real-time Communication

Abstract: Sign language is an essential communication tool for India's Deaf and Hard of Hearing people. This study introduces a novel approach for recognising and synthesising Indian Sign Language (ISL) using Long Short-Term Memory (LSTM) networks. LSTM, a kind of recurrent neural network (RNN), has demonstrated promising performance in sequential data processing. In this study, we leverage LSTM to develop a robust ISL recognition system, which can accurately interpret sign gestures in real-time. Additionally, we employ LSTM-based models for ISL synthesis, enabling the conversion of spoken language into sign language for improved inclusivity and accessibility. We evaluate the proposed approach on a diverse dataset of ISL signs, achieving high recognition accuracy and natural sign synthesis. The integration of LSTM in ISL technology holds significant potential for breaking down communication barriers and improving the quality of life for India's deaf and hard of hearing people

Improving big data analytics data processing speed through map reduce scheduling and replica placement with HDFS using genetic optimization techniques

Big Data Analytics (BDA) is an unavoidable technique in today’s digital world for dealing with massive amounts of digital data generated by online and internet sources. It is kept in repositories for data processing via cluster nodes that are distributed throughout the wider network. Because of its magnitude and real-time creation, big data processing faces challenges with latency and throughput. Modern systems such as Hadoop and SPARK manage large amounts of data with their HDFS, Map Reduce, and In-Memory analytics approaches, but the migration cost is higher than usual. With Genetic Algorithm-based Optimization (GABO), Map Reduce Scheduling (MRS) and Data Replication have provided answers to this challenge. With multi objective solutions provided by Genetic Algorithm, resource utilization and node availability improve processing performance in large data environments. This work develops a novel creative strategy for enhancing data processing performance in big data analytics called Map Reduce Scheduling Based Non-Dominated Sorting Genetic Algorithm (MRSNSGA). The Hadoop-Map Reduce paradigm handles the placement of data in distributed blocks as a chunk and their scheduling among the cluster nodes in a wider network. Best fit solutions with high latency and low accessing time are extracted from the findings of various objective solutions. Experiments were carried out as a simulation with several inputs of varied location node data and cluster racks. Finally, the results show that the speed of data processing in big data analytics was enhanced by 30–35% over previous methodologies. Optimization approaches developed to locate the best solutions from multi-objective solutions at a rate of 24–30% among cluster nodes.

Reducing Model Uncertainty: Optimized Ensemble Methods for High-Stakes Prediction Tasks

Reliable forecasts are critical in high-risk application areas, including healthcare, finance, and autonomous systems. In these critical applications, this research investigates the application of optimized ensemble techniques to minimize model uncertainties. To address such issues, we describe an approach that integrates multiple learning models to enhance real-time data processing performance and overcome the computational hurdles imposed by such methods. Using real-time working examples to generate simulation reports, we see how ensemble methods can help build more accurate predictions. Additionally, this paper presents typical issues in high-stake prediction scenarios and provides actionable advice on tactfully resolving them by model fine-tuning and uncertainty minimization. The findings, therefore, stress the necessity of applying effective AI solutions to increase the reliability of advanced decisions, thus enhancing performance in extended application domains.

Near-real-time Mueller polarimetric image processing for neurosurgical intervention.

Wide-field imaging Mueller polarimetry is a revolutionary, label-free, and non-invasive modality for computer-aided intervention; in neurosurgery, it aims to provide visual feedback of white matter fibre bundle orientation from derived parameters. Conventionally, robust polarimetric parameters are estimated after averaging multiple measurements of intensity for each pair of probing and detected polarised light. Long multi-shot averaging, however, is not compatible with real-time in vivo imaging, and the current performance of polarimetric data processing hinders the translation to clinical practice. A learning-based denoising framework is tailored for fast, single-shot, noisy acquisitions of polarimetric intensities. Also, performance-optimised image processing tools are devised for the derivation of clinically relevant parameters. The combination recovers accurate polarimetric parameters from fast acquisitions with near-real-time performance, under the assumption of pseudo-Gaussian polarimetric acquisition noise. The denoising framework is trained, validated, and tested on experimental data comprising tumour-free and diseased human brain samples in different conditions. Accuracy and image quality indices showed significant ( ) improvements on testing data for a fast single-pass denoising versus the state-of-the-art and high polarimetric image quality standards. The computational time is reported for the end-to-end processing. The end-to-end image processing achieved real-time performance for a localised field of view ( ). The denoised polarimetric intensities produced visibly clear directional patterns of neuronal fibre tracts in line with reference polarimetric image quality standards; directional disruption was kept in case of neoplastic lesions. The presented advances pave the way towards feasible oncological neurosurgical translations of novel, label-free, interventional feedback.

Pengembangan E- Inventory Peralatan dan Perlengkapan Sekolah Pada SMK Satmanggi Siompu

SMK Satmanggi Siompu is a private vocational high school located on Jalan Poros Siompu, Biwinapada Village, Siompu District, South Buton Regency, Southeast Sulawesi Province. The aim of this research is to understand the current inventory system, design a new system, conduct testing, and implement the system. It is hoped that the inventory information system for equipment and supplies at SMK Satmanggi Siompu can address issues related to inventory data collection and processing. The goal is to enhance performance in inventory data collection and processing. The information system encompasses hardware, software, database, and the user interface of the resulting system. The final stage of the research involves testing the software using the Black Box Testing method. The research method used is Research and Development (R&D), which aims to produce a specific product and test its effectiveness. Data collection methods are descriptive and involve using both primary and secondary data sources. The system-oriented approach uses object-oriented tools called Unified Modeling Language (UML). Analytical tools include use case diagrams, activity diagrams, sequence diagrams, collaboration diagrams, class diagrams, object diagrams, and component diagrams. The system development method uses the prototype method. The system is built using the Bootstrap framework, PHP programming language, and MySQL database

MFRLMO: Model-free reinforcement learning for multi-objective optimization of apache spark

Hyperparameter optimization (HO) is a must to figure out to what extent can a specific configuration of hyperparameters contribute to the performance of a machine learning task. The hardware and MLlib library of Apache Spark have the potential to improve big data processing performance when a tuning operation is combined with the exploitation of hyperparameters. To the best of our knowledge, the most of existing studies employ a black-box approach that results in misleading results due to ignoring the interior dynamics of big data processing. They suffer from one or more drawbacks including high computational cost, large search space, and sensitivity to the dimension of multi-objective functions. To address the issues above, this work proposes a new model-free reinforcement learning for multi-objective optimization of Apache Spark, thereby leveraging reinforcement learning (RL) agents to uncover the internal dynamics of Apache Spark in HO. To bridge the gap between multi-objective optimization and interior constraints of Apache Spark, our method runs a lot of iterations to update each cell of the RL grid. The proposed model-free learning mechanism achieves a tradeoff between three objective functions comprising time, memory, and accuracy. To this end, optimal values of the hyperparameters are obtained via an ensemble technique that analyzes the individual results yielded by each objective function. The results of the experiments show that the number of cores has not a direct effect on $speedup$. Further, although grid size has an impact on the time passed between two adjoining iterations, it is negligible in the computational burden. Dispersion and risk values of model-free RL differ when the size of the data is small. On average, MFRLMO produced $speedup$ that is 37% better than those of the competitors. Last, our approach is very competitive in terms of converging to a high accuracy when optimizing Convolutional Neural networks (CNN).

Asymmetric transition of electrical resistance in an all-solid-state redox device with Fe3O4 and Li-ion electrolyte thin films for physical reservoir computing

In recent years, ion-gating devices have been used in artificial neuromorphic computing and achieved high performance for time-series data processing. However, the origin of this performance still needs to be clarified. In this study, we fabricated an all-solid-state redox device with functional material Fe3O4 and Li-ion conducting solid electrolytes, and the transient response of the electrical resistance of the Fe3O4 thin film to time-series data input was investigated. The transition between high and low electrical resistance states was asymmetric, and residual Li-ion in the thin film led to a hysteresis effect. These unique features, which are induced by ion-electron dynamics coupling, contributes to the high performance of physical reservoir computing utilizing an ion-gating device.

Enhancing circular economy practices through the adoption of digital technologies

AbstractIn transitioning from a linear economy model to a circular business, digital technologies can help manage the flow of restorative and regenerative resources. In this context, the present study aims to evaluate the contribution of digital technologies, such as the internet of things (IoT), Big Data and Blockchain in the execution of circular economy (CE) practices and what mechanisms companies are adopting to integrate digital technologies into the circular context. The research method consists of an exploratory case study with a qualitative approach. Two prominent companies took part in the study. The results indicate benefits highlighted by companies, contemplate that Big Data analysis has improved performance in data processing and management, IoT has added monitoring and traceability to companies' packaging, thus bringing efficiency to recycling, recovery, reuse and disposal, and the Blockchain provides traceability, security and reliability for the implemented practice. This study contributes by exploring the factors that will help explain the mechanism for implementing CE practices. The present study integrates two dimensions (i.e., CE and digital technology). In doing so, it expands previous studies on CE practices by adopting digital technologies. As a practical contribution, the companies in the study understood that integration with startups benefits the development of projects. In addition to innovative proposals, they present digital technologies as management solutions for implementing CE practices.

Enhancing patient information performance in internet of things-based smart healthcare system: Hybrid artificial intelligence and optimization approaches

Utilizing cutting-edge technology in the healthcare industry could provide advanced methods for managing medical records for patients’ treatment. Integrating essential technologies like artificial intelligence (AI) can enhance healthcare services. The intersection of the Internet of Things (IoT) and AI presents various prospects within the healthcare industry. This research paper presents a novel model for enhancing the standard of treatment in smart healthcare systems (SHS) based on the convergence of AI and IoT. We investigate optimizing patient data processing performance in smart healthcare centers. The upgraded particle swarm optimization-long short-term memory (PSO-LSTM) algorithm is introduced to optimize the IoT-based SHS model. In order to perform a more suitable classification of the patient medical data, PSO is determined to be compared with PSO-LSTM to tune several metrics and benchmarks to achieve the highest value on patient data process performance. The proposed outcomes discovered that in terms of the test sets, the predicted values of patient health risks, and just six of them are varied, with an accuracy of 92.5%. This accuracy indicates the PSO-LSTM algorithm has more satisfactory performance and higher efficiency and provides a more secure, reliable, and improved overall patient satisfaction experience.

Advanced neuromonitoring powered by ICM+ and its place in the Brand New AI World, reflections at the 20th anniversary boundary

IntroductionAdoption of the ICM+® brain monitoring software by clinical research centres worldwide has been continuously growing over the past 20 years. This has necessitated ongoing updates to accommodate evolving neuromonitoring research needs, including recent explosion of artificial intelligence (AI). Research questionWe sought to provide an update on the current features of the software. In particular, we aimed to highlight the new options of integrating AI models. Material and methodsWe reviewed all currently available ICM+ analytical areas and discussed potential AI based extensions in each. We tested a proof-of-concept integration of an AI model and evaluated its performance for real-time data processing. ResultsICM+ current analytical tools serve both real-time (bed-side) and offline (file based) analysis, including the calculation engine, Signal Calculator, Custom Statistics, Batch tools, ScriptLab and charting. The ICM+ Python plugin engine allows to execute custom Python scripts and take advantage of complex AI frameworks. For the proof-of-concept, we used a neural network convolutional model with 207,000 trainable parameters that classifies morphology of intracranial pressure (ICP) pulse waveform into 5 pulse categories (normal to pathological plus artefactual). When evaluated within ICM+ plugin script on a Windows 10 laptop the classification of a 5 min ICP waveform segment took only 0.19s with a 2.3s of initial, one-off, model loading time required. ConclusionsModernised ICM+ analytical tools, reviewed in this manuscript, include integration of custom AI models allowing them to be shared and run in real-time, facilitating rapid prototyping and validating of new AI ideas at the bed-side.

Tensor Robust Kernel PCA for Multidimensional Data.

Recently, the tensor nuclear norm (TNN)-based tensor robust principle component analysis (TRPCA) has achieved impressive performance in multidimensional data processing. The underlying assumption in TNN is the low-rankness of frontal slices of the tensor in the transformed domain (e.g., Fourier domain). However, the low-rankness assumption is usually violative for real-world multidimensional data (e.g., video and image) due to their intrinsically nonlinear structure. How to effectively and efficiently exploit the intrinsic structure of multidimensional data remains a challenge. In this article, we first suggest a kernelized TNN (KTNN) by leveraging the nonlinear kernel mapping in the transform domain, which faithfully captures the intrinsic structure (i.e., implicit low-rankness) of multidimensional data and is computed at a lower cost by introducing kernel trick. Armed with KTNN, we propose a tensor robust kernel PCA (TRKPCA) model for handling multidimensional data, which decomposes the observed tensor into an implicit low-rank component and a sparse component. To tackle the nonlinear and nonconvex model, we develop an efficient alternating direction method of multipliers (ADMM)-based algorithm. Extensive experiments on real-world applications collectively verify that TRKPCA achieves superiority over the state-of-the-art RPCA methods.