Data Processing Domain Research Articles

A User-Centered Framework for Data Privacy Protection Using Large Language Models and Attention Mechanisms

This paper introduces a user-centered data privacy protection framework utilizing large language models (LLMs) and user attention mechanisms, which are tailored to address urgent privacy concerns in sensitive data processing domains like financial computing and facial recognition. The innovation lies in a novel user attention mechanism that dynamically adjusts attention weights based on data characteristics and user privacy needs, enhancing the ability to identify and protect sensitive information effectively. Significant methodological advancements differentiate our approach from existing techniques by incorporating user-specific attention into traditional LLMs, ensuring both data accuracy and privacy. We succinctly highlight the enhanced performance of this framework through a selective presentation of experimental results across various applications. Notably, in computer vision, the application of our user attention mechanism led to improved metrics over traditional multi-head and self-attention methods: FasterRCNN models achieved precision, recall, and accuracy rates of 0.82, 0.79, and 0.80, respectively. Similar enhancements were observed with SSD, YOLO, and EfficientDet models with notable increases in all performance metrics. In natural language processing tasks, our framework significantly boosted the performance of models like Transformer, BERT, CLIP, BLIP, and BLIP2, demonstrating the framework’s adaptability and effectiveness. These streamlined results underscore the practical impact and the technological advancement of our proposed framework, confirming its superiority in enhancing privacy protection without compromising on data processing efficacy.

Investigating the Role of Machine Leaning and Blockchain for Forecasting Stock Market Trends

There is a sudden deluge of evolutionary approaches in the domain of data processing and computation, which are significantly affecting several facets of applications globally. Artificial intelligence, machine learning and Blockchain happen to be at the forefront of evolutionary computation beating conventional approaches. Finance applications currently are heavily reliant of data driven models. Stock trend analysis happens to be one such approach, which lays the foundation for forecasting decisions to be made. The leeway in such applications is critically small as minimal inaccuracies in forecasting may lead to major losses. This paper presents the current perspective in terms of evolutionary algorithms such as artificial intelligence and Blockchain and how they are transforming software development, the application of machine learning algorithms to regression problems. Finally, the stock trend analysis based on in and out of sample datasets has been performed for standard S&P datasets. A comparative analysis with previous work clearly indicates the improved performance of the proposed work with respect to baseline approaches in the domain.

RUNNING A WORKFLOW WITHOUT WORKFLOWS: A BASIC ALGORITHM FOR DYNAMICALLY CONSTRUCTING AND TRAVERSING AN IMPLIED DIRECTED ACYCLIC GRAPH IN A NON-DETERMINISTIC ENVIRONMENT

This paper introduces a novel algorithm for dynamically constructing and traversing Directed Acyclic Graphs (DAGs) in workflow systems, particularly targeting distributed computation and data processing domains. Traditional workflow management systems rely on explicitly defined, rigid DAGs, which can be cumbersome to maintain, especially in response to frequent changes or updates in the system. Our proposed algorithm circumvents the need for explicit DAG construction, instead opting for a dynamic approach that iteratively builds and executes the workflow based on available data and operations, through a combination of entities like Data Kinds, Operators, and Data Units, the algorithm implicitly forms a DAG, thereby simplifying the process of workflow management. We demonstrate the algorithm’s functionality and assess its performance through a series of tests in a simulated environment. The paper discusses the implications of this approach, especially focusing on cycle avoidance and computational complexity, and suggests future enhancements and potential applications.

Tackling the cold start issue in movie recommendations with a refined epsilon-greedy approach

With the rapid growth of the Internet and the consequent surge in data, the current era is characterized by information overload. As the domain of data processing and storage expands, recommendation systems have become pivotal tools in navigating this deluge, assisting users in filtering through vast information landscapes. A notable segment of this is movie recommendation systems. As living standards rise, so does the demand for cinematic experiences. Enhancing and refining the methodologies of these recommendation systems is, therefore, of significant value. However, a consistent challenge is the cold start problem encountered when new users join. Without prior viewing records or preferences, these users pose a dilemma for the system: how to offer relevant recommendations without historical data? Addressing this challenge, this paper proposes a unique method grounded in the N-armed bandit model, introducing an enhanced Epsilon-greedy algorithm specifically designed for movie recommendations for such users. By adjusting dynamically based on real-time user feedback, the algorithm aims to continuously hone its recommendation quality, ensuring a consistently better user experience.

Construction Tasks Electronic Process Monitoring: Laboratory Circuit-Based Simulation Deployment

The domain of data processing is essential to accelerate the delivery of information based on electronic performance monitoring (EPM). The classification of the activities conducted by craft workers can enhance the mechanisation and productivity of activities. However, research in this field is mainly based on simulations of binary activities (i.e., performing or not performing an action). To enhance EPM research in this field, a dynamic laboratory circuit-based simulation of ten common constructions activities was performed. A circuit feasibility case study of EPM using wearable devices was conducted, where two different data processing approaches were tested: machine learning and multivariate statistical analysis (MSA). Using the acceleration data of both wrists and the dominant leg, the machine-learning approach achieved an accuracy between 92 and 96%, while MSA achieved 47–76%. Additionally, the MSA approach achieved 32–76% accuracy by monitoring only the dominant wrist. Results highlighted that the processes conducted with manual tools (e.g., hammering and sawing) have prominent dominant-hand motion characteristics that are accurately detected with one wearable. However, free-hand performing (masonry), walking and not operating value (e.g., sitting) require more motion analysis data points, such as wrists and legs.

Do Spectra Live in the Matrix? A Brief Tutorial on Applications of Factor Analysis to Resolving Spectral Datasets of Mixtures

In spite of a rapid growth of data processing software, that has allowed for a huge advancement in many fields of chemistry, some research issues still remain problematic. A standard example of a troublesome challenge is the analysis of multi-component mixtures. The classical approach to such a problem consists of separating each component from a sample and performing individual measurements. The advent of computers, however, gave rise to a relatively new domain of data processing – chemometry – focused on decomposing signal recorded for the sample rather than the sample itself. Regrettably, still a very few chemometric methods are practically used in everyday laboratory routines. The Authors believe that a brief ‘user-friendly’ guide-like article on several ‘flagship’ algorithms of chemometrics may, at least partly, stimulate an increased interest in the use of these techniques among researchers specializing in many fields of chemistry. In the paper, five different techniques of factor analysis are used for the analysis of a three-component system of fluorophores. These algorithms, applied on the excitation-emission spectra, recorded for the ‘unknown’ mixture, allowed to unambiguously determine its composition without the need for physical separation of the components. An example of using chemometric methods for physical chemistry research is also provided. For each presented technique of the data analysis, a short description of its theoretical background followed by an example of its practical performance is given. In addition, the Reader is supplemented with a basic information on matrix algebra, detailed experimental ‘recipes’, reference specialist literature and ready-to-use MATLAB codes.Graphical abstract

Toward Distributed, Global, Deep Learning Using IoT Devices

Deep learning (DL) using large scale, high-quality IoT datasets can be computationally expensive. Utilizing such datasets to produce a problem-solving model within a reasonable time frame requires a scalable distributed training platform/system. We present a novel approach where to train one DL model on the hardware of thousands of mid-sized IoT devices across the world, rather than the use of GPU cluster available within a data center. We analyze the scalability and model convergence of the subsequently generated model, identify three bottlenecks that are: high computational operations, time consuming dataset loading I/O, and the slow exchange of model gradients. To highlight research challenges for globally distributed DL training and classification, we consider a case study from the video data processing domain. A need for a two-step deep compression method, which increases the training speed and scalability of DL training processing, is also outlined. Our initial experimental validation shows that the proposed method is able to improve the tolerance of the distributed training process to varying internet bandwidth, latency, and Quality of Service metrics.

Kognitor: Big Data Real-Time Reasoning and Probabilistic Programming

There is a huge increase in the amount of generated data since the explosion of the Internet. This generated data which is usually collected in different formats and from multiple sources is popularly termed Big Data. Big data contains uncertainty. To handle uncertainty in big data, probabilistic reasoning is used to develop probabilistic models that specify generic knowledge in different topics. These models are used in conjunction with an inference algorithm to enable decision makers especially during uncertain situations. Extensive knowledge in fields such as statistics, machine learning and probability theories are employed in the development of these probabilistic models. Thus, it is usually a difficult undertaking. Probabilistic programming was introduced to simplify and enable development of complex models. Again, decision makers often need to use knowledge from historic data as well as current data to make cogent decisions. Thus, the necessity to unify processing of historic and real-time data with low latency. The Lambda architecture was introduced for this purpose. This paper presents a framework called Kognitor that simplifies the design and development of difficult models using probabilistic programming and Lambda architecture. Evaluation of this framework is also presented in this paper using a case study to highlight the crucial potential of probabilistic programming to achieve simplification of model development and enable real-time reasoning on big data. Thus, demonstrating the effectiveness of the framework. Finally, results of this evaluation are presented in this paper. The Kognitor framework can be used to steer effective and easier implementation of complicated real-life situations as probabilistic models. This will be beneficial in the big data processing domain and for decision makers. Kognitor ensures cost-effectiveness using contemporary big data tools and technology on commodity hardware. Kognitor framework will also be beneficial in academia with respect to the use of probabilistic programming.

Big Data, Artificial Intelligence and Machine Learning: A Transformative Symbiosis in Favour of Financial Technology

The financial technology revolution is a reality, as the financial world is gradually transforming into a digital domain of high-volume information and high-speed data transformation and processing. The more this transformation takes place, the more consumer and investor behaviour shifts towards a pro-technology attitude of financial services offered by market participants, financial institutions and financial technology companies. This new norm is confirming that information technology is driving innovation for financial technology. In this framework, the value of big data, artificial intelligence and machine learning techniques becomes apparent. The aim of this chapter is multi-fold. Firstly, a multidimensional descriptive analysis is shown to familiarise the reader with the extent of penetration of the above in the financial technology road-map. A short non-technical overview of the methods is then presented. Next, the impact of data analytics and relevant techniques on the evolution of financial technology is explained and discussed along with their applications’ landscape. The chapter also presents a glimpse of the shifting paradigm these techniques bring forward for several fintech related professions, while artificial intelligence and machine learning techniques are tied with the future challenges of AI ethics, regulation technology and the smart data utilisation.

Phase congruency and ODBTC based image retrieval

In this study, the authors investigate content based image retrieval (CBIR) using ordered-dither block truncation coding (ODBTC) and phase congruency feature (PCF). Relevant feature extraction plays a vital role for retrieving the image in CBIR. The unique reason to choose PCF with ODBTC is that it detects the edges and corners during variation of image while preserving image brightness and contrast. Combining the PCF and ODBTC features improves CBIR system usage in various visual data processing domains. Thus, yields a better CBIR system which assists in the reduction of storage space, decreases retrieval time and increases accuracy of the system. The precision and recall are used as performance metrics to evaluate the proposed method based on retrieval of relevant images. Extensive experimental results with Corel 1 K (1000 images), Corel 10 K (10000 images) and CALTECH 256 (30144 images) proves that the proposed method is more desirable than antecedent proposed CBIR system in terms of accuracy, precision and recall.

Gencore: an efficient tool to generate consensus reads for error suppressing and duplicate removing of NGS data

BackgroundRemoving duplicates might be considered as a well-resolved problem in next-generation sequencing (NGS) data processing domain. However, as NGS technology gains more recognition in clinical application, researchers start to pay more attention to its sequencing errors, and prefer to remove these errors while performing deduplication operations. Recently, a new technology called unique molecular identifier (UMI) has been developed to better identify sequencing reads derived from different DNA fragments. Most existing duplicate removing tools cannot handle the UMI-integrated data. Some modern tools can work with UMIs, but are usually slow and use too much memory. Furthermore, existing tools rarely report rich statistical results, which are very important for quality control and downstream analysis. These unmet requirements drove us to develop an ultra-fast, simple, little-weighted but powerful tool for duplicate removing and sequence error suppressing, with features of handling UMIs and reporting informative results.ResultsThis paper presents an efficient tool gencore for duplicate removing and sequence error suppressing of NGS data. This tool clusters the mapped sequencing reads and merges reads in each cluster to generate one single consensus read. While the consensus read is generated, the random errors introduced by library construction and sequencing can be removed. This error-suppressing feature makes gencore very suitable for the application of detecting ultra-low frequency mutations from deep sequencing data. When unique molecular identifier (UMI) technology is applied, gencore can use them to identify the reads derived from same original DNA fragment. Gencore reports statistical results in both HTML and JSON formats. The HTML format report contains many interactive figures plotting statistical coverage and duplication information. The JSON format report contains all the statistical results, and is interpretable for downstream programs.ConclusionsComparing to the conventional tools like Picard and SAMtools, gencore greatly reduces the output data’s mapping mismatches, which are mostly caused by errors. Comparing to some new tools like UMI-Reducer and UMI-tools, gencore runs much faster, uses less memory, generates better consensus reads and provides simpler interfaces. To our best knowledge, gencore is the only duplicate removing tool that generates both informative HTML and JSON reports. This tool is available at: https://github.com/OpenGene/gencore

Repeating patterns as symbols for long time series representation

A symbolic time series representation using repeating shapes as symbols is proposed.Lower bounding similarity measure properties evaluated on URC datasets.Electricity consumption dataset used to evaluate on very long, seasonal time series.Reconstruction error and dimensionality reduction ability comparable to PAA. Over the past years, many representations for time series were proposed with the main purpose of dimensionality reduction and as a support for various algorithms in the domain of time series data processing. However, most of the transformation algorithms are not directly applicable on streams of data but only on static collections of the data as they are iterative in their nature. In this work we propose a symbolic representation of time series along with a method for transformation of time series data into the proposed representation. As one of the basic requirements for applicable representation is the distance measure which would accurately reflect the true shape of the data, we propose a distance measure operating on the proposed representation and lower bounding the Euclidean distance on the original data. We evaluate properties of the proposed representation and the distance measure on the UCR collection of datasets. As we focus on stream data processing, we evaluate the properties and limitations of the proposed representation on very long time series from the domain of electricity consumption monitoring, simulating the processing of potentially unbound data stream.

Measurement Data Processing in Spectrophotometric Analysers of Food

Abstract Spectrometry, especially spectrophotometry, is getting more and more often the method of choice not only in laboratory analysis of (bio)chemical substances, but also in the off-laboratory identification and testing of physical properties of various products, in particular - of various organic mixtures including food products and ingredients. Specialised spectrophotometers, called spectrophotometric analysers, are designed for such applications. This paper is on the state of the art in the domain of data processing in spectrophotometric analysers of food (including beverages). The following issues are covered: methodological background of food analysis, physical and metrological principles of spectrophotometry, the role of measurement data processing in spectrophotometry. General considerations are illustrated with examples, predominantly related to wine and olive oil analysis.

Template design for file processing applications

AbstractEven though the data processing domain is relatively well‐defined, application‐specific details vary too widely to conceive of totally automating programming and specification support. Conventional tools lack the semantic knowledge to constructively support software development. On the other hand, it is possible to design a prompting and guiding system that can elicit the application details. The limits of automated programming support possible for file and data processing applications were identified by means of a case study of COBOL and PL/I programs. The data gathered from this study facilitated the definition of seventeen widely applicable templates. The resulting template set has immediate value as a manual specification and documentation tool. The templates encourage programmers to make well‐considered design decisions and enforce standardized documentation.

Transformation Between Continuous and Discrete Representations of Images: A Perceptual Approach

The transformation of image signals between the continuous domain of the real world and the discrete domain of modern data processing can have a significant effect on quality and efficiency. Analysis based on perceptual rather than mathematical considerations has been carried out. A series of experiments based on the analysis has shown that substantial improvement over usual techniques can be achieved by the use of a cascade of a presharpening filter combined with Gaussian presampling and interpolation filters. The resulting ``sharpened Gaussian'' filter, although not exactly circularly symmetrical, gives a high degree of isotropy. Each element in the cascade is separable, so that computational efficiency is high. A favorable tradeoff is achieved among sharpness, smoothness, and the effects of aliasing. Subjective testing, in comparison with other commonly used filters, has shown the clear superiority of this filter.

A Microprocessor for Signal Processing, the RSP

Signal processing is a data processing domain that contains a diversity of applications, including speech processing, image processing, radar, sonar, medical imaging, data communications, seismic processing, and many others. Despite the diversity of the applications, this processing domain has a very structured set of characteristics. These include real-time operation, dominance of arithmetic operations, and well-structured data flows. The Real-Time Signal Processor (RSP) is a microprocessor architecture that was created to exploit these characteristics in order to provide an expeditious and economical way to implement signal processing applications. In this paper, the organization and architecture of the RSP are described. Features of the RSP, such as the instruction pipeline and the fractional fixed-point arithmetic, which exploit the characteristics of signal processing to provide additional computational power, are emphasized. Other features, such as the powerful indexing, the saturation arithmetic, the guard bits, and the double-word-width accumulator, which add much to the processor's versatility and programmability, are also highlighted. The performance of the RSP is illustrated through examples.