Data Block Research Articles

Outcome dependent subsampling divide and conquer in generalized linear models for massive data

In order to break the constraints and barriers caused by limited computing power in processing massive datasets, we propose an outcome dependent subsampling divide and conquer strategy in this paper. The proposed strategy can process data on multiple blocks in parallel and concentrate the computing resources of each block on regions with the most information. We develop a distributed statistical inference method and propose a computation-efficient algorithm in the generalized linear models for massive data. The proposed method only need to preserve some summary statistics from each data block and then use them to directly construct the proposed estimator. The asymptotic properties of the proposed method are established. Simulation studies and real data analysis are conducted to illustrate the merits of the proposed method.

BLOCKCHAIN REVOLUTION: Intensifying Data Protection and Blockchain based IPR Systems

Blockchain Technology, also known as Distributed Ledger System, collects the data stored in enciphered blocks, which are closely linked to hash pointers. These hash pointers, links one block to another, which together forms a chain, known as the Block Chain. Blockchain and any other form of distributed ledger technologies, creates a trustworthy and a transparent record, by allowing multiple parties to a transaction to substantiate what will be entered in the ledger advanced. It has a multi-faceted utilization, where it can reserve and protect any type of data and can also, encrypt the Data in stored in Independent Data Blocks. These chains have an unaccompanied and unassisted time snaps, which secures the indivisible data, collected from consumers. This distributed ledger technology, creates a secure, time-stamped and immutable chain of information, which prevents counterfeiting and ensures that, the IP systems, are properly being traced. Blockchain, also supports smart contracts, which are self-executing contracts with the terms of the agreement directly written into code. Smart contracts can automate the licensing and royalty payments for IP, ensuring that creators are compensated fairly and transparently. It also facilitates the creation of decentralized marketplaces for buying, selling, and licensing Intellectual Property. Thus, this reduces the need for intermediaries and can also lower transaction costs, which increases the access to IP for potential buyers and licensees. Therefore, where a particular society is adapting up, to explore the unexplored, protection of Intellectual Property by using Blockchain, could play a vital role in the development and expansion of the Indian Economy. Thus, This research paper, aims to spell out the role of Block Chain technology, in protecting the data embedded in Intellectual Property Rights, which ensures to create a secure, transparent and a decentralized protection, for Intellectual Property Rights.

A unified LOD model for river network and DEM based on an improved 3D_DP algorithm

The level of detail (LOD) modelling of vector and terrain data is individual, resulting in geometric and topological inconsistencies in simplification processes. The three dimension Douglas Peucker (3D_DP)algorithm can realize gradual discrete point selection through threshold, which is mainly used in DEM synthesis, and its simplified process is very suitable for the dynamic establishment of massive data sets. A new LOD modeling method based on 3D_DP algorithm is proposed to simplify the consistency of river network vector elements and DEM in this paper. The specific steps are as follows: Firstly, the “Bending Adjustment Index (BAI)” is introduced to improve the 3D_DP algorithm, called the improved 3D_DP algorithm; Secondly, the DEM data is extracted into a 3D discrete point dataset, and the river line vector data is also converted into a discrete point dataset, assigned with elevation attributes, and merges with the DEM’s 3D discrete points. The merged point datasets are equenced based on the importance of each point, which are computed by the improved 3D_DP algorithm. The order of deleting points is determined by the sequence and the corresponding model spatial errors are calculated with the deletion of points. Then, the DEM’s 3D discrete points are constructed a Delaunay network with the river line as a mandatory constraint condition. The required triangulation is called in real time with the change of sight distance depending on the simple correspondence between screen projection error and model space error, and the unified LOD model for river line vector and DEM is established. The results show that the river’s overall shape and the terrain’s main features can be reserved under the same simplified factor based on the improved 3D_DP algorithm. The unified LOD model for the river network and DEM is feasible under the importance sequence of merged point datasets by the improved 3D_DP algorithm. Under the proper operation of data blocking, the rendering frame rate can meet practical application requirements.

Missing Data Imputation in Balanced Construction for Incomplete Block Designs

Observational data with massive sample sizes are often distributed on many local machines. From an experimental design perspective, investigators often desire to identify the effect of new treatments (even ML algorithms) on many blocks of experimental data. With time requirements or budget constraints, assigning all treatments to each block is not always feasible. This creates incomplete responses with respect to a randomized complete block design (RCBD). These incomplete responses are missing by design. However, whether they can be estimated with missing imputation methods is not well understood. Thus, it is challenging to correctly identify the treatment effects with missing data. To this end, this paper provides a method for imputation and analysis of the responses with missing data. The proposed method consists of three steps: Reconstruction, Imputation, and ‘Complete’-data Analysis (RICA). The incomplete responses are imputed with the expectation-maximization (EM) algorithm. The RCBD model is then fitted by the resulting dataset. The identifiability result suggests that the missing may be nonignorable for each block, but the whole data of an incomplete design are missing by design when the design is balanced. Theoretical results on relative efficiency also inform us when the missingness should be imputed for incomplete designs with the role of balanced variance. Applications on real-world data verify the efficacy of this method.

Response-Aided Score-Matching Representative Approaches for Big Data Analysis and Model Selection under Generalized Linear Models

In this paper, we propose an efficient method called the response-aided score-matching representative (RASMR) approach to facilitate massive data model selection and data analysis with generalized linear models (GLMs) and a predetermined data partition due to data localization. Similar to the original score-matching representative (SMR) approach, RASMR constructs an artificial data point, called the representative, for each data block. It then fits a GLM on the representative dataset, which provides not only an efficient approach for massive data analysis but also an ideal solution in response to privacy concerns by avoiding the transfer of sensitive data. By further splitting the data blocks according to the values of the response variables, RASMR can obtain more accurate parameter estimates than SMR. Furthermore, by theoretical justifications and simulation studies, we show that RASMR can be more efficiently utilized for model selection and variable selection for a massive dataset by approximating the Akaike information criterion (AIC) and the aggregated prediction errors for cross-validation, which are commonly used for choosing the most appropriate statistical model and drawing reliable conclusions. We also apply the proposed RASMR approach to the airline on-time performance data, which consists of 371 data files labeled by month, and show that RASMR can be successfully used for selecting the most appropriate model for real massive data analysis.

A Wireless Ad Hoc Network Communication Platform and Data Transmission Strategies for Multi-Bus Instruments

As automatic test technology advances, the number of programmable instruments in a single test system increases. Traditional wired communication methods have a limited range and involve complex cable layouts. Single-function wireless converters provide a viable alternative, but they have limitations. These include complicated configuration, issues with multi-system collaboration, and data blocking. This paper proposes a wireless ad hoc network platform for multi-bus instruments based on a low-cost ESP-12H WiFi module. The platform supports GPIB, RS232, RS485, and CAN bus interface instrument access. It features easy configuration, ad hoc networking, and self-repairing capabilities. A relay multi-hop network with a tree topology expands capacity and coverage. Additionally, a dynamic window-receiving mode and an improved multi-priority queue ensure data transmission integrity. The experimental results show that the platform’s networking time is less than 10 s, and the coverage range reaches 50 m in complex indoor environments. It also shows good stability when running for a long time. However, due to hardware and software design limitations, the actual upload speeds fall short of the theoretical values. For example, RS232 and RS485 are about 10% slower than the theoretical values, and GPIB is about 80% slower. Further optimization is required in the future.

Utilizing language models for advanced electrocardiogram analysis

Electrocardiography (ECG) signals are often referred to as the language of the heart and have been widely utilized for diagnosing various heart ailments, particularly arrhythmias. Consequently, numerous machine learning models have been employed to automatically detect heart disorders using ECG signals. In this research, the primary objective is to detect arrhythmias using a center-symmetric self-organized textual pattern. A novel feature engineering model has been introduced, which includes the following components: (i) multilevel feature extraction using the proposed center-symmetric self-organized textual pattern (CSSOTP), (ii) iterative neighborhood component analysis (INCA), and (iii) classification using k-nearest neighbors (kNN) with 10-fold cross-validation. During the feature extraction phase, a multilevel feature extraction model incorporating maximum absolute pooling (MAP) and the proposed CSSOTP was applied. The CSSOTP was designed to select the most appropriate pattern for the given data block. A large language model (LLM) was leveraged to generate text for creating patterns, and ChatGPT was used to assist in text generation. To identify the most informative features, the INCA feature selector was employed, and the selected features were subsequently classified using the kNN classifier. An impressive 96.20 % classification accuracy was achieved by the CSSOTP-based feature engineering model when tested on an ECG dataset containing 17 classes. Furthermore, comparisons with state-of-the-art feature engineering models were conducted, demonstrating that the proposed model has superior classification capabilities.

Generalized fiducial inference for the GEV change-point model

Generalized extreme value (GEV) distribution is used to analyse the maximum from a block of data. It is very useful to describe the unusual event rather than the usual event. In this paper, we propose a change-point detection procedure for GEV distribution based on generalized fiducial inference. The fiducial distribution of the change-point location is constructed. Meanwhile, Markov Chain Monte Carlo method combined with Gibbs sampling and the Metropolis–Hastings algorithm is utilised to estimate the location of the change point and its confidence interval. In addition, the generalized fiducial factor is used to test whether there is a change point. Simulation results show that the proposed generalized fiducial method performs better in accuracy, robustness and the length of confidence intervals. Finally, we apply the proposed method to the annual maximum rainfall data in Beijing.

Vibrational spectroscopy data fusion for enhanced classification of different milk types

The aim of this study is to classify seven types of Irish milk (butter, fresh, heart active, lactose free, light, protein, and slimline), supplied by a specific company, using vibrational spectroscopy methods: Near infrared (NIR), mid infrared (MIR), and Raman spectroscopy. In this regard, chemometric methods were used, and the impact of spectral data fusion on prediction accuracy was evaluated. A total of 105 samples were tested, with 21 used in the test set. The study assessed principal component analysis (PCA), partial least square discriminant analysis (PLS-DA), and sequential and orthogonalized partial least squares linear discriminant analysis (SO-PLS-LDA) for classifying different milk types. The prediction accuracy, when applying PLS-DA on individual blocks of data and low-level fused data, did not exceed 85.71 %. However, implementing the SO-PLS-LDA strategy significantly improved the accuracy to 95 %, suggesting a promising method for the development of classification models for milk using data fusion strategies.

GS3: A Lightweight Method of Generating Data Blocks With Shuffling, Scrambling, and Substituting Data for Constrained IoT Devices

GS3: A Lightweight Method of Generating Data Blocks With Shuffling, Scrambling, and Substituting Data for Constrained IoT Devices

Increasing the period of active use of on-board electronic equipment of spacecraft

For electronic equipment of space systems, and primarily memory devices, the task of protection from the effects of ionizing cosmic radiation and other external factors that distort stored and processed information is relevant. This paper proposes a holographic coding method that allows you to restore information in the event of a large number of errors. The method is based on recording into memory, instead of the original digital hologram data, a virtual digital object corresponding to a data block. The divisibility property of a hologram is used, which makes it possible to reconstruct a recorded data block from its fragment. The achieved level of noise immunity is determined by the size of the hologram. For an 8-bit data block, recording a 256-bit hologram provides information recovery if 75 % of the recorded hologram is lost. The developed decoder corrects a package of dependent (grouping) errors that distort all bits of the hologram. The number of random independent errors that the decoder corrects can be up to 40 % of the recorded information. The information storage system, resistant to ionizing radiation, is a memory array of increased capacity, taking into account the selected redundancy factor, and a memory controller that performs holographic encoding when recording information and decoding with automatic error correction when reading information. The operating algorithm of the controller itself can be implemented in the form of a programmable logic integrated circuit, or stored in a read-only memory device that is not affected by ionizing radiation.

DBPBFT: A hierarchical PBFT consensus algorithm with dual blockchain for IoT

The Internet of Things (IoT) is composed of smart devices connected to a network that can send and receive large amounts of data with other devices, generating a lot of data for processing and analysis. Due to the fact that every transaction in blockchain is recorded, placed in a data block, and added to an immutable and secure data chain, blockchain is becoming one of the most promising solutions for enhancing IoT security issues. As more devices become intelligent, the scale of IoT systems, including residential IoT and industrial IoT, is on the rise. Consequently, the issue of resource consumption, stemming from the escalating system communication overhead, is becoming more pronounced. In order to improve the efficiency of the consensus process for residential IoT and reduce the overhead caused by the consensus process, this paper proposes a hierarchical PBFT consensus algorithm With Dual Blockchain for IoT (DBPBFT). Compared to industrial IoT, DBPBFT is more suitable for residential IoT with small scope and clear data classification. DBPBFT separates the responsibilities of dual chains, improving system scalability while also enhancing blockchain security. A chain is divided into several small groups, each responsible for a type of data, reducing system overhead and communication overhead. To avoid unnecessary view-change as much as possible, before consensus begins, each group will select the current view primary node based on reputation values. The simulation results show that the DBPBFT algorithm is superior to traditional algorithms. In terms of reducing communication overhead, compared with EPBFT and DPNPBFT, DBPBFT has increased by 73.8% and 53.1%, respectively. In terms of consensus efficiency, DBPBFT has improved by 34% compared to DPNPBFT.

Optimization of block-scaled integer GeMMs for efficient DNN deployment on scalable in-order vector processors

A continuing rise in DNN usage in distributed and embedded use cases has demanded more efficient hardware execution in the field. Low-precision GeMMs with optimized data formats have played a key role in more memory and computationally-efficient networks. Recently trending formats are block-scaled representations stemming from tight HW-SW co-optimization, that compress network size by sharing exponents per data block. Prior work mostly focuses on deploying such block-scaled GeMM operations on domain-specific accelerators for optimum efficiency at the cost of flexibility and ease of deployment. In this work, we exploit and optimize the deployment of block-scaled GeMMs on fully-programmable in-order vector processors using ARM SVE. We define a systematic methodology for performing design space exploration to optimally match the workload specifications with processor vector-lengths, different microkernels, block sizes and shapes. We introduce efficient intrinsics-based microkernels with effective loop unrollings, and data-transfer efficient fused requantization strategies to maximize kernel performance, while also ensuring several deployment configurations. We enable generalized block-scaled kernel deployments through tunable block sizes and shapes, which helps in accommodating different accuracy-speed trade-off requirements. Utilizing 2D activation blocks instead of conventional 1D blocks, the static and dynamic BS-INT8 configurations yielded on average 3.8x and 2.9x faster speedups over FP32 models respectively, at no accuracy loss for CNN classification tasks on CIFAR10/100 datasets.

A Secure Certificate-Based Data Integrity Auditing Protocol with Cloud Service Providers

With the surge in cloud storage popularity, more individuals are choosing to store large amounts of data on remote cloud service providers (CSPs) to save local storage resources. However, users’ primary worries revolve around maintaining data integrity and authenticity. Consequently, several cloud auditing methods have emerged to address these concerns. Many of these approaches rely on traditional public-key cryptography systems or are grounded in identity-based cryptography systems or certificateless cryptography systems. However, they are vulnerable to the increased costs linked with certificate management, key escrow, or the significant expenses of establishing a secure channel, respectively. To counter these limitations, Li et al. introduced a certificate-based cloud auditing protocol (LZ22), notable for its minimal tag generation overhead. Nonetheless, this protocol exhibits certain security vulnerabilities. In this paper, we devise a counterfeiting technique that allows the CSP to produce a counterfeit data block with an identical tag to the original one. Our counterfeiting method boasts a 100% success rate ∀ data block and operates with exceptional efficiency. The counterfeiting process for a single block of 10 kB, 50 kB, and 100 kB takes a maximum of 0.08 s, 0.51 s, and 1.04 s, respectively. By substituting the exponential component of homomorphic verifiable tags (HVTs) with non-public random elements, we formulate a secure certificate-based cloud auditing protocol. In comparison to the LZ22 protocol, the average tag generation overhead of our proposed protocol is reduced by 6.80%, 13.78%, and 8.66% for data sizes of 10 kB, 50 kB, and 100 kB, respectively. However, the auditing overhead of our proposed protocol shows an increase. The average overhead rises by 3.05%, 0.17%, and 0.45% over the LZ22 protocol’s overhead for data sizes of 10 kB, 50 kB, and 100 kB, correspondingly.

МЕТОД ЗАХИЩЕНОГО ОБМІНУ ТА ЗБЕРІГАННЯ ФАЙЛІВ У ПІРИНГОВИХ МЕРЕЖАХ

The article proposes a method for secure file exchange and storage in peer-to-peer networks based on the IPFS (InterPlanetary File System) protocol and a linear feedback shift register (LFSR). The use of IPFS provides efficient decentralized data storage with guaranteed integrity due to the use of content-oriented addressing based on hash functions. In IPFS, each file or data block is identified by a unique CID (Content Identifier), which changes with any modification of the content. To increase confidentiality before uploading files to the network, it is proposed to use an encryption method based on a linear feedback shift register (LFSR) and counters. This method provides efficient generation of pseudo-random permutations of the file bytes, which makes it impossible to decrypt it without knowing the key. Encryption using the FLSR is characterized by high speed, ease of implementation, and low requirements for computing resources. This is especially important for peer-to-peer networks, where nodes may have limited performance and unstable connections. The proposed approach eliminates the vulnerability of IPFS associated with the possibility of obtaining file contents based on their CIDs, since even with the presence of an identifier, only encrypted content is available. In addition, the method ensures the confidentiality of the transmitted data, since files are transmitted in encrypted form and can only be decrypted by authorized nodes. The method also provides high performance due to the features of IPFS: files or their fragments (IPLD objects) can be downloaded in parallel from several nodes, which speeds up the transfer of large amounts of data. The method can be used in decentralized storage systems, platforms for secure file exchange, as well as in environments with increased requirements for information security.

Block Chain Based Counterfeit Product Identification

In recent years, counterfeit goods have grown to be a significant issue for the product manufacturing sector. They affect the companies' sales, profit margins, and brand. Block chain technology is used to distinguish between authentic and fake goods. It is a distributed, decentralized digital ledger that keeps track of transactions by storing blocks of data across several databases linked together by chains. It is secure, therefore no block can be changed or breached. Block chain technology allows users and customers to verify a product's safety without depending on outside parties. Quick Response (QR) codes, which capitalize on new developments in mobile and wireless technology, provide a potent means of countering the practice of product counterfeiting in this project. It is possible to detect counterfeit goods with a QR code scanner that connects a product's QR code to a block chain. Keywords: Counterfeit products, Block chain technology, QR codes, Manufacturing sectors, Secure, Digital ledger.

Securing the Cloud: A New Horizon of Hybrid Automated System for Storage Protection

While cloud storage seems dependable and scalable, the necessity for it is increasing every day. The intricacies involved in securing data in terms of privacy and confidentiality makes the storing and administration of user information at a remote location problematic. The cloud third party who is in charge of handling the issues concerned with data security present within the cloud systems could end up being an unreliable insider. In order to enhance the protection of data and ensure the confidentiality of information without any sort of interruption from cloud third parties, the current research provides an automated hybrid Cryptographic algorithm for Cloud Data Storage. Improved automated Asynchronous Encryption for Data Streams (AEDS) and enhanced automated Concurrent Encryption for Data Blocks (CEDB) are two encryption techniques that are used to achieve great performance and efficiency. We introduce a unique encryption method in Enhanced Automated Concurrent Encryption by transforming the fixed S-box in the conventional AES to a varying S-box utilising a Ternary Galois field and Complementary Ternary Galois field in the proposed work. Additionally, the Ternary Galois field and Complementary Ternary Galois field are employed to transform the matrix used for the AES mix column technique into a dynamic matrix.

Cybersecurity: Fast Encryption Cipher Block Chaining Mode (FCBC Mode) for Time Series Data

Cipher Block Chaining (CBC) mode is used to make the encryption of each block of data in a data series more secure by making the encryption of the block interdependent. In case of time series data, the size of each data point includes the time information and actual data itself. This means that each block of data that is encrypted may hold only nearly half the actual data values due to timestamp information. This reduces the efficiency of encryption and decryption of data for applications that demand high frequency data. Fast Encryption Cipher Block Chaining Mode (FCBC Mode) provides a solution to overcome this to a great extent

Performance of PBFT Consensus under Voting by Groups

Practical Byzantine Fault Tolerance (PBFT) is the protocol of choice for many applications that require distributed consensus between a number of participant nodes. While PBFT assumes a single voting committee, many applications recognize different groups of participants that need to reach a consensus separately before accepting a proposal. To this end, we propose to count the votes by separate groups or committees of participating nodes, instead of all together as in the original PBFT. We then investigate the performance impact of this approach on the mean time to accept a data block and the number of nodes involved in making the final decision. Our results indicate that the proposed solutions impose a slight performance penalty which may be countermanded by reducing the quorum numbers needed in different subsets of the original committee.

ПОБУДОВА КОДІВ ХЕММІНГА В СКІНЧЕННИХ ПОЛЯХ ГАЛУА

Hamming codes and their modifications are key in numerous technological processes and systems, where minimizing errors is crucial for enhancing reliability and efficiency in data transmission or storage. They allow for the detection and automatic correction of single errors in each data block. Their relative simplicity to implement at the hardware and software levels, renders them useful in systems demanding high reliability. Moreover, they optimize transmission channel use due to their minimal redundancy. At work is the task of applying the properties of modular arithmetic to develop codes that can effectively correct errors in systems with higher bit rates than binary has been considered. Conducted research confirms that such codes can be tailored to meet specific data transmission system requirements, showcasing adaptability. The use of Hamming codes within finite Galois fields, leveraging modular arithmetic, further enhances correction capabilities and efficiency, streamlining implementation and computation processes. Error correction codes in finite Galois fields use the same principles applied in Hamming codes but take into account the mathematical properties of these fields for effective encoding, detection, and correction of errors.The proposed error correction method, through integration with modular arithmetic, opens possibilities for optimizing encoding and decoding processes, allowing for a higher level of data transmission reliability with minimal redundancy, which was previously limited by the properties of traditional codes. The proposed solution demonstrates the potential for using modular arithmetic for error correction but can provide a basis for further research in this area, opening new possibilities for improving information processing and transmission technologies.