Chunking Method Research Articles

Fault diagnosis of complex industrial equipment based on chunked statistical global-local feature fusion

Abstract Industrial process equipment is bulky and complex in structure, which is easy to produce faults during operation and affect production efficiency, cause huge economic losses, and even threaten the safety of workers. To achieve sustainable operation of large-scale industrial processes, timely and accurate monitoring and handling of abnormal situations are essential. However, fault monitoring of large equipment requires the collection of abundant data, which includes many complex related variables, resulting in excessive redundant data generated during the fault monitoring process. Moreover, the existing principal component analysis (PCA) method can only retain the global characteristic of variance information, and cannot obtain the local characteristic that can characterize the topological relationship between the data points, which affects its monitoring reliability and intelligence level. In response to these issues, a fault diagnosis model for complex industrial processes based on chunked statistical global-local feature fusion (CSGLFF) is proposed in this paper. First, considering the correlation characteristics between industrial process variables, a correlation variable chunking method mutual information-based is designed to merge the variables with small correlation to obtain the optimal chunking of variables. Second, PCA and locality preserving projections (LPP) are combined to construct a global-local feature fusion (GLFF) model that can extract global and local features simultaneously. The chunked data are imported into the GLFF for the extraction of its features respectively, and the corresponding CSGLFF is established. In addition, Bayesian inference is used to fuse the statistics of each sub-chunk to establish an overall fault monitoring statistical indicators, and the reason for failure is found through the variable contribution graph. Finally, two cases of Tennessee Eastman process (TEP) and laboratory emulsion pump were used to conduct experimental research on the performance of CSGLFF. The results show that compared with the chunked statistical PCA, chunked statistical LPP, and GLFF algorithms, The accuracy of fault monitoring for TEP mean, flow pulsation impact, and pressure anomaly of this method reached 92.91%, 97%, and 90.30%, respectively. It has good monitoring effect in processing data with large variables, reducing the generation of redundant data, improving the accuracy of industrial monitoring, and accurately identifying the relevant variables of fault occurrence. This provides a theoretical basis for determining the fault location and points out the direction for maintenance by staff.

A novel fish individual recognition method for precision farming based on knowledge distillation strategy and the range of the receptive field.

With the continuous development of green and high-quality aquaculture technology, the process of industrialized aquaculture has been promoted. Automation, intelligence, and precision have become the future development trend of the aquaculture industry. Fish individual recognition can further distinguish fish individuals based on the determination of fish categories, providing basic support for fish disease analysis, bait feeding, and precision aquaculture. However, the high similarity of fish individuals and the complexity of the underwater environment presents great challenges to fish individual recognition. To address these problems, we propose a novel fish individual recognition method for precision farming that rethinks the knowledge distillation strategy and the chunking method in the vision transformer. The method uses the traditional convolutional neural network model as the teacher model, introducing the teacher token to guide the student model to learn the fish texture features. We propose stride patch embedding to expand the range of the receptive field, thus enhancing the local continuity of the image, and self-attention-pruning to discard unimportant tokens and reduce the model computation. The experimental results on the DlouFish dataset show that the proposed method in this paper improves accuracy by 3.25% compared to ECA Resnet152, with an accuracy of 93.19%, and also outperforms other vision transformer models.

Holographic encryption algorithm based on the new integrated chaotic system and chaotic mask

Image encryption algorithms based on chaos theory have rapidly developed in recent years, with many achieving encryption by confusion-diffusion structures. However, the security performance of these algorithms needs to be improved. This paper proposes a holographic encryption algorithm based on the new integrated chaotic system and chaotic mask. The improved Gerchberg-Saxton algorithm transforms plaintext images into pure-phase holograms. The chaotic masks generated by the new integrated chaotic system decompose holograms into sub-images. The sub-images are pixel-wise heterogeneous operations and are finally merged into the complete encrypted image. Compared with one-dimensional chaotic mapping, the generation of NICS is related to kinoform, which enhances the system’s key sensitivity. The chaotic mask chunking method can generate random chunk locations and sizes, effectively preventing attacks against specific chunking rules. The performance analysis shows that the algorithm has a higher key space and security to resist stronger robustness attacks.

Pre-trained transformer model as a surrogate in multiscale computational homogenization framework for elastoplastic composite materials subjected to generic loading paths

A composite material typically exhibits complex behavior at the engineer scale, arising from the interactions between its underlying constituent phases, as well as the competitions between micro-processes. It is generally a daunting task to develop an engineering model to adequately capture the essential micro mechanisms that propagate onto the macro scale. To this end, the multiscale computational homogenization (FE2) method enables a consistent coupling across length scales, to give results that compare well with direct numerical simulations having the full micro-structural details, without the need for any constitutive assumptions nor calibrations at the macro scale. Despite its predictive capabilities, the typical computational homogenization method is still computationally too expensive for most practical problems, as the coupling between micro and macro scales are solved simultaneously during its numerical implementation. In this presentation, focusing on the elastoplastic behavior of fiber-reinforced composite, we address this bottleneck with an offline development of a microscopic surrogate model for a given micro-structure, to be incorporated into a standard nonlinear FE framework, for rapid online implementations at the macro scale. For the offline training phase, we adopt the transformer-based architecture within a pre-training and fine-tuning framework. The proposed pre-trained transformer model is capable of parallelizing computations to effectively capture global dependencies within the strain-stress data sequences. To reduce the data generation cost, a constructed source representative volume element (RVE) having a single central heterogeneity with an identical volume fraction with the target RVE is utilized, to rapidly generate a huge source dataset for a pre-training process. The performance of the surrogate model is first demonstrated by comparing its predictions under random loading paths against the reference homogenized RVE responses. Next, the surrogate model is incorporated into a macro FE framework, and its predictive capabilities illustrated via the generic loading of two specimens with different microstructures, each having a different loading–unloading path. Finally, a chunking method is discussed as a potential remedy for managing very long load sequences.

THE USE OF ROUGH CLASSIFICATION AND TWO THRESHOLD TWO DIVISORS FOR DEDUPLICATION

The data deduplication technique efficiently reduces and removes redundant data in big data storage systems. The main issue is that the data deduplication requires expensive computational effort to remove duplicate data due to the vast size of big data. The paper attempts to reduce the time and computation required for data deduplication stages. The chunking and hashing stage often requires a lot of calculations and time. This paper initially proposes an efficient new method to exploit the parallel processing of deduplication systems with the best performance. The proposed system is designed to use multicore computing efficiently. First, The proposed method removes redundant data by making a rough classification for the input into several classes using the histogram similarity and k-mean algorithm. Next, a new method for calculating the divisor list for each class was introduced to improve the chunking method and increase the data deduplication ratio. Finally, the performance of the proposed method was evaluated using three datasets as test examples. The proposed method proves that data deduplication based on classes and a multicore processor is much faster than a single-core processor. Moreover, the experimental results showed that the proposed method significantly improved the performance of Two Threshold Two Divisors (TTTD) and Basic Sliding Window BSW algorithms.

Measurement of Functional Use in Upper Extremity Prosthetic Devices Using Wearable Sensors and Machine Learning.

Trials for therapies after an upper limb amputation (ULA) require a focus on the real-world use of the upper limb prosthesis. In this paper, we extend a novel method for identifying upper extremity functional and nonfunctional use to a new patient population: upper limb amputees. We videotaped five amputees and 10 controls performing a series of minimally structured activities while wearing sensors on both wrists that measured linear acceleration and angular velocity. The video data was annotated to provide ground truth for annotating the sensor data. Two different analysis methods were used: one that used fixed-size data chunks to create features to train a Random Forest classifier and one that used variable-size data chunks. For the amputees, the fixed-size data chunk method yielded good results, with 82.7% median accuracy (range of 79.3-85.8) on the 10-fold cross-validation intra-subject test and 69.8% in the leave-one-out inter-subject test (range of 61.4-72.8). The variable-size data method did not improve classifier accuracy compared to the fixed-size method. Our method shows promise for inexpensive and objective quantification of functional upper extremity (UE) use in amputees and furthers the case for use of this method in assessing the impact of UE rehabilitative treatments.

Chunking Strategy to Improve Students' Reading Comprehension: An Action Research

The study focused on Chunking Strategy to Improve Student’s Reading Comprehension in the Nonequivalent Control Group Design, a form of quasi-experimental group design. This research consisted of two groups (experimental and controlled groups). Both groups were given pre-test and post-test to determine whether the chunking strategy could improve the students' reading comprehension. This research was conducted on the Junior High School Students of Sibugay Integrated School. Moreover, The researchers used the Reading Comprehension Test Research Instrument by Wigfield and Guthrie (1995) which was developed to evaluate many facets of students' reading motivation. The findings revealed a significant difference in the post-test and pretest scores of the students who underwent the chunking strategy. The findings of this study affirmed that using a chunking method was useful for teaching reading comprehension in the educational process. The pre-test and post-test scores of the students showed significant improvement. By using this strategy, teachers may make it easier for students to comprehend and retain material, which will boost their performance.

Service Recommendations Using a Hybrid Approach in Knowledge Graph with Keyword Acceptance Criteria

Businesses are overgrowing worldwide; people struggle for their businesses and startups in almost every field of life, whether industrial or academic. The businesses or services have multiple income streams with which they generate revenue. Most companies use different marketing and advertisement strategies to engage their customers and spread their services worldwide. Service recommendation systems are gaining popularity to recommend the best services and products to customers. In recent years, the development of service-oriented computing has had a significant impact on the growth of businesses. Knowledge graphs are commonly used data structures to describe the relations among data entities in recommendation systems. Domain-oriented user and service interaction knowledge graph (DUSKG) is a framework for keyword extraction in recommendation systems. This paper proposes a novel method of chunking-based keyword extractions for hybrid recommendations to extract domain-specific keywords in DUSKG. We further show that the performance of the hybrid approach is better than other techniques. The proposed chunking method for keyword extraction outperforms the existing value feature entity extraction (VF2E) by extracting fewer keywords.

A Design of Parallel Content-Defined Chunking System Using Non-Hashing Algorithms on FPGA

Content-defined chunking is a common method in many applications such as data deduplication and data synchronization. In recent years, new CDC algorithms using non-hashing methods have been developed, and positive results have been obtained. However, most of the algorithms are developed for single-thread computation on microprocessors. After analyzing some popular CDC algorithms, we observed that the algorithms using the basic sliding window protocol are more feasible to process in parallel. In this work, we proposed a new parallel chunking method that aims for hardware implementation. Additionally, we used the PCI algorithm, which does not include hash functions, to implement a multi-thread chunking system on FPGA devices. By exploiting the strength of the FPGAs, our proposed design achieves not only high computational speed but also great scalability.

The Comparison of Chunking Methods to Enhance the Cognitive Capacity of Short-term Memory to Retain Textual Information among High School Students

Our short-term memory has a limited capacity of taking in information and retaining it the memory storage. However, this can be enhanced by various memory techniques especially dividing the information into smaller chunks. To investigate this memory enhancement strategy, this study compared the effectiveness of three chunking methods, namely One-Chunk, Two-Chunk, and Three-Chunk to enhance the capacity to retain information in the forms of letters and numbers in the short-term storage. Participants in this study were 50 high school students who took part in an experimental design of online short-term memory assessment. The results revealed that the ability to remember 10 distinct alphabets and 10 distinct digits statistically varied, depending on how the information was chunked. To be more precise, the student participants could memorize the information when divided into 2 or 3 chunks more effectively than one full set of the information (1 chunk) as the mean scores gained in the two chunking methods were statistically greater than that of the One-Chunk delivered information. However, the findings only relied on single types of information so that further research could be done to explore this with more complicated information. Educational implications can be drawn from this present study that in order to assist students to memorize and retain learning materials more effectively, it is essential to help classify them into 2-3 groups of information. This could be done through the use of tree thinking, binary thinking, and computational thinking.

THE EFFECTIVENESS OF CHUNKING METHODS FOR ENHANCING SHORT-TERM MEMORY OF TEXTUAL INFORMATION

The effectiveness of students’ memorization of textual information was investigated in this study with 50 high school students. The information was presented to the participants in three different types: 10 distinct alphabets, 10 distinct numbers, and a combination of 5 distinct alphabets and 5 distinct numbers. This information was divided into three different chunking methods: One-Chunk where the whole information was told all at once, Two-Chunks where the information was divided into 5 and 5, and Three-Chunks where the information was delivered in 3-3-4, 4-3-3, and 3-4-3 fashions. The statistical results revealed that a single type of information (either all alphabets or all numbers) was found to be easier to recall than the combined information. Furthermore, dividing the information into two or three chunks was found to enhance human memorization more significantly. In addition, the study showed that when a combined type of information was shown, grouping the information into two chunks was more effective to enhance short-term memory than providing it in one chunk. Educational implications can be drawn from this study that in order to assist students to memorize and retain learning materials more effectively, it is essential to help classify them into 2-3 groups when being delivered. Also, learning should emphasize more on how to help students learn to take in information more effectively by themselves through the use of tree thinking, binary thinking, and computational thinking.

ANALISIS JALUR PENGARUH TEKNIK HITUNG VEDIC MATH, SUKU MAYA, CHUNKING TERHADAP PERHITUNGAN AKAR KUADRAT METODE BABILONIA

The purpose of this research is to discuss how the influence of several calculation techniques that exist both in the west and in the east, especially the Vedic Math, Maya, Chunking and Square Root Babylonian methods based on the use of path analysis. The approach in this research uses a descriptive approach, the method used is quantitative methods. The quantitative approach uses statistical tests with path analysis to answer the relationship and influence between the 4 variables. From the results of the path analysis, it is found that there is an effect of X1 (mastery of addition and subtraction with Vedic Math), X2 (mastery of multiplication by the Mayan tribe), X3 (mastery of division using the Chunking Method) on X3 (mastery of calculating square roots with Babylonian Middle Iteration). The total effect is 0.887, The total effect of 0.887 shows that the influence of Vedic Math's control of addition and control, mastery of the multiplication of the Maya method, and mastery of the division of the Chunking method based on path analysis have a joint effect on students' ability to calculate questions related to the square root of the Babylonian middle iteration technique.

Search by Screenshots for Universal Article Clipping in Mobile Apps

To address the difficulty in clipping articles from various mobile applications (apps), we propose a novel framework called UniClip, which allows a user to snap a screen of an article to save the whole article in one place. The key task of the framework is search by screenshots , which has three challenges: (1) how to represent a screenshot; (2) how to formulate queries for effective article retrieval; and (3) how to identify the article from search results. We solve these by (1) segmenting a screenshot into structural units called blocks, (2) formulating effective search queries by considering the role of each block, and (3) aggregating the search result lists of multiple queries. To improve efficiency, we also extend our approach with learning-to-rank techniques so that we can find the desired article with only one query. Experimental results show that our approach achieves high retrieval performance ( F 1 = 0.868), which outperforms baselines based on keyword extraction and chunking methods. Learning-to-rank models improve our approach without learning by about 6%. A user study conducted to investigate the usability of UniClip reveals that ours is preferred by 21 out of 22 participants for its simplicity and effectiveness.

A new content-defined chunking algorithm for data deduplication in cloud storage

Chunking is a process to split a file into smaller files called chunks. In some applications, such as remote data compression, data synchronization, and data deduplication, chunking is important because it determines the duplicate detection performance of the system. Content-defined chunking (CDC) is a method to split files into variable length chunks, where the cut points are defined by some internal features of the files. Unlike fixed-length chunks, variable-length chunks are more resistant to byte shifting. Thus, it increases the probability of finding duplicate chunks within a file and between files. However, CDC algorithms require additional computation to find the cut points which might be computationally expensive for some applications. In our previous work (Widodo et al., 2016), the hash-based CDC algorithm used in the system took more process time than other processes in the deduplication system. This paper proposes a high throughput hash-less chunking method called Rapid Asymmetric Maximum (RAM). Instead of using hashes, RAM uses bytes value to declare the cut points. The algorithm utilizes a fix-sized window and a variable-sized window to find a maximum-valued byte which is the cut point. The maximum-valued byte is included in the chunk and located at the boundary of the chunk. This configuration allows RAM to do fewer comparisons while retaining the CDC property. We compared RAM with existing hash-based and hash-less deduplication systems. The experimental results show that our proposed algorithm has higher throughput and bytes saved per second compared to other chunking algorithms.

Isotopic comparison of gelatin extracted from bone powder with that from bone chunk and development of a framework for comparison of different extraction methods

Stable isotope analysis of collagen extracted from ancient bones has become a routine exercise. Collagen can be extracted from either bone powder or chunks, but little research has compared these two procedures in detail. In this study, yield, elemental concentration, and carbon and nitrogen stable isotope ratios of gelatin extracted from bone powder or chunks were compared, and a new framework to compare the quality of gelatin samples extracted from identical bone samples using different extraction methods was presented. Forty adult human skeletons from two sites in Japan dating from the Edo period (1603–1867CE) were used. In the “chunk method,” bone chunks are demineralized, gelatinized, and filtered to obtain gelatin, while bone powder is used in the “powder method.” Using the chunk method, the gelatin yield (+5.29 ± 1.58%), %C (+0.62 ± 0.66%), and %N (+0.22 ± 0.22%) significantly increased, but the C/N ratio (−0.01 ± 0.03), δ13C (0.00 ± 0.09), and δ15N (−0.02 ± 0.21) did not significantly differ for 20 well-preserved bones. In 20 poorly preserved bones, the gelatin yield (+1.27 ± 1.33%), %N (+0.41 ± 0.49%), C/N ratio (−0.07 ± 0.07), and δ15N (−0.12 ± 0.37) significantly changed, but the %C (+0.37 ± 1.11%) and δ13C (−0.01 ± 0.09) were not significantly different using the chunk method. The observed significant differences have a sufficient statistical power (>0.8) with a significance level of 0.01 (except for the significant difference in δ15N of poorly preserved bones). The Fourier transform infrared spectra indicate that the extracted gelatin mostly consists of collagen without apparent contaminants. In general, in the context of palaeodietary studies, there are no actual differences in the δ13C and δ15N values obtained from the powder and chunk methods for gelatin from well preserved as well as poorly preserved bones.

PrIDE: A Protocol-Independent De-Duplication Engine for Packet Recording

Packet recording or capturing is one of the most useful tools for network forensics and surveillance. Since a storage system is of a limited size, de-duplication can be used to save disk space. In this article, we present a new scalable de-duplication engine for packet recording that can eliminate redundant contents over multiple packets. Unlike previous work, our proposed scheme is designed for packet-level de-duplication to support any kinds of network from the current Internet to emerging networks. We also present a new fast chunking method and a new indexing scheme that enable multiple engine instances to execute in parallel. We implement the de-duplication engine, and experimental results show that our proposed scheme can remove up to 65 percent of the packet contents in a real campus network. We also confirm that its throughput scalably increases with the number of CPU cores, which means that the proposed scheme can be implemented in a wide range of computing devices from small home gateways to high-end servers.

Quality enhancement of video on demand implementation in peer-to-peer networks by optimum chunk length in the BitTorrent

BitTorrent in efficiency content distribution is a major concern efficiency among the researchers of this field with respect to streaming video on demand (VoD) production. BitTorrent is not appropriate for real-time applications; therefore, in order to apply it in VoD it should go through the necessary changes. Most of the available studies have greatly focused on changes in methods regarding chunk and peer selections method regarding BitTorrent, which proposed methods have improved the quality of VoD to a certain degree, while, the effect of chunk size on quality of video has been of less concern among them. Noting the fact that the buffer is used on VoD, the specified time for filling the buffer would allow for appropriate management of the chunk’s length. The Bit error rate and the time overhead of the operating algorithm parameters, somehow effect the chunk size. Because of bit error rate, the probability of correctly received chunks with great length is much less, that is, offering shorter pieces, while these pieces would lead to formation of more pieces in a buffer. The results indicate that a specific amount of time is required for obtaining the buffer’s content, and it must be dividable into more chunks. Running algorithms for each chunk generate a greater overhead which would result in of the QoS reduction. This overhead would make the bigger pieces perform better. As for the opposite impacts of these two parameters on the chunk size, in this article the optimal length of the chunks is found by considering both the effective characteristic. This optimal length is an established balance between the correctly received chunks’ rate and the greater rate of the buffer context obtained in a specified time.

Performance Analysis in Web Based Data Uploading using LZ77 Compression and Chunking Method

One of the limitations in data uploading process is the maximum request length, besides that the data size that us transferred is also an issue because it influences the data sending cost. One of the way to cope with the problem of maximum request length is by downsizing the file size (chunking). Another way to do it is by enlarging the maximum reques length. Downsizing the file size can be done by chunking the files into a smaller size or by compressing it. In this paper, the author conducted a research about the file compression process that is done in client server using the technology of AJAX and Webservice. In addition to that, the file compression is combined with file chunking. In this research, the compression method that is used is dictionary based i.e. Lempel Ziv 77(LZ77). This compression is used since it can be performed in AJAX. The analysis that is made by the researcher about the compression ratio, data sending process speed, compression time, decompression time, the compression method capability in handling the maximum request length and the combination method of compression and chunking in uploading process. The result of this research shows that compression method can handle the maximum requet length. Based on the experiment conducted, the relations between the compression ratio and window length is positively corelated. It means that the greater the window length is the more the compression ratio is. Meanwhile, the relation between window length and uploading time is negatively linearly corelated. It means that the greater the window length is the faster the uploading time is. In addition, it can also be observed that the relation between the decompression and the file size is positively linearly correlated. It means that the greater the file size is the more time needed for decompression is.

Framework of Data Deduplication: A Survey

To understand the concept and framework of de-duplication process along with application, various methods and technologies involved during the each level implementation of this process and about the Limitations. Different chunking algorithm such as fixed, variable and content aware chunking methods are used to decide the chunk size of deduplication. To avoid the single point failure in the distributed system, cluster model with parallel process can be used. Comparing with and without deduplication, we can save up to 75% of storage space in the backup system by using with deduplication. Digital data increase happens in all cloud deployment models and this requires more storage capacity, more costs, manpower and more time to handle data information like backup, replication and disaster recovery, more bandwidth utilization in transmitting the data across the network. If we handle the data effectively like remove the redundant data before storing into the storage device we can avoid data handling overhead and we can improve the system performance. By using data deduplication concept we can achieve the above results. The variable chunking method including content aware process yields good throughput in the deduplication system compare with fixed and whole file chunking method. Inline process avoids extra required spaces and increase system performance. Cluster model eliminates single point failure in the distributed deduplication system. Deduplications save more storage spaces, avoid tohandle unnecessary data over head and provides less resources utilization with minimal cost.

Energy Efficient Metadata Management for Cloud Storage System

To effectively handle duplicate files, data deduplication schemes are widely used in many storage systems. Data deduplication algorithms reduce storage space by eliminating data to ensure that only single instance of data is stored in storage device. In this paper, we propose an energy efficient file synchronization scheme that provides hybrid data chunking using variable-length chunking (VLC) and fixed-length chunking (FLC). The main idea is to analyze similarities between old and new versions of data and decide which chunking method to apply in synchronizing the files. In particular, the proposed algorithm exploits the file similarity pattern for calculating the energy efficiency of chunking algorithms. We have developed an Android mobile application for file synchronization and measured energy consumption. The experiment results show that the proposed scheme helps save energy in synchronizing files, regardless of file types or amount of redundancies the files have.