Data Chunks Research Articles

Study of Different Algorithms on Opinion Mining

Abstract: Social media is critical in today's world for exchanging information and disseminating ideas. A person's emotional impact has a significant impact on their day-to-day life. Sentiment analysis is a form of text mining that locates and pulls out subjective information from sources, allowing a company to track discussions online and monitor social sentiment about their brand, product, or service. Simply put, sentiment analysis helps determine the author's attitude towards a topic. Positive, neutral, or negative pieces of writing are classified by sentiment analysis software. Deep learning algorithms and various functions of natural language processing helps to interpret the written or spoken sentiments regarding a topic. An ecosystem where millions of bytes of data are produced daily has enabled sentiment analysis to be a key tool for interpreting these huge chunks of data. The purpose of this work is to conduct a sentiment analysis on "tweets" by making use of a variety of machine learning algorithms. The study will make an attempt to categorise the polarity of the tweet as either positive, negative, or neutral. In the event that a tweet has only positive, negative, or neutral components, the label assigned to the tweet will be determined by the sentiment that predominates.

Data partitioning and scheduling schemes for federated platoon-based vehicular cloud

The organization of vehicles into platoons is very promising due to its contributions to intelligent transportation systems, and the reduction in traffic congestion and fuel consumption. As the powerful processing units and other resources embedded in vehicles may not be fully utilized during the entire travel period of vehicular platoons, we claim that the vehicles can collaborate as a single unit to form a federated platoon-based vehicular cloud to meet the high demand for computing resources and services in vehicular environments. In order to make advancements in the deployment of federated platoon-based vehicular cloud, data partitioning and scheduling schemes that distribute data chunks of large and divisible application data among platoon vehicles are proposed considering the characteristics of vehicular resources, network parameters, and the position of vehicles in the platoon. The data partitioning and scheduling schemes, which are modeled based on the different information flow topologies of vehicular platoon, consist of the Bi-Directional Recursive (BD-R), Bi-Directional Interlaced (BD-I), Bi-Directional Lead Recursive (BDL-R), Bi-Directional Lead Interlaced (BDL-I), Bi-Directional Lead Aggregate Recursive (BDLA-R) and the Bi-Directional Lead Aggregate Interlaced (BDLA-I). Performance analysis carried out through realistic simulations showed that while the BDL-R and BDL-I schemes have the best performance in terms of task execution time, the other schemes have the advantage of enforcing priority in unprocessed data transmission and dependency in the aggregation of processed data chunks by each platoon member. Analysis of the impact of the proposed data partitioning and scheduling schemes on platoon string stability will be examined in future studies.

Feature selection using PRACO method for IDS in cloud environment

Cloud Computing is the distribution of computing resources on demand to the users over Internet. But with virtual existence of data and resources comes the problem of privacy and security. In such environments Intrusion Detection System (IDS) comes in handy. They read huge chunks of data to find out attack patterns. But learning through this huge amount of data is very time consuming. So, data reduction is necessary. Using feature selection methods, number of features can be reduced by eliminating redundant and irrelevant attributes from datasets. In this paper the authors have proposed a Penalty Reward based Ant Colony Optimization (PRACO) method for feature selection. The penalty and reward terms used in this paper help in better exploration-exploitation trade-off by rewarding the useful features and penalizing the other ones. Along with that the concepts of max-relevance and min-redundancy are also used to indicate interactions between selected features. The proposed model is assessed on 10% KDD Cup 99, NSL-KDD and UNSW-NB15 datasets. It was observed that the PRACO method achieved 81.682% and 83.584% accuracy on average during train-test phase using NSL-KDD and UNSW-NB15 datasets. The results provide substantial evidence that the proposed model is effective in finding optimal results and thus provide IDS with increased efficiency.

Forseti: Dynamic chunk-level reshaping for data processing on heterogeneous clusters

Data-intensive computing frameworks typically split job workload into fixed-size chunks, allowing them to be processed as parallel tasks on distributed machines. Ideally, when the machines are homogeneous and have identical speed, chunks of equal size would finish processing at the same time. However, such determinism in processing time cannot be guaranteed in practice. Diverging processing times can result from various sources such as system dynamics, machine heterogeneity, and variable network conditions. Such variation, together with dynamics and uncertainty during task processing, can lead to significant performance degradation at job level, due to long tails in job completion time resulted from residual chunk workload and stragglers.In this paper, we propose Forseti, a novel processing scheme that is able to reshape data chunk size on the fly with respect to heterogeneous machines and a dynamic execution environment. Forseti mitigates residual workload and stragglers to achieve significant improvement in performance. We note that Forseti is a fully online scheme and does not require any a priori knowledge of the machine configuration nor job statistics. Instead, it infers such information and adjusts data chunk sizes at runtime, making the solution robust even in environments with high volatility. In its implementation, Forseti also exploits a virtual machine reuse feature to avoid task start-up and initialization cost associated with launching new tasks. We prototype Forseti on a real-world cluster and evaluate its performance using several realistic benchmarks. The results show that Forseti outperforms a number of baselines, including default Hadoop by up to 68% and SkewTune by up to 50% in terms of average job completion time.

Performance Enhancement of SMR-Based Deduplication Systems

Due to the fast-growing amount of data and cost consideration, shingled-magnetic-recording (SMR) drives are developed to provide low-cost and high-capacity data storage by enhancing the areal-density of hard disk drives, and (data) deduplication techniques are getting popular in data-centric applications to reduce the amount of data that need to be stored in storage devices by eliminating the duplicate data chunks. However, directly applying deduplication techniques on SMR drives could significantly decrease the runtime performance of the deduplication system because of the time-consuming SMR space reclamation caused by the sequential write constraint of SMR drives. In this article, an SMR-aware deduplication scheme is proposed to improve the runtime performance of SMR-based deduplication systems with the consideration of the sequential write constraint of SMR drives. Moreover, to bridge the information gap between the deduplication system and the SMR drive, the lifetime information of data chunks is extracted to separate data chunks of different lifetimes in different places of SMR drives, so as to further reduce the SMR space reclamation overhead. A series of experiments was conducted with a set of realistic deduplication workloads. The results show that the proposed scheme can significantly improve the runtime performance of the SMR-based deduplication system with limited system overheads.

The French Book Trade in Enlightenment Europe I: Selling Enlightenment

The French Book Trade in Enlightenment Europe I: Selling Enlightenment

DB-CMT: A New Concurrent Multi-path Stream Control Transport Protocol

Stream Control Transmission Protocol (SCTP) exploits multiple network interfaces to provide multi-streaming and data chunk ordering in a stream. An extended feature of SCTP, i.e., Concurrent Multi-path Transfer (CMT), bids concurrent data transmission in a multi-path data transfer environment and guarantees bandwidth aggregation, load sharing, robustness, and reliability. In such an environment, the paths usually have distinct characteristics (i.e., delay, Packet Loss Rate (PLR), and bandwidth). Thus, data chunks are received out-of-ordered at the destination. As a result, CMT causes excessive receiver buffer blocking and unnecessary congestion window (cwnd) reductions. Also, during the selection of the retransmission destination path (to resend a lost data chunk), CMT does not take into account vital Quality of Service (QoS) parameters such as the PLR of a path under consideration. This paper introduces a new Delay-Based Concurrent Multi-path Transfer (DB-CMT) approach that transmits data on multiple paths according to their delay. In this scheme, we present a Delay-Based Data chunk Scheduling Policy (DB-DSP), a Retransmission Path Selection Policy (RTX-CL), and a new Delay-Based Fast Retransmission Policy (DB-FRP). The simulation results show that the DB-CMT’s RTX-CL policy performs better than the well-known RTX-CWND and RTX-LOSSRATE retransmission schemes. Also, the overall performance of DB-CMT witnesses improved throughput, fewer timeouts, and reduced File Transfer Time (FTT) performances.

A Text Mining Approach to Determinants of Attitude Towards Syrian Immigration in the Turkish Twittersphere

This study uses novel deep learning-based language models to extract meaningful information from vast chunks of textual data from Twitter on the competing narratives of the recent Syrian immigration to Turkey. Our analysis identifies five main topics in the framing of Syrian immigration in Turkish Twittersphere. In this paper, we demonstrate correlational links between the timing of landmark events and change in the percent share of trends in those topics across time. We highlight two important observations: (a) Social benefit demands of natives on Twitter rose sharply with the COVID-19 pandemic, leading to ever more widespread sentiments of welfare chauvinism and (b) Patriotic feelings and the implementation of an interventionist foreign policy agenda in the immigrants’ country of origin created a relatively tolerant yet patronizing attitude towards migrants. As the COVID-19 pandemic and immigration frequently occupy the center stage in politics of immigrant-hosting societies, our research has international appeal beyond its specific geographical context.

Preemptive Scheduling for Distributed Machine Learning Jobs in Edge-Cloud Networks

Recent advances in 5G and edge computing enable rapid development and deployment of edge-cloud systems, which are ideal for delay-sensitive machine learning (ML) applications such as autonomous driving and smart city. Distributed ML jobs often need to train a large model with enormous datasets, which can only be handled by deploying a distributed set of workers in an edge-cloud system. One common approach is to employ a parameter server (PS) architecture, in which training is carried out at multiple workers, while PSs are used for aggregation and model updates. In this architecture, one of the fundamental challenges is how to dispatch ML jobs to workers and PSs such that the average job completion time (JCT) can be minimized. In this work, we propose a novel online preemptive scheduling framework to decide the location and the execution time window of concurrent workers and PSs upon each job arrival. Specifically, our proposed scheduling framework consists of: i) a job dispatching and scheduling algorithm that assigns each ML job to workers and decides the schedule to train each data chunk; ii) a PS assignment algorithm that determines the placement of PS. We prove theoretically that our proposed algorithm is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$D_{max}(1+1/\epsilon)$ </tex-math></inline-formula> -competitive with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$(1 + \epsilon)$ </tex-math></inline-formula> -speed augmentation, where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$D_{max}$ </tex-math></inline-formula> is the maximal number of data chunks in any job. Extensive testbed experiments and trace-driven simulations show that our algorithm can reduce the average JCT by up to 30% compared with state-of-the-art baselines.

A dynamic ensemble algorithm for anomaly detection in IoT imbalanced data streams

With the rapid development of ambient intelligence (AmI) in the Internet of Things (IoT), many data streams are generated from sensing devices in intelligent scenarios. Due to the deployment issues of IoT devices and the system’s complexity, abnormal behavior is inevitable, resulting in imbalanced data categories. Moreover, data streams generated in IoT systems are dynamic, continuous, and as the environment changes, further increasing the difficulty of anomaly detection. Therefore, we model the monitored historical and current data from the perspective of dynamic imbalanced data streams classification to discover abnormal behaviors in IoT systems. In this paper we propose a dynamic ensemble algorithm for anomaly detection in IoT environments. First the abnormal data samples are synthesized by the borderline-synthetic minority over-sampling technique (Borderline-SMOTE) to relieve the sample imbalance problem. Then considering the dynamics and continuity of data streams we adopt a chunk-based strategy to train a LightGBM classifier for each chunk of data to adapt to the current data distribution. To improve the ensemble model’s processing efficiency and anomaly detection accuracy we adopt a dynamic weighting strategy for base classifiers and remove the classifier whose accuracy performance is lower than the threshold. Finally we evaluate our proposed algorithm by conducting comparative experiments on real-world data streams. Experimental results show that our proposed algorithm outperforms the comparative anomaly detection methods in IoT scenarios.

Scalable Cluster Tendency Assessment for Streaming Activity Data using Recurring Shapelets.

Automatic interpretation of cluster structure in rapidly arriving data streams is essential for timely detection of interesting events. Human activities often contain bursts of repeating patterns. In this paper, we propose a new relative of the Visual Assessment of Cluster Tendency (VAT) model, to interpret cluster evolution in streaming activity data where shapes of recurring patterns are important. Existing VAT algorithms are either suitable only for small batch data and unscalable to rapidly evolving streams, or cannot capture shape patterns. Our proposed incremental algorithm processes streaming data in chunks and identifies repeating patterns or shapelets from each chunk, creating a Dictionary-of-Shapes (DoS) that is updated on the fly. Each chunk is transformed into a lower dimensional representation based on it's distance from the shapelets in the current DoS. Then a small set of transformed chunks are sampled using an intelligent Maximin Random Sampling (MMRS) scheme, to create a scalable VAT image that is incrementally updated as the data stream progresses. Experiments on two upper limb activity datasets demonstrate that the proposed method can successfully and efficiently visualize clusters in long streams of data and can also identify anomalous movements.

Big Data Backup Deduplication : A Survey

The massive explosion in the field of data such as images, video, audio, and text has caused significant problems in data storage and retrieval. Companies and organizations spend a lot of money to store and manage data. Therefore, there is an urgent need for efficient technologies to deal with this massive amount of data. One of the essential techniques to eliminate redundant data is data deduplication and data reduction. The best technique used for this purpose is data deduplication. Data deduplication decreases bandwidth, hard disc drive utilization, and backup costs by removing redundant data. This paper focuses on studying the literature of several research papers related to data deduplication for various techniques that several researchers have proposed. It summarized multiple concepts and techniques related to deduplication and methods used to improve storage. The data deduplication processes were examined in detail, including data chunking, hashing, indexing, and writing. Also, this study discussed the most critical problems faced by the data deduplication algorithm.

Secure Data Deduplication System with Efficient and Reliable Multi-Key Management in Cloud Storage

&lt;p&gt;The revolutionary growth in the processing and storage mechanisms over the Internet has given the enhancement to inexpensive and strong computing properties. Cloud computing is a rising technology, which offers the data storage facility also application accessing facility in online environment. This system stands countless opportunities also challenges. In that, security of data and the increasing similar data in cloud (duplication) are very important issues to be addressed. So, Deduplication method is developed to reduce the similar data that is present in the storage system. In this paper, a novel technique is proposed to remove the duplicate data from cloud also help to save the bandwidth access and storage space. The experimental results demonstrate that the proposed system provide the more security for data in cloud storage and also overcomes the main drawbacks of the existing systems. In one-server storage and distributed storage systems, we have created a solution which provides data security and space efficacy. The chunk data generates encryption keys consistently; the same chunk is therefore always encrypted with the same chip text. In addition, the keys cannot be derived from the chunk data encrypted. Because the information to be accessed and decrypted by each user is encrypted by using a key known to the user alone, even a complete system breach cannot expose which chunks are utilised by which users.&lt;/p&gt; &lt;p&gt;&amp;nbsp;&lt;/p&gt;

Xenon LFP Analysis Platform Is a Novel Graphical User Interface for Analysis of Local Field Potential From Large-Scale MEA Recordings.

High-density multi-electrode array (HD-MEA) has enabled neuronal measurements at high spatial resolution to record local field potentials (LFP), extracellular action potentials, and network-wide extracellular recording on an extended spatial scale. While we have advanced recording systems with over 4,000 electrodes capable of recording data at over 20 kHz, it still presents computational challenges to handle, process, extract, and view information from these large recordings. We have created a computational method, and an open-source toolkit built in Python, rendered on a web browser using Plotly’s Dash for extracting and viewing the data and creating interactive visualization. In addition to extracting and viewing entire or small chunks of data sampled at lower or higher frequencies, respectively, it provides a framework to collect user inputs, analyze channel groups, generate raster plots, view quick summary measures for LFP activity, detect and isolate noise channels, and generate plots and visualization in both time and frequency domain. Incorporated into our Graphical User Interface (GUI), we also created a novel seizure detection method, which can be used to detect the onset of seizures in all or a selected group of channels and provide the following measures of seizures: distance, duration, and propagation across the region of interest. We demonstrate the utility of this toolkit, using datasets collected from an HD-MEA device comprising of 4,096 recording electrodes. For the current analysis, we demonstrate the toolkit and methods with a low sampling frequency dataset (300 Hz) and a group of approximately 400 channels. Using this toolkit, we present novel data demonstrating increased seizure propagation speed from brain slices of Scn1aHet mice compared to littermate controls. While there have been advances in HD-MEA recording systems with high spatial and temporal resolution, limited tools are available for researchers to view and process these big datasets. We now provide a user-friendly toolkit to analyze LFP activity obtained from large-scale MEA recordings with translatable applications to EEG recordings and demonstrate the utility of this new graphic user interface with novel biological findings.

Chunk Dynamic Updating for Group Lasso with ODEs

Group Lasso is an important sparse regression method in machine learning which encourages selecting key explanatory factors in a grouped manner because of the use of L-2,1 norm. In real-world learning tasks, some chunks of data would be added into or removed from the training set in sequence due to the existence of new or obsolete historical data, which is normally called dynamic or lifelong learning scenario. However, most of existing algorithms of group Lasso are limited to offline updating, and only one is online algorithm which can only handle newly added samples inexactly. Due to the complexity of L-2,1 norm, how to achieve accurate chunk incremental and decremental learning efficiently for group Lasso is still an open question. To address this challenging problem, in this paper, we propose a novel accurate dynamic updating algorithm for group Lasso by utilizing the technique of Ordinary Differential Equations (ODEs), which can incorporate or eliminate a chunk of samples from original training set without retraining the model from scratch. Specifically, we introduce a new formulation to reparameterize the adjustment procedures of chunk incremental and decremental learning simultaneously. Based on the new formulation, we propose a path following algorithm for group Lasso regarding to the adjustment parameter. Importantly, we prove that our path following algorithm can exactly track the piecewise smooth solutions thanks to the technique of ODEs, so that the accurate chunk incremental and decremental learning can be achieved. Extensive experimental results not only confirm the effectiveness of proposed algorithm for the chunk incremental and decremental learning, but also validate its efficiency compared to the existing offline and online algorithms.

Online Enhanced Semantic Hashing: Towards Effective and Efficient Retrieval for Streaming Multi-Modal Data

With the vigorous development of multimedia equipments and applications, efficient retrieval of large-scale multi-modal data has become a trendy research topic. Thereinto, hashing has become a prevalent choice due to its retrieval efficiency and low storage cost. Although multi-modal hashing has drawn lots of attention in recent years, there still remain some problems. The first point is that existing methods are mainly designed in batch mode and not able to efficiently handle streaming multi-modal data. The second point is that all existing online multi-modal hashing methods fail to effectively handle unseen new classes which come continuously with streaming data chunks. In this paper, we propose a new model, termed Online enhAnced SemantIc haShing (OASIS). We design novel semantic-enhanced representation for data, which could help handle the new coming classes, and thereby construct the enhanced semantic objective function. An efficient and effective discrete online optimization algorithm is further proposed for OASIS. Extensive experiments show that our method can exceed the state-of-the-art models. For good reproducibility and benefiting the community, our code and data are already publicly available.

Path-Rank-Based Data Chunk Scheduling for Concurrent Multipath Transmission

Abstract The device equipped with a multi-homing feature optimally exploits multiple network interfaces in modern communications networks such as the Internet of Things (IoT) and machine-to-machine communication using the concurrent multipath transfer (CMT). This enhances system performance by concurrently scheduling data chunks on multiple network paths. For a while, several scheduling criteria have been developed to optimize performance. However, it has been identified that CMT still suffers from many serious problems, such as spurious retransmission, receiver buffer blocking, improper congestion window (CWND) growth, re-ordering and long round trip time, resulting in poor performance. These problems occur due to the asymmetric nature of path characteristics. Thus, this paper introduces a path rank-based CMT (R-CMT) that schedules data chunks according to the rank of the path. The proposed scheduling method calculates the rank of each network path based on the ratio of successfully received and transmitted chunks. The simulation results indicate that the proposed R-CMT scheduling achieves higher performance in terms of network latency, throughput and CWND growth.

Online learning of windmill time series using Long Short-term Cognitive Networks

Forecasting windmill time series is often the basis of other processes such as anomaly detection, health monitoring, or maintenance scheduling. The amount of data generated by windmill farms makes online learning the most viable strategy to follow. Such settings require retraining the model each time a new batch of data is available. However, updating the model with new information is often very expensive when using traditional Recurrent Neural Networks (RNNs). In this paper, we use Long Short-term Cognitive Networks (LSTCNs) to forecast windmill time series in online settings. These recently introduced neural systems consist of chained Short-term Cognitive Network blocks, each processing a temporal data chunk. The learning algorithm of these blocks is based on a very fast, deterministic learning rule that makes LSTCNs suitable for online learning tasks. The numerical simulations using a case study involving four windmills showed that our approach reported the lowest forecasting errors with respect to a simple RNN, a Long Short-term Memory, a Gated Recurrent Unit, and a Hidden Markov Model. What is perhaps more important is that the LSTCN approach is significantly faster than these state-of-the-art models.

StoreSim: Optimizing Information Leakage in Multi-Cloud Storage Services

Abstract: Many schemes have been recently advanced for storing data on multiple clouds. Distributing data over different cloud storage providers (CSPs) automatically provides users with a certain degree of information leakage control, as no single point of attack can leak all a user’s information. However, unplanned distribution of data chunks can lead to high information disclosure even while using multiple clouds. In this paper, to address this problem, we present Store-Sim, an information leakage-aware storage system in multi cloud. Store-Sim aims to store syntactically similar data on the same cloud, thus minimizing the user’s information leakage across multiple clouds. We design an approximate algorithm to efficiently generate similarity- preserving signatures for data chunks based on Min-Hash and Bloom filters, and design a function to compute the information leakage based on these signatures. Next, we present an effective storage plan generation algorithm based on clustering for distributing data chunks with minimal information leakage across multiple clouds. Finally, we evaluated our scheme using two real datasets from Wikipedia and GitHub. We show that our scheme can reduce information leakage by up to 60-70 Percent. Keywords: Min-Hash, Bloom-Filter, Multi-Cloud, StoreSim, Data Leakage.

Establishing Pertinence between Sorting Algorithms Prevailing in n log(n) time

Data is the new fuel. With the expansion of global technology, the increasing living standards, and modernization, data values have caught great height. Nowadays, nearly all top MNCs feed on data. Now, storing all this data is a prime concern for all of them, which is relieved by the Data Structures, the systematic way of storing data. Now, once these data are stored and charged in secure vaults, it’s time to utilize them most efficiently. Now, many operations need to be performed on these massive chunks of data, like searching, sorting, inserting, deleting, merging, and so more. In this paper, we would be comparing all the major sorting algorithms, that have prevailed to date. Further, work has been done and inequality in the dimension of time between the three Sorting algorithms, operational in n log(n) time, Merge, Quick, and Heap, that have been discussed in the paper have been proposed.