Large Chunks Of Data Research Articles

GPU optimization techniques to accelerate optiGAN—a particle simulation GAN

The demand for specialized hardware to train AI models has increased in tandem with the increase in the model complexity over the recent years. Graphics processing unit (GPU) is one such hardware that is capable of parallelizing operations performed on a large chunk of data. Companies like Nvidia, AMD, and Google have been constantly scaling-up the hardware performance as fast as they can. Nevertheless, there is still a gap between the required processing power and processing capacity of the hardware. To increase the hardware utilization, the software has to be optimized too. In this paper, we present some general GPU optimization techniques we used to efficiently train the optiGAN model, a Generative Adversarial Network that is capable of generating multidimensional probability distributions of optical photons at the photodetector face in radiation detectors, on an 8GB Nvidia Quadro RTX 4000 GPU. We analyze and compare the performances of all the optimizations based on the execution time and the memory consumed using the Nvidia Nsight Systems profiler tool. The optimizations gave approximately a 4.5x increase in the runtime performance when compared to a naive training on the GPU, without compromising the model performance. Finally we discuss optiGANs future work and how we are planning to scale the model on GPUs.

X-Stream: Accelerating streaming segments on MPSoCs for real-time applications

We are witnessing a race to meet the ever-growing computation requirements of emerging AI applications to provide perception and control in autonomous vehicles — e.g., self-driving cars and UAVs. To remain competitive, vendors are packing more processing units (CPUs, programmable logic, GPUs, and hardware accelerators) into next-generation multiprocessor systems-on-a-chip (MPSoC). As a result, modern embedded platforms are achieving new heights in peak computational capacity. Unfortunately, however, the collateral and inevitable increase in complexity represents a major obstacle for the development of correct-by-design safety-critical real-time applications. Due to the ever-growing gap between fast-paced hardware evolution and comparatively slower evolution of real-time operating systems (RTOS), there is a need for real-time oriented full-platform management frameworks to complement traditional RTOS designs.In this work, we propose one such framework, namely the X-Stream framework, for the definition, synthesis, and analysis of real-time workloads targeting state-of-the-art accelerator-augmented embedded platforms. Our X-Stream framework is designed around two cardinal principles. First, computation and data movements are orchestrated to achieve predictability by design. For this purpose, iterative computation over large data chunks is divided into subsequent segments. These segments are then streamed leveraging the three-phase execution model (load, execute and unload). Second, the framework is workflow-centric: system designers can specify their workflow and the necessary code for workflow orchestration is automatically generated.In addition to automating the deployment of user-defined hardware-accelerated workloads, X-Stream supports the deployment of some computation segments on traditional CPUs. Finally, X-Stream allows the definition of real-time partitions. Each partition groups applications belonging to the same criticality level and that share the same set of hardware resources, with support for preemptive priority-driven scheduling. Conversely, freedom from interference for applications deployed in different partitions is guaranteed by design. We provide a full-system implementation that includes RTOS integration and showcase the proposed X-Stream framework on a Xilinx Ultrascale+ platform by focusing on a matrix-multiplication and addition kernel use-case.

Cryptanalysis of Authentication Protocol for Cloud Assisted IoT Environment

In today's wireless based applications, sensors are playing the vital role due to its pluses like low cost, low maintenance, hostile environment etc. On other end, cloud based technologies are increasing day-by-day to make its presence in the life of people for processing large chunk of data. Also, Internet of Things (IoT) is making its way by utilizing the sensors for various internet based applications and to connect to them with cloud. In all of these technologies, the major issue is authentication i.e. user from distance can access the server and authenticate via insecure channel. Recently, Lee et al. proposed the authentication scheme in IoT based environment using sensors and claimed it to be secure against various attacks. However, in this research we have analyzed the scheme and prove that it is yet susceptible to key control, time synchronization and stolen verifier. In addition, there will be overhead for verification which can lead to the DoS attack for large setup.

Mobile Agent Improvement Testing on A Distributed Network Cluster Using Unsorted Metadata with A Distribution and Delocalization Model (DAME)

There has been growing demand for high performance systems for processing of both structured and unstructured data thus prompting managers and Organizations to find better methods for addressing the data processing needs now and in the future. The trend is projected to increase exponentially, as virtualized and distributed IOT systems are likely to exacerbate the problem as individual nodes will handle large chunks of data; consequently, these organizations are immersing their energies and resources in the research and use of Intelligent tools for data management and analysis which require real time processing, storage and transmission. Our research inspired by the Amidal’s law and Gustafson Barsis law of distribution uses Mobile agent distribution model complimented with map reduce in a virtualized environment to discover the extent to which the distribution of server nodes may improve performance as compared to the centralized server nodes in order to handle large amounts of data that will be produced and transmitted by the individual nodes. The distribution model in our research borrows from the concept of divide and conquer algorithms whose run-time is O (n log n). To test performance improvement, we employed a custom made Simulator called DAME, which has the capability of catching and distributing metadata through its agent based domain controller. Our research indicates that distribution of nodes on a network has a significant performance improvement with throughput increasing by 88 %, Latency decreasing by 23% and Scalability improvement by up to 43 %.

Wireless Sensor Data Fusion Techniques in Estimating Temporal Resource Attributes in Scenarios of Intermittent Connectivity

In data fusion process the fusion centre which lies intermediate distance aggregates the data and forwards to the sink. In scenario of data aggregation at fusion centre an imbalance may occur due to the potential forwarding process of other fusion centres or other sensor nodes to sink. Hence, this back log of time results in forwarding large chunks of data resulting in link imbalance where the associated classical time interval of reporting varies. The problem balancing and coordinating among fusion centres has been achieved in this work using Data Fusion using Generalized Interval Probability Protocol (DFGIPP). DFGIPP is developed considered the duality principle with proper and improper intervals of reporting to provide coherence among the links. Thus allocating and de-allocating links with the quality of fusion metrics in cooperation between fusion centre and sensor nodes within the terrain is being achieved. The simulation using discrete event network simulator-2 provides better fusion capability under data transfer rates and simulation scenarios.

3D convolutional neural networks based automatic modulation classification in the presence of channel noise

AbstractAutomatic modulation classification is a task that is essentially required in many intelligent communication systems such as fibre‐optic, next‐generation 5G or 6G systems, cognitive radio as well as multimedia internet‐of‐things networks etc. Deep learning (DL) is a representation learning method that takes raw data and finds representations for different tasks such as classification and detection. DL techniques like Convolutional Neural Networks (CNNs) have a strong potential to process and analyse large chunks of data. In this work, we considered the problem of multiclass (eight classes) classification of modulated signals, which are, Binary Phase Shift Keying, Quadrature Phase Shift Keying, 16 and 64 Quadrature Amplitude Modulation corrupted by Additive White Gaussian Noise, Rician and Rayleigh fading channels using 3D‐CNN architectures in both frequency and spatial domains while deploying three approaches for data augmentation, which are, random zoomed in/out, random shift and random weak Gaussian blurring augmentation techniques with a cross‐validation (CV) based hyperparameter selection statistical approach. Simulation results testify the performance of 10‐fold CV without augmentation in the spatial domain to be the best while the worst performing method happens to be 10‐fold CV without augmentation in the frequency domain and we found learning in the spatial domain to be better than learning in the frequency domain.

Prevention of Clone Attack in Online Social Media Integrating Secure Data Transmission

Social networking is becoming so essential nowadays and is playing a bigger role in every man’s life for sharing information and knowledge. Also social network is used to see the everyday activities, photos, videos, political agendas and propagandas. Therefore, it is now becoming an important tool to stay updated in this dynamic world. With large chunks of data being generated every second, there is a growing concern about Data protection and user privacy in the Social media network. One of the major concerns being, ‘Fake Users’ - misusing the authorized user’s information like photos and videos without the authorized user’s permission and disguising oneself as a legitimate user. In our contemporary world, many fake profiles are being created for fraudulent activities like money making, malware / virus / Trojan distribution to use user data, especially with malicious intent. In this paper Java static watermarking is proposed. Java static watermark is used in our social media website in order to associate each user's footprint with respect to their unique ID, eliminating the crux of fake users. It is also very evident to say that data present on the cloud is no less prone to cyber-attacks. In this paper, integration of steganography methods for protection of sensitive data on the public cloud server is also proposed to validate its viability and its increased security. The Algorithms used ensure the individual information is kept secret and transmitted in a secure manner with user privacy preserving.

Staged data delivery protocol: A blockchain‐based two‐stage protocol for non‐repudiation data delivery

AbstractFor many years, non‐repudiation in data delivery has not been satisfactorily achieved due to the difficulty of supervising delivery behavior. Existing fair protocols are still not entirely free of third parties who are usually not as trustworthy as expected. Moreover, most of them are built on complex encryption schemes, resulting in inefficiencies when handling large chunks of data and non‐exchanged data sharing. To be able to deliver data without third parties and repudiation, we propose the staged data delivery protocol (SDDP) that integrates fair data delivery into Blockchain consensus processes. SDDP confirms transactions of data delivery in a cascading manner where nodes sign on each transaction one after another. It consists of two stages, in which neither party will have a considerable advantage over the other. In this way, SDDP enables non‐repudiation of directly delivering large chunks of data through an understandable communication strategy; and the Blockchain network replaces third parties to preserve an immutable history of data delivery activities. The results of extensive evaluations show that SDDP is admissible for dealing with fair data delivery on the basis of its good qualities and performance.

Massively scalable density based clustering (DBSCAN) on the HPCC systems big data platform

<span id="docs-internal-guid-919b015d-7fff-56da-f81d-8f032097bce2"><span>Dealing with large samples of unlabeled data is a key challenge in today’s world, especially in applications such as traffic pattern analysis and disaster management. DBSCAN, or density based spatial clustering of applications with noise, is a well-known density-based clustering algorithm. Its key strengths lie in its capability to detect outliers and handle arbitrarily shaped clusters. However, the algorithm, being fundamentally sequential in nature, proves expensive and time consuming when operated on extensively large data chunks. This paper thus presents a novel implementation of a parallel and distributed DBSCAN algorithm on the HPCC Systems platform. The algorithm seeks to fully parallelize the algorithm implementation by making use of HPCC Systems optimal distributed architecture and performing a tree-based union to merge local clusters. The proposed approach* was tested both on synthetic as well as standard datasets (MFCCs Data Set) and found to be completely accurate. Additionally, when compared against a single node setup, a significant decrease in computation time was observed with no impact to accuracy. The parallelized algorithm performed eight times better for higher number of data points and takes exponentially lesser time as the number of data points increases.</span></span>

Sensors Anomaly Detection of Industrial Internet of Things Based on Isolated Forest Algorithm and Data Compression

Aiming at solving network delay caused by large chunks of data in industrial Internet of Things, a data compression algorithm based on edge computing is creatively put forward in this paper. The data collected by sensors need to be handled in advance and are then processed by different single packet quantity K and error threshold e for multiple groups of comparative experiments, which greatly reduces the amount of data transmission under the premise of ensuring the instantaneity and effectiveness of data. On the basis of compression processing, an outlier detection algorithm based on isolated forest is proposed, which can accurately identify the anomaly caused by gradual change and sudden change and control and adjust the action of equipment, in order to meet the control requirement. As is shown by experimental simulation, the isolated forest algorithm based on partition outperforms box graph and K-means clustering algorithm based on distance in anomaly detection, which verifies the feasibility and advantages of the former in data compression and detection accuracy.

Mining Educational Data to Analyze Students’ Performance: A Case Study of Mawuli School, Ho

Data mining is the extraction of prospective valuable information from large chunk of data through the employment of many different data mining techniques. The usefulness of data mining coupled with the huge data generated in scholastic settings has made it an interesting field of research known as Educational Data Mining (EDM). The intent of EDM is to derive understanding from hidden patterns in data collected from institutions of learning to aid in identifying issues that influence students’ scholastic accomplishment, to solution of which will lead to improvement in accomplishment. Because scholastic achievement is dependent on several issues, it is essential to develop predictive models on students’ academic performance. This study’s objective, therefore, is to acquire an insight into performance through knowledge discovery by the use of simple linear regression in order to build a predictive model capable of predicting students’ grades to give a general overview of students’ performance in the WASSCE, and help improve students’ performance. In accomplishing the set objective for which the study was carried out, that is to predict the possible outcome of students in WASSCE, a widely sampled study was applied in undertaking the study (Quantitative research).

Intelligent Traffic Management System

Urbanization has presented opportunities of progress which has attracted people from rural areas to the cities thus leading to mass migration. This migration has been going on for decades all around the globe and has reached a point of saturation. The area of the city remains the same but the population density has increased multiple times. Commuting for work is a scene of chaos on the roads. Though there are modes of public transport, roadway is the major mode of commute and the load on roadways is ever increasing due to the rise in population. There is hardly any scope to expand the area of the roadways. The rise in the number of vehicles each year has saturated the capacity the roads were built to carry. This leads to congestion and long hours of traffic on a daily basis which tests the patience of citizens. This provokes the daily commuters to violate the traffic rules which may sometimes amount to grave accidents. Even on Highways, the empty roads entice drivers to experience the thrill of speed overlooking the fact that they are putting themselves at risk. There have been regulations imposed to reduce the chance of an accidents by implementing rules and levying heavy fines on traffic violations. Traffic cameras have been installed all around the city to monitor for traffic violations and get hold of violators. With the technological advancements to store and process large chunks of data efficiently using techniques like Deep Learning and Computer Vision, this paper proposes an automated system to detect Traffic Violations using YOLOv3 to detect and track vehicles and save a snapshot in case a violation is committed.

Determination of Data Mining Application Design Patterns Booking Raw Food In Restaurant Fountain With Apriori Algorithm

Booking is an activity carried out by certain parties to ensure availability, in carrying out certain activities the company has a supply of material in quantities that exceed the needs. As a result, in the warehouse there is a buildup of raw materials or it can happen otherwise. Inventories of materials that are too small can hinder the company's operations in the form of unavailability of materials when needed. The role of inventory will determine the operation of the company because the inventory will run well if supported by good management. Therefore, the concept of inventory management that affects ordering is very important to be applied by companies so that the goals of effectiveness and efficiency are achieved. So we need a Data Mining that can quickly to determine the Determination of Food Raw Material Ordering Patterns in Restaurant Fountain Using Apriori. Data Mining is the extraction of new information taken from large chunks of data that helps in making decisions. One of the applications of data mining for Determining the Pattern of Ordering Food Raw Materials in Restaurant Fountain Using Apriori. Apriori method is a method for determining frequent itemsets for boolean association rules. The research aims to build the application of Determining the Pattern of Ordering Food Raw Materials in Restaurant Fountain with a web-based application and as a tool for designing applications using the Mysql Database. This data mining is able to determine the ordering of food items in the Restaurant Fountain with the required amount.

Data Science: The Impact of Machine Learning

In this paper, I prove my promise that Machine Learning is one of the most important parts to provide tools and methods to go deeper and nurture the data properly. The most amazing part is to analyze the large chunks of data in a very precise way, and high-value predictions that can guide better decisions and smart actions in real-time without human intervention. I give an overview over different proposed structures of Data Science and mention the impact of Machine learning such as algorithms, model evaluation and selection, pipeline. I also indicate all misconceptions when neglecting Machine learning reasoning.

Data Science: The Impact of Machine Learning

In this paper, I prove my promise that Machine Learning is one of the most important parts to provide tools and methods to go deeper and nurture the data properly. The most amazing part is to analyze the large chunks of data in a very precise way, and high-value predictions that can guide better decisions and smart actions in real-time without human intervention. I give an overview over different proposed structures of Data Science and mention the impact of Machine learning such as algorithms, model evaluation and selection, pipeline. I also indicate all misconceptions when neglecting Machine learning reasoning.

Applying Dempster–Shafer theory for developing a flexible, accurate and interpretable classifier

Two approaches have traditionally been identified for developing artificial intelligence systems supporting decision-making: Machine Learning, which applies general techniques based on statistical analysis and optimization methods to extract information from a large amount of data looking for possible relations among them, and Expert Systems, which codify experts knowledge in rules, which are then applied to a specific situation. One of the main advantages of the first approach is its greater accuracy and wider generality for the application of the methods developed which can be used in various scenarios. By contrast, expert systems are usually more restricted and often applicable only to the domain for which they were originally developed. However, the machine learning approach requires the availability of large chunks of data, and it is much more complicated to interpret the results of the statistical methods to obtain some explanation of why the system decides, classifies, or evaluates a situation in a certain way. This issue may become very important in areas such as medicine, where it is relevant to know why the system recommends a certain treatment or diagnoses a certain illness. Likewise, in the financial sector, it might be legally required to explain that a decision to reject the granting of a mortgage loan to a person is not due to discriminatory causes such as gender or race. In order to be able to have interpretability and extract knowledge of available data we developed a classification method based on Dempster-Shafer’s Plausibility Theory. Mass assignment functions (MAF) must be established to apply this theory and they assign a weight or probability to all subsets of the possible outcomes, given the presence of a certain fact on a decision scenario. Thus MAF assignments encode expert knowledge. The method learns optimal values for the weights of each MAF using the Gradient Descent method. The presented method allows combination of MAF which have been generated by the method itself or defined by an expert with those that are derived from a set of available data. The developed method was first applied to controlled scenarios and traditional data sets to ensure that classifications and explanations are correct. Results show that the model can classify with an accuracy which is comparable to other statistical classification methods, being also able to extract the most important decision rules from the data.

Knowledge management: A breakthrough for preserving implicit knowledge in libraries and information centres

Today's global economy is in the transition towards a knowledge economy. Knowledge is rightly considered as the basic form of capital. Technology and knowledge in compliance are considered as the largest factors for productivity. There has been a rapid increase in the amount of information being produced and mostly standard organizations publish a large chunk of data and information on various web channels. The advancement and growing pace of information environment requires more advanced focus on the content of the documents for its organization and retrieval. This article reflects upon the importance of knowledge management today for individuals, for communities of practice, and for organizations with emerging knowledge management roles and responsibilities needed to ensure successful knowledge management implementations. Moreover, the article discusses key steps for knowledge creation and the drivers towards development of taxonomy.

Improving classification and clustering techniques using GPUs

SummaryClassification and clustering techniques are used in different applications. Large‐scale big data applications such as social networks analysis applications need to process large data chunks in a short time. Classification and clustering tasks in such applications consume a lot of processing time. Improving the performance of classification and clustering algorithms enhances the performance of applications that use such type of algorithms. This paper introduces an approach for exploiting the graphics processing unit (GPU) platform to improve the performance of classification and clustering algorithms. The proposed approach uses two GPUs implementations, which are the pure GPU or GPU‐only implementation and the GPU‐CPU hybrid implementation. The results show that the hybrid implementation, which optimizes the subtask scheduling for both the CPU and the GPU processing elements, outperforms the approach that uses only the GPU.

Intelligent Traffic Management System

Urbanization has presented opportunities of progress which has attracted people from rural areas to the cities thus leading to mass migration. This migration has been going on for decades all around the globe and has reached a point of saturation. The area of the city remains the same but the population density has increased multiple times. Commuting for work is a scene of chaos on the roads. Though there are modes of public transport, roadway is the major mode of commute and the load on roadways is ever increasing due to the rise in population. There is hardly any scope to expand the area of the roadways. The rise in the number of vehicles each year has saturated the capacity the roads were built to carry. This leads to congestion and long hours of traffic on a daily basis which tests the patience of citizens. This provokes the daily commuters to violate the traffic rules which may sometimes amount to grave accidents. Even on Highways, the empty roads entice drivers to experience the thrill of speed overlooking the fact that they are putting themselves at risk. There have been regulations imposed to reduce the chance of an accidents by implementing rules and levying heavy fines on traffic violations. Traffic cameras have been installed all around the city to monitor for traffic violations and get hold of violators. With the technological advancements to store and process large chunks of data efficiently using techniques like Deep Learning and Computer Vision, this paper proposes an automated system to detect Traffic Violations using YOLOv3 to detect and track vehicles and save a snapshot in case a violation is committed.

CASE STUDY OF TRADITIONAL RDBMS AND NOSQL DATABASE SYSTEM

On the basis of reliability, rational models are useful but not in terms of systems which involve huge amount of data; in such cases, non-relational models are much more useful. To store large chunks of data, NoSQL databases are used. NoSQL databases are scalable and wide ranged because they are non-relationally distributed.
 In relational databases, it was not possible to manage data which involved very large number of Big Data applications hence the concept of NoSQL database was introduced.
 There are a lot of advantages of NoSQL which not only involve its own features but also some features of relational database management system. The severe benefit of NoSQL database is that it is an open source system which helps to adapt many numbers of features for newly generated applications.
 This paper is focused on understanding the concepts of non-relational database system architecture with relational database system architecture and figure out the advantages and disadvantages of both simultaneously.