MapReduce Technique Research Articles

Intelligent Healthcare Management: Advancing Healthcare with Integrated AI and ML Solutions

The proposed system, leveraging the power of big data, telecommunication technologies, and wearable sensors, presents a unique opportunity to transform the healthcare industry. It fosters a seamless connection between patients, wearable sensors, caregivers, and providers through the innovative use of Information and Communication Technology (ICT) and software. This is of utmost importance in developing countries, where the healthcare sector grapples with economic challenges amplified by a burgeoning population and a surging demand for quality care, particularly for the elderly. The urgency for new technologies and tools to bolster the healthcare of senior citizens has reached a tipping point. Breakthroughs in wireless technology, miniaturization, computing power, and processing have paved the way for the development of interconnected medical devices. This proposal introduces a cutting-edge healthcare monitoring system for tracking the activities of elderly individuals, harnessing the power of the Hadoop MapReduce technique for the parallel processing of large datasets. Data from wearable sensors placed on the subjects' left ankle, right arm, and chest are transmitted to a cloud platform and a data analytics layer via Internet of Medical Things (IoMT) devices. The data is then divided into small chunks and processed as Map tasks.

A Deep Learning Modified Neural Network(DLMNN) based proficient sentiment analysis technique on Twitter data

ABSTRACT The rapid enhancement in social media over the internet generates massive information in real-time scenarios, which has a striking impact on big data analysis. It resulted in the elevated usage of emotions and sentiments in social media. This paper proffers a proficient sentiment analysis technique in Twitter data. The Twitter database is preprocessed includes, stemming, tokenisation, number removal and stop word removal, etc. The preprocessed words are then passed into the HDFS (Hadoop Distributed File System) to reduce the repeated words and are eliminated using the MapReduce technique. The emoticons and the non-emoticons are extorted as features. The resulted features are ranked with their intended meaning. Then, the classification is performed utilising the DLMNN (Deep Learning Modified Neural Network). The experimental results were examined by using the output parameter such as Accuracy, Recall, Precision, F-Score and Execution Time with other conventional techniques such as ANN, SVM, K-Means and DCNN to show the greatest outcome of the proposed model. Evaluation result shows that DLMNN achieved the greatest performance in terms of precision (95.78%), Recall (95.84%), F-Score (95.87%) and Accuracy (91.65%) when compared with the existing models.

Massively parallelizable approach for evaluating signalized arterial performance using probe-based data

Effective performance of arterial corridors is essential to community safety and vitality. Considering the dynamic nature of traffic demand, efficient management of these corridors require frequent updating of the traffic signal timings through various strategies. Agency resources for these activities are commonly scarce and are primarily by public complaints. This study provides a workflow using probe-based data to measure and compare different segments on arterial corridors in terms of the traffic signal performance measures that can capture travel time dynamics across signalized intersections. The proposed methodology identifies a group of dynamic days followed by evaluation of travel rate based upon remaining non-dynamic days. Dynamic days represent the variability of traffic on a segment. Consequently, a corridor having high number of dynamic segments along with poor performance during normal days would be a candidate for adaptive control. Further, to handle the large-scale data source collected from city-wide or statewide traffic signals, the study adopts parallel computation-based strategy using MapReduce technique. A case study was conducted on 11 corridors within Des Moines, Iowa, to demonstrate the efficacy of the proposed approach, which identified two arterial corridors, Merle Hay Road and University Avenue, to be suitable for adaptive traffic signal control.

Application of Map Reducing Techniques in Online Food Marketing: An Algorithmic Approach

Food industry is dynamic industry as it depends highly on consumer interest and timing. Big Data Analytics can bring considerable revolution in food industry. It can be used to improve operational efficiency, timely delivery and also it can be used to personalize customer experience. In the field of marketing, Big Data Analytics helps in deciding pricing, demand forecasting as well as product promotion. The objective here is to show how Map Reduce technique can be used to process and analyze Big Data to extract useful information. This paper gives insight into use of Big Data Analytics in food marketing by analyzing huge amount of data using Big Data processing tool Apache Hadoop.

Frequent Itemset Mining Based on Development of FP-growth Algorithm and Use MapReduce Technique

The Finding of frequent itemset in big data is an important task in data mining and knowledgediscovery. The exponential daily growth of data, called “Big Data”, mining frequent patterns from the hugevolumes of data has many challenges due to memory requirement, multiple data dimensions, heterogeneityof data and so on. The complexities related to mining frequent item-sets from a Big Data can be minimizedby using Modified FP-growth algorithm and parallelizing the mining task with Map Reduce framework inHadoop. In this paper, a modified FP-growth based on directed graph with Hadoop framework will reducethe execution time for the massive database and works efficiently on number of nodes (computers). Thealgorithm was tested, our experimental results demonstrated that the proposed algorithm could scale welland efficiently process large datasets. In addition, it achieves improvement in memory consumption to storefrequent patterns and time complexity.

An Enhanced Bankruptcy Prediction Model Using Fuzzy Clustering Model and Random Forest Algorithm

The rapid advancements made in Information Technologies (IT) have evolved in the prediction model for financial data. Research in the prediction of bankruptcy is inclining to owe to the growth of related associations with economic and social phenomena. Financial crises in recent scenarios have influenced the growth of financial institutions. Hence, the need for bankruptcy risk prediction at an earlier stage is of prime importance. Though several prediction algorithms were suggested, the predictive models' accuracy is still a challenging task. In this paper, a bankruptcy prediction model is developed by integrating the Fuzzy clustering model and Multi-objective random forest classifiers. The voluminous number of records of the financial dataset and polish bankruptcy dataset is collected from a public repository. It is pre-processed using a MapReduce technique, one of the Big Data approaches. Benefits given by big data approaches help to achieve better flexibility towards a variable declaration. The collected records are pre-processed and organized under efficient index construction. FCM is employed to cluster the data for analytic purposes. Finally, a multi-objective Random Forest classifier helps to develop a prediction model for bankruptcy. Experimental analysis is carried out with accuracy, precision, sensitivity, and specificity compared with existing, Genetic algorithms. Compared to the existing technique, the proposed technique has obtained 80% accuracy.

Communities Detection for Advertising by Futuristic Greedy Method with Clustering Approach

Community detection in social networks is one of the advertising methods in electronic marketing. One of the approaches to find communities in large social networks is to use greedy methods, because these methods perform very fast. Greedy methods are generally designed based on local decisions; thus, inappropriate local decisions may result in an improper global solution. The use of a greedy improved index with a futuristic approach can, to some extent, prevent inappropriate local choices. Our proposed method determines the influential nodes in the social network based on the followers and following and new futuristic greedy index. It classifies the nodes based on the influential nodes by the density-based clustering algorithm with a new distance function. The proposed method can improve clustering precision to detect communities by the futuristic greedy approach. We implemented the proposed algorithm with the map-reduce technique in the Hadoop structure. Experimental results in datasets show that the average of the rand index of clusters was accomplished by 99.32% in the proposed method. In addition, these results illustrate that there is a reduction in execution time by the proposed algorithm.

An Efficient DDoS Attack Detecting System using Levenberg-Marquardt Based Deep Artificial Neural Network Approach for IOT

The Internet of Things model envisions the widespread interconnection and collaboration of smart devices over the present and future Internet environment. Threats and attacks against IoT devices and services are on the rise due to their rapid development. Distributed-Denial-of-Service (DDoS) attacks are one of the main dangerous malwares that attack targeted organizations through infected devices. Many mechanisms are developed for IoT devices in order to detect DDoS attacks. Nonetheless, the prevailing DDoS Attack Detection (DAD) methods involve time-delay and a lower detection rate. This paper proposed an efficient approach using the Levenberg-Marquardt Neural Network (LMDANN) algorithm for detecting the DDoS attacks in order to enhance prediction accuracy. In the proposed system, a MapReduce technique is used to eliminate the redundant copies. In addition, the Entropy-based Fisher’s Discriminate Function (ENTFDF) method was developed to reduce the features from the extracted features, and the system suggests an LMDANN algorithm to classify DDoS attack data separately from the normal data. In this, 80% of the data is used for training, and 20% of the data is used for testing. The performance of the proposed LMDANN method was evaluated in contrast to other art of state algorithms (ANN, SVM, KNN, and ANFIS) in terms of some specific qualitative performance metrics (recall, sensitivity, f-measure, specificity, precision, accuracy, and training time). The results show that the proposed detection approach can efficiently detect the DDoS attack in the IoT environment, achieving 96.35% accuracy.

An intelligent surveillance video analytics framework using NACT-Hadoop/MapReduce on cloud services

Video analytics has gradually increased in recent years. The intelligent CCTV cameras in public places, you-tube videos, etc. generate an enormous amount of video data. Generally, video analytics required more time as it contains several processes like encoding, decoding, etc. There are several existing approaches are evolved in improving the efficiency of video analytics but performance delay and loss of data still existing challenges. With our analysis, we strongly state VM migration will be an effective solution to overcome this delay and performance issues. In this paper, we propose NACT based map reducing mechanism (NACT-Map) for processing the real-time streaming videos. The NACT (Novel Awaiting Computation Time) enables the prediction of VM allocation and automatic migration. The scheduling and allocating of the optimal resource are done by task monitor who utilizes the Task manager (TM) system. The NACT based VM migration and MapReduce technique with Hadoop simplifies the process and minimizes the execution time. The splitting of video into chunks of frames speedup the process. Further efficiency is improved by the Map Reduce technique which uses video and its related content for clusters. The performance of our proposed system is executed in the cloudsim with a large dataset contains two real-time videos. Further, the result is compared with the existing methodologies such as distributed video decoding mechanism with extended FFmpeg and VideoRecordReader (VDMFF) (Yoon et al. in Distributed video decoding on Hadoop. IEICE Trans Inf Syst E101-D(1):2933–2941, 2018) and distributed Video Analytics Framework for Intelligent Video Surveillance (SIAT) (Uddin et al. in SIAT: a distributed video analytics framework for intelligent video surveillance. Symmetry 11:911, 2019). The obtained result shows our proposed NACT_Map consumes minimum Task processing time $$({\text{p}}_{{{\text{tix}}}} )$$ and about 90% of efficiency in overall system performance is increased.

Prefetched wald adaptive boost classification based Czekanowski similarity MapReduce for user query processing with bigdata

With large volumes of data being generated in recent years and the inception of big data analytics on social media necessitates accurate user query processing with minimum time complexity. Several research works have been conducted in this area, to address accuracy and time complexity involved in query processing, in this work, Wald Adaptive Prefetched Boosting Classification based Czekanowski Similarity MapReduce (WAPBC–CSMR) technique is introduced. The WAPBC–CSMR technique uses the big dataset for processing large number of user queries. First, a technique called, Wald Adaptive Prefetched Boosting is employed with the objective of classifying the big dataset into different classes. To reduce the time involved in classification, in this paper a classifier called Gaussian distributive Rocchio is used that achieves significant classification in minimum time. With the classified results, a Likelihood Radio Test is applied to integrate the weak learner results into strong classification results. Then the classified and refined data are stored on the prefetcher cache. Upon reception of multi-dimensional user queries by the prefetch manager, the queries are now split into multiple keywords and are fed into the map phase, where mapping function is performed using Czekanowski Similarity Index with the objective of identifying the repeated jobs with maximum query processing accuracy. Followed by which the relevant data are retrieved from the prefetcher cache and repeated user query task is removed in the reduce phase via statistical function, therefore contributing to minimum time. Result analysis of WAPBC–CSMR is performed with big dataset using different metrics such as query processing accuracy, error rate and processing time for varied number of user queries. The result shows that WAPBC–CSMR technique enhances query processing accuracy and lessens the time as well as the error rate than the conventional methods.

Big Data Analytics in Online Structural Health Monitoring

This manuscript explores the application of big data analytics in online structural health monitoring. As smart sensor technology is making progress and low cost online monitoring is increasingly possible, large quantities of highly heterogeneous data can be acquired during the monitoring, thus exceeding the capacity of traditional data analytics techniques. This paper investigates big data techniques to handle the highvolume data obtained in structural health monitoring. In particular, we investigate the analysis of infrared thermal images for structural damage diagnosis. We explore the MapReduce technique to parallelize the data analytics and efficiently handle the high volume, high velocity and high variety of information. In our study, MapReduce is implemented with the Spark platform, and image processing functions such as uniform filter and Sobel filter are wrapped in the mappers. The methodology is illustrated with concrete slabs, using actual experimental data with induced damage

Predicting Heart Diseases from Large Scale IoT Data Using a Map-Reduce Paradigm

AbstractOver the last few years, the huge amount of data represented a major obstacle to data analysis. Big data implies that the volume of data undergoes a faster progress than computational speeds, thereby demanding a larger data storage capacity. The Internet of Things (IoT) is a main source of data that is closely related to big data, as the former extends to a variety of fields such as healthcare, entertainment, and disaster control. Despite the different advantages associated with the composition of Big Data analytics and IoT, there are a number of complex difficulties and issues involved that need to be resolved and managed to ensure an accurate data analysis. Some of these solutions include the utilization of map-reduce techniques, processing, and large data scale, particularly for the relatively less time that this method requires to process large data from the Internet of Things. Machine learning algorithms of this kind are often implemented in the healthcare sector. Medical facilities need to be advanced so that more appropriate decisions can be made in terms of patient diagnosis and treatment options. In this work, two datasets have been used: the first set, used in the prediction of heart diseases, obtained an accuracy rate of 84.5 for RF and 83 for J48, whereas the second dataset is related to weather stations (automated sensors) and obtained accuracy rates of 88.5 and 86.5 for RF and J48, respectively.

TVD-MRDL: traffic violation detection system using MapReduce-based deep learning for large-scale data

Maintaining a fluid and safe traffic is a major challenge for human societies because of its social and economic impacts. Various technologies have considerably paved the way for the elimination of traffic problems and have been able to effectively detect drivers’ violations. However, the high volume of the real-time data collected from surveillance cameras and traffic sensors along with the data obtained from individuals have made the use of traditional methods ineffective. Therefore, using Hadoop for processing large-scale structured and unstructured data as well as multimedia data can be of great help. In this paper, the TVD-MRDL system based on the MapReduce techniques and deep learning was employed to discover effective solutions. The Distributed Deep Learning System was implemented to analyze traffic big data and to detect driver violations in Hadoop. The results indicated that more accurate monitoring automatically creates the power of deterrence and behavior change in drivers and it prevents drivers from committing unusual behaviors in society. So, if the offending driver is identified quickly after committing the violation and is punished with the appropriate punishment and dealt with decisively and without negligence, we will surely see a decrease in violations at the community level. Also, the efficiency of the TVD-MRDL performance increased by more than 75% as the number of data nodes increased.

Effective Feature Selection Using Hybrid GA-EHO for Classifying Big Data SIoT

Several novel applications and services of networking for the IoT are supported by the Social Internet of Things (SIoT) in a more productive and powerful way. SIoTs are the recent hot topics rather than other extensions of IoTs. In this research, the authors have extracted the Big Data SIoT using the well-known model named MapReduce framework. Moreover, the unwanted data and noise from the database are reduced using the Gabor filter, and the big databases are mapped and reduced using the Hadoop MapReduce (HMR) technique for improving the efficiency of the proposed GA-EHO. Furthermore, the feature selection using GA-EHO is processed on the filtered dataset. The implementation of the proposed system is done by using some machine learning classifiers for classifying the data and the efficiency is predicted for the proposed work. From the simulation results, the specificity, maximum accuracy, and sensitivity of the proposed GA-EHO are produced about 87.88%, 99.1%, and 81%. Also, the results are compared with other existing techniques.

An Implementation of Map Reduce on the Hadoop for Analyzing Big Data

The Speedy development of Internet has led to huge quantities of digital data available online and vast capacity of digital data is increasing and successfully stored. In demand to the process, analyzed, and linked huge volume of stored data to achieve correct Information, some computation is required. Even efficient processing and implementation is needed for scientific data performance analysis. We will compare with already existing MapReduce Technique with Hadoop to afford high performance and efficiency for large volume of dataset. Hadoop distributed architecture with MapReduce programming is analysis here.

Indian Premier League Dataset Analytics using Hadoop-Hive

Big Data is a term used to represent huge volume of both unstructured and structured data which cannot be processed by the traditional data processing techniques. This data is too huge, grows exponentially and doesn't fit into the structure of the traditional database systems. Analyzing Big Data is a very challenging task since it involves the processing of huge amount of data. As the industry or its business grows, the data related to the industries also tend to grow on a larger scale. Prominent data analysis tools are required to analyze the data in order to gain value out of it. Hadoop is a sought-after open source framework that uses MapReduce techniques to store and process huge datasets. However, the programs written using MapReduce techniques are not flexible and also require maintenance. This problem is overcome by making use of HiveQL. In order to execute queries in HiveQL, the platform required is Hive. It is an open-source data warehousing set-up built on Hadoop. HiveQL queries are compiled into MapReduce jobs that are executed utilizing Hadoop. In this paper we have analyzed the Indian Premier League dataset using HiveQL and compared its execution time with that of traditional SQL queries. It was found that the HiveQL provided better performance with larger dataset while SQL performed better with smaller datasets.

An Intelligent Big Data Analytics System using Enhanced Map Reduce Techniques

An Intelligent Big Data Analytics System using Enhanced Map Reduce Techniques include a set of Methods, applications and strategy which helps the organization and industry to bring together the data and information from outside sources and internal systems, as well as it is used to collect , classify, analysis and run the queries against the data and prepare the report for effective decision making. The Enhanced Map Reduced Techniques based on K-Nearest Neighbor (KNN) clustering Strategy works efficient as well as in an effective manner. We found that the existing MR – mafia sub space clustering Strategy have not performed effectively .Many clustering techniques are adopted in real world data analysis for example customer behavior analysis, medical data analysis, digital forensics, etc. The existing MR- mafia sub space clustering Strategy is inefficient because of continuously increase in the data size, and overlaying of the data blocks .The proposed KNN clustering Strategy mainly focused on the enhanced the Map Reduce techniques, and then to avoid the unnecessary input and output data, optimize the data storage in order to achieve the best out sourcing of data privacy. The proposed KNN clustering Strategy works effectively and that can be outsourced to cloud server.

Online Feature Selection (OFS) with Accelerated Bat Algorithm (ABA) and Ensemble Incremental Deep Multiple Layer Perceptron (EIDMLP) for big data streams

Feature selection is mainly used to lessen the dispensation load of data mining models. To condense the time for processing voluminous data, parallel processing is carried out with MapReduce (MR) technique. However with the existing algorithms, the performance of the classifiers needs substantial improvement. MR method, which is recommended in this research work, will perform feature selection in parallel which progresses the performance. To enhance the efficacy of the classifier, this research work proposes an innovative Online Feature Selection (OFS)–Accelerated Bat Algorithm (ABA) and a framework for applications that streams the features in advance with indefinite knowledge of the feature space. The concrete OFS-ABA method is suggested to select significant and non-superfluous feature with MapReduce (MR) framework. Finally, Ensemble Incremental Deep Multiple Layer Perceptron (EIDMLP) classifier is applied to classify the dataset samples. The outputs of homogeneous IDMLP classifiers were combined using the EIDMPL classifier. The projected feature selection method along with the classifier is evaluated expansively on three datasets of high dimensionality. In this research work, MR-OFS-ABA method has shown enhanced performance than the existing feature selection methods namely PSO, APSO and ASAMO (Accelerated Simulated Annealing and Mutation Operator). The result of the EIDMLP classifier is compared with other existing classifiers such as Naïve Bayes (NB), Hoeffding tree (HT), and Fuzzy Minimal Consistent Class Subset Coverage (FMCCSC)-KNN (K Nearest Neighbour). The methodology is applied to three datasets and results were compared with four classifiers and three state-of-the-art feature selection algorithms. The outcome of this research work has shown enhanced performance in accuracy and less processing time.

Big Data Processing and Performance Improvement for Ship Trajectory using MapReduce Technique

Big Data Processing and Performance Improvement for Ship Trajectory using MapReduce Technique

Analysis of KNN Algorithm with Mapreduce Technique on Big Data

Due the fast growth of new technology application like social media analysis, web data analysis and medical information network analysis, here the various types of data are processed frequently. The large amount of effective data management and analysis is very vital goal. To reduce the data processing complexity, time complexity, and space complexity in Big Data, the paper going to propose the k-nearest neighbor join (KNN) operation. KNN is used to find the K nearest points in S. It is a computational task that will handle the large range of applications such as knowledge discovery or data mining. When the volume and the dimension of data increases, then only distributed approaches can perform the big operations in a given time. Recent works have done on implementing the efficient solutions using the map reduce programming model because it is used for distributing the large scale data processing. Although these works provide different solutions for the same problem, each one has particular constraints and properties. This paper compares the existing of different computation of KNN on MapReduce. First the paper compares the solutions in to three steps for KNN computation on MapReduce: 1) Data processing, 2) Data partitioning and 3) Computation. The Experiment in this paper explains the variety of different data sets, and analyzes the data volume, data dimension and the value of k from many perspectives like time and space complexity, and accuracy.