Huge Data Processing Research Articles

USING ARTIFICIAL INTELLIGENCE TO OPTIMIZE GENETIC RESEARCH

The purpose of the article is to analyse the main achievements and prospects for the introduction of artificial intelligence in genetics, to improve the efficiency of research and the reliability of the results obtained. The article analyses publications that reveal the areas of integration of neural networks into DNA fingerprinting, in particular PCR-PCR analysis, sequencing, FISH diagnostics, etc., increasing the sensitivity of these technologies and reducing the cost of research. The author also discusses technologies for processing huge data sets of Big Data and their effectiveness on the example of international projects in theoretical and applied genetics. Attention is paid to artificial intelligence in genomics, proteomics and genetic engineering, in particular, optimisation of GED (genome editing) methods based on CRISPR (short palindromic repeats regularly interspaced in groups) to achieve better accuracy of genome editing. Neural networks are important in creating guide RNAs (gRNAs) for CRISPR-Cas systems, which determine the direction of endonuclease function. In addition to specialised applications and resources, artificial intelligence generally improves research work by optimising many processes that used to take a lot of time and effort. Examples of neural network-based applications that are widely used by the scientific community are presented.

The intelligent prediction and assessment of financial information risk in the cloud computing model

cloud computing (cloud computing) is a kind of distributed computing, referring to the network "cloud" will be a huge data calculation and processing program into countless small programs, and then, through the system composed of multiple servers to process and analyze these small programs to get the results and return to the user. This report explores the intersection of cloud computing and financial information processing, identifying risks and challenges faced by financial institutions in adopting cloud technology. It discusses the need for intelligent solutions to enhance data processing efficiency and accuracy while addressing security and privacy concerns. Drawing on regulatory frameworks, the report proposes policy recommendations to mitigate concentration risks associated with cloud computing in the financial industry. By combining intelligent forecasting and evaluation technologies with cloud computing models, the study aims to provide effective solutions for financial data processing and management, facilitating the industry's transition towards digital transformation.

ARPMEC: an adaptive mobile edge computing-based routing protocol for IoT networks

The Internet of Things (IoT) networks comes with many challenges, especially in network architecture designs. IoT is populated by several kinds of devices with different characteristics that are autonomously managed. These devices do not have enough resources and they require to process data in real-time. Hence, there is a need to design suitable architectures for IoT networks that are as efficient as possible. Previously, Cloud Computing (CC) seemed to provide a good solution of processing data from IoT networks. Recently, Mobile Edge Computing (MEC) seems to be offering a better solution than CC by ensuring a better Quality of Services (QoS) provisioning. As a result, many MEC solutions have emerged for QoS improvement in IoT networks. These solutions mainly focus on device resource management without considering data routing from an end-user device to another, especially when the latter are mobile and need to communicate with each other. In this paper, we propose to design an adaptive routing protocol for a MEC-based network to manage efficiently, the end-user devices’ energy consumption during data routing. The proposed adaptive Mobile Edge Computing-based protocol consists of two main phases: firstly, we subdivide the network’s objects into clusters by exploiting a link quality prediction algorithm. Secondly, we route the data to their destination adaptively by considering the object’s movement during the routing process. As presented in the simulation results, our protocol outperforms other existing routing protocols for IoT networks in terms of energy consumption. We then propose the use of our solution for data routing in IoT networks that require huge data processing and forwarding.

Alternative approach to design Dibit-based XOR and XNOR gate

In this generation, high-speed communication has very demanding. In this respect, optical communication plays a crucial role in meeting the goal of high-speed communication. With the increasing demands of high-speed communication, huge data processing is also needed. Therefore, we have proposed a design of XOR and XNOR gates using five reflective semiconductor optical amplifiers (RSOA). Our proposed gates are dibit logic-based. To increase the reliability of the devise, we have incorporated this logic scheme. Here, we consider the logic state ‘0’ for the absence of pulse and logic state ‘1’ for the presence of pulse. The dibit logic ‘0 1’ and ‘1 0’ are similar as ‘0’ and ‘1’ in digtal states, respectively. To check its practical feasibility, we have simulated the proposed design in Matlabsoftware and also quality factor, contrast, and extinction ratios are calculated for this design.

Large-Scale Data Processing and Machine Learning Analysis Model Based on Distributed Algorithm

The model of large-scale data processing and ML(machine learning) analysis based on DA(distributed algorithm) is a powerful computing method, which aims at processing huge data sets and performing efficient ML analysis. In this paper, a cluster topology driver module based on gradient switching and aggregate communication is designed, and its core goal is to adapt the distributed system to various underlying network topologies. By designing decentralized gradient exchange algorithm and aggregate communication framework, the parallel transmission ability of multi-interface network can be fully exerted, thus improving the model synchronization efficiency of ML task. The experimental results show that the cluster topology driver module can provide better performance than the existing methods in terms of training convergence, cluster scalability and communication overhead. Large-scale data processing and ML analysis model based on DA is widely used in processing massive data and realizing complex analysis tasks.

Building Fake Review Detection Model Based on Sentiment Intensity and PU Learning.

Fake review detection has the characteristics of huge stream data processing scale, unlimited data increment, dynamic change, and so on. However, the existing fake review detection methods mainly target limited and static review data. In addition, deceptive fake reviews have always been a difficult point in fake review detection due to their hidden and diverse characteristics. To solve the above problems, this article proposes a fake review detection model based on sentiment intensity and PU learning (SIPUL), which can continuously learn the prediction model from the constantly arriving streaming data. First, when the streaming data arrive, the sentiment intensity is introduced to divide the reviews into different subsets (i.e., strong sentiment set and weak sentiment set). Then, the initial positive and negative samples are extracted from the subset using the marking mechanism of selection completely at random (SCAR) and Spy technology. Second, building a semi-supervised positive-unlabeled (PU) learning detector based on the initial sample to detect fake reviews in the data stream iteratively. According to the detection results, the data of initial samples and the PU learning detector are continuously updated. Finally, the old data are continually deleted according to the historical record points, so that the training sample data are within a manageable size and prevent overfitting. Experimental results show that the model can effectively detect fake reviews, especially deceptive reviews.

Big Data and Deep Learning Analytics

There has been an enormous growth of the Internet, mobile phone, medical facilities, and many more in the 21st century, which can also be known as the beginning of the knowledge era. Knowledge is defined not for what it is, but for what it can do. In this fast-moving technological era, as a result, a huge amount of data is generated in different regions of the world and it is growing day by day, this growing data is known as “Big Data”. To extract useful information (analyze) from large unstructured data (like Web, sales, customer contact center, social media, mobile data, and so on) is a complex task, as data being generated is a combination of structured, semi-structured and unstructured data. Traditional systems are not capable to handle semi-structured or unstructured data generated whose volume could range in petabytes or exabytes, as the major challenges are limited memory usage, computational hurdles and slower response time, data redundancy, etc. This problem can be overcome with big data analytics having technologies like Apache Hadoop, Apache Spark, Hive, Pig, etc. which can extract useful information from these large data. Authors are going to explore more on them in these chapters. Alongside authors will explore “Deep Learning” also known as “Deep Neural Learning” or “Deep Neural Network”, which is a class of Machine Learning that progressively extract higher-level features from raw data automatically. It performs 'end-to- end learning' and uses layers of algorithms to process data, understand human speech, and visually recognize objects, which is an important part of it. Feature extraction, self-driving cars, fraud detection, healthcare, neural language processing, etc. are some of the areas where it is applied in daily life. Algorithms like RNN, CNN, FNN, Backpropagation, etc, are some of the algorithms used in deep learning. The authors will explore how Machine learning is different from deep learning. Deep learning (DL) is also associated with data science in many ways as the DL algorithms work better than older learning algorithms for prediction or feature extraction etc. Which has brought it, more closer towards one of its main objectives i.e., artificial intelligence (AI)? Hence it is immensely advantageous to the data scientists who aim for making predictions and draw useful information to analyze and interpret it for helping the organization in its growth. The processing of Big Data and the evolution of Artificial Intelligence are both dependent on Deep Learning. Deep learning technology came up along with big data analytics. The concept of deep learning is supportive in the big data analytics due to its efficient use for processing huge and enormous data. This chapter explains about deep learning and big data analytics use in healthcare and alongside authors will study about algorithms used in deep learning and technologies used in big data analytics with its architecture. After reading this chapter, authors must be able to connect deep learning with big data analytics for building new products and contribute to society in a much better way

A Novel Data Management Scheme in Cloud for Micromachines

In cyber-physical systems (CPS), micromachines are typically deployed across a wide range of applications, including smart industry, smart healthcare, and smart cities. Providing on-premises resources for the storage and processing of huge data collected by such CPS applications is crucial. The cloud provides scalable storage and computation resources, typically through a cluster of virtual machines (VMs) with big data tools such as Hadoop MapReduce. In such a distributed environment, job latency and makespan are highly affected by excessive non-local executions due to various heterogeneities (hardware, VM, performance, and workload level). Existing approaches handle one or more of these heterogeneities; however, they do not account for the varying performance of storage disks. In this paper, we propose a prediction-based method for placing data blocks in virtual clusters to minimize the number of non-local executions. This is accomplished by applying a linear regression algorithm to determine the performance of disk storage on each physical machine hosting a virtual cluster. This allows us to place data blocks and execute map tasks where the data blocks are located. Furthermore, map tasks are scheduled based on VM performance to reduce job latency and makespan. We simulated our ideas and compared them with the existing schedulers in the Hadoop framework. The results show that the proposed method improves MapReduce performance in terms of job latency and makespan by minimizing non-local executions compared to other methods taken for evaluation.

Memory aware optimized Hadoop MapReduce model in cloud computing environment

<p>In the last decade, data analysis has become one of the popular tasks due to enormous growth in data every minute through different applications and instruments. MapReduce is the most popular programming model for data processing. Hadoop constitutes two basic models i.e., Hadoop file system (HDFS) and MapReduce, Hadoop is used for processing a huge amount of data whereas MapReduce is used for data processing. Hadoop MapReduce is one of the best platforms for processing huge data in an efficient manner such as processing web logs data. However, existing model This research work proposes memory aware optimized Hadoop MapReduce (MA-OHMR). MA-OHMR is developed considering memory as the constraint and prioritizes memory allocation and revocation in mapping, shuffling, and reducing, this further enhances the job of mapping and reducing. Optimal memory management and I/O operation are carried out to use the resource inefficiently manner. The model utilizes the global memory management to avoid garbage collection and MA-OHMR is optimized on the makespan front to reduce the same. MA-OHMR is evaluated considering two datasets i.e., simple workload of Wikipedia dataset and complex workload of sensor dataset considering makespan and cost as an evaluation parameter.</p>

Dimension Reduction and Redundancy Removal through Successive Schmidt Decompositions

Quantum computers are believed to have the ability to process huge data sizes, which can be seen in machine learning applications. In these applications, the data, in general, are classical. Therefore, to process them on a quantum computer, there is a need for efficient methods that can be used to map classical data on quantum states in a concise manner. On the other hand, to verify the results of quantum computers and study quantum algorithms, we need to be able to approximate quantum operations into forms that are easier to simulate on classical computers with some errors. Motivated by these needs, in this paper, we study the approximation of matrices and vectors by using their tensor products obtained through successive Schmidt decompositions. We show that data with distributions such as uniform, Poisson, exponential, or similar to these distributions can be approximated by using only a few terms, which can be easily mapped onto quantum circuits. The examples include random data with different distributions, the Gram matrices of iris flower, handwritten digits, 20newsgroup, and labeled faces in the wild. Similarly, some quantum operations, such as quantum Fourier transform and variational quantum circuits with a small depth, may also be approximated with a few terms that are easier to simulate on classical computers. Furthermore, we show how the method can be used to simplify quantum Hamiltonians: In particular, we show the application to randomly generated transverse field Ising model Hamiltonians. The reduced Hamiltonians can be mapped into quantum circuits easily and, therefore, can be simulated more efficiently.

Application of the cloud pyramid model and the SaaS method to increase the level of big data security

The digital revolution has changed the way, that business is conducted in many areas. Recently, large volumes of various types of data are used at high speed by a variety of sources, that are processed, often in real time, using information technology tools (powerful processors, software and algorithms. Today’s bank servers contain vast amounts of data collected from online interactions. People in modern times willingly share information about themselves. Traditional banking analytics systems aren’t capable of processing huge data sets from various sources. However, the development of digital technologies and big data analytics allow bank clients to structure and use their personal information to solve existing problems of saving bank data. It was determined, that every client of the bank is worried about the preservation of his personal information, which is provided to the bank managers. The essence of the concept of cloud computing, cloud infrastructure is defined, and the purpose of using cloud interfaces is also considered. Levels of problems, related to the security of saving bank customer data, have been revealed. The levels of opportunities for using the cloud pyramid to increase the level of security of big data in banking activities have been determined. The cloud enables the operation of an ideal platform with massive computing power and storage capacity to process big data with great variety, volume, veracity and speed. The SaaS provider takes care of the performance of applications, provides technical support for users, and installs updates independently. The type of services and the type of organizations, to which Software as a Service (SaaS) technologies can be applied, are considered. The role of Software-as-a-service (SaaS) technologies to increase the level of big data security in banking is defined.

A Survey of Image Classification Algorithms based on Convolution Neural Network

With the deep learning (DL) sweeping the world. Traditional image classification methods are difficult to process huge image data and cannot meet people's requirements for image classification accuracy and speed. The image classification method based on convolutional neural network (CNN) breaks through the bottle neck of traditional image classification methods and becomes the mainstream algorithm of image classification at present, how to effectively use convolutional neural network to classify images has become a hot research topic in the field of computer vision at home and abroad. Convolutional neural network (CNN) has performed well in image classification and segmentation, target detection and other applications, and its powerful feature learning and feature expression capabilities are increasingly respected by researchers. However, CNN still has a few problems, such as incomplete feature extraction and overfitting of sample training. In view of these problems, after in-depth research on the application of convolutional neural network in image processing, this paper gives the mainstream structure model, advantages and disadvantages, time/space complexity, problems that may be encountered in the model training process and corresponding solutions used in image classification based on convolutional neural network. Through the overview of the research status of CNN model in image classification, it provides suggestions for the further development and research direction of CNN.

Evaluation of Open-Source Tools for Big Data Processing

Every day, large terabytes of data repository are being generated which comes mostly from modern information systems, new technologies, Internet of Things (IoT) and cloud computing. With the ever-expanding number of alternatives, the choice of picking machine learning tools for big data to analyse such volume of massive data can be difficult and so necessitates exertions at various stages to excerpt information meant for decision making. As big data analysis is currently the latest researchable area of interest, this paper therefore intended to aid researchers understand machine learning and focus on exploring the impact of open-source tools for the processing of big data. Machine learning was used to analyse three open-source tools of Hadoop, Spark and Presto These open-source tools were evaluated by considering scalability, fault tolerant and latency as the metrics. While Presto as a tool for big data analytic was discovered to be efficient and fast in processing huge data, spark plays greater role in precision and Hadoop was found to be the best in fault tolerance. In conclusion, the paper furnishes the platform with various steps to explore big data that could open latest sphere of research development.

Computational Study of DC Glow Discharge and the Effect of Voltage on the Parameters of the Resulting Plasma

Recently, the computational procedure has become the best and low-cost tool for various studies, both theoretically and practically, due to its ability to process huge data in a short time. In this research, a computer study was conducted in two dimensionsof the DC glow discharge, where a noble reactive gas, argon gas was used to study the effect of voltage on the parameters of the resulting plasma. The particle flow was approximated using the diffusion approximation based on the fluid model, excited and neutral types where it was found that the density of electrons, the temperature of the electrons, the electric potential, and the density of ions will increase with the increase ofthe applied voltage.

A Cost-Optimized Data Parallel Task Scheduling in Multi-Core Resources Under Deadline and Budget Constraints

Large-scale distributed systems have advantages of high processing speeds and large communication bandwidths over the network. The processing of huge real-world data through distributed computing system becomes obscure because the major concern in large-scale distributed systems is to guarantee the completion of data processing task to be done within a budget and time constraints. This paper proposes a cost-optimized data parallel task scheduling in multi-core resources to address the above issue. By running concurrent executions on a multi-core resource, the number of parallel executions could be increased correspondingly, thereby it is able to finish the task within the deadline. A model is developed here to optimize the operational cost of data parallel task by feasibly assigning load fractions to each multi-core resource. This work experimented with data parallel task. The outcome of the work gives better solutions in terms of processing task by deadline at optimised computational cost.

Privacy protection federated learning system based on blockchain and edge computing in mobile crowdsourcing

With the rapid popularization and development of the Internet of Things (IoT) and 5G networks, mobile crowdsourcing (MCS) has become an indispensable part in today’s society. However, when task participants submit tasks, they are likely to expose their data privacy and location privacy. These privacy will be maliciously attacked and exploited by attackers (external attackers and untrusted third party). With the rapid increase of MCS data throughput, traditional cloud platforms can no longer meet the huge data processing needs. To solve these problems, this paper proposes an MCS federated learning system based on Blockchain and edge computing. This paper uses federated learning as the framework of the MCS system. The system protects data privacy and location privacy by using the Double local disturbance Localized Differential Privacy (DLD-LDP) proposed in this paper. Because the sensed data exists in multiple modalities (text, video, audio, etc.), this paper uses the Multi-modal Transformer (MulT) method to merge the multi-modal data before subsequent operations. To solve the problem that the third party is untrusted, we utilize Blockchain to distribute tasks and collect models in a distributed way. A reputation calculation method (Sig-RCU) is proposed to calculate the real-time reputation of task participants. Through conducting experiments on real data sets, the effectiveness and adaptation of the proposed DLD-LDP algorithm and Sig-RCU algorithm are verified.

A Cost-Optimized Data Parallel Task Scheduling with Deadline Constraints in Cloud

Large-scale distributed systems have the advantages of high processing speeds and large communication bandwidths over the network. The processing of huge real-world data within a time constraint becomes tricky, due to the complexity of data parallel task scheduling in a time constrained environment. This paper proposes data parallel task scheduling in cloud to address the minimization of cost and time constraints. By running concurrent executions of tasks on multi-core cloud resources, the number of parallel executions could be increased correspondingly, thereby, finishing the task within the deadline is possible. A mathematical model is developed here to minimize the operational cost of data parallel tasks by feasibly assigning a load to each virtual machine in the cloud data center. This work experiments with a machine learning model that is replicated on the multi-core cloud heterogeneous resources to execute different input data concurrently to accomplish distributive learning. The outcome of concurrent execution of data-intensive tasks on different parts of the input dataset gives better solutions in terms of processing the task by the deadline at optimized cost.

Implementing a Hybrid Recommendation System to Personalize Customer Experience in E-Commerce Domain

With the rapid use of the internet, a huge amount of data is generated over the network with every passing second. On the other hand, the user demands information that is relevant to his personal search. The processing of such huge data is a challenging task. To serve the user with his specific required information, there is a need for an information retrieval mechanism that can process this large data. A recommendation system is such required technology that retrieves information in order to improve users’ access and thereby recommending items that are relevant to his explicitly mentioned behavior and preferences. The recommendation algorithm analyzes the huge dataset and focuses to recommend accurate content to the user. There are several recommendation systems in use today, some popular among them are Netflix, YouTube, Tinder, and Amazon. In this article, discussion is done on various types of recommendation systems, issues of recommendation systems, and use cases of widely used recommendation engines and their potential benefits. The work introduced in this paper is an integration approach of domain-specific and item-based recommendation system. The performance of the proposed algorithm is measured on evaluation metrics precision and recall. Experimental results prove that the approach introduced in this paper performed well as compared to existing methods.

Intelligent Sensing and Computing in Wireless Sensor Networks for Multiple Target Tracking

With sixth generation (6G) communication technologies, target sensing can be finished in milliseconds. The mobile tracking-oriented Internet of Things (MTT-IoT) as a kind of emerging application network can detect sensor nodes and track targets within their sensing ranges cooperatively. Nevertheless, huge data processing and low latency demands put tremendous pressure on the conventional architecture where sensing data is executed in the remote cloud and the short transmission distance of 6G channels presents new challenges into the design of network topology. To cope with the above difficulties, this paper proposes a new resource allocation scheme to perform delicate node scheduling and accurate tracking in multitarget tracking mobile networks. The dynamic tracking problem is formulated as an infinite horizon Markov Decision Process (MDP), where the state space that considers energy consumption, system responding delay, and target important degree is extended. A model-free reinforcement learning is applied to obtain satisfied tracking actions by frequent iterations, in which smart agents interact with the complicated environment directly. The performance of each episode is evaluated by the action-value function in search of the optimal reward. Simulation results demonstrate that the proposed scheme shows excellent tracking performance in terms of energy cost and tracking delay.

Virtual Machine Task Classification Using Support Vector Machine and Improved MFO Based Task Scheduling

The processing of data in Big data computing necessitates a significant number of CPU cycles and network bandwidth. Dataflow is a huge data processing programming model that comprises of jobs structured in a graph structure. Scheduling these jobs is one of the most active study fields, with the primary goal of allocating tasks based on available resources. It is critical to efficiently schedule jobs in a way that minimizes task completion time and maximizes resource utilization. In recent years, many research works on task scheduling problems in cloud computing and various heuristic approaches have been evolved which the thesis focuses upon. Most of these efforts are focused on improving the performance such as minimizing the makespan and efficient utilization of cloud resources that benefits the cloud users and the providers. In this paper, Big Data analysis processing in cloud environment for efficient dynamic task scheduling by using different techniques as machine learning classifier and optimization approach. In machine learning classifier as Support Vector Machine (SVM) for classification of virtual machine task classification. In this classifier can classify the incoming request efficiently and reduce the makespan and execution time. Further, we used moth flame optimization technique for allocating the classified task from SVM classifier. In this proposed system have classification of virtual machine (VM) task and evaluate the resource allocation decision making procedures. In this experiment, we carried out by using cloud simulation environment to evaluate and analysis the proposed method. In this proposed scheme can efficiently reduce the makespan time and load balancing to improve the better VM Classification. And aslo we compare the proposed moth flame optimization (MFO) with min-max algorithm and particle swarm optimization to compare the performance respectively.