Data Processing Performance Research Articles

Refinement and augmentation for data in micro open learning activities with an evolutionary rule generator

AbstractImproving both the quantity and quality of existing data are placed at the center of research for adaptive micro open learning. To cover this research gap, our work targets on the current scarcity of both data and rules that represent open learning activities. An evolutionary rule generator is constructed, which consists of an outer loop and an inner loop. The outer loop runs a genetic algorithm (GA) to produce association rules that can be effective in the micro open learning scenario from a small amount of available data sources; while the inner loop optimizes generated candidates by taking into account both rare and negative association rules (NARs). These optimized rules are further applied in refining and augmenting data denoting learners’ behaviors in open learning into a low‐dimensional, descriptive and interpretable form. The performance of rule discovery and data processing have been empirically evaluated using genuine open learning data.

SkinSim: A Design and Simulation Tool for Robot Skin With Closed-Loop pHRI Controllers

The importance of tactile sensing for physical human–robot interaction (pHRI) and dexterous manipulation is well known. SkinSim is a novel simulation environment for tactile robot skins in which one can study design tradeoffs involved in deploying whole-body, dense sensor arrays. In this article, scalable modeling approaches are presented for simulating pressure-sensitive robot skin patches, with simultaneous consideration of sensing element geometry and mechanical structure, signal quality, data processing, and closed-loop force controller performance. The open-source simulation architecture of SkinSim is compatible with Gazebo and robot operating system (ROS) programming environments and supports both robot skin dynamic models, as well as tactile sensing element models. An experimentally validated force dispersion model was introduced for the simulation of sensors embedded in a mechanical damping layer. Simulation examples of robot skin with different tactile resolutions are presented using parameter values extracted from an experimental testbed. Thus, simulation results were experimentally validated and the skin sensor density impact on a simple pHRI controller performance was evaluated. Performance measures include center of pressure (COP) estimation error and control signal settling time, overshoot, and steady-state errors. Results suggest that while COP errors decrease in denser sensor skins, controller performance also deteriorates. Therefore, optimal robot skin designs will have to consider application-dependent tradeoffs. Similar results were confirmed in simulation with a larger skin patch containing approximately 4000 tactels and deployed on the end-effector of a collaborative mobile manipulator. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This article was motivated by the expensive and time-consuming process of designing tactile skin for robots. When designing pressure-sensitive whole-body sensor arrays for physical human–robot interaction, there are tradeoffs related to sensor resolution, accuracy, and response time, among others. Instead of building prototypes and evaluating them experimentally, the SkinSim simulation environment allows automatic testing of skins with simultaneous consideration of sensing element geometry and mechanical structure, signal quality, data processing, and closed-loop force controller performance. The user can specify several configurations to test and thereby explore the best tradeoffs before actually prototyping any hardware.

Titaneon: the eco smart lamp to degrade cigarette smoke pollutants

The purpose of this study is to find out the results of the “TITANEON” degradation power test on hazardous CO (carbon monoxide) compounds qualitatively and quantitatively and to explain the working system of the lamp. The research methods used in this study are (a) preparation: tools and materials, design testing tools; (b) conducting research: TiO2/Co synthesis, coating, tool performance test and data processing and analysis; (c) evaluation. The test results in this study were divided into two, namely qualitative and quantitative. In the qualitative test results, the “TITANEON” lamp is able to degrade CO compounds for 20 minutes. The quantitative results from 0 minutes amounted to 15526.24 ppm, 5 minutes later the CO concentration was 6202.82, in the next 15 minutes CO concentration was 2549.98 ppm and in the last 20 minutes the CO concentration was 1871.89 ppm. The system works, namely the first light that involves TiO2 nanoparticles will excite electrons from the valence band to the conduction band while producing positive holes in the valence band and negatively charged electron pairs. After that, TiO2 nanoparticles are transformed into super oxide compounds which release active oxygen species (O2 radicals and OH radicals), these compounds will degrade cigarette smoke compounds into CO2 and H2O compounds through photocatalytic mechanisms.

Extended stable factor method for the inverseQ-filter

The commonly used stable factor methods for the inverse [Formula: see text]-filter achieve good performance in seismic data processing; however, the constant gain-limit assumption in these methods is not associated with the effective frequency band of seismic data and cannot obtain desirable results with high resolution and high signal-to-noise ratio (S/N). Our extended stable factor method for the inverse [Formula: see text]-filter extends these methods by introducing two parameters and constant or self-adaptive gain limit to achieve the desirable high-resolution and high-S/N result. The extended stable factor method for the inverse [Formula: see text]-filter can be implemented in the frequency or time-frequency domain; the latter implementation achieves a higher S/N. Analysis of synthetic signals and field seismic data application illustrate that our method can produce a desirable high-resolution and high-S/N result.

Available techniques in hadoop small file issue

Hadoop is an optimal solution for big data processing and storing since being released in the late of 2006, hadoop data processing stands on master-slaves manner [1] that’s splits the large file job into several small files in order to process them separately, this technique was adopted instead of pushing one large file into a costly super machine to insights some useful information. Hadoop runs very good with large file of big data, but when it comes to big data in small files it could facing some problems in performance, processing slow down, data access delay, high latency and up to a completely cluster shutting down [2]. In this paper we will high light on one of hadoop’s limitations, that’s affects the data processing performance, one of these limits called “big data in small files” accrued when a massive number of small files pushed into a hadoop cluster which will rides the cluster to shut down totally. This paper also high light on some native and proposed solutions for big data in small files, how do they work to reduce the negative effects on hadoop cluster, and add extra performance on storing and accessing mechanism.

Factors of Performance for Application of AI Models in GPU Cloud

Introduction. In machine learning (ML) and artificial intelligence (AI) works, the emphasis is usually on the quality of classification or the accuracy of parameter estimation. If the focus is on performance, then it is also mainly about the performance of the model's training phase. However, with the proliferation of AI applications in real-world problems, the problem of ensuring high data processing performance with ready models becomes more important. By its nature, this problem is fundamentally different from the one of model training: the latter deals with intensive calculations and the former with simple calculations, but large flows of data (files) coming from the network or file system for processing. That is, the typical task of parallel processing with intensive input-output. Besides, in terms of application, the AI module that performs classification, evaluation, or other data processing is a "black box": the cost of developing and training the model, as well as the risks of failure, are too high to handle such tasks in a non-professional manner. Therefore, performance optimization primarily involves the selection and balancing of system parameters. Cloud systems with their flexibility, manageability and easy scaling are the ideal platforms for such tasks. Consider in more detail the task of investigating the factors which affect performance on a single, but notable, pattern recognition sample of a subset of ImageNet image collection [1] classified by the 50-layer deep learning neural network ResNet-50 [2]. The purpose of the paper is to experimentally investigate the factors that influence the performance of a ready-to-use neural network model application in GPU cloud systems of various architectures. Results. Overheads related to microservices and distributed architectures, memory, network, batch size, synchronous and asynchronous interactions are estimated. The complex nonlinear nature of the influence of the system parameters in various combinations is demonstrated.

Averaged Bi-LSTM networks for RUL prognostics with non-life-cycle labeled dataset

LSTM network is an effective RNN model to predict the system RUL for its superior performance in sequential data processing. Usually, networks trained by life-cycle labeled dataset would possess similar RUL predicting accuracies, because the network training algorithm could ensure the network optimality for the whole training dataset. However, for networks trained by non-life-cycle labeled samples, the network uncertainty caused by different training conditions could lead to degradation prediction uncertainty for some local points. Further, the RUL predicting results that are computed by these uncertain local points may shows relatively large differences. Therefore, in order to obtain an accurate RUL prediction with networks trained by non-life-cycle labeled samples, our paper proposes a novel network model averaging method to deal with the network uncertainty. What is more, to learn the temporal correlation information of training samples sufficiently, we adopt the Bi-LSTM network to illustrate the application of the proposed network model averaging method. Finally, degradation values of Graphite/LiCoO2 battery are used to verify the effectiveness of the proposed method. The results show that the proposed method could improve the RUL prediction accuracy and reduce the prediction error.

Design and implementation of distributed asynchronous data aided computer information interaction system

Under the influence of novel corona virus pneumonia epidemic prevention and control, it has put forward higher requirements for data storage and processing for personnel management system. The distributed asynchronous data aided computer information interaction system can solve the problem of multi node concurrent data processing. The traditional computer information interaction system has poor real-time performance, low precision and asynchronous data processing ability. The invocation features of message queuing asynchronous caching mode are combined with the standardization of Web services and cross language with cross platform access features in this paper. Through the combination of the two technologies, a flexible and universal asynchronous interaction architecture of distributed system is established. Based on Web service technology and system to system access, the call and response of tasks between modules are carried out in the system, which makes the interaction between the whole system have the characteristics of message driven. The test result shows that the system proposed in this paper has good real-time performance and strong data processing ability. It is suitable for the data interaction of distributed personal management system under the influence of novel corona virus pneumonia epidemic prevention and control.

A Survey on Spark Ecosystem: Big Data Processing Infrastructure, Machine Learning, and Applications

With the explosive increase of big data in industry and academic fields, it is necessary to apply large-scale data processing systems to analysis Big Data. Arguably, Spark is state of the art in large-scale data computing systems nowadays, due to its good properties including generality, fault tolerance, high performance of in-memory data processing, and scalability. Spark adopts a flexible Resident Distributed Dataset (RDD) programming model with a set of provided transformation and action operators whose operating functions can be customized by users according to their applications. It is originally positioned as a fast and general data processing system. A large body of research efforts have been made to make it more efficient (faster) and general by considering various circumstances since its introduction. In this survey, we aim to have a thorough review of various kinds of optimization techniques on the generality and performance improvement of Spark. We introduce Spark programming model and computing system, discuss the pros and cons of Spark, and have an investigation and classification of various solving techniques in the literature. Moreover, we also introduce various data management and processing systems, machine learning algorithms and applications supported by Spark. Finally, we make a discussion on the open issues and challenges for large-scale in-memory data processing with Spark.

Saksham: Resource Aware Block Rearrangement Algorithm for Load Balancing in Hadoop

Big Data Analytics demands huge processing power. Distributed computing seems to be a viable solution with the usage of commodity computers. But, processing big data optimally, on distributed systems poses the number of challenges, such as seamlessly integrating different resources of distributed systems. Hadoop provides this ease and hence is one of the most commonly used tools for the processing of big data. But, it is observed that the performance of Hadoop is compromised, when it comes to the heterogeneous environment. This paper proposes Saksham: The block rearrangement algorithm which optimizes the processing time by efficient file system management in the heterogeneous environment. Our proposed scheme successfully optimizes the big data processing over the homogeneous and heterogeneous environment. To achieve better performance for big data processing, we target two important aspects of heterogeneous distributed computing: file system management and process management. First, file system management basically controls the block placement and allows us to rearrange the blocks on specified nodes, considering the node’s storage and processing capability. Second, we use the concept of node labeling and scheduling to achieve better process management. The results prove that the proposed approach has optimized job execution time along with reduced latency and data-skew substantially.

FENCE: Fast, ExteNsible, and ConsolidatEd Framework for Intelligent Big Data Processing

The proliferation of smart devices and the advancement of data-intensive services has led to explosion of data, which uncovers massive opportunities as well as challenges related to real-time analysis of big data streams. The edge computing frameworks implemented over manycore systems can be considered as a promising solution to address these challenges. However, in spite of the availability of modern computing systems with a large number of processing cores and high memory capacity, the performance and scalability of manycore systems can be limited by the software and operating system (OS) level bottlenecks. In this work, we focus on these challenges, and discuss how accelerated communication, efficient caching, and high performance computation can be provisioned over manycore systems. The proposed Fast, ExteNsible, and ConsolidatEd (FENCE) framework leverages the availability of a large number of computing cores and overcomes the OS level bottlenecks to provide high performance and scalability for intelligent big data processing. We implemented a prototype of FENCE and the experiment results demonstrate that FENCE provides improved data reception throughput, read/write throughput, and application processing performance as compared to the baseline Linux system.

Satellite-Ground Two-Way Measuring Method and Performance Evaluation of BDS-3 Inter-Satellite Link System

From November 2017 to December 2018, Beidou navigation satellite system (BDS) completed the basic system construction of Beidou-3 system (BDS-3) and opened the initial service, BDS-3 satellites are equipped with Ka-band inter-satellite link (ISL) payloads, satellite autonomous orbit determination (OD) and time synchronization (TS) can be realized through ISL technology. At present, 18 BDS-3 medium earth orbit (MEO) satellites have been formally networked, and a series of ground-satellite link (GSL) test and connection test of Ka-band ISL signals are carried out, through the test, the working state and functional performance of the ISL payloads can be evaluated. This article focuses on the GSL measurements and performance evaluation of the ISL signals of BDS-3, the basic architecture of the GSL test platform is given, the principle of Ka-band GSL measurement and basic data processing methods and performance evaluation methods are introduced. Obtained the GSL observation data of 18 BDS-3 MEO satellites using the Ka-band ground station of the National Time Service Center (NTSC), and evaluate the performance of the GSL ranging and time synchronization. The result shows that GSL ranging accuracy of 18 MEO satellites is better than 0.02m and GSL time synchronization accuracy is better than 0.2ns. Thus, this meets the requirements of ISL system. The assessment methods and analysis results presented in this article can provide the reference of the function and performance verification for inter-satellite link system, and provide technical and data accumulation for the time synchronization extension application based on ISL signals.

ANALISIS KUALITAS LAYANAN APLIKASI BERKAS MLAKU DHEWE TIPE B DI KECAMATAN SUKODONO

Sukodono district is one of the sub-districts in Sidoarjo Regency that has first tried to become a pioneer of internet-based service providers through service innovations File Mlaku dheWe (BMW). The fact shows that the quality of application services is such an important issue given the technical aspects of its implementation. This study aims to analyze the quality of administrative services and the factors that influence the quality of administrative services based on E-Government, especially in the BMW type B application service in Sukodono District. This research uses descriptive quantitative research methods. The population of this study took 21 respondents. This analysis uses the e-GovQual dimensional approach. The analysis technique of this research uses the Importance Performance Analysis (IPA) method and data processing techniques. The results show (1) The analysis conducted by the researcher using Importance Performance Analysis (IPA) shows that of the 35 sub-indicators studied, the quadrant A variable consists of 11 sub-indicators, quadrant B consists of 7 sub-indicators, quadrant C consists of 9 sub-indicator, and quadrant D consists of 8 sub indicators. (2) The results of the analysis of the quality of administrative services based on E-Government on the Mlakau dheWe file application service type B in Sukodono District, Sidoarjo Regency are well implemented.

ANALISIS KUALITAS LAYANAN APLIKASI BERKAS MLAKU DHEWE TIPE B DI KECAMATAN SUKODONO

Sukodono district is one of the sub-districts in Sidoarjo Regency that has first tried to become a pioneer of internet-based service providers through service innovations File Mlaku dheWe (BMW). The fact shows that the quality of application services is such an important issue given the technical aspects of its implementation. This study aims to analyze the quality of administrative services and the factors that influence the quality of administrative services based on E-Government, especially in the BMW type B application service in Sukodono District. This research uses descriptive quantitative research methods. The population of this study took 21 respondents. This analysis uses the e-GovQual dimensional approach. The analysis technique of this research uses the Importance Performance Analysis (IPA) method and data processing techniques. The results show (1) The analysis conducted by the researcher using Importance Performance Analysis (IPA) shows that of the 35 sub-indicators studied, the quadrant A variable consists of 11 sub-indicators, quadrant B consists of 7 sub-indicators, quadrant C consists of 9 sub-indicator, and quadrant D consists of 8 sub indicators. (2) The results of the analysis of the quality of administrative services based on E-Government on the Mlakau dheWe file application service type B in Sukodono District, Sidoarjo Regency are well implemented.

Tornado: Enabling Blockchain in Heterogeneous Internet of Things Through a Space-Structured Approach

With the widespread applications of the Internet of Things (IoT), e.g., smart city, business, healthcare, etc., the security of data and devices becomes a major concern. Although blockchain can effectively enhance the network security and achieve fault tolerance, the huge resource consumption and limited performance of data processing restrict its deployments in IoT scenarios. Observing the heterogeneity and resource constraints, we intend to make blockchain accommodate both wimpy and brawny IoT devices. In this article, we present ${\textsf {Tornado}}$ , a high-performance blockchain system based on space-structured ledger and corresponding algorithms, to enable blockchain in IoT. Specifically, we first design a space-structured chain architecture with novel data structures for promoting the network scalability. To address the huge heterogeneity of IoT, a novel consensus mechanism named collaborative-proof of work is developed. Moreover, we propose the space-structured greedy heaviest-observed subtree (S2GHOST) protocol for improving the resource efficiency of IoT devices. Additionally, a dynamic weight assignment mechanism in S2GHOST contributes to reflect the trustworthiness of data and devices. Extensive experiments demonstrate that ${\textsf {Tornado}}$ can achieve a maximum throughput of 3464.76 transactions per second. The optimizations of propagation latency and resource efficiency are 68.14% and 30.56%, respectively.

Optimizing performance of data processing

The industry associates data processing with the application of geophysical algorithms to seismic data, but this definition is too narrow. We should consider a broader meaning particularly in these times of increased automation, and time and cost consciousness. If processing is a method to produce information from the manipulation of data, then we should consider the framework that enables this process. A system must exist to facilitate the work. Generally, systems are a collection of computers, networks, processes and to a lesser extent people. For any given input, they create an output, which in seismic data processing can be the application of a geophysical algorithm, or it can be reordering, reducing or aggregating data. In this system, the headline grabbing geophysical process may only be a small part of the work. The underlying framework needs to be as efficient as it can be, either managing the data, enabling the geophysical process, or both.

Intelligent Missing Shots’ Reconstruction Using the Spatial Reciprocity of Green’s Function Based on Deep Learning

The trace interval in the common shot and receiver gathers is always inconsistent. The inconsistency affects the final performance of seismic data processing, and the reconstruction methods can enhance the consistency. Unfortunately, most interpolation algorithms are suitable in randomly missing cases, and the difficulty increases sharply in regularly missing cases, especially with big gaps. As deep learning (DL) has a strong self-learning ability in nonlinear characterizations to avoid linear events, sparsity, and low rank assumptions, we introduce DL into missing shots’ reconstruction. The spatial reciprocity of Green’s function is used to provide reasonable training data sets. First, the residual learning networks (ResNets) and the interpolation issue are briefly illustrated. Then, the spatial reciprocity is reviewed and illustrated qualitatively using the common shot and receiver gathers. The similar features in the common shot and receiver gathers guarantee the reasonability to regard the common shot gathers as the training sets and to regard the common receiver gathers as the test sets. The common shot gathers are divided into the training sets to train ResNets and the validation sets to verify the performance of the trained ResNets. Finally, the trained ResNets are used to reconstruct missing shots intelligently in the common receiver gather. Three different data sets are used to prove the validity of the proposed strategy. After reconstruction, the events are more continuous with less serrations and serious frequency wavenumber (FK) aliasing is attenuated effectively. The reconstructed data with a better consistency can improve the accuracy of migration and the final reservoir characterization.

Towards secure big data analytic for cloud-enabled applications with fully homomorphic encryption

Cloud computing empowers enterprises to efficiently manage big data and discovery of useful information which are the most fundamental challenges for big data enabled applications. The cloud offered unlimited resources that can store, manage and analyze massive and heterogeneous data to improve the quality assurance of application services. Nevertheless, cloud computing is exposed to enormous external and internal privacy breaches and leakage threats. In this paper, we introduce a privacy-preserving distributed analytics framework for big data in cloud. Fully Homomorphic Encryption (FHE) is used as an emerging and powerful cryptosystem that can carry out analysis tasks on encrypted data. The developed distributed approach has the scalability to partition both data and analysis computations into subset cloud computing nodes that can be run independently. This rapidly accelerates the performance of encrypted data processing while preserving a high level of analysis accuracy. Our experimental evaluation demonstrates the efficiency of the proposed framework, in terms of both analysis performance and accuracy, for building a secure analytics cloud-enabled application.

An implementation of standardization for telecom systems on ships and offshore plants based on the TMS

Telecom systems for ships and offshore plants used to be tools for long-distance communication in the past. Today, telecom systems combined with evolving IT ensure safety, accident prevention and cost saving for ships and offshore plants. In the future, all systems will be managed and controlled by telecom systems. Yet, the IMO and Classification Societies have neither revised nor amended the requirements and certification concerning telecom systems for decades. The failure to establish a standardized scheme applicable to telecom systems hinders ships and offshore plants from adopting advanced next-generation technologies widely used on land. Thus, unstandardized telecom systems are manufactured and installed for different user demands. Consequently, complicated interfaces impede the scalability and compatibility between telecom systems. Ships and offshore plants require advanced stability and diversity compared to their onshore counterparts. Unless the communication systems for ships and offshore plants are duly verified before use, serious adverse effects including marine accidents may ensue. This paper proposes a method of standardization to implement safe, scalable and general-purpose telecom systems for ships and offshore plants. The standardization of communication systems for ships and offshore plants facilitates the simplification and systematization of interfaces, and thus maximizes system inter-operability. Also, ships and offshore plants should adopt the standardized communication systems to enhance data reliability. This paper demonstrates the superior stability and performance of the proposed standardized communication system compared to the existing systems. Moreover, it proves the proposed system ensures the stable performance and fast data processing even in emergency. The standardized system for ships and offshore plants is a requisite for quality improvement, cost saving and stable performance. The findings of this paper are applied to the Smart Ship Solution for standardized integrated communication systems for containers, VLCC and LNG. Also, this paper analyzes the strengths and weaknesses of telecom systems configuration in TMS for offshore plants. In addition, it experimentally demonstrates the advantages of the proposed standardization of telecom systems in light of processing time and system operation cost.

Machine learning in materials science

AbstractTraditional methods of discovering new materials, such as the empirical trial and error method and the density functional theory (DFT)‐based method, are unable to keep pace with the development of materials science today due to their long development cycles, low efficiency, and high costs. Accordingly, due to its low computational cost and short development cycle, machine learning is coupled with powerful data processing and high prediction performance and is being widely used in material detection, material analysis, and material design. In this article, we discuss the basic operational procedures in analyzing material properties via machine learning, summarize recent applications of machine learning algorithms to several mature fields in materials science, and discuss the improvements that are required for wide‐ranging application.