Scientific Big Data Research Articles

Special Issue: Big Scientific Data and Machine Learning in Science and Engineering.

Special Issue: Big Scientific Data and Machine Learning in Science and Engineering.

EGQCY: A smart contract-based scientific big data system approach for incentive sharing and transaction on the cost of data quality

Currently, scientific big data management is generally faced with the problems of scattered data resources, inconsistent data standards, and the inability to share and circulate data safely. Research personnel attaches great importance to whether sharing the first-hand property is secure under clear ownership and whether it can contribute to the large society. The isolation of the data management system is the obvious obstacle to collecting and managing across-disciplinary data. To a large extent, sharing and trading scientific big data is the primary purpose to realize the clarity of property rights, secure data sharing, and the value of the data assets step by step. We propose to construct a public platform for scientific big data management. The system is managed to unify and authorize the on-chain data, on which data sharing and trading is tracked throughout the process. Smart contracts are executed with vital functions and guarantee price matching in data transactions. We design the incentive mechanism which measures the incentive yield of data cost quality based on Evolutionary Game Theory and Data Quality Control Theory (EGQCY), considering how the cost of data quality performs in controlling and impacting the rational release of the incentive yields in the sharing and trading process. The experiments found that the design of incentive yield and incentive coefficients only significantly affected the transition from low-quality data to medium-quality data. Both parameters converged to fixed values as the cost of data quality increased.

Scientific Data Spaces - Experiences from the EGI-ACE project.

This paper presents the approach adopted by the EGI-ACE project for the setup and delivery of Data Spaces for various scientific domains. The work was implemented by members of the EGI e-infrastructure and of several European Research Infrastructures in the context of the European Open Science Cloud programme. Our results are several Data Space services that enable the reuse and exploitation of open, scientific big data for compute intensive use cases. The paper illustrates the EGI-ACE approach through two examples: (1) EMSO ERIC Data Portal for seafloor and water column research and (2) ENES Data Space for climate research.

Parallel Optimization for Large Scale Interferometric Synthetic Aperture Radar Data Processing

Interferometric synthetic aperture radar (InSAR) has developed rapidly over the past years and is considered as an important method for surface deformation monitoring, benefiting from growing data quantities and improving data quality. However, the handing of SAR big data poses significant challenges for related algorithms and pipeline, particularly in large-scale SAR data processing. In addition, InSAR algorithms are highly complex, and their task dependencies are intricate. There is a lack of efficient optimization models and task scheduling for InSAR pipeline. In this paper, we design parallel time-series InSAR processing models based on multi-thread technology for high efficiency in processing InSAR big data. These models concentrate on parallelizing critical algorithms that have high complexity, with a focus on deconstructing two computationally intensive algorithms through loop unrolling. Our parallel models have shown a significant improvement of 10–20 times in performance. We have also developed a parallel optimization tool, Simultaneous Task Automatic Runtime (STAR), which utilizes a data flow optimization strategy with thread pool technology to address the problem of low CPU utilization resulting from multiple modules and task dependencies in the InSAR processing pipeline. STAR provides a data-driven pipeline and enables concurrent execution of multiple tasks, with greater flexibility to keep the CPU busy and further improve CPU utilization through predetermined task flow. Additionally, a supercomputing-based system has been constructed for processing massive InSAR scientific big data and providing technical support for nationwide surface deformation measurement, in accordance with the framework of time series InSAR data processing. Using this system, we processed InSAR data with the volumes of 500 TB and 700 TB in 5 and 7 days, respectively. Finally we generated two maps of land surface deformation all over China.

Stability of scientific big data sharing mechanism based on two-way principal-agent

<abstract> <p>In the era of big data, facing the data-intensive scientific paradigm shift and the explosion of scientific big data, there is an urgent need for alliance cooperation between heterogeneous research groups to actively open and share scientific big data to support China's economic development, technological innovation and national security. Therefore, the study of scientific big data sharing mechanism has very important practical significance. We think science big data sharing is an ecosystem that is constantly evolving to higher-order ecological evolution. Based on the dual perspectives of psychological contract and contractual contract, the scientific big data sharing strategy evolution mechanism and sharing strategy incentive mechanism are explored.The research finds that the cooperation of scientific research groups is bound by psychological contract and contractual contract; stochastic evolutionary game has stronger explanatory power for sharing strategy evolution, complementarity is positive indicator, random interference and moral risk are negative indicators; two-way principal agent can describe Alliance members are mutually entrusted, and the shared strategy incentive contract consists of fixed wages and incentive wages, which are proportional to risk.</p> </abstract>

Epigenomics Scientific Big Data Workflow Scheduling for Cancer Diagnosis in Health Care Using Heterogeneous Computing Environment

HIGHLIGHTS Analysis of different scheduling techniques for Epigenomics computation. An efficient algorithm that minimize computation time of Epigenomics Application. Performance analysis in High Performance Computing (HPC) environment. Shows better performance over state-of-art algorithms.

Open science and Big Data in South Africa.

With the Square Kilometer Array (SKA) project and the new Multi-Purpose Reactor (MPR) soon coming on-line, South Africa and other collaborating countries in Africa will need to make the management, analysis, publication, and curation of "Big Scientific Data" a priority. In addition, the recent draft Open Science policy from the South African Department of Science and Innovation (DSI) requires both Open Access to scholarly publications and research outputs, and an Open Data policy that facilitates equal opportunity of access to research data. The policy also endorses the deposit, discovery and dissemination of data and metadata in a manner consistent with the FAIR principles - making data Findable, Accessible, Interoperable and Re-usable (FAIR). The challenge to achieve Open Science in Africa starts with open access for research publications and the provision of persistent links to the supporting data. With the deluge of research data expected from the new experimental facilities in South Africa, the problem of how to make such data FAIR takes center stage. One promising approach to make such scientific datasets more "Findable" and "Interoperable" is to rely on the Dataset representation of the Schema.org vocabulary which has been endorsed by all the major search engines. The approach adds some semantic markup to Web pages and makes scientific datasets more "Findable" by search engines. This paper does not address all aspects of the Open Science agenda but instead is focused on the management and analysis challenges of the "Big Scientific Data" that will be produced by the SKA project. The paper summarizes the role of the SKA Regional Centers (SRCs) and then discusses the goal of ensuring reproducibility for the SKA data products. Experiments at the new MPR neutron source will also have to conform to the DSI's Open Science policy. The Open Science and FAIR data practices used at the ISIS Neutron source at the Rutherford Appleton Laboratory in the UK are then briefly described. The paper concludes with some remarks about the important role of interdisciplinary teams of research software engineers, data engineers and research librarians in research data management.

Reflections on the Innovation of University Scientific Research Management in the Era of Big Data

In China, universities are important centers for SR (scientific research) and innovation, and the quality of SR management has a significant impact on university innovation. The informatization of SR management is a critical component of university development in the big data environment. As a result, it is crucial to figure out how to improve SR management. As a result, this paper builds a four-tier B/W/D/C (Browser/Web/Database/Client) university SR management innovation information system based on big data technology and thoroughly examines the system’s hardware and software configuration. The SVM-WNB (Support Vector Machine-Weighted NB) classification algorithm is proposed, and the improved algorithm runs in parallel on the Hadoop cloud computing platform, allowing the algorithm to process large amounts of data efficiently. The optimization strategy proposed in this paper can effectively optimize the execution of scientific big data applications according to a large number of simulation experiments and real-world multidata center environment experiments.

RENDA: Resource and Network Aware Data Placement Algorithm for Periodic Workloads in Cloud

The Hadoop enabled cloud platforms are gradually becoming preferred computational environment to execute scientific big data workloads in a periodic manner. However, it is observed that the default data placement approach of such cloud platforms is not the efficient one and often ends up with significant data transfer overhead leading to degradation of the overall job completion time. In this article, a Resource and Network-aware Data Placement Algorithm (RENDA) is proposed to reduce the non-local executions and thereby reduce the overall job completion time for periodic workloads in the cloud environment. The entire job execution is modeled as a two-stage execution characterized as data distribution and data processing. The RENDA reduces the time of the stages as mentioned above by estimating the heterogeneous performance of the nodes on a real-time basis followed by careful allocation of data in several installments to participating nodes. The experimental results show that the proposed RENDA algorithm consistently outperforms over the recent state-of-the-art alternatives with as much as 28 percent reduction in data transfer overhead leading to 16 percent reduction in average job completion time with 27 percent average speedup on average job execution.

Call for Special Issue Papers: Big Scientific Data and Machine Learning in Science and Engineering.

Call for Special Issue Papers: Big Scientific Data and Machine Learning in Science and Engineering.

Call for Special Issue Papers: Big Scientific Data and Machine Learning in Science and Engineering.

Call for Special Issue Papers: Big Scientific Data and Machine Learning in Science and Engineering.

Estigma, género e interseccionalidad en la poesía de Xela Arias

Scientific Big Data (SBD), helps authors in all fields, with a space to collaborate and to disseminate their ideas, and research results to the widest possible audience, adding value to the quality of the work. Open access it's the way we think that research should be in order to accelerate progress in science and medicine thru the widest research communication. It's not just about free and unrestricted access to research, it's also about open data and transparency that should represents the best scientific values, bringing scientists together, to share work as rapidly and as widely as possible, to advance science faster and to benefit society as a whole.

Digital Innovation Risk Management Model of Discrete Manufacturing Enterprise Based on Big Data Analysis

At present, most risk management work mainly relies on manpower, and manpower relies on the professional knowledge of relevant skilled workers to discover hidden safety risks in production activities. This article combines relevant big data theories and 4V characteristics to analyze and investigate safety production and big data, study data structure, data source and data type. Using 5W1H scientific big data and applications, this analysis method analyzes the theoretical basis, applications and beneficiaries of big data related to safety production, some of which are application links and important theoretical issues. Secondly, it studies the security risk management model based on big data, proposes a risk management model based on big data, the technical basis of big data and the idea of a three-dimensional model, and applies the systematic space method, which is reflected in three aspects of risk management. In the end, a risk identification model based on big data, a risk assessment classification model, and a risk early warning and pre-control model are defined.

Call for Special Issue Papers: Big Scientific Data and Machine Learning in Science and Engineering.

Call for Special Issue Papers: Big Scientific Data and Machine Learning in Science and Engineering.

Performance Evaluations of Distributed File Systems for Scientific Big Data in FUSE Environment

Data are important and ever growing in data-intensive scientific environments. Such research data growth requires data storage systems that play pivotal roles in data management and analysis for scientific discoveries. Redundant Array of Independent Disks (RAID), a well-known storage technology combining multiple disks into a single large logical volume, has been widely used for the purpose of data redundancy and performance improvement. However, this requires RAID-capable hardware or software to build up a RAID-enabled disk array. In addition, it is difficult to scale up the RAID-based storage. In order to mitigate such a problem, many distributed file systems have been developed and are being actively used in various environments, especially in data-intensive computing facilities, where a tremendous amount of data have to be handled. In this study, we investigated and benchmarked various distributed file systems, such as Ceph, GlusterFS, Lustre and EOS for data-intensive environments. In our experiment, we configured the distributed file systems under a Reliable Array of Independent Nodes (RAIN) structure and a Filesystem in Userspace (FUSE) environment. Our results identify the characteristics of each file system that affect the read and write performance depending on the features of data, which have to be considered in data-intensive computing environments.

SwCS: Section-wise Content Similarity Approach to Exploit Scientific Big Data

The growing collection of scientific data in various web repositories is referred to as Scientific Big Data, as it fulfills the four “V’s” of Big Data–-volume, variety, velocity, and veracity. This phenomenon has created new opportunities for startups; for instance, the extraction of pertinent research papers from enormous knowledge repositories using certain innovative methods has become an important task for researchers and entrepreneurs. Traditionally, the content of the papers are compared to list the relevant papers from a repository. The conventional method results in a long list of papers that is often impossible to interpret productively. Therefore, the need for a novel approach that intelligently utilizes the available data is imminent. Moreover, the primary element of the scientific knowledge base is a research article, which consists of various logical sections such as the Abstract, Introduction, Related Work, Methodology, Results, and Conclusion. Thus, this study utilizes these logical sections of research articles, because they hold significant potential in finding relevant papers. In this study, comprehensive experiments were performed to determine the role of the logical sections-based terms indexing method in improving the quality of results (i.e., retrieving relevant papers). Therefore, we proposed, implemented, and evaluated the logical sections-based content comparisons method to address the research objective with a standard method of indexing terms. The section-based approach outperformed the standard content-based approach in identifying relevant documents from all classified topics of computer science. Overall, the proposed approach extracted 14% more relevant results from the entire dataset. As the experimental results suggested that employing a finer content similarity technique improved the quality of results, the proposed approach has led the foundation of knowledge-based startups.

Research on the Innovation Path of University Ideological and Political Work Based on Big Data Technology

Ideological and political work is one of the concrete forms of higher education. The effectiveness of education determines the teaching quality of Ideological and political work for college students. In the context of the era of big data, the "5V" characteristics of big data not only bring convenience to the development of the era, but also bring new opportunities to ideological and political work. Ideological and political work in Colleges and universities should combine the characteristics of big data. Through the establishment of scientific big data thinking, colleges and universities can explore and publish innovative ideological and political work paths.

An efficient list scheduling algorithm with task duplication for scientific big data workflow in heterogeneous computing environments

SummaryA high‐performance heterogeneous computing environment provides computing resources for efficient computation of scientific Big Data workflow applications. A Big Data workflow application comprises thousands of interdependent tasks with precedence constraints. Basically, the performance of heterogeneous computing systems mainly depends on the workflow scheduling algorithms. These workflow scheduling algorithms are considered an NP‐complete problem. In this article, a List Scheduling with Task Duplication (LSTD) algorithm is proposed that efficiently minimizes the makespan of workflow applications. The LSTD introduces task duplication strategy in the list scheduling algorithm without increasing the overall time complexity. The overall functionality of LSTD mainly consists of three phases. In the first phase, it calculates the rank of the tasks for deciding the scheduling order. The next step is responsible for duplicating the entry task on the processor only if it increases the overall efficiency and avoids processor overloading. Finally, in the last step, the processor is assigned to the tasks based on the popular insertion‐based policy that attempts to insert the task among two earlier assigned tasks on a given processor in earliest idle time. In order to verify the usefulness of the proposed algorithm, several existing well‐known algorithms, such as Heterogeneous Earliest Finish Time (HEFT), Critical Path on a Processor (CPOP), and Predict Earliest Finish Time (PEFT) are considered for comparison. A non‐duplication version of LSTD named List Scheduling without Task Duplication algorithm is also considered for performance evaluation. The experimental analysis based on scientific Big Data workflows (CyberShake, Montage, and LIGO) proves that LSTD significantly surpasses all considered scheduling heuristics concerning schedule length ratio, the percentage of best results, and average running time metrics.

An Efficient Multiresolution Clustering for Motif Discovery in Complex Networks.

Motif discovery and network clustering in complex networks have received a lot of attention in recent years, also they are still challenging tasks in bioinformatics, big data analytics and data mining applications. Motif discovery in big data networks has a lot of important applications in different domains such as engineering, bioinformatics, cheminformatics, genomics, sociology and ecology for revealing hidden frequent structures, functional building blocks, or knowledge discovery. In this paper, a motif localization method based on a novel clustering algorithm in complex networks is presented. In our method, for each complex network, a novel structure so-called Augmented Multiresolution Network (AMN) is generated, then it is adaptively partitioned into several clusters and their corresponding subnets. Then top ranked subnets are chosen to discover network motifs. We show that the proposed method provides an efficient solution for clustering and motif discovery; It speeds up current motif discovery algorithms by pruning non-promising regions of complex networks. Experimental results show our algorithm efficiently deals with complex networks representing large datasets with high-dimensionality such as big scientific data. Our method also provides motivations for future studies in big data and complex networks.

Segmented In-Advance Data Analytics for Fast Scientific Discovery

Scientific discovery usually involves data generation, data preprocessing, data storage and data analysis. As the data volume exceeds a few terabytes (TB) in a single simulation run, the data movement, which happens during each cycle of the scientific discovery, continues to be the bottleneck in most scientific big data applications. A lot of research works have been conducted on reducing the data movement. Among the existing efforts and based on our previous research, reusing the analysis results shows a significant potential in optimizing the data movement between analysis operations. In this work, we propose the Segmented In-Advance (SIA) data analytics approach for optimizing the data movement and we also provide a cloud-based elastic distributed in-memory database to manage the intermediate analysis results. The fundamental idea of this Segmented In-Advance approach is to analyze the history operations and to predict the future interesting analytics operations. The predicted analysis operation is in-advance performed on the finer segmented dataset and the segmented results are distributed in an in-memory key-value store for future reuse. The evaluation shows that the segmented in-advance data analytics approach achieves 1.2X-6.1X speedup. The evaluation also shows a good scalability of the in-memory distributed data store. The proposed Segmented In-Advance data analytics approach is a promising data movement reduction solution for scientific big data applications and fast scientific discovery.