Big Data Software Research Articles

Pervasive Computing For New Design In Big Data Analytics Software Development: Requirements Engineering Perspective

In recent years, the field of big data has been rapidly growing and has become a crucial aspect of many industries. The ability to collect, analyze, and interpret massive amounts of data has enabled businesses to make more informed decisions and has led to the development of new products and services. However, with the increase in data volume, velocity, and variety, traditional software development methods have become inadequate. This has led to the emergence of new design approaches that can handle the complexities of big data. Big data is very important nowadays for business organizations to make smart decisions based on insight gained from the big data. These benefit the organization to be competent in the market as well as to maximize their profit. However, there is a high rate of big data software project failure. This is due to a lack of new design approaches that can handle different dimensions of requirements for big data. The objective of this paper is to show how the pervasive computing / pervasive requirements engineering approach should be considered for big data software development based on different perspectives in pervasive environments to design new software

ETune: Efficient configuration tuning for big-data software systems via configuration space reduction

Configuration tuning for big-data software systems is generally challenging due to the complex configuration space and costly performance evaluation. The workload changes over time in the context of big data making this problem even more difficult. To address the low efficiency issue caused by the need to retune from scratch after workload changes, we present ETune based on a combination of an Bayesian Optimization (BO) based tuner and configuration space reduction techniques to efficiently find high-performance configurations for big-data systems. For configuration space reduction, we develop two approaches: (1) shrinking the configuration space by trading off the exploitation and exploration between the impactful parameters and other parameters in the tuning process, (2) generating the promising regions of configuration space by transferring knowledge across past tuning tasks. The two configuration space reduction methods aim to reduce the huge configuration space to a compact but promising one, and searching in the reduced configuration space can further accelerate the configuration tuning. The extensive experiments show that our configuration space reduction methods considerably boost the BO-based tuner, and ETune significantly improves the tuning efficiency compared with the transfer learning based state-of-the-arts.

Implementasi Aplikasi SP4N-Lapor! Dalam Pengelolaan Pengaduan Masyarakat di Dinas Komunikasi dan Informatika Kabupaten Tangerang


 SP4N-REPORT! (National Public Service Complaint Management System - People's Online Aspiration and Complaint Service), which is a service for conveying all community aspirations and complaints through several complaint channels. The aim of this research is to find out an overview of the implementation of the SP4N-LAPOR Application! in managing public complaints at the Tangerang Regency Communication and Information Service which has been implemented as a public service innovation so far. The research method uses a qualitative descriptive method with an approach in the form of literature study or literature study, through a collection process obtained from various sources including journals, books, documentation, the internet and libraries. Meanwhile, the type of data in this research is secondary data, namely data obtained indirectly, namely taking data from journals related to research. The research results show that the SP4N-LAPOR Application program in providing convenience to public complaint services as access for the community in conveying aspirations, requests for information on government performance and improving the quality of public services at the Tangerang Regency Communication and Information Service has been running well. Proven through several applications by providing evaluations, collaboration with other institutions such as LSN, subscribing to big data software, and creating a special complaint management structure, with the aim of increasing responsiveness and quality in managing complaints from the public.
 Keywords: Implementation, management, Complaints, SP4N-LAPOR!

System analysis of big data for Earth sciences

Currently, big data is one of the most discussed phenomena in the field of information technology. What is big data? How are they created? How does big data software work? How is big data theory and practice used and can be used in relation to geosciences? How is the issue of big data developing within the Russian Academy of Sciences? These are the questions discussed in an article prepared on the basis of a scientific report presented at a meeting of the Presidium of the Russian Academy of Sciences on March 28, 2023.

Digital Transformation: Exploring big data Governance in Public Administration

As economies become increasingly data-driven, big data technologies and software products are turning into key tools for managing technological processes in real time for more efficient delivery of public services to citizens. The change in the interaction between the state and society implies the creation of a unified state digital ecosystem, centered around big data. Such paradigm calls for rethinking of public administration principles ensuring the transition from an electronic state to a digital one. As a result, the likelihood of creating values ​​that meet the shifting expectations of citizens in relation to public services increases.

Software engineering and big data software

Software engineering is a mature industry of human activity focused on the creation, deployment, marketing and maintenance of software. The fundamental concepts of engineering are life cycle model; three main components of life cycle phases - products, processes and resources; engineering and methodologies for creating, deployment and maintaining software. Software is the foun- dation of technological advances that lead to new high performance products. As the functionality of products grows, so does the need to efficiently and correctly create and maintain the complex software that enables this growth. Therefore, in addition to solving its own problems, software engineering serves the solution of the problems of creating and maintaining software in other domains, which are called application domains. Information technology is a well-known application domain. The basis of this domain is data. Information systems are being implemented in an organization to improve its effectiveness and efficiency. The functionality of information systems has grown dramatically when big data began to be used. This growth has led to the emergence of a wide variety of software-intensive big data information systems. At the same time, the role and importance of software engineering for solving the problems of this application domain has only intensified. Modern possibilities of software engineering are shown. The aspects of interaction between software engineering and big data systems are analyzed. The topics for the study of big data software ecosystems and big data system of systems are outlined.

Risk perception and crisis communication during the Covid-19 pandemic: Analysis based on Twitter hashtags

This research evaluates response strategies to crisis communication, in terms of perceived risks, in organizations during the Covid-19 pandemic by studying the main discussion topics in social media. The data was collected from Twitter between March and April 2020. By using big data software, a total number of 3559255 tweets in different languages were extracted worldwide from Twitter API of popular hashtags on the Covid-19 pandemic. The data processing was carried out through the association of terms in order to identify patterns and relationships in the discussion topics. The results indicate that the relationships of the terms "crisis" and "risks" were statistically significant with seven important topics for businesses, users, and consumers: "business", "economic and financial", "social"," health"," work"," family" and "government"; and in turn these seven topics are related to other terms related to the impact of the crisis, the response to the crisis, aid, the watch out, and support. This research has implications for the situational crisis communication theory by showing that in situations with high perceived risk, such as the Covid-19 pandemic crisis, the use of crisis response strategies predominates in organizations. This research also has implications for managers who can use crisis response strategies to rebuild their reputation and avoid market losses, thus helping to reduce the effects of unpredictable crisis situations.

Relevance of Big Data Analytics in Nigerian Academic Libraries: University of Ilorin Library Experience

This paper reviewed the concept of big data with reference to the University of Ilorin, North-Central, Nigeria. In the study that directed this paper, a descriptive design was employed and thematic analysis was done on the information obtained through key informant interviews. The population of this study consisted of the two systems librarians who work in the electronic library (e-library) unit of the university library. The characteristics of big data, based on the V characteristics, were examined, as well as the various types of data sets found in academic libraries. The findings of the study revealed that while big data sets exist in libraries in Nigeria, none of the big data software was available in the library studied. The findings also revealed the various methods used by the University of Ilorin Library in generating big data. Also, the main challenges confronting the management of big data in the University of Ilorin Library include inadequate funding, insufficient infrastructure, and inadequate software tools for managing big data. The need to re-skill library practitioners in this new technology, as a means of surmounting challenges in service delivery in university libraries, was recommended.

Topic and sentiment analysis of crisis communications about the COVID-19 pandemic in Twitter's tourism hashtags.

The purpose of this paper was to assess Twitter as a means of communication during tourism crises with the following objectives: (a) identify the topics that are discussed, (b) establish the text sentiment, and (c) determine the differences in gender regarding the topics under discussion and the text sentiment. The data were collected from Twitter between March and April 2020. Using big data software, this study extracted 123,868 tweets globally in different languages through the Twitter API of popular tourism hashtags. Two techniques were applied: word association and sentiment analysis. The results show that the communication made through Tweets has the characteristics of a crisis communication related to the effects of the COVID-19 pandemic in the tourism industry. The theoretical contribution of the research is that Twitter in social media is an effective means of communication during pandemic crises and contributes to reducing negative perceptions and adverse effects of the tourism crises in companies and destinations. The practical contribution of the research is that Twitter can be used as a means of communication helping the communication strategies of companies and organizations.

The Internet of Things and Architectures of Big Data Analytics: Challenges of Intersection at Different Domains

The current exponential advancements in the Internet of Things (IoT) technologies pave a vast intelligent computing platform by integrating smart objects with sensing, processing and communication capabilities. The core element of IoT is the complex big data generated from different interconnected sources at real-time, presenting divergent processing and analysis challenges. Best practices in software engineering have been continuously addressed in IoT technologies to handle such big data efficiently at different domains. Despite of the massive studies dedicated for IoT, no explicit processing architecture is proposed based on real investigation of software engineering concepts and big data analytics characteristics in IoT. This paper provides a systematic literature review for the current state-of-the-art of IoT systems in different domains. The study investigates the current techniques and technologies that serve IoT systems from the big data analytics and software engineering perspectives, revealing a matrix for the specific IoT data features and their encountered challenges and gaps for each domain. The review deduces a proposed domain-independent software architecture for big IoT data analytics, maintaining various IoT data processing challenges, including data scalability, timeliness, heterogeneity, inconsistency, confidentiality and correlations. Finally, the main research gaps are emphasized for future considerations.

Exploring parallel formal verification of BIG-DATA systems

Software Engineering is trying to adapt its tools, mechanisms and techniques to cope with the challenges involved when developing BIG DATA software systems. In particular, formal verification in one of the areas that more urgently is required to step in. In this work we introduce two crucial aspects to consolidate the FVS tool to tackle this issue. For one side, FVS’s parallel algorithm is proved to be sound and correct. For the other side, we developed a compelling empirical validation of our approach, employing a communication protocol relevant in the industrial world within a context of parallel systems, introducing a load-balancer process and comparing several implementations.

Data Mining Between Classical and Modern Applications: A Review

Data mining (DM) is an incredible innovation with extraordinary potential to help organizations centre around the main data in the information they have gathered about the conduct of their clients and likely clients. It finds data inside the information that inquiries and reports can't viably uncover. Overall, DM (to a great extent called information or data revelation) is the route toward analysing data according to substitute perspectives and summarizing it into significant information - information that can be used to assemble pay, diminishes costs, or both. DM writing computer programs is one of different logical gadgets for separating data. It grants customers to separate data from a wide scope of estimations or focuses, organize it, and summarize the associations perceived. In reality, DM is the path toward finding associations or models among numerous fields in enormous social datasets. Procedures used in DM measure come from a mix of computational strategies including Artificial Intelligence (AI), Statistics, Machine Learning (ML), and Database (DB) Systems. Aside from the centre techniques used to do the investigation, the cycle of DM can include different pre-handling ventures preceding executing the mining method. Also, a post-preparing stage is normally utilized to picture the aftereffects of the investigation (for example perceived examples or recovered data) in an instinctive and simple to-impart way. From a wide perspective, there are two significant standards of methods: expectation and information disclosure. It includes four sub-groups: a) Classification, Prediction and Regression, b) Clustering, c) Association Rule and Sequence Pattern Mining, and d) Outliers and Anomaly Detection. What's more, there are some generally new and energizing zones of information investigation, for example, spatial DM and graph DM that have been made conceivable through the structure squares of DM techniques. This survey not just advantages analyst to create solid examination subjects and distinguish gaps in the research areas yet additionally helps experts for data mining and Big Data (BD) software framework advancement.

An extended Meta Learning Approach for Automating Model Selection in Big Data Environments using Microservice and Container Virtualizationz Technologies

For a given specific machine learning task, very often several machine learning algorithms and their right configurations are tested in a trial-and-error approach, until an adequate solution is found. This wastes human resources for constructing multiple models, requires a data analytics expert and is time-consuming. Meta learning addresses these problems and supports non-expert users by recommending a promising learning algorithm based on meta features computed from a given dataset. In the present paper, a new concept for enhancing the predictive performance of meta learning classification models by generating new meta examples is introduced. Our concept is realized and evaluated in a microservice-based meta learning framework. This framework makes use of a powerful Big Data software stack, container visualization, modern web technologies and a microservice architecture. In this demonstration and for evaluation purpose, time series model selection is taken as a use case for applying meta learning. It is shown that the proposed microservice-based meta learning framework introduces an excellent performance in assigning the adequate forecasting model for the chosen time series datasets. Moreover, our new concept for generating new meta examples enhances the predictive performance of the meta learner up to 16.77% and 27.07% in the case of using the original and encoded representation forms of meta features respectively. The recommendation of the most appropriate forecasting model results in a well acceptable low framework overhead demonstrating that the framework can provide an efficient approach to solve the problem of model selection in the context of Big Data.

Challenges and Application Development of Industrial Big Data Software

Contemporarily, with the rapid development of science and technology, industrial big data software serves as an important content to realize artificial intelligence and improve the industrial Internet. However, there are enormous challenges confronted by industrial big data software in exorbitant cost for trial and error as well as in theoretical guidance for artificial intelligence. The goal of deep development of industrial big data software can be achieved only by steady development and mobilizing resources for key contents.

Strategy of Indonesian Navy Big Data Development Using SWOT Analysis and Analytic Hierarchy Process

The Indonesian Navy organization has the duties and responsibilities of state sovereignty. In carrying out its duties, very large and large data and real time data include data on ships in all Indonesian waters from satellite and terrestrial AIS, from radar, data from long range cameras, data from other agencies such as weather data, marine accident data, natural disasters data and fishing areas data. The managed data consists of structured, semi-structured and unstructured data. Thus, big data is needed within the Indonesian Navy. Therefore, this study analyzes the strategies that need to be taken to be able to develop a big data in the Indonesian Navy. The method used is qualitative analytic research using SWOT and AHP. The research subjects were Section Heads at command and control center, Head of Section at Information and Data Processing Services. Data With this combined method, the resulting main strategy is to implement Big Data with existing and adequate system resources, manageable costs, set up network infrastructure and sufficient bandwidth using available big data software by optimizing government regulations regarding single data for communication and data sharing from other agencies and optimizing advanced API technology.

Coastal and marine topics and destinations during the COVID-19 pandemic in Twitter's tourism hashtags

Coastal and marine tourism offer various topics discussed within social media communication during a pandemic. The present study analysed the discussion topics in the popular Twitter's tourism hashtags during the COVID-19 crisis related to coastal and marine tourism. The objectives were to (i) determine the discussion topics, (ii) identify the discussion topics of greater hierarchy, and (iii) determine the tourist destinations discussed. The data were collected from Twitter between March and April 2020. A total of 269,722 tweets were extracted in different languages and globally through the Twitter API of popular tourism hashtags about the COVID-19 pandemic using big data software. The word associations technique was used to process the Twitter data. The discussion topics were ‘travel’, ‘beach’, ‘cruise’, ‘sea’, ‘ocean’, ‘nature’, ‘sun’, ‘coast’, and others. The discussion topics of greatest hierarchy were ‘travel’, ‘beach’, ‘sea’, and ‘cruise’. These topics refer to the motivational dimensions‘ ‘escape’, and ‘sun and beach’. The geographic locations discussed were Spain, Canary, Hawaii, Australia, Caribbean, Bermuda, Florida, California, and others. The findings will contribute to the relationship of the topics with the tourist motivational dimensions, and the destinations discussed about marine and coastal tourism in the time of crisis in the COVID-2019 Pandemic in Twitter's tourism hashtags, and companies will also be able to improve their communication strategies and develop post-pandemic products.

Automatic Selection and Parameter Configuration of Big Data Software Core Components Based on Retention Pattern

This paper conducts an in-depth analysis and research on the automatic selection and parameter configuration of the core components of Big Data software by using the retention model and the automatic selection of Big Data components by establishing a standardized requirement index and using the decision tree model to solve the problem of component selection in Big Data application development. By establishing standardized demand indicators and based on the retention model, a data transmission intermediate platform for bidirectional data detection is proposed based on the three demands of user input: storage, computation, and analysis, as well as the problem of undetectable packet loss in data transmission of existing IoT and Web service platforms. The data communication module of the data transmission intermediate platform enables mutual monitoring and detection of data interaction between IoT smart terminals and cloud platforms. The retention mode is built separately to realize the automatic selection of Big Data components. In this paper, we start from several mainstream distributed storage systems and use Cassandra as an example for experiments and tests. We use the multiple regression fitting method to build a corresponding performance model for hardware parameters, take user requirements as input, and use the performance model to configure system hardware parameters; by studying its system principle, architecture, features, and application scenarios, we build a software parameter configuration knowledge base to guide the software. This solves the difficult problem of selecting, deploying, and configuring parameters for Big Data applications.

Big data reliability: A critical review

Big data requires new technologies and tools to process, analyze and interpret the vast amount of high-speed heterogeneous information. A simple mistake in processing software, error in data, and malfunctioning in hardware results in inaccurate analysis, compromised results, and inadequate performance. Thus, measures concerning reliability play an important role in determining the quality of Big data. Literature related to Big data software reliability was critically examined in this paper to investigate: the type of mathematical model developed, the influence of external factors, the type of data sets used, and methods employed to evaluate model parameters while determining the system reliability or component reliability of the software. Since the environmental conditions and input variables differ for each model due to varied platforms it is difficult to analyze which method gives the better prediction using the same set of data. Thus, paper summarizes some of the Big data techniques and common reliability models and compared them based on interdependencies, estimation function, parameter evaluation method, mean value function, etc. Visualization is also included in the study to represent the Big data reliability distribution, classification, analysis, and technical comparison. This study helps in choosing and developing an appropriate model for the reliability prediction of Big data software.

Enhanced Framework for Big Data Requirement Elicitation

Requirement engineering is one of the software development life cycle phases; it has been recognized as an important phase for collecting and analyzing a system’s goals. However, despite its importance, requirement engineering has several limitations such as incomplete requirements, vague requirements, lack of prioritization, and less user involvement, all of which affect requirement quality. With the emergence of big data technology, the complexity of big data, which is defined by large data volume, high velocity, and large data variety, has gradually increased, affecting the quality of big data software requirements. This study proposes a framework with four sequential phases to improve requirement engineering quality through big data software development. By integrating the proposed framework’s phases in which user requirements are collected in a complete vision using traditional requirement elicitation techniques with agile methodology and mind mapping, the collected requirements are displayed via a graphical representation using mind maps to achieve high requirement accuracy with connectivity and modifiability, enabling the accurate prioritization of requirements implemented using agile SCRUM methodology. The proposed framework improves requirement quality in big data software development, which is represented by accuracy, completeness, connectivity, and modifiability to understand the value of the collected requirements and effectively affect the quality of the implementation phase.

Freedom of Expression in Digital Age: An Analysis of Twitter in Context of Pak-China Relationship

This article is aimed to provide an analysis regarding the freedom of expression on Twitter in the digital age. Freedom of expression in the digital age is the capability of an individual through which they are able to express their beliefs, thoughts, ideas and emotions on various issues via different social media platforms that are free from governmental censorship. These freedoms play a significant role, as now, each individual can have his/her own perspectives and school of thoughts; and can live his/her life as per own choice. The present study is a discourse analysis of the Pak-China relations debate as top trend hashtags on Twitter in 2020. The data was collected through Mozdeh Big Data Software. The top twenty tweets with the highest likes in seven trending hashtags have been studied as per Searle's Speech Act Analysis. The study shows that how Twitter as a social media platform provides a forum of free debate for everyone.