Big Data Requirements Research Articles

Four-dimensional hemodynamic prediction of abdominal aortic aneurysms following endovascular aneurysm repair combining physics-informed PointNet and quadratic residual networks

Hemodynamic parameters can provide surveillance for the risk of complication of abdominal aortic aneurysms following endovascular aneurysm repair (EVAR). However, obtaining hemodynamic parameters through computational fluid dynamics (CFD) has disadvantages of complex operation and high computational costs. Recently proposed physics-informed neural networks offer novel solutions to solve these issues by leveraging fundamental physical conservation principles of fluid dynamics. Based on cardiovascular point datasets, we further propose an integration algorithm combining physics-informed PointNet and quadratic residual networks (PIPN-QN) that is capable of mapping sparse point clouds to four-dimensional hemodynamic parameters. The implemented workflow includes generating point cloud datasets through CFD simulation and dynamically reproducing the three-dimensional flow field in the spatial and temporal dimensions through deep learning. Compared with physics-informed PointNet (PIPN), the PIPN-QN reduces the mean square error of pressure and wall shear stress by around 32.1% and 33.1% and anticipates hemodynamic parameters in less than 2 s (14 400 times faster than CFD). To address the challenge of big data requirements, we quantify the universal flow field using a reduced number of supervision points, as opposed to the large number of point clouds generated from the CFD simulation. The PIPN-QN can meet the real-time hemodynamic parameters obtained from patients with abdominal aortic aneurysms following EVAR with higher accuracy, faster speed, and lower training costs.

Learning through Simulation: A Role-Based Learning Exercise to Equip Students in Big Data Requirement Elicitation Skills and Challenges

Bridging the gap between theoretical knowledge and practical application remains challenging in big data education. Our paper proposes a platform where students actively participate in a role-based learning (RBL) exercise designed to simulate real-world big data projects. This RBL exercise assigns each student participant to a business analyst role tasked with eliciting requirements or a customer role that provides those big data requirements. We conducted RBL sessions for postgraduate and undergraduate students learning big data analytics/data warehousing. Through role-playing in a collaborative environment, students face challenges such as unclear objectives, data privacy concerns, and evolving requirements. We evaluated the effectiveness of RBL simulations through brainstorming sessions, which verified that the students achieved the learning outcomes through RBL exercises. We also collected students’ feedback through a survey and found the RBL experience helped us understand the big data requirement elicitation skills, highlighting the significance of communication and collaboration skills. Further, we have evaluated students through a final exam question, and identified that students who participated in the RBL exercise outperformed in the big data requirement elicitation question. In summary, our research demonstrates that this RBL approach offers a valuable learning experience by enabling students to directly experience the complexities of big data requirement elicitation and identify the future requirements or challenges that encourage them to acquire the required skills.

Application of cloud computing platform in industrial big data processing

With the rapid growth and increasing complexity of industrial big data, traditional data processing methods are facing many challenges. This article takes an in-depth look at the application of cloud computing technology in industrial big data processing and explores its potential impact on improving data processing efficiency, security, and cost-effectiveness. The article first reviews the basic principles and key characteristics of cloud computing technology, and then analyzes the characteristics and processing requirements of industrial big data. In particular, this study focuses on the application of cloud computing in real-time data processing, predictive maintenance, and optimization, and demonstrates its practical effects through case studies. At the same time, this article also discusses the main challenges encountered during the implementation process, such as data security, privacy protection, performance and scalability issues, and proposes corresponding solution strategies. Finally, this article looks forward to the future trends of the integration of cloud computing and industrial big data, as well as the application prospects of emerging technologies such as artificial intelligence and machine learning in this field. The results of this study not only provide practical guidance for cloud computing applications in the industry, but also provide a basis for further research in academia.

Implementation of Artificial Intelligence, Machine Learning, and Internet of Things (IoT) in revolutionizing Agriculture: A review on recent trends and challenges

Artificial Intelligence (AI) can revolutionize agriculture which impacts a country’s economy, employing more than 30% of the world’s population directly or indirectly. It can fulfill the needs of an ever-growing world’s population through automation. Traditional farmland practices like weeding, pesticide spraying, irrigation, monitoring soil nutritional and moisture status, etc. can be performed quicker using robots, sensors, drones, and algorithms. It reduces water wastage and pesticide overuse, maintains soil fertility, helps in reducing labor and enhances crop yield and productivity despite world problems. However, its penetration into agriculture is still in its infancy due to its uneconomical nature, lack of expertise and big data requirement for accuracy among others. This paper delves deeper into the various applications and impacts of AI in agriculture, new tools being used, challenges and future scope related to this field. Combined with Artificial Neural Network (ANN) models and Machine Learning (ML), along with Expert systems (ES) and Internet of Things (IoT), AI can do wonders in agriculture in the subsequent years to come.

Load balancing and service discovery using Docker Swarm for microservice based big data applications

Big Data applications require extensive resources and environments to store, process and analyze this colossal collection of data in a distributed manner. Containerization with cloud computing provides a pertinent remedy to accommodate big data requirements, however requires a precise and appropriate load-balancing mechanism. The load on servers increases exponentially with increased resource usage thus making load balancing an essential requirement. Moreover, the adjustment of containers accurately and rapidly according to load as per services is one of the crucial aspects in big data applications. This study provides a review relating to containerized environments like Docker for big data applications with load balancing. A novel scheduling mechanism of containers for big data applications established on Docker Swarm and Microservice architecture is proposed. The concept of Docker Swarm is utilized to effectively handle big data applications' workload and service discovery. Results shows that increasing workloads with respect to big data applications can be effectively managed by utilizing microservices in containerized environments and load balancing is efficiently achieved using Docker Swarm. The implementation is done using a case study deployed on a single server and then scaled to four instances. Applications developed using containerized microservices reduces average deployment time and continuous integration.

Analysis on Data Governance Policy in the Health Sector of Zambia: Perspective of Big Data

The study explored the data governance policy in Zambia's health sector, focusing on big data. To fully realize the potential of data in our digital age, proper data governance is required. However, questions have been expressed concerning Zambia's health sector's ability to exploit data as a valuable resource due to the state of data governance. This study attempts to address this critical issue and shed light on the functions of big data governance aspects in the health sector. This work is significant because it has the potential to improve healthcare service quality, decision-making, and resource allocation through strong data governance. This study aims to bridge the gap between data governance and the requirements of big data in the Zambian health sector by reviewing existing policies, procedures, and practices. A mixed methods approach is used in the research, involving qualitative interviews with key informants and secondary data analysis. To get a full understanding of data governance difficulties, practices, and their alignment with big data demands, thematic and content analysis methodologies were used. The findings show that data governance is critical to managing data in the health sector, yet gaps in governance procedures impede the efficient implementation of ICT efforts. These concerns are exacerbated by challenges relating to competent human resources and ICT infrastructure. Accountability and transparency are also identified as areas for improvement. While this study offers insight on the existing state of affairs, it recognizes its limits and advocates for further research to investigate specific aspects of policy implementation. Nonetheless, this study contributes significantly to the field of big data and data governance in the Zambian health sector, encouraging stakeholders to work for an ICT driven, data-governed future that promotes openness, accountability, and improved healthcare results. Finally, this article emphasizes the importance of improving data governance procedures in Zambia's health sector, as well as the transformative potential of big data in healthcare as a catalyst for good change in the country's healthcare environment.

Identification of unknown faults in chemical processes using few-shot learning

This study investigated the application of deep learning for fault diagnosis of chemical processes involving unknown classification ability. A few-shot learning-based unknown recognition and classification (FSLB-UR&C) was presented to avoid the requirement of big data in training and to deal with the problem with small sample training data using the Tennessee Eastman Process (TEP). The model uses a gated recurrent unit with a dot-product attention layer to extract features and evaluate the feature similarity score with support and query data as inputs. Semi-sequential and random sampling methods were applied to enhance the training efficiency. The results showed that the FSLB-UR&C could distinguish known and unknown labels. Furthermore, t-SNE visualization proved that the model could gather unknown faults as a clustering, whereas known faults were clustered separately from the unknown faults. Compared with the other method, FSLB-UR&C can maintain a higher accuracy of the known classification to distinguish unknown faults.

Artificial intelligence and machine learning in pain research: a data scientometric analysis.

The collection of increasing amounts of data in health care has become relevant for pain therapy and research. This poses problems for analyses with classical approaches, which is why artificial intelligence (AI) and machine learning (ML) methods are being included into pain research. The current literature on AI and ML in the context of pain research was automatically searched and manually curated. Common machine learning methods and pain settings covered were evaluated. Further focus was on the origin of the publication and technical details, such as the included sample sizes of the studies analyzed with ML. Machine learning was identified in 475 publications from 18 countries, with 79% of the studies published since 2019. Most addressed pain conditions included low back pain, musculoskeletal disorders, osteoarthritis, neuropathic pain, and inflammatory pain. Most used ML algorithms included random forests and support vector machines; however, deep learning was used when medical images were involved in the diagnosis of painful conditions. Cohort sizes ranged from 11 to 2,164,872, with a mode at n = 100; however, deep learning required larger data sets often only available from medical images. Artificial intelligence and ML, in particular, are increasingly being applied to pain-related data. This report presents application examples and highlights advantages and limitations, such as the ability to process complex data, sometimes, but not always, at the cost of big data requirements or black-box decisions.

Hybrid approach for flood susceptibility assessment in a flood-prone mountainous catchment in China

• We developed a hybrid approach integrating physical-based flood models with random forest for flood susceptibility assessment. • We identified the relatively significant flood-inducing factors. • The geographical and hydromorphological factors had greater effects on riverine flooding. • This flood susceptibility assessment is suitable in ungauged areas. Flooding is one of the most destructive natural hazards that has caused catastrophic effects worldwide. Recently, machine learning methods have become widespread in flood susceptibility assessments due to their efficiency and robustness. However, flood inventory data acquired from historical flooding records and field surveys in existing studies are sparse and not enough to develop accurate machine learning models. Thus, we developed a hybrid approach that integrates hydrodynamic model (HEC-HMS/RAS), rapid flood model (height above the nearest drainage, HAND) and machine learning model (random forest, RF) for flood susceptibility assessment. The proposed approach was then implemented in the Xinluo watershed, a flood-prone mountainous catchment in China. The results showed that the spatially continuous flood inventory map generated from HEC-HMS/RAS model satisfies the big data requirements of the RF model, and the proposed hybrid approach demonstrates the effectiveness and efficiency in flood susceptibility assessment with overall accuracy and area under the ROC curve values of 0.915 and 0.912, respectively. The geographical and hydromorphological factors had greater effects on riverine flooding with average contribution of 27.8% and 27.6%. In comparison, the average contribution of land cover and meteorological factors were 24.3% and 20.3%, respectively. The method proposed in this study is effective in assessing and forecasting flood susceptibility and can facilitate resilience planning as well as integrated flood management.

Optimal Strategy on Radiation Estimation for Calculating Universal Thermal Climate Index in Tourism Cities of China.

The Universal Thermal Climate Index (UTCI) is believed to be a very powerful tool for providing information on human thermal perception in the domain of public health, but the solar radiation as an input variable is difficult to access. Thus, this study aimed to explore the optimal strategy on estimation of solar radiation to increase the accuracy in UTCI calculation, and to identify the spatial and temporal variation in UTCI over China. With daily meteorological data collected in 35 tourism cities in China from 1961 to 2020, two sunshine-based Angstrom and Ogelman models, and two temperature-based Bristow and Hargreaves models, together with neural network and support vector machine-learning methods, were tested against radiation measurements. The results indicated that temperature-based models performed the worst with the lowest NSE and highest RMSE. The machine-learning methods performed better in calibration, but the predictive ability decreased significantly in validation due to big data requirements. In contrast, the sunshine-based Angstrom model performed best with high NSE (Nash–Sutcliffe Efficiency) of 0.84 and low RMSE (Root Mean Square Error) of 35.4 J/m2 s in validation, which resulted in a small RMSE of about 1.2 °C in UTCI calculation. Thus, Angstrom model was selected as the optimal strategy on radiation estimation for UTCI calculation over China. The spatial distribution of UTCI showed that days under no thermal stress were high in tourism cities in central China within a range from 135 to 225 days, while the largest values occurred in Kunming and Lijiang in southwest China. In addition, days under no thermal stress during a year have decreased in most tourism cities of China, which could be attributed to the asymmetric changes in significant decrease in frost days and slightly increase in hot days. However, days under no thermal stress in summer time have indeed decreased, accompanying with increasing days under strong stress, especially in the developed regions such as Yangze River Delta and Zhujiang River Delta. Based on the study, we conclude that UTCI can successfully depict the overall spatial distribution and temporal change of the thermal environments in the tourism cities over China, and can be recommend as an efficient index in the operational services for assessing and predicting thermal perception for public health. However, extreme cold and heat stress in the tourism cities of China were not revealed by UTCI due to mismatch of the daily UTCI with category at hourly scale, which makes it an urgent task to redefine category at daily scale in the next research work.

Ultrafast flash memory with large self-rectifying ratio based on atomically thin MoS2-channel transistor

Abstract Flash memory with high operation speed and stable retention performance is in great demand to meet the requirements of big data. In addition, the realisation of ultrafast flash memory with novel functions offers a means of combining heterogeneous components into a homogeneous device without considering impedance matching. This report proposes a 20 ns programme flash memory with 108 self-rectifying ratios based on a 0.65 nm-thick MoS2-channel transistor. A high-quality van der Waals heterojunction with a sharp interface is formed between the Cr/Au metal floating layer and h-BN tunnelling layer. In addition, the large rectification ratio and low ideality factor (n = 1.13) facilitate the application of the MoS2-channel flash memory as a bit-line select transistor. Finally, owing to the ultralow MoS2/h-BN heterojunction capacitance (50 fF), the memory device exhibits superior performance as a high-frequency (up to 1 MHz) sine signal rectifier. These results pave the way toward the potential utilisation of multifunctional memory devices in ultrafast two-dimensional NAND-flash applications.

Liquid Software-Based Edge Intelligence for Future 6G Networks

The 6G wireless network is promising to build bridges toward smart society in the digital world, which calls for innovative architectures and new solutions. The future 6G network should be sensing-based and data-driven for near-instant and massive connectivity with distributed intelligence. With a majority of intelligent applications being deployed at the edge, artificial intelligence (AI) is envisioned to play a key role in satisfying key requirements of 6G networks. Edge intelligence, as the marriage of AI and edge computing, is envisioned to fully meet the potential requirements of edge big data with energy, bandwidth, storage, and privacy concerns. However, it is an attractive issue to deal with distributed edge intelligence for the complexities and heterogeneous requirements, especially considering the time-varying channels and network dynamics. Furthermore, the ever increasing number of smart devices present great challenges for intelligent network management and newly modular network design in 6G networks, which needs to enable liquid self-management with comprehensive network intelligence. Hence, in this article, we first comprehensively give an overview on AI toward 6G networks, and characterize the requirements of a 6G network for AI applications. In particular, we investigate distributed edge intelligence challenges, requirements, and trends in future 6G networks. Then a liquid-specific and flexible software-defined network architecture for AI applications is inspired and discussed by 6G networks, which will play a crucial role in both academia and industry.

Physics-Based Deep Learning for Flow Problems

It is the tradition for the fluid community to study fluid dynamics problems via numerical simulations such as finite-element, finite-difference and finite-volume methods. These approaches use various mesh techniques to discretize a complicated geometry and eventually convert governing equations into finite-dimensional algebraic systems. To date, many attempts have been made by exploiting machine learning to solve flow problems. However, conventional data-driven machine learning algorithms require heavy inputs of large labeled data, which is computationally expensive for complex and multi-physics problems. In this paper, we proposed a data-free, physics-driven deep learning approach to solve various low-speed flow problems and demonstrated its robustness in generating reliable solutions. Instead of feeding neural networks large labeled data, we exploited the known physical laws and incorporated this physics into a neural network to relax the strict requirement of big data and improve prediction accuracy. The employed physics-informed neural networks (PINNs) provide a feasible and cheap alternative to approximate the solution of differential equations with specified initial and boundary conditions. Approximate solutions of physical equations can be obtained via the minimization of the customized objective function, which consists of residuals satisfying differential operators, the initial/boundary conditions as well as the mean-squared errors between predictions and target values. This new approach is data efficient and can greatly lower the computational cost for large and complex geometries. The capacity and generality of the proposed method have been assessed by solving various flow and transport problems, including the flow past cylinder, linear Poisson, heat conduction and the Taylor–Green vortex problem.

Review of Big Data Integration in Construction Industry Digitalization

The 2030 agenda for sustainable development has embraced the importance of sustainable practices in the construction industry. Parallel to the Industry revolution 4.0, the construction industry needs to keep pace with technological advances in data management to keep pace with the revolution through the ability to process and extract value from data. This phenomenon attracts the requirement of Big Data (BD). The construction industry deals with large volumes of heterogeneous data, which is expected to increase exponentially following an intense use of modern technologies. This research presents a comprehensive study of the literature, investigating the potential application of BD integration in the construction industry. The adoption of such technologies in this industry remains at a nascent stage and lags broad uptake of these technologies in other fields. The Construction Industry is driving to boost its productivity through the implementation of data technologies; hence, significant research is needed in this area. Currently, there is a lack of deep comprehensive research on BD integration applications that provide insight for the construction industry. This research closes the gap and gives an overview of the literature. The discussion presented the current utilization, the issues, and ways for potential works along with the challenges companion with the implementation.

Current Trends in Cloud Computing for Data Science Experiments

Recent trends in data-intensive experiments require extensive computing and storage resources that are now handled using cloud resources. Industry experts and researchers use cloud-based services and resources to get analytics of their data to avoid inter-organizational issues including power overhead on local machines, cost associated with maintaining and running infrastructure, etc. This article provides detailed review of selected metrics for cloud computing according to the requirements of data science and big data that includes (1) load balancing, (2) resource scheduling, (3) resource allocation, (4) resource sharing, and (5) job scheduling. The major contribution of this review is the inclusion of these metrics collectively which is the first attempt towards evaluating the latest systems in the context of data science. The detailed analysis shows that cloud computing needs research in its association with data-intensive experiments with emphasis on the resource scheduling area.

Research on the Optimal Deployment and Scheduling Strategy of Big Data Application in Cloud Computing Platform

With the progress and development of science and technology and society, all walks of life in China have started to actively apply big data. Big data has become the most important part and plays an indispensable role in the development of today’s society. In recent years, in the application of big data based on cloud computing platform, the flexibility of big data has been fully reflected in the resource allocation. At the same time, in the actual application of big data, cloud computing platform can make corresponding reflection based on the requirements of big data for relevant platforms. Moreover, the charging behavior based on cloud service will be carried out in accordance with the actual demand. For users in today’s era, this is completely in line with their consumption pattern, especially for small and medium-sized users in today’s market. The advantages and characteristics of this applicable business are particularly prominent. However, in the application process of big data technology based on cloud computing platform, there are still some problems that need our attention. This paper makes an in-depth discussion on the performance of big data in the cloud computing platform to make a brief analysis on the optimization deployment and scheduling strategy of big data application in the cloud computing platform.

Research on System Data Security Under Network Technology

Today’s society is a digital society, all data can be stored digitally, which leads to the explosion of data growth. However, the security storage of cloud data is the primary problem that restricts the development of cloud storage. Malicious hackers, software and hardware vulnerabilities and so on will pose a serious threat to users’ data stored in the cloud. This paper combines the typical big data system technical framework and builds the big data security technical framework around the security requirements of big data. Under this framework, the research status of key technologies of big data security is systematically summarized from multiple aspects, including the main security mechanisms involved in the technical framework of big data system. Finally, the core problems and development trends of big data security technology to be solved are summarized.

An Empirical Study on Transmission Performance of IPv6 over WAN Latency

Compared to IPv4, IPv6 provides more internet protocol (IP) addresses and higher security and has easier routing capability; and these advantages lead to tendency of the IP protocol used by a network to be upgraded from IPv4 to IPv6. With the global transmission requirements of Big Data increasing rapidly, wide area network (WAN) will play a crucial role in data transfer, but majority of previous research studies are limited to local area network (LAN). Therfore, in this study, a WAN experiment is implemented to estimate the transmission control protocol (TCP) throughput of IPv4 and IPv6, which is used as an index to evaluate the efficiency of data transmission. Four latest operating systems were tested to assess performance, and a pair of operating systems was chosen to estimate the transmission efficiency of WAN by time delay. In this experiment, performance degradation in latency was quantified. The results will be a useful reference for followup research in the future.

Importance of Process Mining for Big Data Requirements Engineering

Requirements engineering (RE), as a part of the project development life cycle, has increasingly been recognized as the key to ensure on-time, on-budget, and goal-based delivery of software projects. RE of big data projects is even more crucial because of the rapid growth of big data applications over the past few years. Data processing, being a part of big data RE, is an essential job in driving big data RE process successfully. Business can be overwhelmed by data and underwhelmed by the information so, data processing is very critical in big data projects. Employing traditional data processing techniques lacks the invention of useful information because of the main characteristics of big data, including high volume, velocity, and variety. Data processing can be benefited by process mining, and in turn, helps to increase the productivity of the big data projects. In this paper, the capability of process mining in big data RE to discover valuable insights and business values from event logs and processes of the systems has been highlighted. Also, the proposed big data requirements engineering framework, named REBD, helps software requirements engineers to eradicate many challenges of big data RE.

Importance of Process Mining for Big Data Requirements Engineering

Importance of Process Mining for Big Data Requirements Engineering