Distributed Data Collection Research Articles

Federated Learning of Generalized Linear Causal Networks.

Causal discovery, the inference of causal relations among variables from data, is a fundamental problem of science. Nowadays, due to an increased awareness of data privacy concerns, there has been a shift towards distributed data collection, processing and storage. To meet the pressing need for distributed causal discovery, we propose a novel federated DAG learning method called distributed annealing on regularized likelihood score (DARLS) to learn a causal graph from data stored on multiple clients. DARLS simulates an annealing process to search over the space of topological sorts, where the optimal graphical structure compatible with a sort is found by distributed optimization. This distributed optimization relies on multiple rounds of communication between local clients and a central server to estimate the graphical structure. We establish its convergence to the solution obtained by an oracle with access to all the data. To the best of our knowledge, DARLS is the first distributed method for learning causal graphs with such finite-sample oracle guarantees. To establish the consistency of DARLS, we also derive new identifiability results for causal graphs parameterized by generalized linear models, which could be of independent interest. Through extensive simulation studies and a real-world application, we show that DARLS outperforms existing federated learning methods and is comparable to oracle methods on pooled data, demonstrating its great advantages in estimating causal networks from distributed data.

Cost-Effective Scalability in Cloud Monitoring Systems: A Comparative Stud

This research article explores strategies for cost-effective scalability in cloud monitoring systems. As the complexity and scale of IT-based infrastructure are growing, an efficient monitoring system becomes vital for maintaining performance and optimizing resources while managing costs. The current study aims to shed light on various approaches to cost-effective scalability by considering factors such as data collection methods, storage optimization, and adaptive monitoring techniques. A comparison between cloud monitoring tools, Amazon Cloud Watch and Datadog, has been made to gain a better understanding of the monitoring tools. The research indicates that a multi-faceted approach is necessary for cost-effective scalability in cloud monitoring systems, and there must be a holistic approach in the selection of cloud monitoring tools, depending on the organization's requirements. In the later sections, strategies, like distributed data collection, hierarchical aggregation, adaptive sampling, and machine learning- based predictive scaling, can significantly improve monitoring system scalability while optimizing resource utilization.

Life, death, and statins: association of statin prescriptions and survival in older general practice patients.

This study serves as an exemplar to demonstrate the scalability of a research approach using survival analysis applied to general practice electronic health record data from multiple sites. Collection of these data, the subsequent analysis, and the preparation of practice-specific reports were performed using a bespoke distributed data collection and analysis software tool. Statins are a very commonly prescribed medication, yet there is a paucity of evidence for their benefits in older patients. We examine the relationship between statin prescriptions for general practice patients over 75 and all-cause mortality. We carried out a retrospective cohort study using survival analysis applied to data extracted from the electronic health records of five Australian general practices. The data from 8025 patients were analysed. The median duration of follow-up was 6.48 years. Overall, 52015 patient-years of data were examined, and the outcome of death from any cause was measured in 1657 patients (21%), with the remainder being censored. Adjusted all-cause mortality was similar for participants not prescribed statins versus those who were (HR 1.05, 95% CI 0.92-1.20, P = 0.46), except for patients with diabetes for whom all-cause mortality was increased (HR = 1.29, 95% CI: 1.00-1.68, P = 0.05). In contrast, adjusted all-cause mortality was significantly lower for patients deprescribed statins compared to those who were prescribed statins (HR 0.81, 95% CI 0.70-0.93, P < 0.001), including among females (HR = 0.75, 95% CI: 0.61-0.91, P < 0.001) and participants treated for secondary prevention (HR = 0.72, 95% CI: 0.60-0.86, P < 0.001). This study demonstrated the scalability of a research approach using survival analysis applied to general practice electronic health record data from multiple sites. We found no evidence of increased mortality due to statin-deprescribing decisions in primary care.

Pengaruh Kualitas Layanan &amp; Citra Merek Terhadap Loyalitas Pelanggan Melalui Kepuasan Pelanggan di Nagadigit

This study aims to identify the effect of service quality &amp; brand image on customer loyalty through Nagadigit customer satisfaction. This study uses a quantitative method using SPSS. This research technique uses survey research techniques with online questionnaire distribution data collection methods. The sampling technique used was the Slovin technique with 94 respondents. This study uses validity, reliability, classical assumptions and hypotheses. In the hypothesis test, the results of the path analysis test conducted show that the effect of service quality (X1) and brand image (X2) on customer loyalty (Z) through customer satisfaction (Y) has a smaller coefficient when compared to the effect of service quality (X1) and brand image (X2) on customer loyalty (Z) directly, so that customer satisfaction (Y) cannot be an intervening variable between service quality (X1) and brand image (X2) on customer loyalty (Z).

Collecting Multi-type and Correlation-Constrained Streaming Sensor Data with Local Differential Privacy

Local differential privacy (LDP) is a promising privacy model for distributed data collection. It has been widely deployed in real-world systems (e.g. Chrome, iOS, macOS). In LDP-based mechanisms, an aggregator collects private values perturbed by each user and then analyses these values to estimate their statistics, such as frequency and mean. Most existing works focus on simple scalar value types, such as boolean and categorical values. However, with the emergence of smart sensors and Internet of Things, high-dimensional data are gaining increasing popularity. In many cases where more than one type of sensor data are collected simultaneously, correlations exist between various attributes of such data, e.g. temperature and luminance. To ensure LDP for high-dimensional data, existing solutions either partition the privacy budget ϵ among these correlated attributes or adopt sampling, both of which dilute the density of useful information and thus result in poor data utility. In this paper, we propose a relaxed LDP model, namely, univariate dominance local differential privacy (UDLDP), for high-dimensional data. We quantify the correlations between attributes and present a correlation-bounded perturbation (CBP) mechanism that optimizes the partitioning of privacy budget on each correlated attribute. Furthermore, we extend CBP to support sampling, which is a common bandwidth reduction technique in sensor networks and Internet of Things. We derive the best allocation strategy of sampling probabilities among attributes in terms of data utility, which leads to the correlation-bounded perturbation mechanism with sampling (CBPS). Finally, we discuss how to collect and leverage the correlation from real-time data stream with a by-round algorithm to enhance the utility. The performance of the proposed mechanisms is evaluated and compared with state-of-the-art LDP mechanisms on real-world and synthetic datasets.

디지털 농업을 위한 딥러닝 기반의 환경 인자 추천 기술 연구

Smart Farm means creating new value in various fields related to agriculture, including not only agricultural production but also distribution and consumption through the convergence of agriculture and ICT. In Korea, a rental smart farm is created to spread smart agriculture, and a smart farm big data platform is established to promote data collection and utilization. It is pushing for digital transformation of agricultural products distribution from production areas to consumption areas, such as expanding smart APCs, operating online exchanges, and digitizing wholesale market transaction information. As such, although agricultural data is generated according to characteristics from various sources, it is only used as a service using statistics and standardized data. This is because there are limitations due to distributed data collection from agriculture to production, distribution, and consumption, and it is difficult to collect and process various types of data from various sources. Therefore, in this paper, we analyze the current state of domestic agricultural data collection and sharing for digital agriculture and propose a data collection and linkage method for artificial intelligence services. And, using the proposed data, we propose a deep learning-based environmental factor recommendation method.

Decentralized E-KYC Blockchain Network for Central Bank

Abstract: Know Your Customer or KYC processes are the backbones of a financial institution’s anti-money laundering efforts. KYC processes are mandatory in most of Europe and India. Almost all countries have their own process for identifying and keeping a record of their citizens like the USA has Social Security Protocol. According to current estimates, the amount of KYC spending rose to up to $1.8 Billion in 2021 on a global level. Despite the importance of the process, KYC continues to operate inefficiently. KYC processes are labor-intensive and time-consuming tasks. It is estimated that 80% of KYC efforts go in gathering information and processing while only 20% of efforts are evaluating and monitoring focused. This centralization of data is causing an inefficient KYC process and creating issues like :- Misidentification of fraudulent data , Inability of tracking customers, customers entering fake data(address, age etc) , delayed processing time and very high cost of processing. Reimagining KYC Using Blockchain Technology will enable seamless and secure data exchange at a fraction of the cost. A decentralized blockchain network will Facilitate near real-time data exchange. Blockchain when used with other technologies can indicate great potential to help organizations reduce the cost and time linked with the KYC process. The introduction of blockchain in KYC brings data on a decentralized network which also has its own pros like distributed data collection ,higher operational efficiency , validation of accuracy of the information. The customer will build their profile on the DCT KYC network. There are multiple options to store the data like Centralized encrypted servers, DCT platforms like IPFS(Interplanetary File System). Customers will perform transactions with Financial Institutions by giving them access to users profile( restricted non-appendable access),Using hashing and cryptographic processes the data would safely be called and returned to the DCT network . Thus making sure the whole process is immutable .

Decentralized privacy-preserving truth discovery for crowd sensing

Truth discovery is an efficient technique for tackling data conflict problems in crowd sensing for distributed data collection. As the sensory data to be collected may include sensitive information about users, privacy-preserving truth discovery has attracted significant attention in recent years. Most existing studies apply a centralized architecture based on a cryptographic system, which may be vulnerable to single-point attacks and also has a very high computational cost. In this paper, we propose DPriTD, a decentralized privacy-preserving framework for truth discovery in crowd sensing. The proposed approach leverages the additively homomorphic property of Shamir's Secret Sharing scheme to protect user's privacy. DPriTD provides a strict privacy guarantee for crowd sensing applications. Because each sensitive data point, considered to be a secret, is split into a batch of shares, and the secret cannot be recovered unless a sufficient number of shares are aggregated, DPriTD achieves effective truth discovery while protecting sensitive data from collusion attacks. Furthermore, DPriTD is independent of a centralized server and can perform reliably when not all participants are online in real time. It thus enhances the robustness of a crowd sensing system. Extensive experiments conducted on real-world datasets demonstrate the high performance of our method compared with existing mechanisms.

Pedestrian Behaviour at Unsignalized Zebra Crossing: A Case Study in UMP Gambang Campus

The interest toward promoting walking culture has been increased dramatically especially in many cities across the nation. This is also included universities worldwide that have started seeking ways to increase pedestrian activities. Hence, campus planners must address the mobility and accessibility needs of pedestrian in their communities to ensure safety, functionality and conducive living and learning environment. This study was conducted to evaluate the pedestrian perception and behaviour towards the unsignalized zebra crossing in campus environment. To achieve the objective of this study, the unsignalized zebra crossing in UMP Gambang campus was selected as study location. This study was conducted using a quantitative study by means of questionnaires distribution and pedestrian movement data collection. Then Average Index Method was performed to indicate the pedestrian perception towards the crossing facility performances. The result shows that the zebra crossing was efficient and safe from the respondent’s perspective. Other than that, gender and platoon significantly influenced the crossing speeds. The outcomes from this study were hoped to bring some understanding to the university on the pedestrian’s behaviour for future planning and pedestrian safety.

Distributed Data Collection Algorithm With Unknown Objective Function in Adaptive Duty-Cycle WSNs

With the rapid development of wireless sensor networks (WSNs), higher requirements are raised for the flexibility and applicability of data collection algorithms. Therefore, a novel distributed algorithm for data collection in adaptive duty-cycle WSNs is proposed in this article. On the one hand, the priority in data transmission is considered, in which the queues are used to realize the data storage and data transmission with different priorities and then a scheme for selecting the optimal transmission path for the sensor nodes is developed to ensure that high-priority data can be transmitted more efficiently. Next, a method that can adaptively adjust the duty cycle of the sensor nodes is designed to reduce the transmission delay, where each sensor can update its own duty cycle according to the amount of data with different priorities. On the other hand, the relation between the objective function and the variables in the data collection problem may be difficult to establish accurately or even unknown due to the complexity of WSNs, while the most existing data collection algorithms require the objective function to be explicitly expressed, making them inapplicable to the above scenarios. Hence, the data collection problem with unknown objective function in adaptive duty-cycle WSNs is also investigated in this article. Since the objective function cannot be established, a distributed optimization algorithm with randomized gradient-free is developed to solve that problem. In particular, the proposed algorithm solely relies on the row-stochastic weight matrix, where each sensor needs to communicate only with its neighbors to make decision and locally allocates the information weight of its neighbors, making the algorithm easier to implement. Under some reasonable assumptions, the explicit analysis for the convergence of the proposed algorithm is provided. Finally, the convergence and effectiveness of the proposed algorithm are substantiated through numerical simulations.

Performance Enhancement in Federated Learning by Reducing Class Imbalance of Non-IID Data.

Due to the distributed data collection and learning in federated learnings, many clients conduct local training with non-independent and identically distributed (non-IID) datasets. Accordingly, the training from these datasets results in severe performance degradation. We propose an efficient algorithm for enhancing the performance of federated learning by overcoming the negative effects of non-IID datasets. First, the intra-client class imbalance is reduced by rendering the class distribution of clients close to Uniform distribution. Second, the clients to participate in federated learning are selected to make their integrated class distribution close to Uniform distribution for the purpose of mitigating the inter-client class imbalance, which represents the class distribution difference among clients. In addition, the amount of local training data for the selected clients is finely adjusted. Finally, in order to increase the efficiency of federated learning, the batch size and the learning rate of local training for the selected clients are dynamically controlled reflecting the effective size of the local dataset for each client. In the performance evaluation on CIFAR-10 and MNIST datasets, the proposed algorithm achieves 20% higher accuracy than existing federated learning algorithms. Moreover, in achieving this huge accuracy improvement, the proposed algorithm uses less computation and communication resources compared to existing algorithms in terms of the amount of data used and the number of clients joined in the training.

Coscheduling Spatial Self-organization and Distributed Data Collection in Multi-agent System

The problem of coscheduling spatial self-organization control processes and distributed data collection processes in a multi-agent system has been considered. The goal of coscheduling is to find and use the possibilities of functional coordination of these processes and increase the efficiency of the multi-agent system due to their parallel execution. An analysis of the main features of spatial self-organization tasks that affect the solution of the problem of coscheduling has been carried out. Variants of the mobile agent robotic platform configuration and the problem of the dependence of spatial self-organization algorithms on the type of robotic platform have been considered. A method of coscheduling of spatial self-organization and distributed data collection by coordinated parallel execution of the corresponding data collection process and the process of controlling mobile agent motion has been proposed. The method of coscheduling is implemented using the interaction protocol of these processes and the algorithm for planning their parallel execution using functional decomposition. The simulation results of the proposed method of coscheduling are given. It is proved that the proposed method of coscheduling provides acceleration of computations in the decision-making module of the mobile agent due to more efficient parallelization. On average, for typical values of parameters of control processes, the proposed method of coscheduling provides acceleration of computations in the decision-making module of the mobile agent by 40.6 %.

Exploring Environmental and Geographical Factors Influencing the Spread of Infectious Diseases with Interactive Maps

Environmental problems due to human activities such as deforestation, urbanisation, and large scale intensive farming are some of the major factors behind the rapid spread of many infectious diseases. This in turn poses significant challenges not only in as regards providing adequate healthcare, but also in supporting healthcare workers, medical researchers, policy makers, and others involved in managing infectious diseases. These challenges include surveillance, tracking of infections, communication of public health knowledge and promotion of behavioural change. Behind these challenges lies a complex set of factors which include not only biomedical and population health determinants but also environmental, climatic, geographic, and socioeconomic variables. While there is broad agreement that these factors are best understood when considered in conjunction, aggregating and presenting diverse information sources requires effective information systems, software tools, and data visualisation. In this article, we argue that interactive maps, which couple geographical information systems and advanced information visualisation techniques, provide a suitable unifying framework for coordinating these tasks. Therefore, we examine how interactive maps can support spatial epidemiological visualisation and modelling involving distributed and dynamic data sources and incorporating temporal aspects of disease spread. Combining spatial and temporal aspects can be crucial in such applications. We discuss these issues in the context of support for disease surveillance in remote regions, utilising tools that facilitate distributed data collection and enable multidisciplinary collaboration, while also providing support for simulation and data analysis. We show that interactive maps deployed on a combination of mobile devices and large screens can provide effective means for collection, sharing, and analysis of health data.

Distributed and Privacy Preserving Graph Data Collection in Internet of Thing Systems

Internet of Thing (IoT) systems have been treated as a novel platform for graph data acquisition. Contents like dynamic network topology, organization and control flows, and interactions among monitored objects all contribute to the huge volumes of graph data generated in IoT. These data are believed to brought significant benefits to both the operation and functionalities of IoT systems, especially when combined with cutting-edge Artificial Intelligence techniques. However, these graph data are usually locally collected by data contributors with sensing devices, which could be both partially overlapped as they record same environment, and sensitive as they can indicate private physical status of contributors. Considering all challenges, current solutions for graph data collection in IoT are incapable. Therefore, this article proposes a novel framework for privacy-preserving distributed graph data collection for IoT. The framework allows the graphs kept by data contributors to be partially overlapped, and can help the data broker to efficiently derive the universal view by combining these graphs. The differential privacy is applied for privacy preservation during data collection. The proposed problem aims at minimizing the total bandwidth consumption for graph collection, which is proved to be NP-complete. Then three algorithms are proposed for different circumstances, based on the diverse knowledge and purposes held by the data broker. Finally, both theoretical and numerical analysis have demonstrated the advancement of these methods.

Using big data and federated learning for generating energy efficiency recommendations

Internet of Things (IoT) devices are becoming popular solutions for smart home and office environments and contribute the most to energy efficiency. The most common implementation of such solutions relies on smart home systems that are hosted on the cloud. They collect data from a multitude of sensors, process it in real-time on the cloud and deliver immediate actions to sets of actuators that are installed locally. In this work, we present the (EM) $$^3$$ project (Consumer Engagement towards Energy Saving Behaviour by Means of Exploiting Micro Moments and Mobile Recommendation Systems), which combines data collection, information abstraction, timed recommendations for energy saving actions and automations that promote energy saving in a household or office setup. The advantage of the (EM) $$^3$$ project is that each room or office setup is controlled locally on an edge device, thus removing the need to share data to the cloud. The current article details on the data and information processing aspects of the (EM) $$^3$$ solution, which efficiently handles thousands of sensor events on a daily basis and provides useful analytics and recommendations to the end user to support habit change. It also demonstrates the scalability of the solution by simulating a simple scenario of distributed data collection and processing on the edge nodes, which takes advantage of federated learning in order to adapt to the needs of multiple users without exposing their privacy.

Supporting Remote Experimentation Through Computer-Mediated Communication

With the emergence of COVID-19 and the resultant reduction in direct personal contact, we were faced with a novel challenge of how to conduct studies that involved both semi-structured elicitation of detailed cognitive information and the evaluation of software interfaces and algorithms. We leveraged distributed communication technologies to support data collection across three separate human-computer interaction studies. We describe these studies in terms of specific computer-mediated communication methods and features and how we used them. We also present lessons learned, potential risks and detriments of using such approaches, (e.g., connectivity dependence, reduced ability to observe non-verbal cues, and diminished interpersonal interaction), along with how the research community can compensate for the shortcomings of distributed data collection in human-centered research.

TrackInk: An IoT-Enabled Real-Time Object Tracking System in Space

Nowadays, there is tremendous growth in the Internet of Things (IoT) applications in our everyday lives. The proliferation of smart devices, sensors technology, and the Internet makes it possible to communicate between the digital and physical world seamlessly for distributed data collection, communication, and processing of several applications dynamically. However, it is a challenging task to monitor and track objects in real-time due to the distinct characteristics of the IoT system, e.g., scalability, mobility, and resource-limited nature of the devices. In this paper, we address the significant issue of IoT object tracking in real time. We propose a system called ‘TrackInk’ to demonstrate our idea. TrackInk will be capable of pointing toward and taking pictures of visible satellites in the night sky, including but not limited to the International Space Station (ISS) or the moon. Data will be collected from sensors to determine the system’s geographical location along with its 3D orientation, allowing for the system to be moved. Additionally, TrackInk will communicate with and send data to ThingSpeak for further cloud-based systems and data analysis. Our proposed system is lightweight, highly scalable, and performs efficiently in a resource-limited environment. We discuss a detailed system’s architecture and show the performance results using a real-world hardware-based experimental setup.

Passive citizen science: The role of social media in wildlife observations.

Citizen science plays an important role in observing the natural environment. While conventional citizen science consists of organized campaigns to observe a particular phenomenon or species there are also many ad hoc observations of the environment in social media. These data constitute a valuable resource for 'passive citizen science'-the use of social media that are unconnected to any particular citizen science program, but represent an untapped dataset of ecological value. We explore the value of passive citizen science, by evaluating species distributions using the photo sharing site Flickr. The data are evaluated relative to those submitted to the National Biodiversity Network (NBN) Atlas, the largest collection of species distribution data in the UK. Our study focuses on the 1500 best represented species on NBN, and common invasive species within UK, and compares the spatial and temporal distribution with NBN data. We also introduce an innovative image verification technique that uses the Google Cloud Vision API in combination with species taxonomic data to determine the likelihood that a mention of a species on Flickr represents a given species. The spatial and temporal analyses for our case studies suggest that the Flickr dataset best reflects the NBN dataset when considering a purely spatial distribution with no time constraints. The best represented species on Flickr in comparison to NBN are diurnal garden birds as around 70% of the Flickr posts for them are valid observations relative to the NBN. Passive citizen science could offer a rich source of observation data for certain taxonomic groups, and/or as a repository for dedicated projects. Our novel method of validating Flickr records is suited to verifying more extensive collections, including less well-known species, and when used in combination with citizen science projects could offer a platform for accurate identification of species and their location.

RETRACTED ARTICLE: Application of hardware system of temporal big data in enterprise human resource management in distributed environment

Distributed data collection systems are often used in many fields, such as industrial control, power systems, seismic monitoring, and ocean exploration. In a distributed data collection system, the time of data collection and the efficiency of data collection are all related to the accuracy of the data. Subsequent analysis and processing show that the distributed data collection system is a very important performance indicator: this article combines the timing factors of the timing rules in the dataset, and after normalization and expansion, the rules will be edited according to the effective voting rate. Perform a weighted sum calculation on the rules in the edited sub-database to obtain the final timing rules. Finally, the algorithm is implemented by pseudo code. With the rapid development of information technology, we have entered the era of big data. Big data has the characteristics of resource sharing, large storage capacity, and high speed, so it has been widely used in various fields. In the context of the rapid development of the modern economy, the human resources department has provided a relatively complete infrastructure system for many large companies. It can be seen that the work of the personnel department is gradually becoming an important factor in the development of the company. Appropriate implementation of big data in the company’s human resource model can integrate resources into the industry and the company, improving operational efficiency, and laying a solid foundation for the company’s sustainable and long-term development. This article first outlines the hardware system and company management of temporal big data in a distributed environment, discusses in detail the needs of human resource management in companies in the era of big data, and proposes a suitable human resource management model.

An Unstructured Supplementary Service Data System for Daily Tracking of Patient Samples and Diagnostic Results in a Diagnostic Network in Malawi: System Development and Field Trial.

BackgroundDiagnostics in many low- and middle-income countries are conducted through centralized laboratory networks. Samples are collected from patients at remote point-of-care health facilities, and diagnostic tests are performed at centralized laboratories. Sample transportation systems that deliver diagnostic samples and test results are crucial for timely diagnosis and treatment in such diagnostic networks. However, they often lack the timely and accurate data (eg, the quantity and location of samples prepared for collection) required for efficient operation.ObjectiveThis study aims to demonstrate the feasibility, adoption, and accuracy of a distributed data collection system that leverages basic mobile phone technology to gather reports on the quantity and location of patient samples and test results prepared for delivery in the diagnostic network of Malawi.MethodsWe designed a system that leverages unstructured supplementary service data (USSD) technology to enable health workers to submit daily reports describing the quantity of transportation-ready diagnostic samples and test results at specific health care facilities, free of charge with any mobile phone, and aggregate these data for sample transportation administrators. We then conducted a year-long field trial of this system in 51 health facilities serving 3 districts in Malawi. Between July 2019 and July 2020, the participants submitted daily reports containing the number of patient samples or test results designated for viral load, early infant diagnosis, and tuberculosis testing at each facility. We monitored daily participation and compared the submitted USSD reports with program data to assess system feasibility, adoption, and accuracy.ResultsThe participating facilities submitted 37,771 reports over the duration of the field trial. Daily facility participation increased from an average of 50% (26/51) in the first 2 weeks of the trial to approximately 80% (41/51) by the midpoint of the trial and remained at or above 80% (41/51) until the conclusion of the trial. On average, more than 80% of the reports submitted by a facility for a specific type of sample matched the actual number of patient samples collected from that facility by a courier.ConclusionsOur findings suggest that a USSD-based system is a feasible, adoptable, and accurate solution to the challenges of untimely, inaccurate, or incomplete data in diagnostic networks. Certain design characteristics of our system, such as the use of USSD, and implementation characteristics, such as the supportive role of the field team, were necessary to ensure high participation and accuracy rates without any explicit financial incentives.