Data Silos Research Articles

Improving detection of scanning attacks on heterogeneous networks with Federated Learning

Scanning attacks are the first step in the attempt to compromise the security of systems. Machine learning (ML) has been used for network intrusion detection systems (NIDS) to protect systems by learning misbehavior based on network traffic. This paper demonstrates that Federated Learning (FL) is a promising approach to achieve better detection performance than traditional local training and inference on distributed agents. Also, this FL approach brings privacy, efficiency, and it is suitable for distributed ML-based NIDS solutions. We present a horizontal FL setup using Logistic Regression with FedAvg strategy applied to 13 agents (data silos) capable of providing an iterative process of constant learning improvement. Our results indicate a more stable learning process when observed the F1-score average, whereas the traditional NIDS approach (local trained models) present lesser performance and bigger variability to classify scanning and benign traffic. We tested our model performance on the TON_IoT dataset containing network traffic from a virtualized heterogeneous network composed of cloud, fog, and edge layers.

As Data Flows Surge Data Infrastructure Groans

As Data Flows Surge Data Infrastructure Groans

Engage with empathy: Improving customer experience with artificial intelligence

Change is inevitable, and sometimes transformation happens gradually. At other times, as the global pandemic showed us, it happens without warning. Either way, traditional marketing tactics have no place in modern customer experiences (CX). So how can brands build trust and develop deeper relationships through ever-changing sentiment or market conditions? They need to show customers that they understand their unique situation and can provide tangible value. Marketing and customer engagement practitioners can achieve true one-to-one engagement with artificial intelligence (AI)-powered, ‘always-on’ models that continuously engage customers during calculated ‘moments of need’. Legacy marketing techniques will no longer be sustainable in a rapidly evolving business landscape riddled with data constraints, regulatory challenges and labour shortages. Organisations need to move away from data silos and channel-specific strategies and instead rely on a single ‘brain’ that helps orchestrate engagement with each customer. Brands can survive only by moving beyond sales and mixing additional messages that support retention, service, nurture and resilience actions into their marketing framework; only then can they deliver truly personalised customer experiences across channels while always remaining sensitive to their circumstances.

Simulation for predictive maintenance using weighted training algorithms in machine learning

<span>In the production, the efficient employment of machines is realized as a source of industry competition and strategic planning. In the manufacturing industries, data silos are harvested, which is needful to be monitored and deployed as an operational tool, which will associate with a right decision-making for minimizing maintenance cost. However, it is complex to prioritize and decide between several results. This article utilizes a synthetic data from a factory, mines the data to filter for an insight and performs machine learning (ML) tool in artificial intelligence (AI) to strategize a decision support and schedule a plan for maintenance. Data includes machinery, category, machinery, usage statistics, acquisition, owner’s unit, location, classification, and downtime. An open-source ML software tool is used to replace the short of maintenance planning and schedule. Upon data mining three promising training algorithms for the insightful data are employed as a result their accuracy figures are obtained. Then the accuracy as weighted factors to forecast the priority in maintenance schedule is proposed. The analysis helps monitor the anticipation of new machines in order to minimize mean time between failures (MTBF), promote the continuous manufacturing and achieve production’s safety.</span>

Integration of real-world clinical data into the Munich Mental Health Biobank – clinical and scientific potential and challenges

Introduction New insights into the pathophysiology of mental disorders and innovations in psychiatric care depend on the availability of representative, longitudinal and multidimensional datasets across diverse, transdiagnostic populations. Biobanks usually attempt to collect such data in parallel to clinical routine, which is resource-intensive, puts additional burden on health-care providers, and may reduce the generalizability of the results. Despite containing rich phenotypic and biological information, data generated in routine clinical care is seldomly used for research purposes, because it is usually unstructured and locked in data silos. To truly link clinical practice and research, solutions that optimize the generation and scientific utilization of real-world clinical data are needed.ObjectivesEvaluation of a new digital infrastructure which warrants the efficient, automatized, and structured collection of real-world data in psychiatric care, and integrates the generated data into existing biobanking efforts.MethodsWe have developed a new documentation system which augments the existing IT-structures, enables the collection of routine clinical data in a structured format and involves patients in the data generation process. In an implementation science approach, to replicate and extend the findings of Blitz et al. (JMIR Ment Health 2021), we are investigating the acceptance, efficacy, and safety of the system in our outpatient clinic for affective disorders.ResultsFirst results describing the technical safety, usage metrics, and acceptance of the system, and the quality of the collected data will be presented.ConclusionsChallenges of collecting real-world data for biobanking and research purposes and perspectives on future digital solutions will be discussed.DisclosureNo significant relationships.

Distribution Grid Resource Business Center Integrating Dynamic Migration Algorithm and Lightweight System

The existing power system does not have a unified grid resource management model. The basic data of equipment assets cannot be stored, managed, and maintained in a unified manner. The production systems of different departments are developed by different manufacturers at different times, which leads to the problem of production data silos and high maintenance costs. To solve this problem, a grid resource business middle platform based on State grid-corporation public data model (SG-CIM, CIM) model is designed in this paper. Firstly, the concept and core technology of the middle platform in the power grid resource business are introduced. Then a dynamic data migration strategy based on the FA algorithm is proposed. Finally, the model based on SG-CIM and the application scenario of power transmission and distribution are built and its effectiveness is analyzed.

Optimizing Multi-Objective Federated Learning on Non-IID Data with Improved NSGA-III and Hierarchical Clustering

Federated learning (FL) can tackle the problem of data silos of asymmetric information and privacy leakage; however, it still has shortcomings, such as data heterogeneity, high communication cost and uneven distribution of performance. To overcome these issues and achieve parameter optimization of FL on non-Independent Identically Distributed (non-IID) data, a multi-objective FL parameter optimization method based on hierarchical clustering and the third-generation non-dominated sorted genetic algorithm III (NSGA-III) algorithm is proposed, which aims to simultaneously minimize the global model error rate, global model accuracy distribution variance and communication cost. The introduction of a hierarchical clustering algorithm on non-IID data can accelerate convergence so that FL can employ an evolutionary algorithm with a low FL client participation ratio, reducing the overall communication cost of the NSGA-III algorithm. Meanwhile, the NSGA-III algorithm, with fast greedy initialization and a strategy of discarding low-quality individuals (named NSGA-III-FD), is proposed to improve the convergence efficiency and the quality of Pareto-optimal solutions. Under two non-IID data settings, the CNN experiments on both MNIST and CIFAR-10 datasets show that our approach can obtain better Pareto-optimal solutions than classical evolutionary algorithms, and the selected solutions with an optimized model can achieve better multi-objective equilibrium than the standard federated averaging (FedAvg) algorithm and the Clustering-based FedAvg algorithm.

Federated Learning with Dynamic Model Exchange

Large amounts of data are needed to train accurate robust machine learning models, but the acquisition of these data is complicated due to strict regulations. While many business sectors often have unused data silos, researchers face the problem of not being able to obtain a large amount of real-world data. This is especially true in the healthcare sector, since transferring these data is often associated with bureaucratic overhead because of, for example, increased security requirements and privacy laws. Federated Learning should circumvent this problem and allow training to take place directly on the data owner’s side without sending them to a central location such as a server. Currently, there exist several frameworks for this purpose such as TensorFlow Federated, Flower, or PySyft/PyGrid. These frameworks define models for both the server and client since the coordination of the training is performed by a server. Here, we present a practical method that contains a dynamic exchange of the model, so that the model is not statically stored in source code. During this process, the model architecture and training configuration are defined by the researchers and sent to the server, which passes the settings to the clients. In addition, the model is transformed by the data owner to incorporate Differential Privacy. To trace a comparison between central learning and the impact of Differential Privacy, performance and security evaluation experiments were conducted. It was found that Federated Learning can achieve results on par with centralised learning and that the use of Differential Privacy can improve the robustness of the model against Membership Inference Attacks in an honest-but-curious setting.

On the Internet of Things, Blockchain Technology for Supply Chain Management (IoT)

Patient’s medical records are now accessible from anywhere and at any time, thanks to the Internet and the changes it has wrought in the healthcare industry. Electronic health records were dogged by a lack of standards, but that was not the only issue. Decentralized online ledgers were already being proposed and used to solve interoperability and privacy issues when blockchain-based systems were first built. As far as technical issues go, scaling, usability, and accessibility stand out. On the one hand, it is difficult to keep secure access control measures on-chain while simultaneously keeping a wide range of medical data off-chain. Finding out who owns what and spreading access control of data is the second challenge in medical settings. Using temporal blockchain, the Secured Healthcare System (SHS) aims to address these problems (TB). As an SHS fundamental building piece, the context-based Merkle tree emphasizes privacy, enhanced integrity management, and access control methods (CBMT). For interoperability and scale control, the framework uses temporal features, HL7 criteria, and IPFS data management (IPFS). Personalized micro booklet (PML) security was found to be affected by the SHS framework, namely, on the time-based shadow notions and the contextual components of the PML (PML). Taking advantage of the architecture’s enormous potential to solve the challenges of siloed data and enabling tamper-proof, secure healthcare transaction has been sought.

Study on Fraud Transaction Detection System

Fraudulent transactions are very common problem in banking systems internationally, accounting for $5.1 trillion dollars every year. Many financial institutions are facing the common problem of being targeted by transactions of fraudulent nature and its becoming more and more obvious that advanced technology, such as Machine Learning (ML) is needed to counter such acts. Machine learning is the most effective technique against these complex bank frauds when approaches relying on fragmented and siloed data, rules-based approaches or traditional point-solutions are not only costly but also not as effective as needed. Complex algorithms powered by ML can be used to reduce manual investigations in Financial Institutions. Volume of these transactions is huge, lots of current solutions do not focus on big data the proposed model will work on big data with the help of ‘Apache Spark’ using latest machine learning technology. The proposed model will try to find pattern in given data set and flag the transaction as fraudulent or not with probability score and then banking system can decide further course of action. We are also going to include different machine learning algorithms used to detect fraudulent transactions and provide a comparative study between those algorithms to show which is more effective.

Privacy-preserving federated neural network learning for disease-associated cell classification

SummaryTraining accurate and robust machine learning models requires a large amount of data that is usually scattered across data silos. Sharing or centralizing the data of different healthcare institutions is, however, unfeasible or prohibitively difficult due to privacy regulations. In this work, we address this problem by using a privacy-preserving federated learning-based approach, PriCell, for complex models such as convolutional neural networks. PriCell relies on multiparty homomorphic encryption and enables the collaborative training of encrypted neural networks with multiple healthcare institutions. We preserve the confidentiality of each institutions’ input data, of any intermediate values, and of the trained model parameters. We efficiently replicate the training of a published state-of-the-art convolutional neural network architecture in a decentralized and privacy-preserving manner. Our solution achieves an accuracy comparable with the one obtained with the centralized non-secure solution. PriCell guarantees patient privacy and ensures data utility for efficient multi-center studies involving complex healthcare data.

A Perspective on Digital Knowledge Representation in Materials Science and Engineering

The amount of data generated worldwide is constantly increasing. These data come from a wide variety of sources and systems, are processed differently, have a multitude of formats, and are stored in an untraceable and unstructured manner, predominantly in natural language in data silos. This problem can be equally applied to the heterogeneous research data from materials science and engineering. In this domain, ways and solutions are increasingly being generated to smartly link material data together with their contextual information in a uniform and well‐structured manner on platforms, thus making them discoverable, retrievable, and reusable for research and industry. Ontologies play a key role in this context. They enable the sustainable representation of expert knowledge and the semantically structured filling of databases with computer‐processable data triples.

It's the data, stupid: Inflection point for Artificial Intelligence in Indian healthcare

Indian healthcare is fast growing and with significant chunk of it being in small, fragmented, informal sector; Artificial Intelligence (AI) is pegged as a magical tool for a better healthcare system. There is an inclination to merely mimic the US approach in the on-going policy making and legislative exercises, which can have serious fallouts for Indian healthcare.India needs a different approach to suite her unique requirements. In this regard, each of the five stages in AI development lifecycle has been analyzed in the light of current on-ground realities. These boil down to three fold challenges of how to increase adoption of digital health, prevent data silos and create maximum value from data.Availability of quality data for value addition without barriers and restrictions is the common denominator for leveraging the full potential of AI. This requires liberal policies enabling secondary use of data in developing countries with rapidly growing healthcare sector akin to India. This has to be carefully balanced with data privacy and security.Restrictive healthcare data policies and laws can slow down adoption of digitization, perpetuate status-quo, be biased towards the incumbent players, cause Industry stagnation and thus will do more harm than good. It is therefore the data policies that will make or break AI in Indian healthcare.

A framework for mapping and monitoring human-ocean interactions in near real-time during COVID-19 and beyond

The human response to the COVID-19 pandemic set in motion an unprecedented shift in human activity with unknown long-term effects. The impacts in marine systems are expected to be highly dynamic at local and global scales. However, in comparison to terrestrial ecosystems, we are not well-prepared to document these changes in marine and coastal environments. The problems are two-fold: 1) manual and siloed data collection and processing, and 2) reliance on marine professionals for observation and analysis. These problems are relevant beyond the pandemic and are a barrier to understanding rapidly evolving blue economies, the impacts of climate change, and the many other changes our modern-day oceans are undergoing. The “Our Ocean in COVID-19″ project, which aims to track human-ocean interactions throughout the pandemic, uses the new eOceans platform (eOceans.app) to overcome these barriers. Working at local scales, a global network of ocean scientists and citizen scientists are collaborating to monitor the ocean in near real-time. The purpose of this paper is to bring this project to the attention of the marine conservation community, researchers, and the public wanting to track changes in their area. As our team continues to grow, this project will provide important baselines and temporal patterns for ocean conservation, policy, and innovation as society transitions towards a new normal. It may also provide a proof-of-concept for real-time, collaborative ocean monitoring that breaks down silos between academia, government, and at-sea stakeholders to create a stronger and more democratic blue economy with communities more resilient to ocean and global change.

Publishing and Interlinking COVID-19 Data Using Linked Open Data Principles: Toward Effective Healthcare Planning and Decision-Making

The COVID-19 data is critical to support countries and healthcare organizations for effective planning and evidence-based practices to counter the pressures of cost reduction, improved coordination, and outcome and produce more with less. Several COVID-19 datasets are published on the web to support stakeholders in gaining valuable insights for better planning and decision-making in healthcare. However, the datasets are produced in heterogeneous proprietary formats, which create data silos and make data discovery and reuse difficult. Further, the data integration for analysis is difficult and is usually performed by the domain experts manually, which is time-consuming and error-prone. Therefore, an explicit, flexible, and widely acceptable methodology to represent, store, query, and visualize COVID-19 data is needed. In this paper, we have presented the design and development of the Linked Open COVID-19 Data system for structuring and transforming COVID-19 data into semantic format using explicitly developed ontology and publishing on the web using Linked Open Data (LOD) principles. The key motivation of this research is the evaluation of LOD technology as a potential option and application of the available Semantic Web tools (i.e., Protégé, Excel2RDF, Fuseki, Silk, and Sgvizler) for building LOD-based COVID-19 information systems. We have also underpinned several use-case scenarios exploiting the LOD format of the COVID-19 data, which could be used by applications and services for providing relevant information to the end-users. The effectiveness of the proposed methodology and system is evaluated using the system usability scale and descriptive statistical methods and results are found promising.

Optimization algorithms as training approach with hybrid deep learning methods to develop an ultraviolet index forecasting model

The solar ultraviolet index (UVI) is a key public health indicator to mitigate the ultraviolet-exposure related diseases. This study aimed to develop and compare the performances of different hybridised deep learning approaches with a convolutional neural network and long short-term memory referred to as CLSTM to forecast the daily UVI of Perth station, Western Australia. A complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) is incorporated coupled with four feature selection algorithms (i.e., genetic algorithm (GA), ant colony optimization (ACO), particle swarm optimization (PSO), and differential evolution (DEV)) to understand the diverse combinations of the predictor variables acquired from three distinct datasets (i.e., satellite data, ground-based SILO data, and synoptic mode climate indices). The CEEMDAN-CLSTM model coupled with GA appeared to be an accurate forecasting system in capturing the UVI. Compared to the counterpart benchmark models, the results demonstrated the excellent forecasting capability (i.e., low error and high efficiency) of the recommended hybrid CEEMDAN-CLSTM model in apprehending the complex and non-linear relationships between predictor variables and the daily UVI. The study inference can considerably enhance real-time exposure advice for the public and help mitigate the potential for solar UV-exposure-related diseases such as melanoma.

Data Quality Barriers for Transparency in Public Procurement

Governments need to be accountable and transparent for their public spending decisions in order to prevent losses through fraud and corruption as well as to build healthy and sustainable economies. Open data act as a major instrument in this respect by enabling public administrations, service providers, data journalists, transparency activists, and regular citizens to identify fraud or uncompetitive markets through connecting related, heterogeneous, and originally unconnected data sources. To this end, in this article, we present our experience in the case of Slovenia, where we successfully applied a number of anomaly detection techniques over a set of open disparate data sets integrated into a Knowledge Graph, including procurement, company, and spending data, through a linked data-based platform called TheyBuyForYou. We then report a set of guidelines for publishing high quality procurement data for better procurement analytics, since our experience has shown us that there are significant shortcomings in the quality of data being published. This article contributes to enhanced policy making by guiding public administrations at local, regional, and national levels on how to improve the way they publish and use procurement-related data; developing technologies and solutions that buyers in the public and private sectors can use and adapt to become more transparent, make markets more competitive, and reduce waste and fraud; and providing a Knowledge Graph, which is a data resource that is designed to facilitate integration across multiple data silos by showing how it adds context and domain knowledge to machine-learning-based procurement analytics.

Promoting the application of federated learning in medical imaging artificial intelligence

Medical image-based artificial intelligence (AI) systems have shown great potential in assisting disease diagnosis and treatment. However, the challenges, such as data silos, privacy security and standardization, seriously impedes the application of AI in disease diagnosis and treatment. By integrating federated learning technology and FAIR data principle, it is possible to resolve the aforementioned obstacles. Then, it is able to maximize the value of multicenter data to develop a more efficient and accurate disease diagnosis and treatment AI systems, and promote the clinical application of medical image-based AI systems in the field of disease diagnosis and treatment.

GIScience research challenges for realizing discrete global grid systems as a Digital Earth

ABSTRACT Increasing data resources are available for documenting and detecting changes in environmental, ecological, and socioeconomic processes. Currently, data are distributed across a wide variety of sources (e.g. data silos) and published in a variety of formats, scales, and semantic representations. A key issue, therefore, in building systems that can realize a vision of earth system monitoring remains data integration. Discrete global grid systems (DGGSs) have emerged as a key technology that can provide a common multi-resolution spatial fabric in support of Digital Earth monitoring. However, DGGSs remain in their infancy with many technical, conceptual, and operational challenges. With renewed interest in DGGS brought on by a recently proposed standard, the demands of big data, and growing needs for monitoring environmental changes across a variety of scales, we seek to highlight current challenges that we see as central to moving the field(s) and technologies of DGGS forward. For each of the identified challenges, we illustrate the issue and provide a potential solution using a reference DGGS implementation. Through articulation of these challenges, we hope to identify a clear research agenda, expand the DGGS research footprint, and provide some ideas for moving forward towards a scaleable Digital Earth vision. Addressing such challenges helps the GIScience research community to achieve the real benefits of DGGS and provides DGGS an opportunity to play a role in the next generation of GIS.

Tackling Health Disparities in Radiology: A Practical Conceptual Framework.

Tackling Health Disparities in Radiology: A Practical Conceptual Framework.