Black-box Data Research Articles

Greedy identification of latent dynamics from parametric flow data

Projection-based reduced-order models (ROMs) play a crucial role in simplifying the complex dynamics of fluid systems. Such models are achieved by projecting the Navier-Stokes equations onto a lower-dimensional subspace while preserving essential dynamics. However, this approach requires prior knowledge of the underlying high-fidelity model, limiting its effectiveness when applied to black-box data. This article introduces a novel, non-intrusive, data-driven method–Greedy Identification of Latent Dynamics (GILD)–for constructing parametric fluid ROMs. Unlike traditional methods, GILD constructs models directly from data, without relying on specific high-fidelity model information. It also employs interpolation within the manifold R∗N×q/Oq to accommodate parameter variability. Numerical experiments on various fluid dynamics scenarios, including lid-driven cavity flow, flow past a cylinder with varying Reynolds number, and Ahmed body flow with variable geometry, demonstrate GILD’s robust performance across both training and unseen parameter values. GILD’s ability to accurately capture system dynamics and its adaptability to diverse data sources highlight its potential as a powerful tool for constructing parametric reduced-order models in an easy and general way for complex fluid dynamics and beyond.

Federated Learning-Based Prediction of Energy Consumption from Blockchain-Based Black Box Data for Electric Vehicles

In modern society, the proliferation of electric vehicles (EVs) is continuously increasing, presenting new challenges that necessitate integration with smart grids. The operational data from electric vehicles are voluminous, and the secure storage and management of these data are crucial for the efficient operation of the power grid. This paper proposes a novel system that utilizes blockchain technology to securely store and manage the black box data of electric vehicles. By leveraging the core characteristics of blockchain—immutability and transparency—the system records the operational data of electric vehicles and uses federated learning (FL) to predict their energy consumption based on these data. This approach allows the balanced management of the power grid’s load, optimization of energy supply, and maintenance of grid stability while reducing costs. Additionally, the paper implements a searchable black box data storage system using a public blockchain, which offers cost efficiency and robust anonymity, thereby enhancing convenience for electric vehicle users and strengthening the stability of the power grid. This research presents an innovative approach to the integration of electric vehicles and smart grids, exploring ways to enhance the stability and energy efficiency of the power grid. The proposed system has been validated through real data and simulations, demonstrating its effectiveness and performance in managing black box data and predicting energy consumption, thereby improving the efficiency and stability of the power grid. This system is expected to empower electric vehicle users with data ownership and provide power suppliers with more accurate energy demand predictions, promoting sustainable energy consumption and efficient power grid operations.

“Counting sheep PSG”: EEGLAB-compatible open-source matlab software for signal processing, visualization, event marking and staging of polysomnographic data

BackgroundProgress in advancing sleep research employing polysomnography (PSG) has been negatively impacted by the limited availability of widely available, open-source sleep-specific analysis tools. New methodHere, we introduce Counting Sheep PSG, an EEGLAB-compatible software for signal processing, visualization, event marking and manual sleep stage scoring of PSG data for MATLAB. ResultsKey features include: (1) signal processing tools including bad channel interpolation, down-sampling, re-referencing, filtering, independent component analysis, artifact subspace reconstruction, and power spectral analysis, (2) customizable display of polysomnographic data and hypnogram, (3) event marking mode including manual sleep stage scoring, (4) automatic event detections including movement artifact, sleep spindles, slow waves and eye movements, and (5) export of main descriptive sleep architecture statistics, event statistics and publication-ready hypnogram. Comparison with existing methodsCounting Sheep PSG was built on the foundation created by sleepSMG (https://sleepsmg.sourceforge.net/). The scope and functionalities of the current software have made significant advancements in terms of EEGLAB integration/compatibility, preprocessing, artifact correction, event detection, functionality and ease of use. By comparison, commercial software can be costly and utilize proprietary data formats and algorithms, thereby restricting the ability to distribute and share data and analysis results. ConclusionsThe field of sleep research remains shackled by an industry that resists standardization, prevents interoperability, builds-in planned obsolescence, maintains proprietary black-box data formats and analysis approaches. This presents a major challenge for the field of sleep research. The need for free, open-source software that can read open-format data is essential for scientific advancement to be made in the field.

BLOCKBOX: Blockchain based black box designing and modeling

SummaryWith the development of technology, data has become more accessible. The storage of critical and valuable information is getting harder and harder with the increase of cyber attacks and vulnerabilities of network and internet. The centralized storage of data causes security and privacy problems. As a remedy to these problems, blockchain has occurred. The reliability, transparency, integrity and confidentiality of the data have been increased with the blockchain technology. Blockchain maintains the immutability of the data with cryptographic hash algorithms. The data is kept in distributed structures with the consensus algorithms so that it is not lost. Projects and studies using blockchain technology have shown that data stored in centralized structures is not as reliable as data that is distributed and secured by cryptographic algorithms, smart contracts, and consensus. The data of flights, which is a critical data, has a great place in world transportation. The parameters inside the black boxes guide us on why the aircrafts have crashed: whether the responsibility belongs to the manufacturer, to the pilot, or to the aircraft technicians. In this way, the real reasons of the accident may be determined and measures can be taken for the accidents that may occur afterwards. In addition, the safe delivery of all black box data to the interested parties and its storage in a way that is immutable will not only benefit to the manufacturer's self‐development by processing the data, but also help flight companies to find optimum flight data. Both the flight companies and the manufacturer will be able to read the black box data, thus the suspicion of data manipulation will be eliminated and the data will become more transparent. In this study, a system is proposed in which an aircraft's black box data can be saved safely and securely with the help of blockchain technology.

Multispecies Benchmark Analysis for LC-MS/MS Validation and Performance Evaluation in Bottom-Up Proteomics.

We present an instrument-independent benchmark procedure and software (LFQ_bout) for the validation and comparative evaluation of the performance of LC-MS/MS and data processing workflows in bottom-up proteomics. The procedure enables a back-to-back comparison of common and emerging workflows, e.g., diaPASEF or ScanningSWATH, and evaluates the impact of arbitrary and inadequately documented settings or black-box data processing algorithms. It enhances the overall performance and quantification accuracy by recognizing and reporting common quantification errors.

Detection of Suspicious Activity using Mobile Sensor Data and Modified Sub-space K-NN for Criminal Investigations

With the bulk availability of mobile sensors, the data collected from them mustn’t be wasted. Nowadays the creation of black-box software that collects this data is not a very difficult task. It is possible to detect suspicious unlawful events using this black-box data. In this paper, we present a novel way of doing forensic investigation using a modified sub-space K-NN (MSK) algorithm. The MSK algorithm is capable of detecting suspicious activities from mobile sensor data. Using this technique, we could detect any normal activity versus suspicious activity with 99.7 % accuracy. This study lays the foundation for future explorations, envisioning potential applications in diverse fields, including zoology. By adapting and expanding the proposed methodology, researchers in zoology could harness mobile sensor data to study animal behavior, offering an innovative approach to understanding and monitoring wildlife activities. Such interdisciplinary bridges highlight the versatility of technological advancements, where tools developed for criminal investigations may find unexpected yet valuable applications in the study of zoological phenomena

Construction of Reciprocal Macromodels in the Loewner Matrix Framework

Many of the passive interconnect systems that arise in applications for analysis of power and signal integrity (PI/SI) are reciprocal by nature, and the transfer matrices of such systems are characterized by symmetry constraints. Therefore, it is natural to think in using reciprocal macromodels for the representation of reciprocal systems. Reciprocal macromodels do not only represent in a more faithful way the systems in which they are based upon, but given a reciprocal stable macromodel, it is computationally cheaper to find its passivity violations. This work shows how to construct reciprocal descriptor systems macromodels from black box data using the Loewner matrix (LM) framework, and also, it shows that it is possible to enforce stability and passivity in reciprocal stable macromodels, keeping its size and structure. Various examples of printed circuit board-based high-speed links and power delivery networks described by scattering parameters are used to demonstrate the effectiveness of the proposed approach.

Sistem Pakar Diagnosa Gizi Buruk Pada Balita Berbasis Mobile Menggunakan Metode Certainty Factor

Maltnutrition has a significant impact on children’s development and is a common problem in developing countries, including Indonesia. There are many factors that contribute to malnutrition, one of which is the lack of understanding and knowledge among parents regarding childcare and proper nutrition. This has motivated the author to develop an expert system application for diagnosing malnutrition in toddlers, aiming to facilitate the community, especially mothers with toddlers, in early diagnosis of malnutrition symptoms and diseases through mobile devices. This expert system application is built using Java Programming language with the assistance of Android studio as the development tool. The system analysis employed is the Unified Modeling Language (UML) to provide an overview of the application to be created. Testing is conducted using the Black Box method and data validation yields nearly 100% accuracy. The calculation for diagnosing symptoms and diseases utilizes the Certainty Factor methode, which serves as the calculation of value within the expert system application. The testing results based on symptoms and diseases through the applied calculation method achieve a 92% accuracy rate. The development of this application is expected to assist the community, especially mothers with toddlers, in identifying early symptoms and diseases of malnutrition in children, as well as obtaining solutions for the experienced illnesses.

On learning agent-based models from data

Agent-Based Models (ABMs) are used in several fields to study the evolution of complex systems from micro-level assumptions. However, a significant drawback of ABMs is their inability to estimate agent-specific (or “micro”) variables, which hinders their ability to make accurate predictions using micro-level data. In this paper, we propose a protocol to learn the latent micro-variables of an ABM from data. We begin by translating an ABM into a probabilistic model characterized by a computationally tractable likelihood. Next, we use a gradient-based expectation maximization algorithm to maximize the likelihood of the latent variables. We showcase the efficacy of our protocol on an ABM of the housing market, where agents with different incomes bid higher prices to live in high-income neighborhoods. Our protocol produces accurate estimates of the latent variables while preserving the general behavior of the ABM. Moreover, our estimates substantially improve the out-of-sample forecasting capabilities of the ABM compared to simpler heuristics. Our protocol encourages modelers to articulate assumptions, consider the inferential process, and spot potential identification problems, thus making it a useful alternative to black-box data assimilation methods.

Classification and Prediction of Highway Accident Characteristics Using Vehicle Black Box Data

Classification and Prediction of Highway Accident Characteristics Using Vehicle Black Box Data

Detection of suspicious activity using mobile sensor data and Modified Sub-space K-NN for criminal investigations

With the bulk availability of mobile sensors, the data collected from them mustn’t be wasted. Nowadays the creation of black-box software that collects this data is not a very difficult task. It is possible to detect suspicious unlawful events using this black-box data. In this paper, we present a novel way of doing forensic investigation using a modified sub-space K-NN (MSK) algorithm. The MSK algorithm is capable of detecting suspicious activities from mobile sensor data. Using this technique, we could detect any normal activity versus suspicious activity with 99.7 % accuracy. We expect the future researcher to develop on this idea and build a solid digital forensic system capable of doing bias-free decisions. Keywords: Forensic, Mobile sensor data, Black box, mobile data collection

A GAN-Based Data Injection Attack Method on Data-Driven Strategies in Power Systems

With the expansion of system scale and data size in power systems, data-driven methods are gradually becoming widely used. However, compared with traditional methods, data-driven methods face more threats in data security and algorithm security. This paper proposes a black box data injection attack method against data-driven strategies in power systems. To obtain a stealthy attack scheme with the greatest impact, an attack vector generator based on deep convolutional neural networks (DCNN) is designed. The generator includes a node selector, a filter, an encoder, and a decoder. The node selector is used to select the most likely successful attack scheme for the filter, the encoder is used to extract sample features, and the decoder is used to generate disturbances based on the features. The proposed generator is trained with an improved generative adversarial network (GAN) and can generate minimal disturbances in real time based on measurement data from the power grid. The effectiveness of the proposed method is demonstrated with an attack experiment on an online transient stability application.

An incremental clustering method for anomaly detection in flight data

Safety is a top priority for civil aviation. Data mining in digital Flight Data Recorder (FDR) or Quick Access Recorder (QAR) data, commonly referred to as black box data on aircraft, has gained interest for proactive safety management. New anomaly detection methods, primarily clustering methods, have been developed to monitor pilot operations and detect any risks from such flight data. However, all existing anomaly detection methods are offline learning — the models are trained once using historical data and used for all future predictions. In practice, new flight data are accumulated continuously and analyzed every month at airlines. Clustering such dynamically growing data is challenging for an offline method because it is memory and time intensive to re-train the model every time new data come in. If the model is not re-trained, false alarms or missed detections may increase since the model cannot reflect changes in data patterns. To address this problem, we propose a novel incremental anomaly detection method based on Gaussian Mixture Model (GMM) to identify common patterns and detect outliers in flight operations from digital flight data. It is a probabilistic clustering model of flight operations that can incrementally update its clusters based on new data rather than to re-cluster all data from scratch. It trains an initial GMM model based on historical offline data. Then, it continuously adapts to new incoming data points via an expectation–maximization (EM) algorithm. To track changes in flight operation patterns, only model parameters need to be saved, not the raw flight data. The proposed method was tested on three sets of simulation data and two sets of real-world flight data. Compared with the traditional offline GMM method, the proposed method can generate similar clustering results with significantly reduced processing time (57 %–99 % time reduction in testing sets) and memory usage (91 %–95 % memory usage reduction in testing sets). Preliminary results indicate that the incremental learning scheme is effective in dealing with dynamically growing data in flight data analytics.

Poisonous Label Attack: Black-Box Data Poisoning Attack with Enhanced Conditional DCGAN

Poisonous Label Attack: Black-Box Data Poisoning Attack with Enhanced Conditional DCGAN

The governance-production nexus of eco-efficiency in Chinese resource-based cities: A two-stage network DEA approach

For decades, resource-based cities in China have significantly contributed to China's socio-economic development. The heavy resource dependence of resource-based cities inevitably leads to a series of environmental problems. Mitigating environmental impacts in an unthinking manner might be disruptive for economic development. Improving eco-efficiency has been a crucial solution for protecting the environment while mitigating its negative economic impact. However, the method commonly used to evaluate the eco-efficiency – that is, the black-box data envelopment analysis (DEA) – cannot examine the inefficiencies of the internal structure, and as a result, the underlying management defects are unclear. To open the black box, this study presents a two-stage network DEA framework incorporating government and industrial sectors and measures the eco-efficiency of 84 resource-based cities during the post-financial crisis period (2007–2015). The results indicate that the average eco-efficiency of China's resource-based cities shows a promising increase, and there is a positive relationship between governance efficiency and production efficiency. The decreasing trend of governance efficiency in the Central, Western, and Northeast regions after 2014 shows the low quality of the government sector in the usage of fiscal income. Proactive disclosure of how the government sector conducts public business and spends taxpayers' money should be made to increase transparency, attract more entrepreneurial resources to carry out production activities, and further improve sustainability. The two-stage network DEA framework helps obtain more insights into the internal management defects of the government and industrial sectors and enhance their cooperation to improve the eco-efficiency precisely.

Black box operation optimization of basic oxygen furnace steelmaking process with derivative free optimization algorithm

Mastering the smelting status and making the material adding strategy are important and challenging issue in the basic oxygen furnace (BOF) steelmaking process. Unfortunately, complex physicochemical reaction with multi-input and multi-output makes the first-principles models unusable. To this end, a black box operation optimization (BBOO) problem integrated black box data analytics modeling and optimization is abstracted from the BOF steelmaking process. Moreover, a derivative free optimization (DFO) method based on the trust region framework is developed to solve the operation optimization problem. In the proposed algorithm, we integrate quadratic interpolation and sparse modeling to construct the surrogate model. Wherein, l0-norm optimization, which is solved through converting it into a convex optimization problem with introducing the slack variables, is used to determine the sparse surrogate model. Finally, we apply the proposed DFO algorithm and other DFO algorithms to solve the BBOO problem in BOF steelmaking process. The results demonstrate that the proposed method can be an alternative for providing solutions and insights of material addition in BOF process and the proposed algorithm is comparable.

Black box modelling of a latent heat thermal energy storage system coupled with heat pipes

This paper presents black box models to represent a LHTESS (Latent Heat Thermal Energy Storage System) coupled with heat pipes, aimed at increasing the storage performance and at decreasing the time of charging/discharging. The presented storage system is part of a micro solar CHP plant and the developed model is intended to be used in the simulation tool of the overall system, thus it has to be accurate but also fast computing. Black box data driven models are considered, trained by means of numerical data obtained from a white box detailed model of the LHTESS and heat pipes system. A year round simulation of the system during its normal operation within the micro solar CHP plant is used as dataset. Then the black box models are trained and finally validated on these data. Results show the need for a black box model that can take into account the different seasonal performance of the LHTESS. In this analysis the best fit was achieved by means of Random Forest models with an accuracy higher than 90%.

PENERAPAN TEKNOLOGI DATAWAREHOUSE NOSQL DAN BUSINESS INTELLIGENCE UNTUK ANALISA TRANSAKSI PENJUALAN

Business process are increasingly developing, causing data generated will be even greater. The application of datawarehouse technology and MongoDB as one of the NoSQL technologies can solve large historical data processing problems. In addition, business intelligence technology that is able to visualize data in into graphics and tables will make it easier for stakeholders to analyze sales transactions. The process for transforming transaction data into a datawarehouse is the Extract Transform Load (ETL) process. The data generated from the ETL process is quality data and is really needed for analysis. Black box testing results and data validity indicate the system can be accepted and can produce the right information for decision support.

Efficiency Measurement for Hierarchical Network Systems Using Network DEA and Intuitionistic Fuzzy ANP

According to the high importance of the university in the growth and development of the country, the evaluation performance of educational and research groups in universities is very important. The black-box Data Envelopment Analysis (DEA) model is mathematical programming for measuring the relative efficiency of a set of DMUs, without considering the operations of the component processes that may have misleading results. To overcome this defect, network models are recommended. This paper intends to propose a hybrid Intuitionistic Fuzzy Analysis Network Process (IFANP) and Network DEA (NDEA) technique to evaluate the efficiency of the faculty of basic sciences of Islamic Azad University. The IFANP method is used to evaluate the overall weights among all the criteria and sub-criteria and these weights are used in the NDEA model to measure the relative efficiency of a system. The hypothetical example shows that the efficiency of all DMUs is equal to 1 by using the DEA, and there is no ranking between DMUs. The results of the IFANP-NDEA are more meaningful because of the full ranking of DMUs, considering component process operations. Also, it can prioritize efficient DMUs, provide the efficiencies of the DMU’s functions which enables managers to identify areas of weakness.

Research on the Whole Process Engineering Consulting Service of Risk Sharing based on CIM Reward Model

In order to fully study the CIM reward model risk sharing process of the whole process of engineering consulting services, this paper analyzes the three common outsourcing modes of engineering consulting by AHP, and analyzes the work flow by black box data flow. Finally, this paper believes that the policy analysis and case study of response engineering should be strengthened before the risk analysis of specific projects under CIM reward model can be made.