Data Flow Management Research Articles

SensorHUB – a Novel, Open-source Software Stack for Enhanced Accessibility and Secure Interoperability in IoT Project Management

Abstract. The Internet of Things (IoT) seeks to achieve seamless device connectivity and data sharing, thereby enhancing automation, efficiency, and insights in various fields of application. By delivering real-time data, IoT is essential for systems that aim to model and map the physical world to digital formats, such as digital twins or metaverses. In the recent years, there has been a surge in high-quality commercial IoT products available to various sectors. However, these solutions often come with high costs and proprietary interfaces, which hinder seamless interoperability. For users seeking cost-effective and flexible solutions, open-source software and international standards present viable alternatives. Nonetheless, despite technological progress, the adoption of open and standardized IoT software solutions is still predominantly seen among technology-affine groups or within research and development initiatives. We propose the sensorHUB, a novel IoT stack, which builds upon the Open Geospatial Consortium’s (OGC) data standard SensorThings API and enhances existing solutions towards more accessible, production-oriented solutions for diverse application domains and user groups. Key features include the provision of a new, state-of-the-art graphical user interface for managing data flows and the access to related software modules, and a secure and flexible user and access management.

Uncertainty Calculation as a Service: Integrating Cloud-Based Microservices for Enhanced Calibration and DCC Generation.

The calibration industry is renowned for its diverse and sophisticated equipment and complex processes, which necessitate innovative solutions to keep pace with rapidly advancing technology. This paper introduces an enhancement to an existing microservice-based cloud architecture, aimed at effectively managing the inherent complexity within this field. The enhanced architecture seamlessly integrates various equipment types and communication technologies, aligning diverse stakeholder expectations into a unified system that ensures efficient and accurate calibration processes. It highlights the integration of microservices to facilitate various methods of uncertainty calculation and the generation of digital calibration certificates (DCCs). A case study on RF power measurement illustrates the practical application and benefits of the enhanced architecture. Although initially focused on RF power measurement, the flexible architecture allows for future expansions to accommodate new standards and measurement techniques. The enhanced system offers a comprehensive approach to managing data flow from calibration equipment to the final generation of DCCs, utilizing cloud-based services for efficient data processing. As a future direction, this extension sets the groundwork for broader applicability across multiple measurement types, ensuring readiness for upcoming advancements in metrology.

Data flow management in information systems using blockchain technology

Data flow management in information systems using blockchain technology

Analysis of methods of data flow management in mobile radio communication means

The article analyzes the existing methods of managing data flows in mobile radio communications. During the analysis, the peculiarities of the functioning and construction of these networks were considered. It has been established that such networks have features different from other networks, such as mobility, dynamic topology, independent organization, use of mobile means of communication, lack of fixed data transmission routes, etc. The task of providing routing in mobile radio communication devices, mobile radio networks, and the types of routing in communication networks are considered. The conditions for the implementation of the routing task and the features that characterize the routing process have been established. The levels of the OSI open systems model and the most well-known protocols that work at these levels and are used in data flow management are considered. Data flow management methods are classified by type and a list of requirements for the process of data flow transmission in mobile radio networks is defined, taking into account the peculiarities of the functioning of mobile radio communication means, which must be taken into account when building a mobile radio network management system. It has been established that at the current stage of information technology development, data flow management methods are mostly adapted for use in computer, wired or fixed networks and do not take into account the conditions of use in mobile radio communication devices. It has been established that the construction and operation of an effective data flow control subsystem in the control systems of mobile radio networks and mobile radio communication means requires the use of the latest technologies and modern approaches in the development of data flow control methods and methods that will ensure the functioning of the corresponding data flow control subsystem. When choosing protocols, methods, methods of managing data flows, it is proposed to focus on adaptation, modification of approaches or their combined use based on the target function, taking into account the peculiarities of the functioning of mobile radio networks and mobile radio communication means.

Information Flow Management regarding the Functioning of Public Authorities in Crisis Situations

For the management of information flow, the use of the functional modelling methodology is proposed, as part of the graph theory for describing the functioning of systems, such as public authorities as organizational-management structures. The methodology is based on concepts such as functional block and interface arc (flow). In organizational-management structures, one of the functional blocks is the manager, and the others are managed. Each of the blocks is a source of information and generates information flows, transferring information to a managed functional block or several ones. For the implementation and management of information exchange by the public authorities in crisis management, processing and protection of data flow, when working in a group environment, it is proposed to build an adequate communication and information system using the methodology of functional modelling.

Managementul fluxului de informații privind funcționarea autorităților publice în situații de criză

For the management of information flow, the use of the functional modelling methodology is proposed, as part of the graph theory for describing the functioning of systems, such as public authorities as organizational-management structures. The methodology is based on concepts such as functional block and interface arc (flow). In organizational-management structures, one of the functional blocks is the manager, and the others are managed. Each of the blocks is a source of information and generates information flows, transferring information to a managed functional block or several ones. For the implementation and management of information exchange by the public authorities in crisis management, processing and protection of data flow, when working in a group environment, it is proposed to build an adequate communication and information system using the methodology of functional modelling.

CHALLENGES AND SOLUTIONS FOR SCALING DATA FLOWS IN THE INDUSTRY 4.0 ECOSYSTEM IN INDUSTRY AND BUSINESS

The information technology integration processes into manufacturing and logistics are often described by the term "Industry 4.0". This term includes various approaches in digitalization, automation and interconnection in various industrial sectors. The integration of advanced technologies into manufacturing creates large flows of data that conventional industry information systems are often unable to process and respond to in a timely manner. That can create delays and errors that can lead to inefficiencies in supply chains, delays in production, and communication between industrial sectors in addition in problems of transparency and accountability of information systems in the "Industry 4.0" ecosystem, problems with security, data integrity, real-time process tracking, etc. Therefore, it is an important task to research new methods of managing data flows in information systems in the "Industry 4.0" ecosystem, which can improve interaction and accelerate the adoption of information technologies in industry and logistics.

Cross-Border Data Forensics: Challenges and Strategies in the Belt and Road Initiative Digital Era

Cross-border data forensics in the era of the Belt and Road Initiative (BRI) are facing increased complexity. Multinational enterprises are encountering legal and technical challenges due to the fragmentation of global data regulations. The different data protection standards in major jurisdictions such as the European Union, China and the United States have created varying approaches to data privacy, national security, and cross-border data flow management. A study was conducted to explore the intricate framework of traditional Mutual Legal Assistance (MLA) in criminal cases. It highlights the inefficiency of cross-border data forensics and proposes reform proposals to strengthen international data-sharing cooperation. The study suggests that data localization is an effective alternative, but developing digital forensics standards in line with the goals of the Belt and Road Initiative would be a better option. To achieve this, a comprehensive regulatory framework needs to be established that balances national security, personal privacy and international cooperation in data exchange. The framework should emphasize that cross-border data management needs to coordinate and integrate technical, legal and business considerations.

Advancing signal integrity and application analysis through data-flow mapping

Data-flow mapping is a crucial method in signal processing and optimization, managing data flow within systems. Its essential in signal compensation, particularly in telecommunications, audio processing, and biomedical signal processing. Four main algorithm categories underpin data-flow mapping: heuristics, meta-heuristics, Integer Linear Programming (ILP), and Constraint Satisfaction Problems (CSP). Heuristic and meta-heuristic methods like Genetic Algorithms (GA) and Ant Colony Optimization (ACO) provide approximate solutions, crucial for complex problems. ILP and Branch and Bound (B&B) methods offer precise solutions by exhaustive searches under constraints. CSP focuses on satisfying imposed conditions. These methodologies have practical applications, such as signal compensation in communication systems and improving medical imaging like MRI and ultrasound. Theyre also integrated with machine learning, quantum computing, and specialized hardware for 5G/6G communications and IoT. Real-time processing and noise reduction advancements enhance consumer audio and diverse sectors. In summary, data-flow mapping and its algorithms drive signal processing innovations across domains, with evolving technology integration ensuring their lasting importance.

Optimizing IoT Device Connectivity and Data Flow Management Using Azure Functions and Event Hubs: A Scalable Framework for Modern Applications

Optimizing IoT Device Connectivity and Data Flow Management Using Azure Functions and Event Hubs: A Scalable Framework for Modern Applications

СТРУКТУРИРОВАНИЕ ГРУППЫ ПРОЕКТА ЦИФРОВОЙ ТРАНСФОРМАЦИИ ОРГАНИЗАЦИИ

The article discusses ways of structuring roles and reporting relationships for such key persons in the field of digital transformation management of a company as: project manager, technical director, functional managers, business project managers, solution architects, data flow management manager, change management manager, support project quality and other key roles of the digital business transformation management team that ensure its effectiveness.

Networks method of hybrid construction and support of data transmission routes in programmed tools of mobile radio communication based on neural networks

The article proposes a method of hybrid construction and support of data transmission routes in programmable means of mobile radio communication based on neural networks. Taking into account the peculiarities of the operation of mobile radio communication networks and means of communication, the article proposes to intellectualize the processes of managing data flows by using knowledge processing technologies in the construction of a control system. The construction of the control system is based on the use of neural networks, the methodology proposes three main steps of its construction, namely the method of hybrid construction of data transmission routes, the method of maintaining data transmission routes and modeling the process of construction and maintenance of data transmission routes in the network using neural networks. Unlike similar methods, which: are used in stationary communication networks with a small number of end nodes, without the possibility of monitoring the functions of data transmission routes; which are characterized by the absence of a clear decision-making hierarchy and the presence of unpredictability when applied in data transmission systems of mobile radio communication networks; do not take into account the peculiarities of the functioning of mobile radio communications. The proposed technique is able to build data transmission routes, increases the time of network operation and reduces the loading of network channels based on the application of a hybrid structure of neural network construction, by minimizing service traffic in the information direction; selection of the type, number and method of control of data transmission routes; introduction of the hierarchy of the decision-making process for the construction of the data transmission route and the selection of data parameters, taking into account the characteristic features of the use of programmable mobile communication tools to ensure the given quality of network service under different conditions of operation of mobile radio communication tools, using neural networks.

Publish/Subscribe Method for Real-Time Data Processing in Massive IoT Leveraging Blockchain for Secured Storage.

In the Internet of Things (IoT) era, the surge in Machine-Type Devices (MTDs) has introduced Massive IoT (MIoT), opening new horizons in the world of connected devices. However, such proliferation presents challenges, especially in storing and analyzing massive, heterogeneous data streams in real time. In order to manage Massive IoT data streams, we utilize analytical database software such as Apache Druid version 28.0.0 that excels in real-time data processing. Our approach relies on a publish/subscribe mechanism, where device-generated data are relayed to a dedicated broker, effectively functioning as a separate server. This broker enables any application to subscribe to the dataset, promoting a dynamic and responsive data ecosystem. At the core of our data transmission infrastructure lies Apache Kafka version 3.6.1, renowned for its exceptional data flow management performance. Kafka efficiently bridges the gap between MIoT sensors and brokers, enabling parallel clusters of brokers that lead to more scalability. In our pursuit of uninterrupted connectivity, we incorporate a fail-safe mechanism with two Software-Defined Radios (SDR) called Nutaq PicoLTE Release 1.5 within our model. This strategic redundancy enhances data transmission availability, safeguarding against connectivity disruptions. Furthermore, to enhance the data repository security, we utilize blockchain technology, specifically Hyperledger Fabric, known for its high-performance attributes, ensuring data integrity, immutability, and security. Our latency results demonstrate that our platform effectively reduces latency for 100,000 devices, qualifying as an MIoT, to less than 25 milliseconds. Furthermore, our findings on blockchain performance underscore our model as a secure platform, achieving over 800 Transactions Per Second in a dataset comprising 14,000 transactions, thereby demonstrating its high efficiency.

IMGA: Efficient In-Memory Graph Convolution Network Aggregation With Data Flow Optimizations

Aggregating features from neighbor vertices is a fundamental operation in Graph Convolution Network (GCN). However, the sparsity in graph data creates poor spatial and temporal locality, causing dynamic and irregular memory access patterns and limiting the performance of aggregation on the Von Neumann architecture. The emerging processing-in-memory (PIM) architecture is based on emerging non-volatile memory (NVM), like Spin-orbit torque Magnetic RAM (SOT-MRAM), and demonstrates promising prospects in alleviating the Von Neumann bottleneck. However, the limited memory capacity of PIM medium still incurs non-negligible data movements between PIM architecture and external memory. To solve this challenge, we propose a SOT-MRAM based in-memory computing architecture, called IMGA, for efficient in-situ graph aggregation. Specifically, we design adaptive data flow management strategies that reuse vertex data in MRAM when processing graphs of different scales and adopt edge data as the control signal source to utilize the graph’s structural information. A reordering optimization strategy leveraging hardware-software co-design principle is proposed to further reduce the costly data movement. Experimental results demonstrate that IMGA achieves an average 2523x and 21x speedup, and 1.03E+6 and 1.04E+3 energy efficiency compared with CPU and GPU, respectively.

Management of data flows in modern industry using blockchain

Recent research and publications. "Industry 4.0" is a concept of the industrial revolu-tion, which is based on the use of modern technologies and digital innovations in production and distribution processes. The introduction of the concept of "Industry 4.0" was designed to improve the competitiveness of European industry and increase productivity and product quality. A blockchain is a distributed data structure that is replicated and distributed among network members. The purpose of the study is to improve automation processes, increase efficiency, re-duce delays and errors in information systems of industry and supply chains by using block-chain technologies in the construction of information systems. Main material of the study. The paper makes an analysis of approaches and algorithms to data management in "Industry 4.0" information systems. Blockchain algorithms are com-pared to classical approach with other databases in the client-server architecture. Conclusions. By implementing algorithms based on blockchain technology, namely by using the Merkle Tree, digital signature technology, and by using consensus algorithms in the framework of decentralized data storage in Distributed Ledger Technology, the processes of automation and efficiency in data flow management are improved, providing a secure and transparent way to store and share data that reduces delays and errors in industry informa-tion systems and supply chains.

Development of floating point operating devices

The paper shows a well-known approach to the construction of cores in multi-core microprocessors, which is based on the application of a data flow graph-driven calculation model. The architecture of such kernels is based on the application of the reduced instruction set level data flow model proposed by Yale Patt. The object of research is a model of calculations based on data flow management in a multi-core microprocessor. The results of the floating-point multiplier development that can be dynamically reconfigured to handle five different formats of floating-point operands and an approach to the construction of an operating device for addition-subtraction of a sequence of floating-point numbers are presented, for which the law of associativity is fulfilled without additional programming complications. On the basis of the developed circuit of the floating-point multiplier, it is possible to implement various variants of the high-speed multiplier with both fixed and floating points, which may find commercial application. By adding memory elements to each of the multiplier segments, it is possible to get options for building very fast pipeline multipliers. The multiplier scheme has a limitation: the exponent is not evaluated for denormalized operands, but the standard for floating-point arithmetic does not require that denormalized operands be handled. In such cases, the multiplier packs infinity as the result. The implementation of an inter-core operating device of a floating-point adder-subtractor can be considered as a new approach to the practical solution of dynamic planning tasks when performing addition-subtraction operations within the framework of a multi-core microprocessor. The limitations of its implementation are related to the large amount of hardware costs required for implementation. To assess this complexity, an assessment of the value of the bits of its main blocks for various formats of representing floating-point numbers, in accordance with the floating-point standard, was carried out.

Energy Optimization Using Reptile Search in Wireless Sensor Networks

Energy saving in wireless sensor networks (WSNs) is a critical problem for diversity of applications. Data aggregation between sensor nodes is huge unless a suitable sensor data flow management is adopted. Clustering the sensor nodes is considered an effective solution to this problem. Each cluster should have a controller denoted as a cluster head (CH) and a number of nodes located within its supervision area. Clustering demonstrated an effective result in forming the network into a linked hierarchy. Thus, balancing the load distribution in WSNs to make efficient use of the available energy sources and reducing the traffic transmission can be achieved. In solving this problem we need to find the optimal distribution of sensors and CHs; thus, we can increase the network lifetime while minimizing the energy consumption. In this paper, a Reptile Search Algorithm (RSA) for preserving location privacy and congestion avoidance with less delay guaranteed is proposed. With this routing technique, the complete sensor field is divided into different subdivisions and each subdivision elects a target area by computing its transmission distance. The backbone of the dynamic routing protocol consists of a virtual ring called bell nodes and a radial line called tentacle nodes employs more nodes to construct the network. The amount of radial line and radius of the virtual ring in a network are conjointly determined to ease the communication path from the node to sink. The radial line paths are routed directionally and bell nodes are routed with angular directions probabilistically. From the routing path, the tentacle nodes collect the data to dynamic sink which will assure that the information is going to be collected with less delay and attacker cannot guess their positions. The experimental results show that the proposed RSA method accomplishes enhanced performance in terms of energy consumption, packet delivery delay and lifetime.

Development of a mathematical model for assessing the quality of service on a packet switching subnet

Currently, data traffic is growing rapidly, and ensuring optimal network performance and effective data flow management have become the most important tasks. In this context, the quality of network service plays a crucial role in achieving these goals. This article suggests an approach to solving the problem of efficient service in ISDN. Namely, optimization of resource distribution between channel switching and packet switching subnets in ISDN to calculate optimal quality of service characteristics. In the process of ISDN design analysis, an optimization problem is compiled, where the evaluation of the packet-switched subnet service is used as an objective function, and the evaluation of the circuit-switched subnet service is used as one of the constraints for this task. To calculate the main characteristics of a packet-switched subnet, the subnet is considered as a service system with a delay. During the study, the methods of optimal movement of the generalized channel boundary between the subnets of channel switching and packet switching were identified, depending on the data parameters and the state of the integrated network, which made it possible to develop an optimal mathematical model of optimal control of the generalized boundary. To calculate the bandwidth for channel switching and packet switching subnets, an algorithm has been compiled to implement the resulting model and a program in C++ has been compiled. The study of the generalized boundary and the dynamic redistribution of bandwidth between subnets represents a new approach to network optimization. The results are based on the use of the classical Erlang formula for systems with service failures and on load distribution plans, which makes it possible to effectively manage the maintenance process in the network

An unified CAD/CAE/VR tool for ship structure design and evaluation based on multi-domain feature mapping

Computer-aided design (CAD), computer-aided engineering (CAE), and virtual reality (VR) are three distinct disciplines. For the collaborative design and interactive evaluation of ship structures, the main obstacle lies in the large differences in model data structures. In this regard, a unified tool for ship structure design/analysis/virtual evaluation based on Multi-Domain Feature Mapping (MDFM) is proposed. The multi-layer transmission and matching mechanism between data, information and knowledge established by Extensible Markup Language (XML) can realize the feature association among design, analysis and evaluation domain, so that CAD, CAE and VR become an organic entity. In the virtual scene, a visual expression strategy of CAD model and FEA data based on Unified Mesh Model (UMM) is proposed. The advantage of this unified tool lies in the intelligent generation of finite element models, as well as the virtual visualization and model reconstruction and evaluation of models with different topologies. Intelligent finite element (FE) meshing and analysis applications integrating knowledge templates represent a major subset of unified CAD/CAE/VR tools for structural models from CAD systems. The structure and data flow management of this unified tool is based on a common design process, which allows designers to link anticipated scenarios with proposed engineering changes. Finally, the effectiveness of the proposed tool is demonstrated by engineering examples.

A 40nm 2TOPS/W Depth-Completion Neural Network Accelerator SoC With Efficient Depth Engine for Realtime LiDAR Systems

Light Detection and Ranging (LiDAR) is becoming a critical requirement for future computer vision applications, such as AR/VR (iPhone-LiDAR) and ADAS (Automotive-LiDAR). A depth point-cloud input has different characteristics than a conventional RGB image input, such that the CNN depthinference implementation is unique when compared with a standard super-resolution CNN(SR-CNN). In this brief, we present a heterogeneous AI-accelerator SoC, which is specific to depth image completion computation. Three key innovations are introduced to improve SoC’s performance. First, to accommodate the unique input data structure of a depth input, a fully-filled dataflow management engine is proposed to pre-process the RGB+Depth input, significantly improving processing element utilization (PEU). Second, to improve the efficiency of the instruction configurations of the CNN accelerator, a hardware-tiling co-processor is proposed that performs the tiling strategy of the CNN accelerator, assigning each sub-job to the PE array directly, therefore reducing the time for task assignments. Third, due to the large number of vector operations required for the postprocess in the neural network, a RISC-V core is incorporated to execute vector computations better. The SoC is implemented in 40nm CMOS process, achieving 2TOPs/W energy efficiency with 34fps throughput under VGA-resolution output for realtime LiDAR systems.