Data Processing Flow Research Articles

Research and application of AMT in tunnel hidden goaf under complex conditions

In order to detect the distribution, development scale and shape of mining area in tunnel hidden coal measure strata, and provide intuitive and reliable geological basis for later design and construction, geological mapping combined with audio frequency magnetotelluric method was used for exploration. In this paper, firstly, the characteristics of tunnel hidden goaf and the basic principle of audio frequency magnetotelluric (AMT) method are briefly described; secondly, the key technologies of scientific acquisition and data processing flow of the method are analyzed; secondly, the geological characteristics of goaf under different conditions are analyzed, and the geological models are established for forward modeling; finally, the method is applied in yuchanshan tunnel of Ronglu expressway The results of electromagnetic imaging directly and accurately reflect the distribution position and scale of goaf, and provide reliable basic data for later design. Through theoretical analysis and practical application, the results show that audio frequency magnetotelluric method combined with geological mapping can play an important role in the investigation of tunnel hidden goaf, and can directly reflect the location, shape and other basic elements of goaf, providing more reliable basic data for subsequent investigation and design.

Obstacle detection and recognition in farmland based on fusion point cloud data

Obstacle detection and recognition in the farmland environment is of great significance to unmanned agricultural machinery navigation technology. In this study, a tractor platform based on 3D/2D LiDAR and GNSS/AHRS was built to acquire fusion point cloud data. A complete data processing flow is proposed, including fusion point cloud data acquisition, obstacle detection, and obstacle recognition. The fusing point cloud data acquisition method realized the time synchronization between multiple sensors and the space registration of point cloud at different coordinate systems. The obstacle detection method removed ground, sparse, and ego platform points based on RANSAC and crop-box filtering. Then, a Euclidean clustering method based on the platform width was used for obstacle clustering. The obstacle recognition method established a box model based on PCA for each cluster and extracted its geometric feature information. Next, a MCB classification method is proposed to distinguish false-positive, non-interest, and trackable obstacles (e.g., tractor and pedestrian). To evaluate the results of data collection and processing, a series of experiments was designed and implemented. The average absolute deviation of the fusion point cloud was 0.015 m, and the average rate of lost was 1.24%. The proportion of residual ground points in the ground point cloud was less than 0.012%. The accuracy, precision, recall, and F1-score exceeded 80%. Therefore, the proposed method could meet the operation requirements in the farmland environment and provide highly precise and stable detection and recognition results.

Program diagramming and fundamental programming patterns for a polymorphic computing dataflow processor

In this era where the growth of computer performance is significantly flattening, new parallel computer architectures are needed to continue to fuel that growth. One candidate architecture to fuel this growth is a recently proposed dataflow processor that is meant to be tiled together into a fabric to create a polymorphic computer (i.e. a reconfigurable computer whose architecture can be changed to match the software) where program instructions can be independently migrated in a fine-grained manner around the fabric to logically change the computer architecture. This work provides a summary of the dataflow processor’s instruction set and details how to write programs for this processor. The techniques that are presented to write programs for this processor are diagramming and 12 building block programming patterns. The 12 patterns are: Operations with a Constant, Simple Copy, Pitch (Discard), Counters, Critical Sections, Funnels (Gathers), Distribution Tree (Scatters), Lookup Tables, Single Value Selection, Single Value Steering, Switch (Multi-Value Steering), and Loops. These patterns are expected to benefit future compilers. In addition, the ideal timing results of several test algorithms that are implemented in this processor and that utilize the above patterns are presented. • Background summary of a polymorphic computing dataflow processor’s instruction set • Program diagramming methods for a polymorphic computing dataflow processor • 12 Programming patterns in a polymorphic computing dataflow processor • Ideal timing results of several test algorithms in polymorphic computing fabric

Recent Developments in Parallel and Distributed Computing for Remotely Sensed Big Data Processing

This article gives a survey of state-of-the-art methods for processing remotely sensed big data and thoroughly investigates existing parallel implementations on diverse popular high-performance computing platforms. The pros/cons of these approaches are discussed in terms of capability, scalability, reliability, and ease of use. Among existing distributed computing platforms, cloud computing is currently the most promising solution to efficient and scalable processing of remotely sensed big data due to its advanced capabilities for high-performance and service-oriented computing. We further provide an in-depth analysis of state-of-the-art cloud implementations that seek for exploiting the parallelism of distributed processing of remotely sensed big data. In particular, we study a series of scheduling algorithms (GSs) aimed at distributing the computation load across multiple cloud computing resources in an optimized manner. We conduct a thorough review of different GSs and reveal the significance of employing scheduling strategies to fully exploit parallelism during the remotely sensed big data processing flow. We present a case study on large-scale remote sensing datasets to evaluate the parallel and distributed approaches and algorithms. Evaluation results demonstrate the advanced capabilities of cloud computing in processing remotely sensed big data and the improvements in computational efficiency obtained by employing scheduling strategies.

Non-Statistical Method for Validation the Time Characteristics of Digital Control Systems with a Cyclic Processing Algorithm

The paper discusses the problem of performance and timing parameters with respect to the validation of digital instrumentation and control systems (I&C). Statistical methods often implicitly assume that the probability distribution law of the estimated parameters is close to normal. Thus, the confidence intervals for the parameter are determined on the grounds of this assumption. However, we encountered cases when the delay distribution law in I&C is not normal. In these cases, we used the non-statistical network calculus method for time parameters estimation. The network calculus method is well elaborated for lossless digital system models with seamless processing algorithm depending only on data volume. We consider the extension of the method to the case of I&C systems with considerable changes in the data flow and content-dependent processing disciplines. The model is restricted to systems with cyclic processing algorithms and fast network connections. Network calculus describes the data flow and system parameters in terms of flow envelopes and service curves that are generally unknown in advance. In this paper, we define equations that allow the calculation of these characteristics from experimental data. The correspondence of the Network Calculus and classical statistical estimation methods is discussed. Additionally, we give an example of model application to a real I&C system.

The Next Wave of Passive Acoustic Data Management: How Centralized Access Can Enhance Science

Passive acoustic data collection has grown exponentially over the past decade resulting in petabytes of data that document our ocean soundscapes. This effort has resulted in two big data challenges: (1) the curation, management, and global dissemination of passive acoustic datasets and (2) efficiently extracting critical information and comparing it to other datasets in the context of ecosystem-based research and management. To address the former, the NOAA National Centers for Environmental Information recently established an archive for passive acoustic data. This fast-growing archive currently contains over 100 TB of passive acoustic audio files mainly collected from stationary recorders throughout waters in the United States. These datasets are documented with standards-based metadata and are freely available to the public. To begin to address the latter, through standardized processing and centralized stewardship and access, we provide a previously unattainable comparison of first order sound level-patterns from archived data collected across three distinctly separate long-term passive acoustic monitoring (PAM) efforts conducted at regional and national scales: NOAA/National Park Service Ocean Noise Reference Station Network, the Atlantic Deepwater Ecosystem Observatory Network, and the Sanctuary Soundscape Monitoring Project. Nine sites were selected from these projects covering the Alaskan Arctic, Northeast and Central Pacific, Gulf of Mexico, Caribbean Sea, and Mid and Northwest Atlantic. Sites could generally be categorized into those strongly influenced by anthropogenic noise (e.g., vessel traffic) and those that were not. Higher sound levels, specifically for lower frequencies (<125 Hz), and proximity to densely populated coastal zones were common characteristics of sites influenced by anthropogenic noise. Conversely, sites with lower overall sound levels and away from dense populations resulted in soundscape patterns influenced by biological sources. Seasonal variability in sound levels across selected decidecade bands was apparent for most sites and often represented changes in the presence or behavior of sound-producing species. This first order examination of levels across projects highlights the utility of these initial metrics to identify patterns that can then be examined in more detail. Finally, to help the PAM community collectively and collaboratively move forward, we propose the next frontier for scalable data stewardship, access, and processing flow.

Designing a Traceability System for Rice Distribution Process Using QR Code Bassed Android Application at Perum Bulog Subdivre III Surakarta

The state-owned Enterprises (SEO) that manages businesses in the food sector, especially rice is Perum Bulog. However, there were often reports from public about quality of Perum Bulog's rice. Such as the rice had contaminated with gravel or plastic and did not match with premium rice standard. Moreover, The distribution process of premium rice to outlet (RPK) found uncertainty over the arrival time of orders. It caused out of stock and lost sales at the outlet. This paper introduces a q traceability system rice distribution process based on Android with QR Code technology. Furthermore, this paper discusses the system architecture and the development of traceability system design using the company's business process data flow diagram. The developed prototype system shows the functional requirements of the system and can be used by stakeholders to monitor the production process and assist the decision-making process.

Possibilities of Using Building Information Model Data in Reinforcement Processing Plant

While the AEC industry is moving towards digitalization off-site rebar prefabrication became a common practice. Now most companies use a long-established standard order processing method, where the customer submits 2D paper or PDF-based drawings. Subsequently, the manufacturers are obligated to make additional detailing, redrawing, calculations, and preparation of other required information for manufacturing. Thus, in this typical scenario, there is a great repetition of the same tasks, with the obvious loss of time and increased likelihood of human error. However, improvements can be made by the application of advanced digital production workflow and the use of open BIM standards (e.g., IFC, XML, BVBS). Therefore, this paper presents the typical data flow algorithm in contrast to the automated data flow for reinforcement manufacturing. Further, the two approaches are compared and analyzed based on Multi-Criteria Decision Making (MCDM) methods. The results have shown promising prospects for companies willing to automate their data flow processes by the use of 3D drawings and digital data from the BIM model in their plants.

Data Management and Interactive Visualizations for the Evolving Marine Biodiversity Observation Network

Assessing the current state of and predicting change in the ocean’s biological and ecosystem resources requires observations and research to safeguard these valuable public assets. The Marine Biodiversity Observation Network (MBON) partnered with the Global Ocean Observing System Biology and Ecosystems Panel and the Ocean Biodiversity Information System to address these needs through collaboration, data standardization, and data sharing. Here, we describe the generalized MBON data processing flow, which includes several steps to ensure that data are findable, accessible, interoperable, and reusable. By following this flow, data collected and managed by MBON have contributed to our understanding of the Global Ocean Observing System Essential Ocean Variables and demonstrated the value of web-based, interactive tools to explore and better understand environmental change. Although the MBON’s generalized data processing flow is already in practice, work remains in building ontologies for biological concepts, improving processing scripts for data standardization, and speeding up the data collection-to-sharing timeframe.

TARGET RECOGNITION BASED ON CENTRAL AND LOGARITHMIC CENTRAL IMAGE CHORD TRANSFORMATIONS

Object recognition is of great importance for many civil and military applications and supposes the identification and classification of the object in real-time, regardless of spatial position, angular orientation, etc. In the article, the methods of central and logarithmic central image chord transformations are described. The system for invariant object recognition and space parameters determination was developed based on the proposed image chord transformation operations, which is of a multiprocessor functionally distributed architecture, in which various modules implement separate image processing operations, and allows to recognize the objects regardless of their position, scale, or rotation, and to determine the parameters for targeting The model of a data processing flow in the system was elaborated to permit the determination of the signal processing stages which can be realized in parallel. An analytical estimation and analysis of the system’s data processing time and throughput is carrying out. The approach of the system’s optimization was applied and allows to decrease the total data processing time by 24.5 times and increase the system’s throughput by 75 times.

A New Compact Metal Mountable Dual-Band UHF RFID Tag Antenna With an Adapted Middleware for Transport and SCM Fields

Today, RFID, "Radio Frequency Identification", is a key technology for tracking assets in real-time. It has become one of the areas for improvement in several applications. An RFID system consists of three parts: RFID Tags, RFID Readers, and RFID Middleware. This study aims to improve the advantages of RFID applications as an applied technology in several fields, first, by modeling and designing a metal mountable tag antenna that is applied to cover two major UHF RFID bands, i.e., European (865-867 MHz) and U.S. bands (902-928 MHz), secondly, by setting out the RFID Middleware main tasks and suggesting a new RBAC & NoSQL-Based RFID Middleware architecture as a solution to manage the accessibility to RFID data, thus rendering the authentication process much more efficient. The proposed tag antenna configuration utilizes microstrip configuration with open bent stub feed network in order to attain conjugate matching w.r.t. Monza R6 chip impedance. The proposed microstrip patch-based tag antenna structure is simple and does not require any shorting pin/holes, making it easy and inexpensive to manufacture. Additionally, the proposed tag's impedance can be easily tuned in order to achieve conjugate matching concerning employed chip impedance. The proposed tag antenna is fabricated and experimentally characterized to measure its read range performance in the desired bands. The Proposed Middleware Platform architecture combines RFID technology with MongoDB to monitor and process a large number of inbound RFID events generated by RFID tags in order to provide high volume real-time data flow processing and better interoperability through the data presented in JSON format.

Relative traffic management scheme for robot-assisted healthcare applications

Abstract Autonomous and interactive healthcare applications exist for providing patients with customized services through user equipment (UE). For user interaction, these applications take the form of chatbots or physical robots. A user's request/query is fed as input for fetching and delivering user services. Due to the wide range of connectivity and application services, heavy data traffic can deteriorate the quality of application service. This paper proposes a relative traffic management scheme (RTMS) to address traffic issues in applications involving delicate healthcare data. The proposed scheme uses healthcare data flow and query processing to improve the responsiveness of autonomous robot-based services. Dilated traffic at any interconnecting edge is identified based on flow concentration; for this purpose, ant colony optimization (ACO) is used. The premature convergence issue in ACO is addressed by using a linear sigmoid function, which identifies the converging points based on the query processing factor to reinitiate the application service discovery and responsive deliveries. This helps to reduce both outages in healthcare service provisioning and processing time. It additionally improves response with controlled processing complexity.

Data Processing and Analysis Flow of Internet of Things System

The data processing and analysis flow of Internet of things system is the basic process of system design ideas and methods. It includes a series of operations such as requirement analysis, software function design, algorithm and method of implementation, overall structure design, module design, coding and debugging, program linking and testing, compilation and submission of the system. Data processing and analysis process can meet the needs of the system, and solve the problem of the main content of the process. This paper discusses the basic mechanism, principle, type and application of data processing of Internet of things, discusses the composition and elements of data analysis system of Internet of things, and explains the construction process and analysis process and steps of data processing of Internet of things.

Machine Learning Technologies for Secure Vehicular Communication in Internet of Vehicles: Recent Advances and Applications

Recently, interest in Internet of Vehicles’ (IoV) technologies has significantly emerged due to the substantial development in the smart automobile industries. Internet of Vehicles’ technology enables vehicles to communicate with public networks and interact with the surrounding environment. It also allows vehicles to exchange and collect information about other vehicles and roads. IoV is introduced to enhance road users’ experience by reducing road congestion, improving traffic management, and ensuring the road safety. The promised applications of smart vehicles and IoV systems face many challenges, such as big data collection in IoV and distribution to attractive vehicles and humans. Another challenge is achieving fast and efficient communication between many different vehicles and smart devices called Vehicle-to-Everything (V2X). One of the vital questions that the researchers need to address is how to effectively handle the privacy of large groups of data and vehicles in IoV systems. Artificial Intelligence technology offers many smart solutions that may help IoV networks address all these questions and issues. Machine learning (ML) is one of the highest efficient AI tools that have been extensively used to resolve all mentioned problematic issues. For example, ML can be used to avoid road accidents by analyzing the driving behavior and environment by sensing data of the surrounding environment. Machine learning mechanisms are characterized by the time change and are critical to channel modeling in-vehicle network scenarios. This paper aims to provide theoretical foundations for machine learning and the leading models and algorithms to resolve IoV applications’ challenges. This paper has conducted a critical review with analytical modeling for offloading mobile edge-computing decisions based on machine learning and Deep Reinforcement Learning (DRL) approaches for the Internet of Vehicles (IoV). The paper has assumed a Secure IoV edge-computing offloading model with various data processing and traffic flow. The proposed analytical model considers the Markov decision process (MDP) and ML in offloading the decision process of different task flows of the IoV network control cycle. In the paper, we focused on buffer and energy aware in ML-enabled Quality of Experience (QoE) optimization, where many recent related research and methods were analyzed, compared, and discussed. The IoV edge computing and fog-based identity authentication and security mechanism were presented as well. Finally, future directions and potential solutions for secure ML IoV and V2X were highlighted.

Measuring collaborative problem-solving skills (CPSS) of vocational high school students using web-based assessment

This study describes an online method for measuring students’ collaborative problem-solving skills individually using two interactive web-based assessment on the webcps program, with students working in pairs. The method used in this research is a qualitative method of case studies of vocational high school students on dynamic electric material. Process flow data was taken from 30 students (17 of them male students and 13 female students) who completed the assignment at one of the vocational high schools in Riau province. Data is transformed into indicators of collaborative problem-solving skills and is categorized based on four levels namely beginner (level 1), emerging (level 2), intermediate (level 3), and advanced (level 4). The results show that students are at various levels with different characteristics in both the social domain and cognitive domain in CPS skills. The assessment used in this study can be used as a measurement tools electrical circuit and as an instrument of measurement for collaborative problem-solving skills.

Data Model for Residential and Commercial Buildings. Load Flexibility Assessment in Smart Cities

Demand response (DR) programs were usually designed to provide load peak reduction and flatten the load curve, but in the context of rapid adoption of emerging technologies, such as smart metering and sensors, load flexibility will address current trends and challenges (such as grid modernization, demand, and renewables growth) encountered by the evolving power systems. The uncertainty of the renewable energy sources (RES) and electric vehicle (EV) fleet operation has increased the importance of load flexibility that can be managed to provide more support for the stable operation of power systems, including balancing. In this paper, we propose a data model to handle load flexibility and take advantage of its benefits. We also develop a methodology to collect and organize data, combining the consumption profile with several auxiliary datasets such as climate characteristics of the location, independent system operator (ISO) to which the consumer is affiliated, geographical coordinates, assessed flexibility coefficients, tariff rates, weather forecast for day-ahead flexibility forecast, DR-enabling technology costs, and DR programs. These multiple features are stored into a flexibility relational database and NoSQL database for large consumption data collections. Then, we propose a data processing flow to obtain valuable insights from numerous .csv files and an algorithm to assess the load flexibility using large residential and commercial profile datasets from the USA, estimating plausible values of the flexibility provided by two categories of consumers.

Study on the Key Technology and Application of UAV Surveying and Mapping Data Processing

With the rapid development of computer technology, the development of UAV in China is faster and faster, which has become an important tool in modern surveying and mapping. Through the integrated system composed of remote sensing technology, UAV can more accurately map out the relevant data, which will better solve the data difficult to be measured manually, including land, mine, environment, emergency, urban construction, etc. By improving the key technology of UAV surveying and mapping data processing, we can improve the accuracy of measurement, which is the focus of this paper. Firstly, the composition and data processing flow of UAV are analyzed. Then, this paper analyzes the key technology of UAV surveying and mapping data processing. Finally, this paper puts forward the main application of UAV in surveying and mapping.

NFC-Based Wearable Optoelectronics Working with Smartphone Application for Untact Healthcare.

With growing interest in healthcare, wearable healthcare devices have been developed and researched. In particular, near-field communication (NFC) based wearable devices have been actively studied for device miniaturization. Herein, this article proposes a low-cost and convenient healthcare system, which can monitor heart rate and temperature using a wireless/battery-free sensor and the customized smartphone application. The authors designed and fabricated a customized healthcare device based on the NFC system, and developed a smartphone application for real-time data acquisition and processing. In order to achieve compact size without performance degradation, a dual-layered layout is applied to the device. The authors demonstrate that the device can operate as attached on various body sites such as wrist, fingertip, temple, and neck due to outstanding flexibility of device and adhesive strength between the device and the skin. In addition, the data processing flow and processing result are presented for offering heart rate and skin temperature. Therefore, this work provides an affordable and practical pathway for the popularization of wireless wearable healthcare system. Moreover, the proposed platform can easily delivery the measured health information to experts for contactless/personal health consultation.

A Perfect Match: Converging and Automating Privacy and Security Impact Assessment On-the-Fly

As the upsurge of information and communication technologies has become the foundation of all modern application domains, fueled by the unprecedented amount of data being processed and exchanged, besides security concerns, there are also pressing privacy considerations that come into play. Compounding this issue, there is currently a documented gap between the cybersecurity and privacy risk assessment (RA) avenues, which are treated as distinct management processes and capitalise on rather rigid and make-like approaches. In this paper, we aim to combine the best of both worlds by proposing the APSIA (Automated Privacy and Security Impact Assessment) methodology, which stands for Automated Privacy and Security Impact Assessment. APSIA is powered by the use of interdependency graph models and data processing flows used to create a digital reflection of the cyber-physical environment of an organisation. Along with this model, we present a novel and extensible privacy risk scoring system for quantifying the privacy impact triggered by the identified vulnerabilities of the ICT infrastructure of an organisation. We provide a prototype implementation and demonstrate its applicability and efficacy through a specific case study in the context of a heavily regulated sector (i.e., assistive healthcare domain) where strict security and privacy considerations are not only expected but mandated so as to better showcase the beneficial characteristics of APSIA. Our approach can complement any existing security-based RA tool and provide the means to conduct an enhanced, dynamic and generic assessment as an integral part of an iterative and unified risk assessment process on-the-fly. Based on our findings, we posit open issues and challenges, and discuss possible ways to address them, so that such holistic security and privacy mechanisms can reach their full potential towards solving this conundrum.

An Empirical Study on Data Flow Bugs in Business Processes

An increasing number of service-based business processes are being developed with the booming of BPaaS (Business Process as a Service) in cloud computing. The profits and performance of enterprises strongly depend on the soundness of their processes being bereft of control flow and data flow bugs. Although some work has focused on the detection of control flow bugs, few studies have comprehensively and empirically investigated data flow bugs in business processes. To this end, we report an empirical study on data flow bugs in business (BPEL) processes. Our analysis of 178 real-world BPEL processes reveals that data flow bugs are surprisingly common: 94 BPEL processes involve data flow bugs, among which redundant output is predominant. The distribution and common scenarios of data flow bugs provide a reference for BPEL process designers. We also investigate the correlation between process complexity metrics and data flow bugs. Based on the statistics of the process complexity metrics and data flow bugs in our empirical study, we present a method to select appropriate metrics as features of BPEL processes and utilize state-of-the-art supervised learning algorithms to predict data flow bugs in an unseen BPEL process. The prediction accuracies of the different classification algorithms exceed 90 percent on average when using our selected metrics.