Monitoring Data Processing Research Articles

АНАЛИЗ ИНФ0РМАТИВНЫХр1РИЗНАК0В В ЗАДАЧАХ ОБРАБОТКИ ДАННЫХ АЭРОКОСМИЧЕСКОГО МОНИТОРИНГА

АНАЛИЗ ИНФ0РМАТИВНЫХр1РИЗНАК0В В ЗАДАЧАХ ОБРАБОТКИ ДАННЫХ АЭРОКОСМИЧЕСКОГО МОНИТОРИНГА

New seismic array solution for earthquake observations and hydropower plant health monitoring

We present the novel fusion of seismic safety monitoring data of the hydropower plant in Chirkey (Caucasus Mountains, Russia). This includes new hardware solutions and observation methods, along with technical limitations for three types of applications: (a) seismic monitoring of the Chirkey reservoir area, (b) structure monitoring of the dam, and (c) monitoring of turbine vibrations. Previous observations and data processing for health monitoring do not include complex data analysis, while the new system is more rational and less expensive. The key new feature of the new system is remote monitoring of turbine vibration. A comparison of the data obtained at the test facilities and by hydropower plant inspection with remote sensors enables early detection of hazardous hydrodynamic phenomena.

Detection of Outliers and Imputing of Missing Values for Water Quality UV-VIS Absorbance Time Series

Context: The UV-Vis absorbance collection using online optical captors for water quality detection may yield outliers and/or missing values. Therefore, data pre-processing is a necessary pre-requisite to monitoring data processing. Thus, the aim of this study is to propose a method that detects and removes outliers as well as fills gaps in time series.Method: Outliers are detected using Winsorising procedure and the application of the Discrete Fourier Transform (DFT) and the Inverse of Fast Fourier Transform (IFFT) to complete the time series. Together, these tools were used to analyse a case study comprising three sites in Colombia ((i) Bogotá D.C. Salitre-WWTP (Waste Water Treatment Plant), influent; (ii) Bogotá D.C. Gibraltar Pumping Station (GPS); and, (iii) Itagüí, San Fernando-WWTP, influent (Medellín metropolitan area)) analysed via UV-Vis (Ultraviolet and Visible) spectra.Results: Outlier detection with the proposed method obtained promising results when window parameter values are small and self-similar, despite that the three time series exhibited different sizes and behaviours. The DFT allowed to process different length gaps having missing values. To assess the validity of the proposed method, continuous subsets (a section) of the absorbance time series without outlier or missing values were removed from the original time series obtaining an average 12% error rate in the three testing time series.Conclusions: The application of the DFT and the IFFT, using the 10% most important harmonics of useful values, can be useful for its later use in different applications, specifically for time series of water quality and quantity in urban sewer systems. One potential application would be the analysis of dry weather interesting to rain events, a feat achieved by detecting values that correspond to unusual behaviour in a time series. Additionally, the result hints at the potential of the method in correcting other hydrologic time series.

Методы анализа и обработки данных мониторинга гидротехнических сооружений мелиоративного комплекса

Методы анализа и обработки данных мониторинга гидротехнических сооружений мелиоративного комплекса

Quality prediction modeling for multistage manufacturing based on classification and association rule mining

For manufacturing enterprises, product quality is a key factor to assess production capability and increase their core competence. To reduce external failure cost, many research and methodology have been introduced in order to improve process yield rate, such as TQC/TQM, Shewhart CycleDeming's 14 Points, etc. Nowadays, impressive progress has been made in process monitoring and industrial data analysis because of the Industry 4.0 trend. Industries start to utilize quality control (QC) methodology to lower inspection overhead and internal failure cost. Currently, the focus of QC is mostly in the inspection of single workstation and final product, however, for multistage manufacturing, many factors (like equipment, operators, parameters, etc.) can have cumulative and interactive effects to the final quality. When failure occurs, it is difficult to resume the original settings for cause analysis. To address these problems, this research proposes a combination of principal components analysis (PCA) with classification and association rule mining algorithms to extract features representing relationship of multiple workstations, predict final product quality, and analyze the root-cause of product defect. The method is demonstrated on a semiconductor data set.

AUTOMATED DATA PROCESSING AND INTEGRATION OF LARGE MULTIPLE DATA SOURCES IN GEOHAZARDS MONITORING

The development of geohazard information management system has greatly promoted the wireless automation monitoring technology for geohazards. More monitoring instruments are increasingly used in geohazard monitoring. Consequently, the types of monitoring data become more and more complicated, and massive amount of monitoring data are collected, which raises new demands in data storage and retrieval. In order to meet the requirements of data processing in geohazard monitoring, this paper presents a method of geohazard monitoring data processing, realizing the heterogeneous data integration, data access optimization, and abnormal data processing. Having analyzed the wireless automation monitoring process and the features of geohazard monitoring data, we defined the data integration standards of multiple data sources. Based on this, we developed a Geohazard Monitoring Data Integration System, with optimization in both hardware and software. This system allows automatic integration of large monitoring data from multiple sources. It has important significance for geohazard monitoring and early warning. A Geohazard Monitoring Data Analyzing System based on the monitoring data integrated by this system and data mining technology is developed to fully explore the hidden values of Big Data. Through field tests in Guizhou province with 92 sets of monitoring equipment and 5 types of databases, this method is proven to meet the system requirements with satisfactory performance.

The Numerical Probabilistic Approach to the Processing and Presentation of Remote Monitoring Data

The Numerical Probabilistic Approach to the Processing and Presentation of Remote Monitoring Data

Introduction to the Special Issue on New Approaches for Infrastructure Services

The special issue is dedicated to the current research and innovation challenges encountered at infrastructure-as-a-service level generated by the desire to improve the user experiences and the efficient use of the available resources. The current trends are including the integration of special devices from high performance computing ones to mobile devices, the design of decentralised service-oriented systems, the improvement of the virtualization technologies, the overcome of portability and interoperability issues, or the automation the organisation and management of the back-end resources. Cloud-based applications from the fields of Internet-of-Things and Big Data are expected to challenge the new services. The first paper, entitled SLA-based Secure Cloud Application Development, reports an implementation of the concept of security service level agreements (Security SLAs) and presents a framework that allows application developers to intervine in the secure provisioning of cloud resources and services. The second paper, with the title Impact of Single Parameter Changes on Ceph Cloud Storage Performance, shows how a change of a global parameter of Ceph distributed file system can effect the performance for a range of access patterns when tested with an OpenStack cloud system. The third paper, entitled Multi-objective middleware for distributed VMI repositories in federated Cloud environment, explains the design of easy-to-use interface capable of receiving unmodified and functionally complete virtual machine images from its users, as well as of a system that transparently distribute them to a specific Cloud infrastructure in a federation achieving an improved quality of service. The fourth paper, with the title Architecture of a Scalable Platform for Monitoring Multiple Big Data Frameworks is dedicated to a new, distributed, scalable software platform able to collect, store, query and process monitoring data obtained from multiple Big Data frameworks. The fifth paper, entitled Exposing HPC services in the Cloud: the CloudLightning Approach, refers to a novel a self-organizing and self-managing cloud service delivery system with capabilities to deliver dynamic and tailored services offered by coalitions of heterogeneous cloud resources.

Architecture of a Scalable Platform for Monitoring Multiple Big Data Frameworks

Latest advances in information technology and the widespread growth in different areas are producing large amounts of data. Consequently, in the past decade a large number of distributed platforms for storing and processing large datasets have been proposed. Whether in development or in production, monitoring applications running on these platforms is not an easy task and dedicated tools and platforms were proposed for this scope. In this paper we present a distributed, scalable, highly available platform able to collect, store, query and process monitoring data obtained from multiple Big Data frameworks. We present its architecture and initial results obtained.

Making Large-Scale Systems Observable - Another Inescapable Step Towards Exascale

The effective mastering of extremely parallel HPC system is impossible without deep understanding of all internal processes and behavior of the whole diversity of the components: computing processors and nodes, memory usage, interconnect, storage, whole software stack, cooling and so forth in detail. There are numerous visualization tools that provide information on certain components and system as a whole, but most of them have severe issues that limit appliance in real life, thus becoming inacceptable for the future system scales. Predefined monitoring systems and data sources, lack of dynamic on-the-fly reconfiguration, inflexible visualization and screening options are among the most popular issues.The proposed approach to monitoring data processing resolves the majority of known problems, providing a scalable and flexible solution based on any available monitoring systems and other data sources. The approach implementation is successfully used in every-day practice of the largest in Russia supercomputer center of Moscow State University.

PipsCloud: High performance cloud computing for remote sensing big data management and processing

Massive, large-region coverage, multi-temporal, multi-spectral remote sensing (RS) datasets are employed widely due to the increasing requirements for accurate and up-to-date information about resources and the environment for regional and global monitoring. In general, RS data processing involves a complex multi-stage processing sequence, which comprises several independent processing steps according to the type of RS application. RS data processing for regional environmental and disaster monitoring is recognized as being computationally intensive and data intensive.We propose pipsCloud to address these issues in an efficient manner, which combines recent cloud computing and HPC techniques to obtain a large-scale RS data processing system that is suitable for on-demand real-time services. Due to the ubiquity, elasticity, and high-level transparency of the cloud computing model, massive RS data management and data processing for dynamic environmental monitoring can all be performed on the cloud via Web interfaces. A Hilbert-R+-based data indexing method is employed for the optimal querying and access of RS images, RS data products, and interim data. In the core platform beneath the cloud services, we provide a parallel file system for massive high-dimensional RS data, as well as interfaces for accessing irregular RS data to improve data locality and optimize the I/O performance. Moreover, we use an adaptive RS data analysis workflow management system for on-demand workflow construction and the collaborative processing of a distributed complex chain of RS data, e.g., for forest fire detection, mineral resources detection, and coastline monitoring. Our experimental analysis demonstrated the efficiency of the pipsCloud platform.

Adaptive data processing for real-time nutrition monitoring.

The issue of time-series segmentation refers to the division of continuous sensor data into discrete windows, each of which are processed individually and assigned a class label. Unlike offline data processing scenarios, segmentation for low-power wearable devices must balance the juxtaposed aims of accuracy and computational efficiency. In this paper, we propose a novel scheme for segmentation of sparse sensor data using an adaptive window size approach. Our results are benchmarked on an audio-based nutrition monitoring dataset, and show a reduction in processing overhead of 68% compared to the baseline with fixed window sizes.

JTangCMS: An efficient monitoring system for cloud platforms

Cloud computing has attracted increasing interest in industry and academia. However, due to the constantly expanding scale of cloud platforms, the monitoring of clouds encounters a number of critical challenges related to flexibility, scalability, efficiency, and performance. In this paper, we present JTang Cloud Monitoring System (JTangCMS), an efficient monitoring system for cloud platforms. Our contributions cover the collection, delivery, and processing of monitoring data. For data collection, a flexible and scalable agent is implemented with pluggable monitoring components to collect runtime information from different entities. For data delivery, an efficient and robust data dissemination framework is implemented to transmit the runtime information reliably with high throughput and low latency, based on the Data Distribution Service (DDS). For data processing, a cloud action platform is developed to support cloud management decision-making, based on complex event processing (CEP). Finally, detailed experimental evaluations show the feasibility and efficiency of JTangCMS.

Integration of seismic and image data processing for rockfall monitoring and early warning along transportation networks

The occurrence of rockfall incidents on the transportation network may cause injuries, and even casualties, as well as severe damage to infrastructure such as dwellings, railways, road corridors, etc. Passive protective measures (i.e., rockfall barriers, wire nets, etc.) are mainly deployed by operators of ground transport networks to minimize the impact of detrimental effects on these networks. In conjunction with these passive measures, active rockfall monitoring should ideally include the magnitude of each rockfall, its initial and final position, and the triggering mechanism that might have caused its detachment from the slope. In this work, the operational principle of a low-cost rockfall monitoring and alerting system is being presented. The system integrates measurements from a multi-channel seismograph and commercial cameras as the primary equipment for event detection. A series of algorithms analyze these measurements independently in order to reduce alarms originated by surrounding noise and sources other than rockfall events. The detection methodology employs two different sets of algorithms: Time–frequency analyses of the rockfall event’s seismic signature are performed using moving window pattern recognition algorithms, whereas image processing techniques are utilized to deliver object detection and localization. Training and validation of the proposed approach was performed through field tests that involved manually induced rockfall events and recording of sources (i.e., passing car, walking people) that may cause a false alarm. These validation tests revealed that the seismic monitoring algorithms produce a 4.17 % false alarm rate with an accuracy of 93 %. Finally, the results of a 34-day operational monitoring period are presented and the ability of the imaging system to identify and exclude false alarms is discussed. The entire processing cycle is 10–15 s. Thus, it can be considered as a near real-time system for early warning of rockfall events.

Analysis of camouflage cover spectral characteristics by imaging spectrometer

Subject of Research.The paper deals with the problems of detection and identification of objects in hyperspectral imagery. The possibility of object type determination by statistical methods is demonstrated. The possibility of spectral image application for its data type identification is considered. Method. Researching was done by means of videospectral equipment for objects detection at "Fregat" substrate. The postprocessing of hyperspectral information was done with the use of math model of pattern recognition system. The vegetation indexes TCHVI (Three-Channel Vegetation Index) and NDVI (Normalized Difference Vegetation Index) were applied for quality control of object recognition. Neumann-Pearson criterion was offered as a tool for determination of objects differences. Main Results. We have carried out analysis of the spectral characteristics of summer-typecamouflage cover (Germany). We have calculated the density distribution of vegetation indexes. We have obtained statistical characteristics needed for creation of mathematical model for pattern recognition system. We have shown the applicability of vegetation indices for detection of summer camouflage cover on averdure background. We have presented mathematical model of object recognition based on Neumann-Pearson criterion. Practical Relevance. The results may be useful for specialists in the field of hyperspectral data processing for surface state monitoring.

Case Study of BIM and Cloud–Enabled Real-Time RFID Indoor Localization for Construction Management Applications

AbstractIn the field of construction, indoor localization of mobile construction resources remains a universal challenge. Although discussions have focused on the tracking accuracy and affordability of indoor localization technologies, few efforts have focused on practical criteria such as ease of use, latency, and data visualization and remote sharing. To close this gap in knowledge, this study introduces a building information modeling (BIM) and cloud-enabled radio-frequency identification (RFID) localization system. The system consists of three main components: the passive RFID localization system, the BIM visualization system, and the cloud computing system. The proposed system is tested in a full-scale implementation on an actual construction site. The test was designed and conducted to evaluate the localization accuracy, data latency, and real-time data processing and visualization for remote monitoring. A comprehensive analysis is made of various practical issues based on the test results and panel...

Electronic monitoring devices and data processing

Criminal justice authorities may wish to monitor or to control the location of particular person without resorting to imprisonment. An electronic monitoring device provides pretrial services, probation, or parole the chance to closely supervise convicted person by keeping track of where person are at or in some cases, determine if convicted person is consuming prohibited substances like alcohol or narcotics. Some systems provide tracing within predefined restricted areas. Systems like this are in use for different situations, for example before a criminal trial or after conviction. Authors described different ankle monitor systems that are in use, their advantages, disadvantages and data that collects. For each convicted person data from electronic monitor device is very important. Authors described machine learning method, which based on data could predict whether the convicted person will repeat the criminal offense or not. Methods like this works with hypothesis and gradient descent. At the end authors explored the role of these devices by the law of enforcement sanctions and measures outside the prison.

Feature Selection for Anomaly Detection Using Optical Emission Spectroscopy

To maintain the pace of development set by Moore’s law, production processes in semiconductor manufacturing are becoming more and more complex. The development of efficient and interpretable anomaly detection systems is fundamental to keeping production costs low. As the dimension of process monitoring data can become extremely high anomaly detection systems are impacted by the curse of dimensionality, hence dimensionality reduction plays an important role. Classical dimensionality reduction approaches, such as Principal Component Analysis, generally involve transformations that seek to maximize the explained variance. In datasets with several clusters of correlated variables the contributions of isolated variables to explained variance may be insignificant, with the result that they may not be included in the reduced data representation. It is then not possible to detect an anomaly if it is only reflected in such isolated variables. In this paper we present a new dimensionality reduction technique that takes account of such isolated variables and demonstrate how it can be used to build an interpretable and robust anomaly detection system for Optical Emission Spectroscopy data.

Novel Power Quality Data Analysis and Reporting Framework for Wide-area system of registration and processing of power quality data

This paper presents a novel method for data analysis and visualization, including real-time visual monitoring and proposal for combined area PQ indices on the example of the developed and operational comprehensive system of registration, archiving and data processing for the wide-area monitoring of power quality in a separated part of real power grid with distributed renewable generation. Real case studies related to power quality disturbances are presented.

Joint processing of surface and underground microseismic monitoring data in hard mineral mining

The article discusses optimization of microseismic monitoring in hard mineral mining. A procedure has been developed to estimate accuracy of positioning of seismic event hypocenters. The advantage of combination observation system with distributed underground receiving antennas and additional sensors installed at a distance from a productive seam plane is illustrated. The obtained results are important for optimizing observation systems and improving efficiency of microseismic monitoring in hard mineral mining.