Multi-sensor Data Processing Research Articles

A panoramic driving perception fusion algorithm based on multi-task learning.

With the rapid development of intelligent connected vehicles, there is an increasing demand for hardware facilities and onboard systems of driver assistance systems. Currently, most vehicles are constrained by the hardware resources of onboard systems, which mainly process single-task and single-sensor data. This poses a significant challenge in achieving complex panoramic driving perception technology. While the panoramic driving perception algorithm YOLOP has achieved outstanding performance in multi-task processing, it suffers from poor adaptability of feature map pooling operations and loss of details during downsampling. To address these issues, this paper proposes a panoramic driving perception fusion algorithm based on multi-task learning. The model training involves the introduction of different loss functions and a series of processing steps for lidar point cloud data. Subsequently, the perception information from lidar and vision sensors is fused to achieve synchronized processing of multi-task and multi-sensor data, thereby effectively improving the performance and reliability of the panoramic driving perception system. To evaluate the performance of the proposed algorithm in multi-task processing, the BDD100K dataset is used. The results demonstrate that, compared to the YOLOP model, the multi-task learning network performs better in lane detection, drivable area detection, and vehicle detection tasks. Specifically, the lane detection accuracy improves by 11.6%, the mean Intersection over Union (mIoU) for drivable area detection increases by 2.1%, and the mean Average Precision at 50% IoU (mAP50) for vehicle detection improves by 3.7%.

Combined onshore and offshore wide-scale seismic data acquisition and imaging for carbon capture and storage exploration in Havnsø, Denmark

Strong global actions for climate change include carbon capture and storage (CCS) as a feasible solution to reach carbon neutrality and raise opportunities for detailed subsurface investigations. An acquisition set-up designed for onshore-offshore zones was maximized for wide-scale high-resolution structural imaging and implemented to cover a domal structure of interest for CCS utilization close to the town of Havnsø (Denmark). The challenges of the combined acquisition and processing of land and marine multisensor data along a 42 km seismic profile are analyzed, the suggested solutions are applied, and the limitations are discussed. On the onshore side, a nodal array and a seismic landstreamer system were simultaneously used, whereas along the transition zone, a marine seismic streamer and ocean-bottom seismometers were added to record the seismic response generated by two seismic vibrator sources. The adopted sensing domains (velocity, acceleration, and pressure) were studied, and different processing steps were evaluated to enable their processing and subsequent data set merging. Results suggest, as the best approach, a separate prestack processing of the different data sets and the computation of new geometries and new surface-consistent residual static correction after their merging. The data acquired in the transition zone illuminate, for the first time, the subsurface geology of the region, delineating an expected domal closure. The final seismic section shows high continuity of the reflections with good resolution along the entire profile, identifying the main reservoir structure and the surrounding areas, which are important to ensure reservoir integrity and safe exploitation over longer time scales. Shallow and deep reflections are consistent with the stratigraphic column from a well log near the profile. The presented study shows a comprehensive workflow for processing such a multisensor data set in onshore and transition zone settings.

Development of Multi Modal Image Fusion Techniques in Night Vision Technology Using Deep Learning

This study is aimed at exploring new development in nighttime image processing based on nighttime photography/night vision. We use various learning methodologies based on neural networks to develop multisensor data processing techniques. The ability of Convolutional Neural Networks (CNN) to enhance image quality and clarity under conditions of low light or at night is the focus of our research. To do this A large database has been created, containing 3200 images both taken from the internet as well as our own. This data is preprocessed, normalized and features expanded using some common methods of data augmentation in order to optimize model performance. We compared our proposed CNN model with existing literature through a series of experiments. Performance was measured using Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Feature Similarity Index (FSIM). The Results are now in. They showed that our model can improve various metrics with higher scores. And they show the real-time monitoring capabilities in urban streets for our model. This research brings multisensor image fusion based on night-adjacent techniques forward from night vision technology and provides insight into practical and potential future directions for using deep learning techniques to improve safety and security in many different environments.

Nonlinear dimensionality reduction methods for potentiometric multisensor systems data analysis

AbstractElectrochemical multisensor systems were proven to be a very perspective research direction in modern analytical chemistry. The multisensor approach assumes an employment of cross‐sensitive chemical sensors in combination with multivariate data processing methods. Dimensionality reduction of the data obtained from multisensor systems is a very important step and it is mostly based on the traditional tools of chemometrics, such as Principal Component Analysis (PCA). In case of chemically complex samples, the response of multisensor systems may have a complex nonlinear nature and the use of linear modelling methods does not seem optimal. However, the potential of nonlinear dimensionality reduction methods in the processing of multisensor data has not yet been systematically studied. In this report we aim to fill this gap and assess the performance of various nonlinear dimensionality reduction tools: Isomap, Self‐Organizing Kohonen Maps, and Autoencoder. These methods were explored using three datasets from potentiometric multisensor systems obtained in various real applications. It was shown that nonlinear dimensionality reduction methods give the possibility to obtain additional and more detailed information about the analyzed objects/processes compared to PCA. However, calculation time for nonlinear dimensionality reduction methods essentially exceeds that for PCA, and it can be a limiting factor for application of such algorithms.

Gaussian Mixture Cardinalized Probability Hypothesis Density(GM-CPHD): A Distributed Filter Based on the Intersection of Parallel Inverse Covariances

A distributed GM-CPHD filter based on parallel inverse covariance crossover is designed to attenuate the local filtering and uncertain time-varying noise affecting the accuracy of sensor signals. First, the GM-CPHD filter is identified as the module for subsystem filtering and estimation due to its high stability under Gaussian distribution. Second, the signals of each subsystem are fused by invoking the inverse covariance cross-fusion algorithm, and the convex optimization problem with high-dimensional weight coefficients is solved. At the same time, the algorithm reduces the burden of data computation, and data fusion time is saved. Finally, the GM-CPHD filter is added to the conventional ICI structure, and the generalization capability of the parallel inverse covariance intersection Gaussian mixture cardinalized probability hypothesis density (PICI-GM-CPHD) algorithm reduces the nonlinear complexity of the system. An experiment on the stability of Gaussian fusion models is organized and linear and nonlinear signals are compared by simulating the metrics of different algorithms, and the results show that the improved algorithm has a smaller metric OSPA error than other mainstream algorithms. Compared with other algorithms, the improved algorithm improves the signal processing accuracy and reduces the running time. The improved algorithm is practical and advanced in terms of multisensor data processing.

Application of Intelligent Sensor Algorithm in Student Management Information Fusion

In order to improve the effectiveness of college student management and promote the integration of college student management information, this paper applies intelligent sensor algorithms to student management. Moreover, this paper combines uncertainty theory with multisensor data fusion technology to establish a complete set of multisensor data processing tools for student information and provides a complete mathematical theoretical framework for the principles of student management information fusion. In addition, in view of the problem of comparing a large number of mixed data of information sources, it is necessary to transfer the information fragments obtained by each sensor to a common set so that the information fragments expressed in different sets can be integrated. Finally, this paper constructs an intelligent student management model and conducts research in combination with simulation experiments. Through simulation research, it can be known that the method proposed in this paper can effectively improve the effect of student management.

Multisensor data processing in dimensional metrology for collaborative measurement of a laser plane sensor combined to a touch probe

Dimensional inspection, which consists in verifying the geometric conformity of parts in terms of ISO specifications, is an essential step in the product life cycle. In recent years, new optical measurement technologies, such as laser sensors or structured light sensors, have become increasingly important. However, the quality of the data limits their expansive use for metrology applications and touch probes with CMM (Coordinate Measurement Machine) remain the reference in terms of measurement quality. On the other hand, the cooperative use of an optical system and a contact sensor presents a good solution which takes advantage of the characteristics of both systems while maintaining a quality level satisfying the requirements of metrology. However, measurement with a laser sensor and a touch probe mounted on a CMM results in two heterogeneous point clouds that must be processed before the final evaluation of specifications. This paper presents a methodology that addresses the processing of measurement data in this scenario. The proposed approach consists of four phases. In the first phase, the measurement data resulting from the laser sensor is processed to remove outliers, reduce noise and partition the point cloud. In the second phase, the point cloud resulting from both sensors are unified in a unique coordinate system. In the third phase, a fusion method is proposed to fuse the point clouds. The specifications are finally evaluated based on the fused point clouds in the fourth phase. A case study is conducted to illustrate the proposed approach with the measurement data of a test part defined by its CAD model and specifications. The results are compared with a reference inspection report obtained by a touch probe measurement system.

Remote Sensing of Ecosystem Structure: Fusing Passive and Active Remotely Sensed Data to Characterize a Deltaic Wetland Landscape

A project was constructed to integrate remotely sensed data from multiple sensors and platforms to characterize range of ecosystem characteristics in the Peace–Athabasca Delta in Northern Alberta, Canada. The objective of this project was to provide a framework for the processing of multisensor data to extract ecosystem information describing complex deltaic wetland environments. The data used in this study was based on a passive satellite-based earth observation multispectral sensor (Sentinel-2) and airborne discrete light detection and ranging (LiDAR). The data processing strategy adopted here allowed us to employ a data mining approach to grouping of the input variables into ecologically meaningful clusters. Using this approach, we described not only the reflective characteristics of the cover, but also ascribe vertical and horizontal structure, thereby differentiating spectrally similar, but ecologically distinct, ground features. This methodology provides a framework for assessing the impact of ecosystems on radiance, as measured by Earth observing systems, where it forms the basis for sampling and analysis. This final point will be the focus of future work.

A Multi-Sensor Unoccupied Aerial System Improves Characterization of Vegetation Composition and Canopy Properties in the Arctic Tundra

Changes in vegetation distribution, structure, and function can modify the canopy properties of terrestrial ecosystems, with potential consequences for regional and global climate feedbacks. In the Arctic, climate is warming twice as fast as compared to the global average (known as ‘Arctic amplification’), likely having stronger impacts on arctic tundra vegetation. In order to quantify these changes and assess their impacts on ecosystem structure and function, methods are needed to accurately characterize the canopy properties of tundra vegetation types. However, commonly used ground-based measurements are limited in spatial and temporal coverage, and differentiating low-lying tundra plant species is challenging with coarse-resolution satellite remote sensing. The collection and processing of multi-sensor data from unoccupied aerial systems (UASs) has the potential to fill the gap between ground-based and satellite observations. To address the critical need for such data in the Arctic, we developed a cost-effective multi-sensor UAS (the ‘Osprey’) using off-the-shelf instrumentation. The Osprey simultaneously produces high-resolution optical, thermal, and structural images, as well as collecting point-based hyperspectral measurements, over vegetation canopies. In this paper, we describe the setup and deployment of the Osprey system in the Arctic to a tundra study site located in the Seward Peninsula, Alaska. We present a case study demonstrating the processing and application of Osprey data products for characterizing the key biophysical properties of tundra vegetation canopies. In this study, plant functional types (PFTs) representative of arctic tundra ecosystems were mapped with an overall accuracy of 87.4%. The Osprey image products identified significant differences in canopy-scale greenness, canopy height, and surface temperature among PFTs, with deciduous low to tall shrubs having the lowest canopy temperatures while non-vascular lichens had the warmest. The analysis of our hyperspectral data showed that variation in the fractional cover of deciduous low to tall shrubs was effectively characterized by Osprey reflectance measurements across the range of visible to near-infrared wavelengths. Therefore, the development and deployment of the Osprey UAS, as a state-of-the-art methodology, has the potential to be widely used for characterizing tundra vegetation composition and canopy properties to improve our understanding of ecosystem dynamics in the Arctic, and to address scale issues between ground-based and airborne/satellite observations.

Magnetic Induction Tomography Spectroscopy for Structural and Functional Characterization in Metallic Materials.

Magnetic induction tomography (MIT) is a powerful imaging system for monitoring the state of metallic materials. Tomographic methods enable automatic inspection of metallic samples making use of multi-sensor measurements and data processing of eddy current-based sensing from mutual inductances. This paper investigates a multi-frequency MIT using both amplitude and phase data. The image reconstruction algorithm is based on a novel spectrally-correlative total variation method allowing an efficient and all-in-one spectral reconstruction. Additionally, the paper shows the rate of change in spectral images with respect to the excitation frequencies. Using both spectral maps and their spectral derivative maps, one can derive key structural and functional information regarding the material under test. This includes their type, size, number, existence of voids and cracks. Spectral maps can also give functional information, such as mechanical strains and their thermal conditions and composition.

Fog-Based Marine Environmental Information Monitoring Toward Ocean of Things

The deepening of ocean measurement work requires higher transmission bandwidth and information calculation efficiency, which provides an opportunity for fog computing. Compared with cloud computing, fog computing shows distribution because it concentrates data processing and application on devices at the edge of the network. In this article, the Ocean of Things (OoT) framework is designed for marine environment monitoring based on the Internet of Things technology. The OoT is divided into three layers: 1) data acquisition layer; 2) fog layer; and 3) cloud layer. In the fog layer, in order to complete the quality control of the sensor measurement data, we use the numerical gradient-based method to process the original acquisition data. An improved D-S algorithm is designed for multisensor information fusion, reducing the data capacity and improving data quality. In the cloud layer, we build ocean information change models based on the fog layer data to predict the dynamic ocean environment. The designed fog layer is evaluated based on marine multisensor information. The results have shown that fog-based multisensor data processing shows low time consumption and high reliability. Moreover, this article uses real temperature data sets to evaluate the prediction accuracy of the cloud model. Finally, we tested the performance of the designed OoT framework with multiple data sets. The simulation results show that the framework can improve the efficiency of data utilization at sea and improve the efficiency of information utilization.

Guest Editorial Special Issue on Next-Generation Smart Body Sensor Networks: From Autonomic Body Sensors to Cognitive Body Sensor Network Ecosystems

Body sensor networks (BSNs) have originally emerged as a potentially disruptive technology for continuous physiological and behavioral monitoring of users. After more than a decade of their introduction, researchers have addressed a number of important research challenges. The research community, however, further needs to deal with more innovative concepts focusing on autonomic, cognitive, and smart BSNs at each level of the BSN architecture: sensors, device, networking, middleware, multi-sensor data processing and fusion, systems, and data and semantics. In addition, the proposed Special Issue targets the latest research findings on the analysis, definition, implementation, and innovative solutions related to models, architectures, algorithms, protocols, signal processing, hardware at sensor, system, and systems-of-systems level toward the design and development of cognitive BSN ecosystems. Next generation of BSN architectures (namely, BSN 2.0) will be also synergistically merged with the Internet of Things technology to foster several application domains: healthcare, fitness and wellness, smart cities, smart factories, on-field emergency management systems, and so on.

A Safety Computer System Based on Multi-Sensor Data Processing.

The safety computer in the train control system is designed to be the double two-vote-two architecture. If safety-critical multi-input data are inconsistent, this may cause non-strict multi-sensor data problems in the output. These kinds of problems may directly affect the decision making of the safety computer and even pose a serious threat to the safe operation of the train. In this paper, non-strict multi-sensor data problems that exist in traditional safety computers are analyzed. The input data are classified based on data features and safety computer features. Then, the input data that cause non-strict multi-sensor data problems are modeled. Fuzzy theory is used in the safety computer to process multi-sensor data and to avoid the non-strict multi-sensor problems. The fuzzy processing model is added into the onboard double two-vote-two architecture safety computer platform. The fuzzy processing model can be divided into two parts: improved fuzzy decision tree and improved fuzzy weighted fusion. Finally, the model is verified based on two kinds of data. Verification results indicate that the fuzzy processing model can effectively reduce the non-strict identical problems and improve the system efficiency on the premise of ensuring the data reliability.

Machine learning approaches to understand the influence of urban environments on human’s physiological response

This research proposes a framework for signal processing and information fusion of spatial-temporal multi-sensor data pertaining to understanding patterns of humans physiological changes in an urban environment. The framework includes signal frequency unification, signal pairing, signal filtering, signal quantification, and data labeling. Furthermore, this paper contributes to human-environment interaction research, where a field study to understand the influence of environmental features such as varying sound level, illuminance, field-of-view, or environmental conditions on humans’ perception was proposed. In the study, participants of various demographic backgrounds walked through an urban environment in Zürich, Switzerland while wearing physiological and environmental sensors. Apart from signal processing, four machine learning techniques, classification, fuzzy rule-based inference, feature selection, and clustering, were applied to discover relevant patterns and relationship between the participants’ physiological responses and environmental conditions. The predictive models with high accuracies indicate that the change in the field-of-view corresponds to increased participant arousal. Among all features, the participants’ physiological responses were primarily affected by the change in environmental conditions and field-of-view.

Implementation of asymmetric processing on multi core processors to implement IOT applications on GNU/Linux framework

Internet of Things brought in a bigger computing challenges where there came a need for running tasks in a multi-sensor and large data processing is involved. In order to implement this requirement multiprocessors are being used for implementation of IoT Gateways. There comes a need for specific tasks having a resource dedicated for its job. To fulfill this we face a hurdle in choosing dedicated processor or shared processor in a Symmetric Processing Architecture. Dedicated processor are the one in which all the tasks are being processed on a single core where as in fair share processors specific processes are assigned to specific cores. Symmetric processing makes use of dedicated processors where as Asymmetric processor makes use of shared processors. Asymmetric Multi Processing can be used in real time applications in order to solve real time problems, one such platform is IOT. In this paper we have evaluated Asymmetric processing on GNU/Linux Platform to test multiple threads running on different multi-core processors architectures to realize the same for running IOT applications having higher computational requirements in the future.

Multi-Dimensionality of Uncertainty in big Geospatial Sensor Data

Developments in sensor technology have contributed immensely to the growth of big geospatial sensor data. Moreover, advances in telecommunications have made it possible to use in-situ sensors to capture and transfer data about the environment in near real-time. The combination of various sensor types within a predefined geographic space and the possibility of making measurements at high temporal resolution contributes to a better understanding of our environment while also generating big geospatial sensor data. Big data from multi-sensor networks, particularly those that capture dynamic characteristics of the environment, have not been spared the challenges that face other types of big data. Specifically, the quality of data and the propagation of uncertainty through the multi-sensor data processing workflows have remained a major concern in the big geospatial sensor data research community. Attempts to document, quantify and communicate the uncertainty associated with sensor data and related sensor network outputs have been made mainly in the context of individual projects. This paper aims to document the state-of-art in defining uncertainty with regard to multi-sensor geospatial data. In particular, we analyse the current literature to outline different types of uncertainty, and document methods for handling uncertainty in the different stages of multi-sensor geospatial data collection, processing and delivery.

Enhanced frame expansion via configuring filterbank topology for rapid processing of multi-sensor vibration data with applications to turbo-machinery fault diagnosis

Multi-sensor deployment is usually utilized in recent structural health monitoring of rotating machinery. Conventional wavelet transform is widely applied in relevant data processing tasks. However, wavelet transform still suffers from deficiencies such as lacking of frequency adjustability and shift variance property. In this article, an enhanced frame expansion is constructed via filterbank topology approach, which addresses these problems but at the same time can be implemented via fast numerical algorithm. In this scheme, different discrete complex wavelet frames are deployed at different processing stages within the filterbank structure. Enhanced frame expansion is obtained via integration of multiple translation-invariant frames. The derived timescale analyzing properties of enhanced frame expansion, whose wavelet functions form approximate Hilbert transform pairs, were investigated. It is also verified that the enhanced frame expansion satisfies the necessary constraints of complex-valued wavelet frame and other beneficial merits. Another major advantage presented by the enhanced frame expansion, which is different from complex-valued wavelet frame, dwells in its nondyadic frequency-scale paving. Moreover, the constructed enhanced frame expansion is also of low computational burden due to its inheriting of tree structure filterbank. The proposed enhanced frame expansion is applied to vibration feature extraction from multi-sensor data of a turbo-machinery. The processing results from the displacement signal as well as the acceleration signal indicate the rub-impact fault.

The Design and Development of an Omni-Directional Mobile Robot Oriented to an Intelligent Manufacturing System.

In order to transport materials flexibly and smoothly in a tight plant environment, an omni-directional mobile robot based on four Mecanum wheels was designed. The mechanical system of the mobile robot is made up of three separable layers so as to simplify its combination and reorganization. Each modularized wheel was installed on a vertical suspension mechanism, which ensures the moving stability and keeps the distances of four wheels invariable. The control system consists of two-level controllers that implement motion control and multi-sensor data processing, respectively. In order to make the mobile robot navigate in an unknown semi-structured indoor environment, the data from a Kinect visual sensor and four wheel encoders were fused to localize the mobile robot using an extended Kalman filter with specific processing. Finally, the mobile robot was integrated in an intelligent manufacturing system for material conveying. Experimental results show that the omni-directional mobile robot can move stably and autonomously in an indoor environment and in industrial fields.

Multi-Sensor Geomagnetic Prospection: A Case Study from Neolithic Thessaly, Greece

Multi-sensor prospecting is a fast-emerging paradigm in archaeological geophysics. Given suitable ground conditions for navigation, sensor arrays drastically increase efficiency in data collection. In particular, geomagnetic prospecting benefits from this development. Despite these advancements, data processing still lacks a best-practice approach. Conventional processing methods developed for gridded data has been challenged by sensor arrays “roaming” in the landscape. In realization of the issue, the Innovative Geophysical Approaches for the Study of Early Agricultural Villages of Neolithic Thessaly (IGEAN) Project explored various innovative techniques for the betterment of the multi-sensor geomagnetic data processing. As a result, a modular pipeline is produced with minimal user intervention. In addition to standard steps, such as data clipping, various other algorithms have been introduced. This pipeline is tested over 20 Neolithic settlements in Thessaly, Greece, three of which are presented here in detail. The proposed workflow provides drastic improvements over raw data. As a result of these improvements, the IGEAN project revealed astonishing details on architectural elements, settlement enclosures, and paleolandscapes, changing completely the existing perspective of the Neolithic habitation in Thessaly.

Multi-sensor Data Processing and Fusing Based on Kalman Filtering

The background of this paper is the warehouse target localization and tracking system which is composed of a number of wireless sensor nodes. Firstly this paper established a model of warehouse target localization and tracking system, then a model of multi-sensor data preprocessing and data fusion was established, and self-adaptive linear recursive method was used to eliminate outliers of the original measured data. Then least squares fitting filter was used to do filtering and denoising for the measured data. In the end, the data which were measured by multi-sensor can be fused by Kalman Filtering algorithm. Data simulation analysis shows that the use of kalman filtering algorithm for the fusion of the data measured by multi-sensor is to obtain more accurate warehouse target location data, so as to increase the positioning and tracking accuracy of the warehouse target localization and tracking system. Key Words: Wireless Sensor Network,Data Fusion,Kalman Filtering DOI: http://dx.doi.org/10.11591/telkomnika.v11i3.2195