Visualization Of Sensor Data Research Articles

Fuzzy based Irrigation Control System for Indian Subcontinent

Water resource usage should be optimized as there is always a scarcity. This paper aims to provide an efficient way of water using sense and weather data and implementing a fuzzy decision model. An automated intelligent watering system is proposed in this paper using the internet of things and fuzzy logic. The weather data, coupled with Temperature, Relative Humidity, and soil moisture sensor data, is used to decide whether to switch on/off the motor. In-house-created prototypes of ground-moving robots have soil moisture, digital humidity, and temperature sensors implanted in them. The soil moisture sensor is attached to the Rack and pinion mechanism. The soil moisture sensor is pushed into the soil when the pinion rotates. It minimizes the use of sensors by using a distributed sensing method. Based on data obtained from sensors and meteorological information, the system will use this information to decide whether to Switch on/off the sprinkler motor. A fuzzy logic-based system decision is implemented on the input sensor and weather data, and the model will decide to switch on/off the actuator. An accuracy of 97% is achieved. The Android app is used to visualize sensor data, based on which the farmer can manually control the motor.

Tiny and error-tolerant ConvLSTM for event-based vision sensor with optimized event representation and ReRAM computation-in-memory

This paper proposes a tiny and error-tolerant convolutional long short-term memory network (ConvLSTM) for event-based vision sensor data processing with computation-in-memory (CiM) that utilizes resistive random-access memory (ReRAM). To handle the event data by ConvLSTM, the type and parameter of event representation are optimized to pre-process event data. To realize tiny and error-tolerant ConvLSTM CiM, reductions of the bit precision and error tolerance in each ConvLSTM layer and weight are considered. On the other hand, reducing the bit precision degrades inference accuracy and error tolerance. ReRAM has non-idealities such as conductance variation and wearing out by Set/Reset cycles that induces bit inversion. Thus, the bit error rate at each Set/Reset cycle is investigated, and the bit precision of the weight is optimized to accept a bit error rate of 0.1% in simulation. The proposed tiny ConvLSTM can reduce the size of memory by 91% compared with non-optimized ConvLSTM for which weight is represented in 32-bit precision.

A sensor data mining process for identifying root causes associated with low yield in semiconductor manufacturing

PurposeThe purpose of this paper is to identify the root cause of low yield problems in the semiconductor manufacturing process using sensor data continuously collected from manufacturing equipment and describe the process environment in the equipment.Design/methodology/approachThis paper proposes a sensor data mining process based on the sequential modeling of random forests for low yield diagnosis. The process consists of sequential steps: problem definition, data preparation, excursion time and critical sensor identification, data visualization and root cause identification.FindingsA case study is conducted using real-world data collected from a semiconductor manufacturer in South Korea to demonstrate the effectiveness of the diagnosis process. The proposed model successfully identified the excursion time and critical sensors previously identified by domain engineers using costly manual examination.Originality/valueThe proposed procedure helps domain engineers narrow down the excursion time and critical sensors from the massive sensor data. The procedure's outcome is highly interpretable, informative and easy to visualize.

Photoelectric Memristor-Based Machine Vision for Artificial Intelligence Applications

With the rapid development of next-generation artificial intelligence technology, research on advanced machine vision has received extensive attention. It is well-known that significant progress has been made in artificial vision systems based on light sensors, but the separate light sensor and memory require additional time for information transfer to realize computation due to the limitation of the von Neumann architecture, which delays the computational speed and hinders large-scale integration. In recent years, the emergence of photoelectric memristors has brought new inspiration to the study of machine vision, which is expected to overcome the above problems. Photoelectric memristors can not only respond directly to light stimuli but also perform temporary memory and real-time processing of visual information and sensory data, providing a promising hardware foundation for the development of artificial vision systems. In this review, the background and related theory of photoelectric memristors and machine vision are first introduced. Then, the research progress of photoelectric memristors and machine vision based on them is reviewed. Finally, the existing problems impeding the progress of machine vision based on photoelectric memristors are summarized, and the future development is predicted.

Vision-Based Semantic Segmentation in Scene Understanding for Autonomous Driving: Recent Achievements, Challenges, and Outlooks

Scene understanding plays a crucial role in autonomous driving by utilizing sensory data for contextual information extraction and decision making. Beyond modeling advances, the enabler for vehicles to become aware of their surroundings is the availability of visual sensory data, which expand the vehicular perception and realizes vehicular contextual awareness in real-world environments. Research directions for scene understanding pursued by related studies include person/vehicle detection and segmentation, their transition analysis, lane change, and turns detection, among many others. Unfortunately, these tasks seem insufficient to completely develop fully-autonomous vehicles i.e., achieving level-5 autonomy, travelling just like human-controlled cars. This latter statement is among the conclusions drawn from this review paper: scene understanding for autonomous driving cars using vision sensors still requires significant improvements. With this motivation, this survey defines, analyzes, and reviews the current achievements of the scene understanding research area that mostly rely on computationally complex deep learning models. Furthermore, it covers the generic scene understanding pipeline, investigates the performance reported by the state-of-the-art, informs about the time complexity analysis of avant garde modeling choices, and highlights major triumphs and noted limitations encountered by current research efforts. The survey also includes a comprehensive discussion on the available datasets, and the challenges that, even if lately confronted by researchers, still remain open to date. Finally, our work outlines future research directions to welcome researchers and practitioners to this exciting domain.

A New Positioning Method for Climbing Robots Based on 3D Model of Transmission Tower and Visual Sensor.

With the development of robot technology and the extensive application of robots, the research on special robots for some complex working environments has gradually become a hot topic. As a special robot applied to transmission towers, the climbing robot can replace humans to work at high altitudes to complete bolt tightening, detection, and other tasks, which improves the efficiency of transmission tower maintenance and ensures personal safety. However, it is mostly the ability to autonomously locate in the complex environment of the transmission tower that limits the industrial applications of the transmission tower climbing robot. This paper proposes an intelligent positioning method that integrates the three-dimensional information model of transmission tower and visual sensor data, which can assist the robot in climbing and adjusting to the designated working area to guarantee the working accuracy of the climbing robots. The experimental results show that the positioning accuracy of the method is within 1 cm.

Integration of Wireless Sensor Networks in Environmental Monitoring Cyber Infrastructure

Abstract: Wireless sensor networks (WSNs) have great potential to revolutionize many science and engineering domains. We present a novel environmental monitoring system with a focus on overall system architecture for seamless integration of wired and wireless sensors for long-term, remote, and near-real-time monitoring. We also present a unified framework for sensor data collection, management, visualization, dissemination, and exchange, conforming to the new Sensor Web Enablement standard. Some initial field testing results are also presented. The monitoring system is being integrated into the Texas Environmental Observatory infrastructure for long-term operation. As part of the integrated system, a new WSN-based soil moisture monitoring system is developed and deployed to support hydrologic monitoring and modeling research. This work represents a significant contribution to the empirical study of the emerging WSN technology. We address many practical issues in real-world application scenarios that are often neglected in the existing WSN research

Driver awareness collision/proximity detection system for heavy vehicles based on deep neural network

Heavy vehicles are essentially used to perform several critical tasks in construction industry. These heavy vehicles are different in shape and structure according to their capabilities and needs on the construction sites. Bigger size and different design of these vehicles result in extended blind spot compare to the normal vehicles, thus risking the equipment and workers operating around them. Each year, a large number of incidents are reported involving human injuries or loss of life due to improper handling of blind spots in heavy vehicles. In this paper we developed a deep learning based collision detection and driver assistance methodology by removing the blind spots in haul truck. In the proposed methodology, vision sensor data is used for automatic detection and classification of objects, employing the YOLO-v5 network architecture. For distance measurement of the objects we used stereo imaging camera Intel RealSense D455. The proposed algorithm detects and classify the object around the vehicle and provide the captioning to describe the nature of object to the driver. In addition, the algorithm also measures the distance of objects from the vehicle. The experimental results validate the performance of our proposed methodology in real time.

Health monitoring of a conveyor belt system using machine vision and real-time sensor data

To truly achieve prognostic health monitoring of complex manufacturing systems, real-time data from multiple sensors must be continuously monitored in order to have a system-wide view of the health of the machine and the effect of different operating conditions on system performance. Belt misalignment is a critical factor in the conveyor belt system traditionally assessed through mechanical means. This research successfully implements multiple sensors and communication protocols along with machine vision for monitoring the health of a conveyor belt system. Data is captured using industrial-grade sensors and conditioned in a PLC. This information is sent to a wireless radio frequency module that transmits wirelessly to a remote receiver paired to an IoT gateway. Images taken by a camera are simultaneously processed by a local computer running a machine vision algorithm used to establish if the conveyor belt is operating normally. Both the sensor data and the machine vision information are sent to the cloud for remote users to monitor the system's operating conditions and to detect potential failure prior to its occurrence. The system communicates using a set of Radio Frequency modules, Zumlink™, and the range of this system was tested and determined to be greater than 2.5 miles with obstructions typical in a manufacturing environment.

Research on Positioning Method in Underground Complex Environments Based on Fusion of Binocular Vision and IMU.

Aiming at the failure of traditional visual slam localization caused by dynamic target interference and weak texture in underground complexes, an effective robot localization scheme was designed in this paper. Firstly, the Harris algorithm with stronger corner detection ability was used, which further improved the ORB (oriented FAST and rotated BRIEF) algorithm of traditional visual slam. Secondly, the non-uniform rational B-splines algorithm was used to transform the discrete data of inertial measurement unit (IMU) into second-order steerable continuous data, and the visual sensor data were fused with IMU data. Finally, the experimental results under the KITTI dataset, EUROC dataset, and a simulated real scene proved that the method used in this paper has the characteristics of stronger robustness, better localization accuracy, small size of hardware equipment, and low power consumption.

Unknown Object Detection Using a One-Class Support Vector Machine for a Cloud-Robot System.

Inter-robot communication and high computational power are challenging issues for deploying indoor mobile robot applications with sensor data processing. Thus, this paper presents an efficient cloud-based multirobot framework with inter-robot communication and high computational power to deploy autonomous mobile robots for indoor applications. Deployment of usable indoor service robots requires uninterrupted movement and enhanced robot vision with a robust classification of objects and obstacles using vision sensor data in the indoor environment. However, state-of-the-art methods face degraded indoor object and obstacle recognition for multiobject vision frames and unknown objects in complex and dynamic environments. From these points of view, this paper proposes a new object segmentation model to separate objects from a multiobject robotic view-frame. In addition, we present a support vector data description (SVDD)-based one-class support vector machine for detecting unknown objects in an outlier detection fashion for the classification model. A cloud-based convolutional neural network (CNN) model with a SoftMax classifier is used for training and identification of objects in the environment, and an incremental learning method is introduced for adding unknown objects to the robot knowledge. A cloud–robot architecture is implemented using a Node-RED environment to validate the proposed model. A benchmarked object image dataset from an open resource repository and images captured from the lab environment were used to train the models. The proposed model showed good object detection and identification results. The performance of the model was compared with three state-of-the-art models and was found to outperform them. Moreover, the usability of the proposed system was enhanced by the unknown object detection, incremental learning, and cloud-based framework.

Determination of ewe behaviour around lambing time and prediction of parturition 7 days prior to lambing by tri-axial accelerometer sensors in an extensive farming system

Context Lamb loss and dyctocia are two major challenges in extensive farming systems. While visual observation can be impractical due to the large sizes of paddocks, number of animals and high labour cost, wearable sensors can be used to monitor the behaviour of ewes as there might be changes in their activities prior to lambing. This provides sufficient time for the farm manager to nurse those ewes that are at risk of dyctocia. Aim The objective of this study was to determine whether the behaviour of a pregnant ewe could predict the time of parturition. Methods Two separate trials were conducted: the first trial (T1), with 32 ewes, included human/video observations, and the second trial (T2), with 165 ewes, conducted with no humans present, to emulate real extensive farming settings. The ewes were fitted with tri-axial accelerometer sensors by means of halters. Three-dimensional movement data were collected for a period of at least 7 and 14 days in T1 and T2 respectively. The sensor units were retrieved, and their data downloaded using ActiGraph software. Ewe behaviour was determined through support vector machine learning (SVM) algorithm, including licking, grazing, rumination, walking, and idling. The behaviours of ewes predicted by analysis of sensor data were compared with behaviours determined using visual observation (video recordings), with time synchronisation to validate the results. Deep learning and neural-network algorithms were used to predict lambing time. Key results The concordance percentages between visual observation and sensor data were 90 ± 11, 81 ± 15, 95 ± 10, 96 ± 6, and 93 ± 8% ± s.d. for grazing, licking, rumination, idling, and walking respectively. The deep-learning model predicted the time of lambing with 90% confidence via a quantile regression method, which can be interpereted as 90% prediction intervals, and shows that the time of lambing can be predicted with reasonable confidence approximately 240 h before the actual lambing events. Conclusion It was possible to predict the time of parturition up to 10 days before lambing. Implications The behaviour of ewes around lambing time has a direct effect on the survival of the lambs and therefore plays an important part in animal management. This knowledge could improve the productivity of sheep and considerably decrease lamb mortality rates.

Lossless Compression of Neuromorphic Vision Sensor Data Based on Point Cloud Representation

Lossless Compression of Neuromorphic Vision Sensor Data Based on Point Cloud Representation

Cloud Server and Internet of Things Assisted System for Stress Monitoring

Currently, the Internet of Things (IoT) has gained attention for its capability for real-time monitoring. The advancement in sensor and wireless communication technology has led to the widespread adoption of IoT technology in distinct applications. The cloud server, in conjunction with the IoT, enables the visualization and analysis of real-time sensor data. The literature concludes that there is a lack of remote stress-monitoring devices available to assist doctors in observing the real-time stress status of patients in the hospital and in rehabilitation centers. To overcome this problem, we have proposed the use of the IoT and cloud-enabled stress devices to detect stress in a real-time environment. The IoT-enabled stress device establishes piconet communication with the master node to allow visualization of the sensory data on the cloud server. The threshold value (volt) for real-time stress detection by the stress device is identified by experimental analysis using MATLAB based on the results obtained from the performance of three different physical-stress generating tasks. In addition, the stress device is interfaced with the cloud server, and the sensor data are recorded on the cloud server. The sensor data logged into the cloud server can be utilized for future analysis.

Seeing under the cover with a 3D U-Net: point cloud-based weight estimation of covered patients

PurposeBody weight is a crucial parameter for patient-specific treatments, particularly in the context of proper drug dosage. Contactless weight estimation from visual sensor data constitutes a promising approach to overcome challenges arising in emergency situations. Machine learning-based methods have recently been shown to perform accurate weight estimation from point cloud data. The proposed methods, however, are designed for controlled conditions in terms of visibility and position of the patient, which limits their practical applicability. In this work, we aim to decouple accurate weight estimation from such specific conditions by predicting the weight of covered patients from voxelized point cloud data.MethodsWe propose a novel deep learning framework, which comprises two 3D CNN modules solving the given task in two separate steps. First, we train a 3D U-Net to virtually uncover the patient, i.e. to predict the patient’s volumetric surface without a cover. Second, the patient’s weight is predicted from this 3D volume by means of a 3D CNN architecture, which we optimized for weight regression.ResultsWe evaluate our approach on a lying pose dataset (SLP) under two different cover conditions. The proposed framework considerably improves on the baseline model by up to {16}{%} and reduces the gap between the accuracy of weight estimates for covered and uncovered patients by up to {52}{%}.ConclusionWe present a novel pipeline to estimate the weight of patients, which are covered by a blanket. Our approach relaxes the specific conditions that were required for accurate weight estimates by previous contactless methods and thus constitutes an important step towards fully automatic weight estimation in clinical practice.

Towards Robust Robot Control in Cartesian Space Using an Infrastructureless Head- and Eye-Gaze Interface.

This paper presents a lightweight, infrastructureless head-worn interface for robust and real-time robot control in Cartesian space using head- and eye-gaze. The interface comes at a total weight of just 162 g. It combines a state-of-the-art visual simultaneous localization and mapping algorithm (ORB-SLAM 2) for RGB-D cameras with a Magnetic Angular rate Gravity (MARG)-sensor filter. The data fusion process is designed to dynamically switch between magnetic, inertial and visual heading sources to enable robust orientation estimation under various disturbances, e.g., magnetic disturbances or degraded visual sensor data. The interface furthermore delivers accurate eye- and head-gaze vectors to enable precise robot end effector (EFF) positioning and employs a head motion mapping technique to effectively control the robots end effector orientation. An experimental proof of concept demonstrates that the proposed interface and its data fusion process generate reliable and robust pose estimation. The three-dimensional head- and eye-gaze position estimation pipeline delivers a mean Euclidean error of mm for head-gaze and mm for eye-gaze at a distance of 0.3–1.1 m to the user. This indicates that the proposed interface offers a precise control mechanism for hands-free and full six degree of freedom (DoF) robot teleoperation in Cartesian space by head- or eye-gaze and head motion.

Classification of assembly operations using machine learning algorithms based on visual sensor data

Image processing enables the acquisition and transfer of data from manual assembly work spaces into a digital environment. The following paper addresses the question to what extent processes of manual assembly can be reliably derived from visual sensor data and classified by machine learning algorithm. Complex assembly processes are transformed into short, discrete assembly operations which are detected by 3D-image processing and processed by a multilayer neuronal network. In order to differentiate assembly operations based on sensor data, methods of machine learning algorithms are implemented and further developed. A recurrent neural network algorithm is investigated regarding its applicability in combination with the sensor data and the robustness in image recognition.

Hydrodynamic Modeling and Design of Robotic Fish using Slender Body Theory

Design of bio-inspired robotic fish has been an important area of research for applications such as water quality monitoring, aquatic animal behavioral study and underwater exploration. The modeling of robotic fish requires high dimensional analysis, which increases the design complexity. To address this complexity, approximate models are preferred and frequently used in the design of robotic fishes. The accuracy of these models mainly relies on various hydrodynamic parameter identification of which drag coefficient estimation is highly challenging. This paper utilizes a simple deceleration motion model to estimate the drag coefficient effectively so that the linearization error is reduced. The non-linearity in the deceleration model is simplified and mapped as a linear model to predict the drag coefficient. The estimated drag coefficient is used for dynamic modeling of robotic fish using slender body theory. The dynamic model is validated through experimental setup using accelerometer and visual sensor data. The maximum swimming speed of 0.32 ms-1 is achieved at 1.2 Hz caudal fin oscillations.

Graph Attention Layer Evolves Semantic Segmentation for Road Pothole Detection: A Benchmark and Algorithms.

Existing road pothole detection approaches can be classified as computer vision-based or machine learning-based. The former approaches typically employ 2D image analysis/ understanding or 3D point cloud modeling and segmentation algorithms to detect (i.e., recognize and localize) road potholes from vision sensor data, e.g., RGB images and/or depth/disparity images. The latter approaches generally address road pothole detection using convolutional neural networks (CNNs) in an end-to-end manner. However, road potholes are not necessarily ubiquitous and it is challenging to prepare a large well-annotated dataset for CNN training. In this regard, while computer vision-based methods were the mainstream research trend in the past decade, machine learning-based methods were merely discussed. Recently, we published the first stereo vision-based road pothole detection dataset and a novel disparity transformation algorithm, whereby the damaged and undamaged road areas can be highly distinguished. However, there are no benchmarks currently available for state-of-the-art (SoTA) CNNs trained using either disparity images or transformed disparity images. Therefore, in this paper, we first discuss the SoTA CNNs designed for semantic segmentation and evaluate their performance for road pothole detection with extensive experiments. Additionally, inspired by graph neural network (GNN), we propose a novel CNN layer, referred to as graph attention layer (GAL), which can be easily deployed in any existing CNN to optimize image feature representations for semantic segmentation. Our experiments compare GAL-DeepLabv3+, our best-performing implementation, with nine SoTA CNNs on three modalities of training data: RGB images, disparity images, and transformed disparity images. The experimental results suggest that our proposed GAL-DeepLabv3+ achieves the best overall pothole detection accuracy on all training data modalities. The source code, dataset, and benchmark are publicly available at mias.group/GAL-Pothole-Detection.

Making Sense of Neuromorphic Event Data for Human Action Recognition

Neuromorphic vision sensors provide low power sensing and capture salient spatial-temporal events. The majority of the existing neuromorphic sensing work focus on object detection. However, since they only record the events, they provide an efficient signal domain for privacy aware surveillance tasks. This paper explores how the neuromorphic vision sensor data streams can be analysed for human action recognition, which is a challenging application. The proposed method is based on handcrafted features. It consists of a pre-processing step for removing the noisy events followed by the extraction of handcrafted local and global feature vectors corresponding to the underlying human action. The local features are extracted considering a set of high-order descriptive statistics from the spatio-temporal events in a time window slice, while the global features are extracted by considering the frequencies of occurrences of the temporal event sequences. Then, low complexity classifiers, such as, support vector machines (SVM) and K-Nearest Neighbours (KNNs), are trained using these feature vectors. The proposed method evaluation uses three groups of datasets: Emulator-based, re-recording-based and native NVS-based. The proposed method has outperformed the existing methods in terms of human action recognition accuracy rates by 0.54%, 19.3%, and 25.61% for E-KTH, E-UCF11 and E-HMDB51 datasets, respectively. This paper also reports results for three further datasets: E-UCF50, R-UCF50, and N-Actions, which are reported for the first time for human action recognition on neuromorphic vision sensor domain.