Camera-based Sensor Research Articles

A novel camera-based sensor for real-time wastewater quality monitoring

ABSTRACT Recent advancements have significantly improved turbidity and absorbance measurement techniques, crucial for municipal and industrial wastewater quality monitoring. This experimental system utilizes image analysis and machine learning on monochrome-camera images of real secondary wastewater effluent samples, irradiated with six LEDs, to classify turbidity and predict absorbance in the visible range. It focuses on low turbidity measurements (0–15 nephelometric turbidity units [NTUs]), the hardest challenge for conventional turbidity sensors. Specifically, this camera-based technique was able to classify within a 2 NTU class, 96 turbidity samples collected from a real wastewater treatment plant with precision and accuracy of over 96%. Additionally, it effectively predicted turbidity and absorbance with a neural network, achieving R-squared coefficients of 0.76 and 0.72, respectively. This innovative monitoring system, deployable in several locations of a wastewater treatment plant, not only addresses the limitations of the existing methods for the low turbidity range but also brings the potential for plant-wide process monitoring. Further testing is in progress to validate the proposed approach in other wastewater applications, such as combined sewer overflow monitoring and waste-activated sludge upset detection where more extreme and rapid changes are expected to occur.

Security in Transformer Visual Trackers: A Case Study on the Adversarial Robustness of Two Models.

Visual object tracking is an important technology in camera-based sensor networks, which has a wide range of practicability in auto-drive systems. A transformer is a deep learning model that adopts the mechanism of self-attention, and it differentially weights the significance of each part of the input data. It has been widely applied in the field of visual tracking. Unfortunately, the security of the transformer model is unclear. It causes such transformer-based applications to be exposed to security threats. In this work, the security of the transformer model was investigated with an important component of autonomous driving, i.e., visual tracking. Such deep-learning-based visual tracking is vulnerable to adversarial attacks, and thus, adversarial attacks were implemented as the security threats to conduct the investigation. First, adversarial examples were generated on top of video sequences to degrade the tracking performance, and the frame-by-frame temporal motion was taken into consideration when generating perturbations over the depicted tracking results. Then, the influence of perturbations on performance was sequentially investigated and analyzed. Finally, numerous experiments on OTB100, VOT2018, and GOT-10k data sets demonstrated that the executed adversarial examples were effective on the performance drops of the transformer-based visual tracking. White-box attacks showed the highest effectiveness, where the attack success rates exceeded 90% against transformer-based trackers.

Automatic Post-Stroke Severity Assessment Using Novel Unsupervised Consensus Learning for Wearable and Camera-Based Sensor Datasets

Stroke survivors often suffer from movement impairments that significantly affect their daily activities. The advancements in sensor technology and IoT have provided opportunities to automate the assessment and rehabilitation process for stroke survivors. This paper aims to provide a smart post-stroke severity assessment using AI-driven models. With the absence of labelled data and expert assessment, there is a research gap in providing virtual assessment, especially for unlabeled data. Inspired by the advances in consensus learning, in this paper, we propose a consensus clustering algorithm, PSA-NMF, that combines various clusterings into one united clustering, i.e., cluster consensus, to produce more stable and robust results compared to individual clustering. This paper is the first to investigate severity level using unsupervised learning and trunk displacement features in the frequency domain for post-stroke smart assessment. Two different methods of data collection from the U-limb datasets—the camera-based method (Vicon) and wearable sensor-based technology (Xsens)—were used. The trunk displacement method labelled each cluster based on the compensatory movements that stroke survivors employed for their daily activities. The proposed method uses the position and acceleration data in the frequency domain. Experimental results have demonstrated that the proposed clustering method that uses the post-stroke assessment approach increased the evaluation metrics such as accuracy and F-score. These findings can lead to a more effective and automated stroke rehabilitation process that is suitable for clinical settings, thus improving the quality of life for stroke survivors.

Accurate single image depth detection using multiple rotating point spread functions.

In this article we present the simulation and experimental implementation of a camera-based sensor with low object-space numerical aperture that is capable of measuring the distance of multiple object points with an accuracy of 8.51 µm over a range of 20 mm. The overall measurement volume is 70 mm × 50 mm × 20 mm. The lens of the camera is upgraded with a diffractive optical element (DOE) which fulfills two tasks: replication of the single object point to a predefined pattern of K spots in the image plane and adding a vortex point spread function (PSF), whose shape and rotation is sensitive to defocus. We analyze the parameters of the spiral phase mask and discuss the depth reconstruction approach. By applying the depth reconstruction to each of the K replications and averaging the results, we experimentally show that the accuracy of the reconstructed depth signal can be improved by a factor of up to 3 by the replication approach. This replication method (also called multipoint method) not only improves accuracy of depth reconstruction but also of lateral position measurement. Therefore, the presented concept can be used as a single camera 3D position sensor for multiple points with high lateral as well as depth resolution.

Detection of Foreign Bodies in Soft Foods Employing Tactile Image Sensor.

In the inspection work involving foodstuffs in food factories, there are cases where people not only visually inspect foodstuffs, but must also physically touch foodstuffs with their hands to find foreign or undesirable objects mixed in the product. To contribute to the automation of the inspection process, this paper proposes a method for detecting foreign objects in food based on differences in hardness using a camera-based tactile image sensor. Because the foreign objects to be detected are often small, the tactile sensor requires a high spatial resolution. In addition, inspection work in food factories requires a sufficient inspection speed. The proposed cylindrical tactile image sensor meets these requirements because it can efficiently acquire high-resolution tactile images with a camera mounted inside while rolling the cylindrical sensor surface over the target object. By analyzing the images obtained from the tactile image sensor, we detected the presence of foreign objects and their locations. By using a reflective membrane-type sensor surface with high sensitivity, small and hard foreign bodies of sub-millimeter size mixed in with soft food were successfully detected. The effectiveness of the proposed method was confirmed through experiments to detect shell fragments left on the surface of raw shrimp and bones left in fish fillets.

Towards an Occupancy-Oriented Digital Twin for Facility Management: Test Campaign and Sensors Assessment

This study focuses on calibration and test campaigns of an IoT camera-based sensor system to monitor occupancy, as part of an ongoing research project aiming at defining a Building Management System (BMS) for facility management based on an occupancy-oriented Digital Twin (DT). The research project aims to facilitate the optimization of building operational stage through advanced monitoring techniques and data analytics. The quality of collected data, which are the input for analyses and simulations on the DT virtual entity, is critical to ensure the quality of the results. Therefore, calibration and test campaigns are essential to ensure data quality and efficiency of the IoT sensor system. The paper describes the general methodology for the BMS definition, and method and results of first stages of the research. The preliminary analyses included Indicative Post-Occupancy Evaluations (POEs) supported by Building Information Modelling (BIM) to optimize sensor system planning. Test campaign are then performed to evaluate collected data quality and system efficiency. The method was applied on a Department of Politecnico di Milano. The period of the year in which tests are performed was critical for lighting conditions. In addition, spaces’ geometric features and user behavior caused major issues and faults in the system.Incorrect boundary definition: areas that are not covered by boundaries; thus, they are not monitored

Learning Local-Global Multiple Correlation Filters for Robust Visual Tracking with Kalman Filter Redetection.

Visual object tracking is a significant technology for camera-based sensor networks applications. Multilayer convolutional features comprehensively used in correlation filter (CF)-based tracking algorithms have achieved excellent performance. However, there are tracking failures in some challenging situations because ordinary features are not able to well represent the object appearance variations and the correlation filters are updated irrationally. In this paper, we propose a local–global multiple correlation filters (LGCF) tracking algorithm for edge computing systems capturing moving targets, such as vehicles and pedestrians. First, we construct a global correlation filter model with deep convolutional features, and choose horizontal or vertical division according to the aspect ratio to build two local filters with hand-crafted features. Then, we propose a local–global collaborative strategy to exchange information between local and global correlation filters. This strategy can avoid the wrong learning of the object appearance model. Finally, we propose a time-space peak to sidelobe ratio (TSPSR) to evaluate the stability of the current CF. When the estimated results of the current CF are not reliable, the Kalman filter redetection (KFR) model would be enabled to recapture the object. The experimental results show that our presented algorithm achieves better performances on OTB-2013 and OTB-2015 compared with the other latest 12 tracking algorithms. Moreover, our algorithm handles various challenges in object tracking well.

Model-based predictive brake control during weft insertion in air-jet weaving

Air-jet weaving is one of the most efficient manufacturing processes for producing textile fabrics. During weft insertion, a yarn brake influences the quality of textile fabrics significantly. This brake is set up manually by a machine operator and the duration of the set-up process depends essentially on his experience. Furthermore, an inappropriate braking process might induce high tension in the weft thread which could cause backward movements of the thread as well as a defect in the fabric. For this reason, a braking system is developed which consists of a camera-based sensor and a continuously adjustable brake. The camera-based sensor enables the estimation of the weft velocity which is then controlled by a Model-based Predictive Controller (MPC). In this contribution, a weft insertion model is derived and validated experimentally. After that, this model is reduced in order to ensure real-time capability for its application in a Kalman filter as well as in the MPC. Finally, an MPC based on piecewise linearisation is proposed and implemented on an air-jet weaving machine. The presented experimental results show that yarn braking system compensates the changing behaviour of weft threads in different machine cycles and brakes the weft thread appropriately without a backward movement.

Crack identification method for concrete structures considering angle of view using RGB-D camera-based sensor fusion

Cracks on concrete structures are an important indicator for assessing concrete durability and structural safety. Although such cracks are typically monitored by manual visual inspection, this method has drawbacks in terms of inspection time, safety, cost-effectiveness, and measurement accuracy. An innovative alternative is digital image processing, which can be used to obtain crack information from images captured using a digital camera. However, in image-based crack detection, the crack width may vary depending on the angle of the camera with respect to the concrete surface. A skewed angle of view is often encountered, particularly when capturing images from unmanned aerial vehicles or from higher locations. This study proposes a crack identification strategy using a combination of RGB-D and high-resolution digital cameras to accurately measure cracks regardless of the angle of view. The camera system is equipped with a tailored sensor fusion algorithm for crack identification, enabling a high measurement resolution and a robust depth estimation considering the skewed angle problem. An approximate plane corresponding to the concrete surface is introduced to effectively handle the high noise in the depth measurement data of the RGB-D camera. Subsequently, the crack image captured using the high-resolution digital camera is mapped onto the obtained plane model, allowing the crack width to be determined using the three-dimensional coordinates of each crack pixel. The measurement accuracy of the proposed approach is experimentally validated on an actual concrete structure.

Wearable Egocentric Camera as a Monitoring Tool of Free-Living Cigarette Smoking: A Feasibility Study.

Wearable sensors may be used for the assessment of behavioral manifestations of cigarette smoking under natural conditions. This paper introduces a new camera-based sensor system to monitor smoking behavior. The goals of this study were (1) identification of the best position of sensor placement on the body and (2) feasibility evaluation of the sensor as a free-living smoking-monitoring tool. A sensor system was developed with a 5MP camera that captured images every second for continuously up to 26 hours. Five on-body locations were tested for the selection of sensor placement. A feasibility study was then performed on 10 smokers to monitor full-day smoking under free-living conditions. Captured images were manually annotated to obtain behavioral metrics of smoking including smoking frequency, smoking environment, and puffs per cigarette. The smoking environment and puff counts captured by the camera were compared with self-reported smoking. A camera located on the eyeglass temple produced the maximum number of images of smoking and the minimal number of blurry or overexposed images (53.9%, 4.19%, and 0.93% of total captured, respectively). During free-living conditions, 286,245 images were captured with a mean (±standard deviation) duration of sensor wear of 647(±74) minutes/participant. Image annotation identified consumption of 5(±2.3) cigarettes/participant, 3.1(±1.1) cigarettes/participant indoors, 1.9(±0.9) cigarettes/participant outdoors, and 9.02(±2.5) puffs/cigarette. Statistical tests found significant differences between manual annotations and self-reported smoking environment or puff counts. A wearable camera-based sensor may facilitate objective monitoring of cigarette smoking, categorization of smoking environments, and identification of behavioral metrics of smoking in free-living conditions. The proposed camera-based sensor system can be employed to examine cigarette smoking under free-living conditions. Smokers may accept this unobtrusive sensor for extended wear, as the sensor would not restrict the natural pattern of smoking or daily activities, nor would it require any active participation from a person except wearing it. Critical metrics of smoking behavior, such as the smoking environment and puff counts obtained from this sensor, may generate important information for smoking interventions.

Motorcycle That See: Multifocal Stereo Vision Sensor for Advanced Safety Systems in Tilting Vehicles.

Advanced driver assistance systems, ADAS, have shown the possibility to anticipate crash accidents and effectively assist road users in critical traffic situations. This is not the case for motorcyclists, in fact ADAS for motorcycles are still barely developed. Our aim was to study a camera-based sensor for the application of preventive safety in tilting vehicles. We identified two road conflict situations for which automotive remote sensors installed in a tilting vehicle are likely to fail in the identification of critical obstacles. Accordingly, we set two experiments conducted in real traffic conditions to test our stereo vision sensor. Our promising results support the application of this type of sensors for advanced motorcycle safety applications.

Coverage problem in camera-based sensor networks using the CUDA platform

Closed-circuit televisions serve as prevention against crime, and many studies for closed-circuit television deployment have been conducted. The closed-circuit television deployment in downtown is ...

Emotion recognition using Kinect motion capture data of human gaits.

Automatic emotion recognition is of great value in many applications, however, to fully display the application value of emotion recognition, more portable, non-intrusive, inexpensive technologies need to be developed. Human gaits could reflect the walker’s emotional state, and could be an information source for emotion recognition. This paper proposed a novel method to recognize emotional state through human gaits by using Microsoft Kinect, a low-cost, portable, camera-based sensor. Fifty-nine participants’ gaits under neutral state, induced anger and induced happiness were recorded by two Kinect cameras, and the original data were processed through joint selection, coordinate system transformation, sliding window gauss filtering, differential operation, and data segmentation. Features of gait patterns were extracted from 3-dimentional coordinates of 14 main body joints by Fourier transformation and Principal Component Analysis (PCA). The classifiers NaiveBayes, RandomForests, LibSVM and SMO (Sequential Minimal Optimization) were trained and evaluated, and the accuracy of recognizing anger and happiness from neutral state achieved 80.5% and 75.4%. Although the results of distinguishing angry and happiness states were not ideal in current study, it showed the feasibility of automatically recognizing emotional states from gaits, with the characteristics meeting the application requirements.

Improved Continuous Tube Welding Due to Unique Process Sensor System and Process Control

Abstract A unique camera-based triple sensor system increases productivity, yield and quality of continuous welding of tubes and profiles. It combines high-precision seam tracking and beam positioning with weld spot visualization and characterization, and seam geometry measurement. The higher overall precision allows operating the process closer to its limits, online quality monitoring detects faults immediately. The process setup time is greatly reduced, and also the waste during startup is reduced. Finally, full documentation sets the basis for data traceability.

Measurement of initial motion of a flying golf ball with multi-exposure images for screen-golf

Recently, realistic sports games have been one of the growth industries of the indoor entertainment field. In particular, screen-golf, which is played indoors with real golf clubs and balls, has shown an outstanding growth rate in South Korea. It consists of a sensor, a computer graphics engine, an interior, and other elements. The sensor is the most essential component among them. It measures the initial conditions of a flying golf ball, the velocity, the horizontal angle, and the vertical angle. Until 2008, most commercialized products were infrared LED type sensors, but these are associated with relatively low accuracy and a narrow measurement range compared to those that utilize a camera. In spite of the better performance, camera-based screen-golf sensors are not broadly employed in the screen-golf market because they remain expensive compared with other screen-golf sensors. In this study, a new type of camera-based sensor system that captures multi-exposure images and ball tracking algorithms are suggested. These measure the initial condition of a flying golf ball from captured images. The suggested system was verified in an environment equal to that of a real screen-golf store.

High-Resolution Images with Minimum Energy Dissipation and Maximum Field-of-View in Camera-Based Wireless Multimedia Sensor Networks

High-resolution images with wide field of view are important in realizing many applications of wireless multimedia sensor networks. Previous works that generally use multi-tier topology and provide such images by increasing the capabilities of camera sensor nodes lead to an increase in network cost. On the other hand, the resulting energy consumption is a considerable issue that has not been seriously considered in previous works. In this paper, high-resolution images with wide field of view are generated without increasing the total cost of network and with minimum energy dissipation. This is achieved by using image stitching in WMSNs, designing a two-tier network topology with new structure, and proposing a camera selection algorithm. In the proposed two-tier structure, low cost camera sensor nodes are used only in the lower-tier and sensor nodes without camera are considered in the upper-tier which decreases total network cost as much as possible. Also, since a simplified image stitching method is implemented and a new algorithm for selecting active nodes is utilized, energy dissipation in the network is decreased by applying the proposed methods. The results of simulations supported the preceding statements.