Vision-based Monitoring System Research Articles

Research on Stay Cable Tension Estimation Based on Vision-Based Monitoring System

The cables in cable-stayed bridges are the primary load-bearing components, making the accurate monitoring of cable tension crucial for assessing the service reliability of the cables and the overall safety of the bridge. This study presents the development of a non-contact, vision-based monitoring system utilizing computer vision technology for monitoring cable tension in cable-stayed bridges. The monitoring system comprises an industrial camera equipped with an infrared filter and an infrared target lamp, employing a sub-pixel template matching algorithm to achieve high-precision cable tension measurement under long-distance and multi-frequency motion conditions. Model experiments were conducted to validate the accuracy and stability of the vision-based monitoring system. Results from long-distance model experiments indicate that the system can accurately measure frequencies across various distances, with a maximum frequency error of only 0.05%. Additionally, results from multi-frequency model experiments demonstrate that the system can accurately measure multiple vibration frequencies, with a maximum frequency error of just 0.4%.

Advances in Structural Health Monitoring: Bio-Inspired Optimization Techniques and Vision-Based Monitoring System for Damage Detection Using Natural Frequency

This paper introduces the improvements in natural-frequency-based SHM by applying bio-inspired optimization methods and a vision-based monitoring system for effective damage detection. This paper proposes a natural frequency extraction method using a motion magnification-based vision monitoring system with bio-inspired optimization techniques to estimate the damage location and depth in a cantilever beam. The proposed optimization techniques are inspired by natural processes and biological evolution including genetic algorithms, particle swarm optimization, sea lion optimization, and coral reefs optimization. To verify the performance of each bio-inspired optimization method, the eigenvalues of a two-bay truss structure are used for estimating the damaged elements. Then, using the proposed video motion magnification method, the natural frequency for each undamaged and damaged cantilever beam is extracted and compared with the LDV sensor to verify the proposed vision-based monitoring system. The performance of each bio-inspired optimizer in damage detection is compared. As a result, coral reefs optimization shows the lowest average error, around 1%, in damage detection using the natural frequency.

An underwater vest containing an antioxidant MXene hydrogel for sensitive recognition of fish locomotion

The perception of fish locomotion is important for understanding their adaptive behaviors and ethological characteristics. However, the main strategy used for extracting fish attitudes involves the use of a vision-based monitoring system, which is limited in its range of observation and cannot perform tracking for long times. Here, we report the use of a wearable tagging electronic device, referred to as an underwater vest, to capture the surrounding flow field disturbances triggered by swimming or momentary postural changes. All of these goals were achieved by integrating a pair of pseudocapacitive pressure-sensing units and a flexible circuit board. Notably, additional conditions, such as variable hydraulic pressures and minimal changes in fish posture, require high stability and sensitivity of the sensing units. Thus, hybrid hydrogel electrodes were developed through cross-linking MXene with holey-reduced graphene oxide nanosheets and further modification with 1-ethyl-3-methylimidazolium dicyanamide ionic liquids, which increased the interfacial capacitance and long-term interfacial activity of the MXene. Consequently, the sensing unit exhibited ultrahigh sensitivity (Smax~136,207 kPa−1) in an aquatic environment for 60 days and superior high-pressure resolution (10 Pa) within a wide working range of 1 MPa. Ultimately, an underwater vest integrated with such sensing units clearly distinguished and recorded fish locomotion. We believe that the designed device may open avenues in flow field monitoring and ocean current detection and provide new insights into the development of sensitive underwater tagging.

Addressing the challenge of process stability control in wire DED-LB/M process

The wire DED-LB/M AM process is fast, cost-effective and it creates high-quality, dense parts. However, its industrial adoption is limited since process stability needs complex monitoring devices and tools. This work proposes a twofold strategy to resolve this issue; A fast-running, physics-based simulation tool calculates the temperature field during the building of single walls, while, an affordable vision-based monitoring system captures the melt pool dimensions that are correlated to the temperature and reveal heat accumulation. Therefore, real-time process stability control is enabled. An infrared thermal camera is used to validate the model and calibrate the monitoring system.

Toddler monitoring system in vehicle using single shot detector mobilenet and single shot detector-inception on Jetson Nano

<p>Road vehicles are today’s primary form of transportation; the safety of children passengers must take precedence. Numerous reports of toddler death in road vehicles, include heatstroke and accidents caused by negligent parents. In this research, we report a system developed to monitor and detect a toddler's presence in a vehicle and to classify the toddler's seatbelt status. The objective of the toddler monitoring system is to monitor the child's conditions to ensure the toddler's safety. The device senses the toddler's seatbelt status and warns the driver if the child is left in the car after the vehicle is powered off. The vision-based monitoring system employs deep learning algorithms to recognize infants and seatbelts, in the interior vehicle environment. Due to its superior performance, the Nvidia Jetson Nano was selected as the computational unit. Deep learning algorithms such as faster region-based convolutional neural network (R-CNN), single shot detector (SSD)- MobileNet, and single shot detector (SSD)-Inception was utilized and compared for detection and classification. From the results, the object detection algorithms using Jetson Nano achieved 80 FPS, with up to 82.98% accuracy, making it feasible for online and real-time in-vehicle monitoring with low power requirements</p>

Detection and measurement of beam deflection in the Madeira Airport runway extension using digital image correlation

PurposePart of the runway at Madeira Airport is a platform above the sea at a 60 m height, supported by a series of frames. When aircraft land on this section, a load is exerted on the structure, resulting in bending of the beams which constitute the frames. A vision-based monitoring system was devised and implemented to measure the deflection of the runway's beams when a landing occurs.Design/methodology/approachAn area on the midspan of two beams, located on the area where aircraft are most likely to land, was prepared with a speckle pattern, and a camera was assembled above a column on each of the adjacent frames, enabling the computation of displacements using digital image correlation (DIC). The camera continuously acquires images of the monitored area and compares them to a reference using DIC. If a displacement is detected, a number of frames before and after this event are saved for further DIC processing.FindingsThe installed systems successfully detected several events corresponding to landings and, for each of those events, measured the deflection of the beams over time and computed displacement fields for critical images, with strain values obtained up to this point being too small to measure using the current system.Originality/valueThis work provides novel insights into the behaviour of a unique structure and constitutes the first use of a vision system in its structural monitoring operations. It is also a valuable development in the implementation of automated DIC monitoring systems in locations of difficult access.

Designing a digitalized cell for remanufacturing of automotive frames

Circular economy introduction in industry is globally recognized as one of the most effective green strategies towards sustainability and resources reduction. As the automotive sector has been historically one of the biggest energy consumers, significant efforts are being constantly made to reduce the environmental impact and to increase energy efficiency at the manufacturing stage, both at production and process level. This study focuses on proposing a framework for automotive defective parts identification and remanufacturing in the main production line. In particular, the concept of an automotive microfactory is taken into consideration. To achieve this, a vision-based monitoring system performing the defect identification drives the whole procedure that is considered to be realized through a remanufacturing cell. The software tool is considered to be utilized right before the scheduling process of orders and tasks performed in the remanufacturing cell, taking into account specific performance parameters. The architecture of the software is presented along with some discussion regarding the types of control that can be enforced at both processing and line level, while the use of synthetic data is regarded to test the efficiency of the monitoring system.

Application of one-stage instance segmentation with weather conditions in surveillance cameras at construction sites

On-site surveillance camera systems have been utilized for safety monitoring remotely to reduce occupational accidents and injuries. Even though previous studies for workspace safety monitoring using computer vision have been undertaken, weather-related conditions such as raindrops and snows affecting pan-tilt-zoom camera visibility have not been considered in depth. Developing a robust detector that is reliable in different weather conditions is a challenge for vision-based monitoring outdoor works in construction sites. This study proposes a deep learning-based real time one-stage instance segmentation model for monitoring systems and data augmentation methods to improve the detector's performance under five weather conditions: brightness, darkness, rainy, snowy, and foggy. Experimental results indicate that the proposed model with weather augmentation outperformed the baseline model without augmentation by around 0.05 mAP05:95. Therefore, this work verified the applicability of the model for detecting and segmenting instances robustly in construction sites. This weather augmentation approach could be applicable to any type of vision-based monitoring system such as quality inspection, productivity assessment, schedule monitoring, and other work management as well as safety management affected by the different weather conditions. There are still room for the future researches in the development of fully autonomous vision-based monitoring systems in the applications mentioned.

Urban dual mode video detection system based on fisheye and PTZ cameras

This work presents an artificial vision-based monitoring system for urban environments. It comprises a fisheye camera monitoring the scenes 180ox360o hemisphere and a Pan-Tilt-Zoom camera capturing narrower regions of interest in high-resolution. The ONVIF protocol standard is used to interface both IP-cameras, allowing for the integration of camera control (image acquisition and movement) and geometric calculations on a single device. The events of interest (motion of vehicles and pedestrians) are assumed to happen on the ground plane. This assumption is required to solve the back-projection, the function that maps coordinates in the highly distorted images of the fisheye camera to the ground plane. A calibration strategy estimates the poses of the cameras without placing restrictions on their orientations or relative distance. It optimizes the back-projection error in the ground plane instead of the re-projection error in the image. Finally, a simple pointing and zoom adjustment strategy controls the Pan-Tilt-Zoom camera. The system is tested in controlled laboratory conditions and shows accurate outdoor performance for pedestrian observation.

Reducing Falls in Dementia Inpatients Using Vision-Based Technology.

ObjectivesFalls have a significant negative impact on the health and well-being of people with dementia and increase service costs related to staff time, paramedic visits, and accident and emergency (A&E) admissions. We examined whether a remote digital vision-based monitoring and management system had an impact on the prevention of falls.MethodsOur study was conducted within the Manor dementia inpatient wards at the Coventry and Warwickshire Partnership Trust. Data were retrieved from incident reports before and 22 months after installation of the system. We examined number of night time falls, severity of fall, number of paramedic visits and A&E admissions, and the number of enhanced observations during both time periods.ResultsThere was a significant 48% reduction in the number of nighttime falls (P < 0.01), a 49% reduction in visits from paramedics (P < 0.2), and a 68% reduction in A&E admissions (P < 0.02). In addition, the data indicated an 82% reduction in the number of moderate severity falls and that enhanced one-to-one observation hours were reduced by 71%.ConclusionsThe study demonstrated that a contact-free, remote digital vision-based monitoring and management system reduced falls, fall-related injuries, emergency services time, clinician time, and disruptive night time observations. This benefits the clinicians by allowing them to undertake other clinical duties and promotes the health and safety of patients who might normally experience injury-related stress and disruption to sleep.

Non-invasive technique for real-time myocardial infarction detection using faster R-CNN

The medical history explores that Myocardial Infarction has been one of the leading factors of death in human beings since several decades globally. The researchers’ key tasks are to emerge a novel real-time health vision-based monitoring system with added measurement features like high accuracy, robust, reliable, low-cost, low power with high data security. The main purpose of this research is to bestow an advanced non-invasive algorithmic approach for detecting the chest pain posture and fall posture based vital signs of Myocardial Infarction and analyzing the performance of a Faster Region-based Convolution Neural Network algorithm. This object detection computer vision technique is simulated for 3000 three-dimensional real-life indoor environment RGB color images for two datasets Nanyang Technological University Red Blue Green, and Depth dataset and private dataset-RMS trained datasets using TensorFlow object detection Application Programming Interface. The 3D RGB Images of NTU RGB database used for Vital Signs of Myocardial Infarction performance analysis is an improved approach. The simulation results have been compared with the existing works. The demonstrated results of ResNet-101 Faster RCNN showed the evaluated metric values: high mean precision and average recall value is a major contribution in this work.

A Machine Vision Based Monitoring System for the LCD Panel Cutting Wheel Degradation

In Liquid Crystal Display (LCD) panel cutting, in-line monitoring of tool wear is important to maintain the dimension precision of finished products and to avoid possible damages to the workpiece. However, limited by the space for camera installation and the miniature structure of the tool itself, monitoring the wear of LCD panel cutting wheel is not a simple task. In this research, we proposed a machine-vision based instrumentation system and a systematic methodology for cutting wheel degradation monitoring. The proposed method describes the degradation of cutting wheel by estimating the tooth height of the cutting wheel blade based on the partially observed random samples. A series of methods are proposed to improve the reliability of the results. The effectiveness of the proposed method is validated based on the field data that is collected from three maintenance cycles. The validation results demonstrate consistent degradation trend of the cutting wheel over production cycles.

Artificial Intelligence Vision-Based Monitoring System for Ship Berthing

This paper proposes a novel artificial intelligence vision-based monitoring system (AVMS) for ship berthing. To dock a ship, it is necessary to accurately estimate the relative distance between the quay wall and the ship. However, maneuvering large ships near a port is a complicated and difficult procedure. Thus, tugboats push the ship and dock it at the berth under the supervision of a pilot, who receives distance information from a berthing aid system (BAS). The conventional BAS based on laser distance sensors, which is the most widely used approach, is high-priced and limited by the size of the ship. Additionally, if there is an obstacle between the ship and the berth, the distance cannot be measured, since it obscures the laser signal. To address this problem, we develop an AVMS sensor module composed of a low-priced camera, a differential global positioning system (DGPS) receiver, and an inertial measurement unit (IMU) with an algorithm to estimate the distance between ship and berth. To evaluate the performance of the proposed AVMS, field tests are performed at Ulsan port in Korea, and the results are compared with a conventional BAS. From the field test results, the AVMS provides highly accurate estimates and shows robust performance in poor weather conditions compared to the conventional BAS. The AVMS can measure the distance between ship and berth regardless of the size of the ship, since it has a wide field of view. In addition, it provides the pilot with real-time image information of the ship approaching the berth to obtain safe ship berthing.

A machine vision system for early detection and prediction of sick birds: A broiler chicken model

The occurrence of poultry diseases not only affects farm production economics but also leads to poor poultry welfare, food safety concerns, and zoonotic infections. Therefore, timely detection of these diseases is of paramount importance in poultry production. This study proposes a machine vision-based monitoring system for broiler chicken as they walk through a test area. Data were collected from two groups of broilers; control group and treatment group (inoculated intramuscularly with virulent Newcastle disease virus) housed in fully isolated chambers for comparative monitoring. The broilers were monitored by video surveillance for data labelling and depth camera for the automated health status classifier development. Feature variables were extracted based on 2D posture shape descriptors (circle variance, elongation, convexity, complexity, and eccentricity) and mobility feature (walk speed). A statistical analysis of the feature variables established that all investigated features were statistically significant ( p 0.05 ) with time after challenge in the treatment group. The earliest possible infection detection time was on the 4th day based on circle variance and elongation, and the 6th day based on eccentricity and walk speed. However, convexity and complexity could not provide early detection. Two sets of classifiers were then developed based on only the posture shape descriptors, and on all the feature variables, The Support Vector Machine (RBF-SVM) outperformed all the other models with an accuracy of 0.975 and 0.978 respectively. The proposed system can serve as an automatic broiler monitoring system by providing an early warning and prediction of an occurrence of disease continuously and non-intrusively.

A hybrid gray wolf and genetic whale optimization algorithm for efficient moving object analysis

Object detection in realistic situations needs various essential applications. The foremost applications of machine vision like vision based monitoring system, object tracking etc. require background subtraction (BS) complied with identification of motion objects. Segregating forefront from background is a challenging task in videos discovered through motion cam since either forefront or background relevant information varies in each successive frame of the video series; hence a pseudo-motion is sensitive with background. Modified Kernel fuzzy c-means technique still bears a few drawbacks, for example, decreased convergence rate, acquiring stuck in the local minima and also in risk to instatement level of sensitivity. To overcome the above problems, here we recommend a technique for information clustering utilizing the OWC (Optimal Weighted Centroid) procedure that decides the optimum centroid for playing out the clustering procedure. To overcome the issues, here we recommend a technique Optimal Background separation using Optimal Weighted Centroid (OWC) - Modified Kernel Fuzzy C Means Algorithm (MKFCM) for information clustering utilizing the OWC (Optimal Weighted Centroid) procedure that decides the optimum centroid for playing out the clustering procedure. The OWC procedure makes use of the procedural activities of the Whale Optimization algorithm (WOA) with the fusion of the Grey Wolf Optimization (GWO). The prescribed new procedure is dynamic clustering technique for splitting up of moving object. Moving object tracking is accomplished through the blob detection which comes under the tracking stage. The examination stage has attribute extraction and also classification. Appearance-based as well as high quality based attributes are drawn out from the refined frames which are provided for classification. Since classification we are making use of J48 (C4.5) i.e., decision tree based classifier. The efficiency of the recommended strategy is examined with preceding strategies k-NN as well as MLP in regard to accuracy, f-measure, ROC as well as recall.

Vision-based melt pool monitoring system setup for additive manufacturing

This work outlines a reference guideline for the realization and the optimization of a vision-based monitoring system for metal Additive Manufacturing (AM) processes. We perform the selection of equipment and image processing methods in the spirit of providing a monitoring solution that could be cost affordable and easily integrated into any laser-based AM machine. By experimental tests conducted on a commercial Direct Energy Deposition machine, we assess the equipment configuration and camera acquisition settings that, in addition to the proposed image-processing algorithm, could optimize the anomaly detection during the deposition.

Frequency-based damage detection in cantilever beam using vision-based monitoring system with motion magnification technique

This article proposes a method for damage detection using vision-based monitoring with motion magnification technique. The methods based on the vibration characteristics of structures such as natural frequency, mode shapes, and modal damping have been applied to structural damage detection. However, the conventional methods have limitations for practical applications. Vision-based monitoring system can be employed as a new structural monitoring system because of its simplicity, potentially low cost, and unique capability of collecting high-resolution data. A methodology called video motion magnification has been developed to amplify non-visible small motions in a video to reveal the dynamic response. The video motion magnification method can be applied to measure small displacements to calculate the natural frequencies and the operational deflection shapes of the structures. Unlike conventional optimization methods, a genetic algorithm explores the entire solution space and can obtain the global optimum. In this article, identification of the location and magnitude of damage in a cantilever beam is formulated as an optimization problem using a real-value genetic algorithm by minimizing the objective function, which directly compares the first three natural frequencies changes from the phase-based motion magnification measurement and from the analytical model of a damaged cantilever beam.

Study of CCD vision-based monitoring system for NC lathes

Reliability of machining system plays an important role in product quality. Although the modern numerical control tools that have their own sensors to do condition monitor, could do a kinds of tasks with high accuracy, but accident defects still are unavoidable and unpredictable, such as deformation in spindle and workpiece due to great force, heat and vibration. It sometimes results in heavy production quality problem, particular for some important parts or components with high precise requirement. To address this difficulty, a vision based online monitoring system is proposed. In this system, an object called as “mark” that is set up on the moving components of the cutter is used to help to locate its position during machining, and a laser is set up and applied to trace the change of position of workpiece center axis. Since the mark always is seen by the camera (optical sensor) in whole cutting procedure, current position of the cutter at any second can be recorded for the monitoring purpose. Considering that the path of the mark obtained from the sequences of images should match the desired one in fault-free condition, the accident fault can be recognized by checking their consistency. To demonstrate the validation, a numerical experiment is carried out for the verification in theory, and a test part is used to validate the proposed approach in practice.

Vision-based stress estimation model for steel frame structures with rigid links

This paper presents a stress estimation model for the safety evaluation of steel frame structures with rigid links using a vision-based monitoring system. In this model, the deformed shape of a structure under external loads is estimated via displacements measured by a motion capture system (MCS), which is a non-contact displacement measurement device. During the estimation of the deformed shape, the effective lengths of the rigid link ranges in the frame structure are identified. The radius of the curvature of the structural member to be monitored is calculated using the estimated deformed shape and is employed to estimate stress. Using MCS in the presented model, the safety of a structure can be assessed gauge-freely. In addition, because the stress is directly extracted from the radius of the curvature obtained from the measured deformed shape, information on the loadings and boundary conditions of the structure are not required. Furthermore, the model, which includes the identification of the effective lengths of the rigid links, can consider the influences of the stiffness of the connection and support on the deformation in the stress estimation. To verify the applicability of the presented model, static loading tests for a steel frame specimen were conducted. By comparing the stress estimated by the model with the measured stress, the validity of the model was confirmed.

Stay cable tension estimation using a vision-based monitoring system under various weather conditions

Recently, the construction of long-span bridges has increased due to continuous developments in materials and construction technologies. For long-span bridges that use cables, it is important to continuously monitor bridge safety by measuring cable tensions under construction and in traffic use. Among the many methods of measuring cable tensions, the vibration method, which estimates tensions based on cable shapes and natural frequencies, is currently popular because it is more cost-efficient and convenient than direct methods. Studies on the estimation of cable tensions using image measurement systems are actively underway, but they are used for routine inspections rather than in fixed systems for long-term monitoring of cable tensions. In this study, a vision-based monitoring system to measure cable tensions is installed. The system is applied to a cable-stayed bridge in traffic use. To verify the possibility of long-term monitoring cable tensions, images acquired under various weather conditions were used to estimate the cable tensions.