Camera Vision System Research Articles

Inferred wind speed and direction during the descent and landing of Perseverance on Mars

Perseverance successfully parachuted into Jezero crater close to its western rim on a Martian spring afternoon. In situ observations of the wind conditions during the parachute and powered descent are unavailable. These observations are important for characterising the Planetary Boundary Layer (PBL) and to better constrain atmospheric conditions in Jezero crater.We infer the winds during Perseverance's parachute descent from an altitude of 12 km down to 2 km by fitting a trajectory model to the reconstructed trajectory data published by the M2020 Entry, Descent and Landing team. Below 2 km altitude we infer the winds by analysing a High Resolution Imaging Science Experiment (HiRISE) image of the landing site that was obtained about one sol after the landing.Our inferred wind speed and direction profile indicate high speed winds at an altitude of 7 km blowing from the east and at 3 km blowing from the southwest. Below an altitude of 2 km, the winds are inferred to be easterlies. The heat shield impact on the surface was observed by Perseverance's Lander Vision System Camera (LCAM). Afterwards, ejecta clouds were observed moving towards the north of the impact site. These observations, together with the orientation of the parachute canopy on the surface, suggest near-surface winds at the time were southerlies.We have inferred the wind speed and direction during Perseverance's descent that are consistent with its horizontal motion as revealed by trajectory data and HiRISE image. Comparisons with atmospheric models suggest the high speed winds could be related to topographic forcing. This study provides constraints for atmospheric modelling and advances the characterisation of winds in Jezero crater, focusing on the vertical structure of the complex wind field present in the crater.

Validation of machine vision and action sport cameras for 3D motion analysis model reconstruction

The study investigated the feasibility of using action sport cameras for motion analysis research. Data acquired from two different marker-based motion capture systems and six different camera combinations were analyzed for motion reconstruction accuracy. Two different calibration procedures were used to determine the influence on marker position reconstruction. Static and dynamic calibration mean merit score differences between the reference and experimental camera systems were 0.4 mm and 1.3 mm, respectively. Angular displacement difference between the reference and experimental camera systems range between 0.1 and 2.0 degrees. A systematic bias (− 0.54 to 0.19 degrees) was determined between the reference and the experimental camera systems for range of motion. The mean of the multi-trial findings suggests the machine vision camera system calibrated with a dynamic procedure generated highly accurate three-dimensional reconstructed ROM data (0.5 degree) followed closely by the four action sport cameras implementing a static calibration procedure (0.5 degree). The overall findings suggest the selected machine vision and action sport camera systems produced comparable results to the reference motion analysis system. However, the combination of camera type, processing software, and calibration procedure can influence motion reconstruction accuracy.

Feasibility of the EDICAM camera for runaway electron detection in JT-60SA disruptions

The visible camera system EDICAM (Event Detection Intelligent Camera), recently installed on JT-60SA, is simulated to assess whether it can be used for measuring synchrotron radiation from relativistic runaway electrons. In this simulation, the SOFT synthetic synchrotron diagnostic framework is used to compute the synthetic synchrotron camera images from a JT-60SA-like disruption modelled with the DREAM disruption simulation code. In the studied scenario, a large amount of argon is added to the plasma, and a disruption is simulated by starting a prescribed exponential temperature drop and finishing with further cooling provided by the argon in a self-consistent simulation of the current quench. The background plasma evolution is calculated by DREAM self-consistently with the fast electron population, which is modelled kinetically. The resulting runaway electron distribution function along with the parameters of the EDICAM visible camera system are used as an input to the SOFT framework to assess the feasibility of the camera for runaway electron detection. We find that the runaway electron beam formed in the disruption can produce synchrotron radiation observable by the EDICAM system, thus enabling the use of the EDICAM for the characterization of runaway electron beams.

A Method for Predicting Canopy Light Distribution in Cherry Trees Based on Fused Point Cloud Data

A proper canopy light distribution in fruit trees can improve photosynthetic efficiency, which is important for improving fruit yield and quality. Traditional methods of measuring light intensity in the canopy of fruit trees are time consuming, labor intensive and error prone. Therefore, a method for predicting canopy light distribution in cherry trees was proposed based on a three-dimensional (3D) cherry tree canopy point cloud model fused by multiple sources. First, to quickly and accurately reconstruct the 3D cherry tree point cloud model, we propose a global cherry tree alignment method based on a binocular depth camera vision system. For the point cloud data acquired by the two cameras, a RANSAC-based orb calibration method is used to externally calibrate the cameras, and the point cloud is coarsely aligned using the pose transformation matrix between the cameras. For the point cloud data collected at different stations, a coarse point cloud alignment method based on intrinsic shape signature (ISS) key points is proposed. In addition, an improved iterative closest point (ICP) algorithm based on bidirectional KD-tree is proposed to precisely align the coarse-aligned cherry tree point cloud data to achieve point cloud data fusion and obtain a complete 3D cherry tree point cloud model. Finally, to reveal the pattern between the fruit tree canopy structure and the light distribution, a GBRT-based model for predicting the cherry tree canopy light distribution is proposed based on the established 3D cherry tree point cloud model, which takes the relative projected area features, relative surface area and relative volume characteristics of the minimum bounding box of the point cloud model as inputs and the relative light intensity as output. The experiment results show that the GBRT-based model for predicting the cherry tree canopy illumination distribution has good feasibility. The coefficient of determination between the predicted value and the actual value is 0.932, and the MAPE is 0.116, and the model can provide technical support for scientific and reasonable cherry tree pruning.

Quantitatively evaluate the cylindricity of Large size pipe fitting via laser displacement sensor and Digital twin technology

The rapid development of intelligent detection technology guarantees accuracy in the assembly process. Aiming to solve the quantitative inspection of the cylindricity of Large size pipe fitting, this paper presents an online method technology using a laser sensor and Digital twin technology. The detection system comprises visual positioning, servo drive, and laser displacement sensor information acquisition systems. After positioning by the machine vision camera system, the servo system and laser displacement sensor scan the large-size pipe fitting layer by layer and establish the information interaction between physical space and virtual space through digital twinning technology. The digital twin scatters information model was found to calculate the maximum tangent circle and minimum peripheral circle, fitting the curve area of the closed point cloud layer by layer. And the least square method was used to calculate the center of the layered circle. The radial deviation of pipe fitting was evaluated comprehensively. The system makes a linear fitting of the center of the most miniature square circle of each layer, compares its deflection Angle with the solid center line, and calculates the center line deviation of the hole. Finally, the precision of the proposed method was validated by varying experiments. The cylindricity measurement of pipe fittings with an inner diameter of 300mm and height of 270mm was carried out by a pair of C-shaped semicircles fixed by the radial assembly. The cylindricity of the pipe fitting is 0.2372mm.

A Comparative Study of Vetiveria zizanioides Leaf Segmentation Techniques Using Visible, Infrared, and Thermal Camera Sensors in an Outdoor Environment

A camera vision system is a fast and effective approach to monitoring leaves. It can be used to monitor plant growth, detect diseases, and conduct plant phenotyping. However, due to the outdoor environment of plants, it becomes challenging to use. This paper addresses the problems of Vetiveria zizanioides leaf segmentation by comparing different camera types and segmentation techniques. Visible, no infrared filter (NoIR), and thermal cameras interfaced in an embedded device were used to capture plants during the day and at night. Several popular thresholding techniques and the K-Means algorithm were employed for leaf segmentation, and their performance was measured using Recall, Precision, and F1 score. The comparison results show that the visible camera achieved the best performance on daytime images, with the highest Recall of 0.934 using Triangle thresholding, the highest Precision of 0.751 using K-Means (K = 3), and the highest F1 score of 0.794 using Multi-Otsu thresholding. For nighttime images, the highest Recall of 0.990 was achieved by the thermal camera using Isodata and Otsu thresholding, the highest Precision of 0.572 was achieved by the NoIR camera using K-Means (K = 3), and the highest F1 score of 0.636 was achieved by the NoIR camera using K-Means (K = 3). To compare the leaf segmentation performance of the thresholding techniques and the K-Means algorithm between our image dataset and the well-known plant image dataset, we also evaluated the methods using the Ara2012 image dataset. The results showed that K-Means (K-3) achieved the best performance. The execution time of K-Means was about 3 s, which was longer than the thresholding techniques. However, it is still acceptable for the real-time plant monitoring system.

ELMs filaments extraction on divertor visible high-speed camera system of EAST

In order to study the ELMs filamentary structure generated by plasma in H-mode, a tangential visible camera system was established on EAST Tokamak. However, the obtained ELMs filamentary structure cannot be accurately and quickly identified, which is not conducive to the study of ELMs filamentary structure. In this paper, the ELMs filamentary structure is preliminarily extracted by the background subtraction method. The background modeling method adopts multi-frame averaging followed by contrast stretching of the image. Although the ELMs filamentary structure can be clearly seen in the obtained image, there exists a lot of noise, so further extraction of the ELMs filamentary structure is needed. In this paper, the neural network model U-Net is used for image segmentation. Compared with the traditional threshold segmentation, the ELMs filamentary structure and noise can be accurately distinguished, and ELMs filaments are better extracted. The results show that the method presented in this paper is suitable for the extraction of ELMs filamentary structure and beneficial to the study of ELMs filamentary structure.

Design Laparoscopy Camera Vision System by Using Raspberry PI

AIM: Design a laparoscopy camera system by using a small electronic board raspberry pi and including it with the appropriate programs to improve the doctor’s image or video while using the device, in addition to the ease of carrying the device and transporting it in the workplace or training and with an efficiency that is no less than the traditional device available. Methods: key component in this project is the 3-in-1 camera laparoscopy compatibility lens ,The light source with camera in front of the scope makes the training platform more realistic for the trainee .The design involves considering several important criteria, including the compromise with conventional one an Having enough workspace in the cameras field of view, easy Setup for users, making it more user-friendly and cost-effective for individual use by the trainee. Results: evaluation of the overall system vision qualities , measurement , color of the video and image in hospital . The paper introduces an innovative viewing system that can be combined with a traditional laparoscope and provides the surgeon with a general view of the abdominal cavity. Conclusion: Laparoscopy as a surgical technique has continue grow. The improvements to our video imaging laparoscopy system that is inexpensive, portable, and camera compact (Laparoscopy with Camera and Light (LDS) did not differ significantly from those obtained with conventional video laparoscopic. Keywords: laparoscopic camera system , image vision ,raspberry pi .

A deep learning-based method to identify and count pelagic and mesopelagic fishes from trawl camera images

Abstract Fish counts and species information can be obtained from images taken within trawls, which enables trawl surveys to operate without extracting fish from their habitat, yields distribution data at fine scale for better interpretation of acoustic results, and can detect fish that are not retained in the catch due to mesh selection. To automate the process of image-based fish detection and identification, we trained a deep learning algorithm (RetinaNet) on images collected from the trawl-mounted Deep Vision camera system. In this study, we focused on the detection of blue whiting, Atlantic herring, Atlantic mackerel, and mesopelagic fishes from images collected in the Norwegian sea. To address the need for large amounts of annotated data to train these models, we used a combination of real and synthetic images, and obtained a mean average precision of 0.845 on a test set of 918 images. Regression models were used to compare predicted fish counts, which were derived from RetinaNet classification of fish in the individual image frames, with catch data collected at 20 trawl stations. We have automatically detected and counted fish from individual images, related these counts to the trawl catches, and discussed how to use this in regular trawl surveys.

Vision-based human joint angular velocity estimation during squat and walking on a treadmill actions.

Elderly health monitoring, rehabilitation training, and sport supervision could benefit from continuous assessment of joint angle, and angular velocity to identify the joint movement patterns. However, most of the measurement systems are designed based on special kinematic sensors to estimate angular velocities. The study aims to measure the lower limb joint angular velocity based on a 2D vision camera system during squat and walking on treadmill action using deep convolution neural network (CNN) architecture. Experiments were conducted on 12 healthy adults, and six digital cameras were used to capture the videos of the participant actions in lateral and frontal view. The normalized cross-correlation (Ccnorm) analysis was performed to obtain a degree of symmetry of the ground truth and estimated angular velocity waveform patterns. Mean Ccnorm for angular velocity estimation by deep CNN model has higher than 0.90 in walking on the treadmill and 0.89 in squat action. Furthermore, joint-wise angular velocities at the hip, knee, and ankle joints were observed and compared. The proposed system gets higher estimation performance under the lateral view and the frontal view of the camera. This study potentially eliminates the requirement of wearable sensors and proves the applicability of using video-based system to measure joint angular velocities during squat and walking on a treadmill actions.

Hardhat Detection Using IR and Depth Frames

The construction industry is one of the largest sectors and 20% of the total deaths happen here. Safety hardhat helmets are one of the precautionary methods recommended and followed in a construction site. Detecting and classifying people wearing and not wearing helmets is an important task. Safety and Security is one of the most researched fields in recent years and acquiring 3D geometric information from any real environment is an important task for such applications. Well-known methods like stereo vision camera systems suffer from high time consumption or from the inability to match corresponding points inhomogeneous regions. Time of Flight (ToF)technology, being a recent development, fulfills features desired for real-time distance acquisition along with the compact size and higher frame rate. A safety application based on the ToF technology ad IR imaging is proposed in the paper. The ToF sensor provides depth information for each pixel as opposed to RGB values in case stereo camera-based systems. The depth sense cameras provide IR images along with the depth information. A YOLO framework is used to classify images. YOLO being faster than RCNN and the faster RCNN is much suitable for real-time classification. The model was trained on 50,000 images. Weights obtained during the 6000th epoch were chosen. A Mean Average Precision of 56% was obtained while testing.

Drone-Based StereoDIC: System Development, Experimental Validation and Infrastructure Application

Digital Image Correlation (DIC) is widely used for remote and non-destructive structural health evaluation of infrastructure. Current DIC applications are limited to relatively small areas of structures and require the use of stationary stereo vision camera systems that are not easy to transfer and deploy in remote areas. The enclosed work describes the development and validation of an Unmanned Aircraft System (UAS, commonly known as drone) with an onboard stereo-vision system capable of acquiring, storing and transmitting images for analysis to obtain full-field, three-dimensional displacement and strain measurements. The UAS equipped with a StereoDIC system has been developed and tested in the lab. The drone system, named DroneDIC, autonomously hovers in front of a prestressed railroad tie under pressure and DIC data are collected. A stationary DIC system is used in parallel to collect data for the railroad tie. We compare the data to validate the readings from the DroneDIC system. We present the analysis of the results obtained by both systems. Our study shows that the results we obtain from the DroneDIC system are similar to the ones gathered from the stationary DIC system. This work serves as a proof of concept for the successful integration of DIC and drone technologies into the DroneDIC system. DroneDIC combines the high accuracy inspection capabilities of traditional stationary DIC systems with the mobility offered by drone platforms. This is a major step towards autonomous DIC inspection in portions of a structure where access is difficult via conventional methods.

Metoda detekcji wad spawalniczych w stanowisku zrobotyzowanym z wykorzystaniem głębokiej sieci neuronowej

Modern automation and robotization of production processes requires new and fast methods of product quality control. In the case of arc welding in robotic systems, where the production process takes place in large series, it is important to quickly control the correctness of the weld. Based on visual data, the system should be able to automatically determine whether a given weld meets the basic quality requirements, and thus be able to stop the process in the event of identified defects. The article presents the results of research on the creation of a visual method for assessing the correctness of the weld seam based on the deep neural network classifying, locating and segmenting welding defects. The proposed detection method was extended by using a combination of a vision system camera with a six-axis industrial robot in order to enable detection of a larger number of welding defects and positioning in a six-dimensional workspace. The research results presented in this article were obtained during the implementation of the project entitled „Development of a method based on the use of deep neural networks for visual inspection of welded joints in the course of R&D works” implemented at the company ZAP-Robotyka Sp. z o.o. in Ostrów Wielkopolski.

Vision servo positioning control of robot manipulator for distribution line insulation wrapping

Traditional visual servo control based on location needs to convert two-dimensional information into three-dimensional space information, and the conversion process is subject to many interference factors, leading to large control errors. Aiming at the above problems, the visual servo positioning control of distribution line insulated wrapping robot and manipulator is studied. The position and pose of the manipulator are described and homogeneous transformed so as to calibrate the camera vision system of the manipulator. The image Jacobian matrix is established to realize the visual servo positioning control of the manipulator. Compared with the traditional control method, the experimental verification shows that the control method can reduce the pose error by at least 39.7%, and the control effect is more stable and the reliability is better.

DESIGN OF BALL COLLECTING ROBOT

A concept of four-wheeled mobile service robot capable of collecting balls from the Tennis court is proposed in this article. The robot is capable of manoeuvring on its own in an autonomous way and also it can be operated wirelessly. The robot is built using the Arduino Uno R3 microcontroller board along with sensors like LiDAR, ultrasound sensor, optical encoders, infrared sensor, camera for vision system. Sensor fusion can be implemented for localization of the robot while detection of the tennis ball can be attained by learning based object detection algorithm using OpenCV. A* algorithm is proposed for the navigation of the robot towards the object. Wireless operation of the robot can be achieved with the Bluetooth technology using the Android application via a smartphone. The article is an overview and information.

Automated inspection system for automobile bearing seals

Automation plays a wide role in industries in order to maintain high productivity, reliability and repeatability. Automated quality inspection system yields cost effective, high reliable and high endurance quality inspection process. The objective of the system is to eliminate errors occurred during manual quality inspection by means of camera vision system and Leak Down Test [LDT] integrated together in a single machine to serve the purpose. The system comprises the location and a physical model of the object to be inspected along with its quality parameters fed into the system to be achieved. This paper exhibits the results of the improved inspection accuracy by eliminating manual inspection and implementing automated inspection system. Each and every activities of the system are controlled and coordinated using a Programmable logic Controller [PLC]. The system achieves efficiency by reducing the false rejection rate from 27% to 10% and false acceptance rate from 10% to 2%.

Robotic Detection and Grasp of Maize and Sorghum: Stem Measurement with Contact

Frequent measurements of the plant phenotypes make it possible to monitor plant status during the growing season. Stem diameter is an important proxy for overall plant biomass and health. However, the manual measurement of stem diameter in plants is time consuming, error prone, and laborious. The use of agricultural robots to automatically collect plant phenotypic data for trait measurements can overcome many of the drawbacks of manual phenotyping. The objective of this research was to develop a robotic system that can automatically detect and grasp the stem, and measure its diameter of maize and sorghum plants. The robotic system comprises of a four degree of freedom robotic manipulator, a time-of-flight camera for vision system, and a linear potentiometer sensor to measure the stem diameter. Deep learning and conventional image processing were used to detect stem in images and find grasping point of stem, respectively. An experiment was conducted in a greenhouse using maize and sorghum plants to evaluate the performance of the robotic system. The system demonstrated successful grasping of stem and a high correlation between manual and robotic measurements of diameter depicting its ability to be used as a prototype to integrate other sensors to measure different physiological and chemical attributes of the stem.

SOLPS-ITER simulations of the TCV divertor upgrade

The effect of the upcoming TCV divertor upgrade on the distribution of neutrals and the onset of detachment is studied using 2D transport code simulations. The divertor upgrade is centered around the installation of a gas baffle to form a divertor chamber of variable closure. SOLPS-ITER simulations predict that the baffle geometry selected to be installed in TCV in 2019 increases the divertor neutral density by a factor ∼5 and the neutral compression by one order of magnitude in typical TCV single null, Ohmic heated scenarios (330 kW). The compression increases further with the addition of auxiliary heating systems (1.2 MW). Simulations show that volumetric power losses in the divertor increase giving access to deeper detachment for given upstream densities and heating power. Predictions for observations by various TCV diagnostics, including baratrons, divertor spectrometer and visible camera systems, are presented to guide the experimental verification of the efficiency of the divertor baffles.

Full thermal panorama from a long wavelength infrared and visible camera system

We present a method of constructing a full thermal panorama from a collection of images taken by a long wavelength infrared and visible camera system. Due to recent development in joint stereo calibration, our method is able to project a thermal image onto a corresponding visible image. After both images are in the same domain, panoramic projection from visible images can also be applied to thermal images. Our method also extends the scope of the projection function by allowing nonzero distance between center of rotation and optical center of rotating camera as well as arbitrary camera’s primary axes that are no longer parallel to the world axes. We show that our method works on a dataset with various camera movements and compare our results with standard software made specifically for a visible panorama and a thermal panorama. Furthermore, we are able to combine thermal panoramas of the same scene with a simple calculation. With an addition of guided upsampling, we can simultaneously improve thermal image resolution and field of view with a single run of the method.

The research of implementing safety driving system based on camera vision system

The research of implementing safety driving system based on camera vision system