Consumer-grade Digital Camera Research Articles

Dynamic performance verification of the Rędziński Bridge using portable camera-based vibration monitoring systems

The assessment of dynamic performance of large-scale bridges typically relies on the deployment of wired instrumentation systems requiring direct contact with the tested structures. This can obstruct their operation and create unnecessary risks to the involved personnel and equipment. These problems can be readily avoided by using non-contact instrumentation systems. However, the cost of off-the-shelf commercial products often prevents their wide adoption in engineering practice. To this end, the dynamic performance of the biggest one-pylon cable-stayed bridge in Poland is investigated based on data from a consumer-grade digital camera and open access image-processing algorithms. The quality of these data is benchmarked against data obtained from conventional wired accelerometers and a high-end commercial optical motion capture system. Operational modal analysis is conducted to extract modal damping, which has a potential to serve as an indicator of structural health. The dynamic properties of the bridge are evaluated against the results obtained during a proof loading exercise undertaken prior to the bridge opening. It is shown that a vibration monitoring system based on consumer-grade digital camera can indeed provide an economically viable alternative to monitoring the complex time-evolving dynamic behaviour patterns of large-scale bridges.

An RGB-D multi-view perspective for autonomous agricultural robots

Automated in-field data gathering is essential for crop monitoring and management and for precision farming treatments. To this end, consumer-grade digital cameras have been shown to offer a flexible and affordable sensing solution. This paper describes the integration and development of a cost-effective multi-view RGB-D device for sensing and modeling of agricultural environments. The system features three RGB-D sensors, arranged to cover a horizontal field of view of about 130 deg in front of the vehicle, and a suite of localization sensors consisting of a tracking camera, an RTK-GPS sensor and an IMU device. The system is intended to be mounted on-board an agricultural vehicle to provide multi-channel information of the surveyed scene including color, infrared and depth images, which are then combined with localization data to build a multi-view 3D geo-referenced map of the traversed crop. The experimental demonstrator of the multi-sensor system is presented along with the steps for the integration of the different sensor data into a unique multi-view map. Results of field experiments conducted in a commercial vineyard are included, as well, showing the effectiveness of the proposed system. The resulting map could be useful for precision agriculture applications, including crop health monitoring, and to support autonomous driving.

Vibration Analysis and Digital Image Correlation Techniques for a Suspension Bridge

Bridge inspection is critical to safe transportation, especially in highly urbanized areas. Frequent inspection invoked by the un-scheduled maintenance for bridges is often subjected to the availability of NDT service providers. Hence the objective of the current research is to provide an economical NDT technique for the maintenance crew. Digital image correlation techniques can be cost-effective for the displacement measurement of highway bridges using merely one high-resolution camera. The vibration analysis based on the remote measurements of high-speed camera systems is not only challenging but also prohibitively expensive. An alternative approach using two consumer-grade digital cameras has been developed. The field test has been successfully carried out for a 50-meter-span suspension bridge. Satisfactory results have been obtained in both the vertical displacement caused by a moving load and the mode shapes of the first and second vibration modes.

TRIMMING AND ROAD ORTHO IMAGING FOR NIGHT IMAGES BY ONBOARD HIGH SENSITIVITY CONSUMER GRADE DIGITAL CAMERAS

Abstract. Various kinds of cameras have been utilizing as the onboard cameras in the construction of Intelligent Transport Systems. In recent years, utilization of the high sensitivity consumer grade digital cameras at night is attracting attention from the viewpoint of avoiding the effects of sunlight and congestion of people and cars. However, due to the image taken by the onboard cameras is a perspective projection image, the image is projected small at the far from car and the effect of the lens distortion will be greater at points far from the image center. In order to avoid these issues, the lower part of the projection image or a bird's-eye view image is used, but the imaging of the bonnet part caused by the car models and tilts of the cameras becomes a new issue. Furthermore, a bird's-eye view image at night has to be trimmed to coincide with the irradiation range because the irradiation distance and range of the headlights are limited. On the other hand, feature quantities such as vanishing points and feature points on the lane have been used for projective transformation from a perspective projection image to a bird's-eye view image, but the projective transformation based on feature quantities is an ill-posed problem.Therefore, this paper discusses a quantitative trimming method based on projective transformation that does not depend on feature quantities and coincide with the irradiation range of the headlights.

Marker-free fatigue crack detection and localization by integrating the optical flow and information entropy

Fatigue cracks caused by repetitive loads are one of the major threats to the structural integrity of civil infrastructure. Human inspection is the most common method for detecting fatigue cracks, but it is time-consuming, labor-intensive, and unreliable. In this paper, we propose a new vision-based fatigue crack detection and localization method that can detect the fatigue crack with marker-free and high precision using a consumer-grade digital camera. A motion tracking technology called optical flow algorithm is applied to the video for tracking the surface motion of the monitored structure under repetitive load. Then, a crack detection and localization algorithm based on optical flow information entropy are developed to search differential features at different video frames caused by the crack opening and closing. The proposed method’s precision is first validated by doing two experiments and then comparing its precision and efficiency to the existing crack detection methods, including image processing and digital image correlation. The results show that, when compared to the existing vision-based methods, the proposed method can accurately and efficiently identify the fatigue crack even when the crack is surrounded by other crack-like edges, covered by complex surface textures, or invisible to human eyes. In addition, based on the proposed methods, a practical application for calculating the stress intensity factor is given to track crack development.

A computational method for rapid orthographic photography of lake sediment cores

Photographs of sediment cores are an important dataset, often containing visual evidence for environmental change via sediment composition and structure. These photographs may be used to stratigraphically correlate adjacent cores or for automated image analysis, and can facilitate collaboration amongst researchers through sharing and annotation of the image files. Here we describe the use of computational photogrammetry (also known as Structure from Motion–Multi-View Stereo) to generate orthographic imagery of sediment cores. Computational photogrammetry is a rapid and economical technique, typically requiring only a few minutes for each metre of core, using consumer-grade digital camera equipment. The photogrammetric methodology corrects for topographic distortion caused by sediment surfaces that are not perfectly flat, and can also record features of the scene surrounding the core, such as notes, colour reference cards and measurement tapes or rulers. As the photogrammetric process also generates a three-dimensional reconstruction of the sediment core, spatial-based analysis can be used to identify damaged or non-representative sections of the core that are to be avoided during image analysis. Using an intermittently laminated sediment core from Lake Surprise, Australia, we tested 22 scenarios using control points in a variety of configurations, as well as calibrated and uncalibrated cameras, to identify techniques that can reconstruct the core accurately and generate orthophotos. Multiple techniques were able to achieve suitable accuracy. In particular, targets placed on the table alongside the core, combined with a calibrated camera, achieved high accuracy and enabled a simple, rapid, and repeatable method for generating high-quality sediment core images.

Two-step approach for fatigue crack detection in steel bridges using convolutional neural networks

The advent of parallel computing capabilities, further boosted through the exploitation of graphics processing units, has resulted in the surge of new, previously infeasible, algorithmic schemes for structural health monitoring (SHM) tasks, such as the use of convolutional neural networks (CNNs) for vision-based SHM. This work proposes a novel approach for crack recognition in digital images based on coupling of CNNs and suited image processing techniques. The proposed method is applied on a dataset comprising images of the welding joints of a long-span steel bridge, collected via high-resolution consumer-grade digital cameras. The studied dataset includes photos taken in sub-optimal light and exposure conditions, with several noise contamination sources such as handwriting scripts, varying material textures, and, in some cases, under presence of external objects. The reference pixels representing the cracks, together with the crack width and length, are available and used for training and validating the proposed model. Although the proposed framework requires some knowledge of the “damaged areas”, it alleviates the need for precise labeling of the cracks in the training dataset. Validation of the model by means of application on an unlabeled image set reveals promising results in terms of accuracy and robustness to noise sources.

The feasibility of using a low-cost near-infrared, sensitive, consumer-grade digital camera mounted on a commercial UAV to assess Bambara groundnut yield

ABSTRACT Accurate, timely, and non-destructive early crop yield prediction at the field scale is essential in addressing changing crop production challenges and mitigating impacts of climate variability. Unmanned aerial vehicles (UAVs) are increasingly popular in recent years for agricultural remote sensing applications such as crop yield forecasting and precision agriculture (PA). The objective of this study was to evaluate the performance of a low-cost UAV-based remote sensing technology for Bambara groundnut yield prediction. A multirotor UAV equipped with a near-infrared sensitive consumer-grade digital camera was used to collect image data during the 2018 growing season (April to August). Flight missions were carried out six times during critical phenological stages of the life-cycle of the monitored crop. Yield was recorded at harvest. Four vegetation indices (VIs) namely normalized difference vegetation index (NDVI), enhanced vegetation index 2 (EVI2), green normalized difference vegetation index (GNDVI), and simple ratio (SR) generated from the Red-Green-Near Infrared bands were calculated using the georeferenced orthomosaic UAV images. Pearson’s product-moment correlation coefficient (r) and Bland–Altman testing showed a significant agreement between remotely and proximally sensed VIs. Significant and positive correlations were found between the four VIs and yield, with the strongest relationship observed between SR and yield at podfilling stage (r = 0.81, P < 0.01). Multi-temporal accumulative VIs improved yield prediction significantly with the best index being ∑SR and the best interval being from podfilling to maturity (r = 0.88, P < 0.01). The accumulated ∑SR from podfilling to maturity resulted in higher prediction accuracy with a coefficient of determination (R 2) of 0.71, root mean square error (RMSE) of 0.20 and mean absolute percentage error (MAPE) of 14.2% than SR spectral index at a single stage (R 2 = 0.68, RMSE = 0.24, MAPE = 15.1%). Finally, a yield map was generated using the model developed, to better understand the within-field spatial variations of yield for future site-specific or variable-rate application operations.

Performance of Camera-Based Vibration Monitoring Systems in Input-Output Modal Identification Using Shaker Excitation

Despite significant advances in the development of high-resolution digital cameras in the last couple of decades, their potential remains largely unexplored in the context of input-output modal identification. However, these remote sensors could greatly improve the efficacy of experimental dynamic characterisation of civil engineering structures. To this end, this study provides early evidence of the applicability of camera-based vibration monitoring systems in classical experimental modal analysis using an electromechanical shaker. A pseudo-random and sine chirp excitation is applied to a scaled model of a cable-stayed bridge at varying levels of intensity. The performance of vibration monitoring systems, consisting of a consumer-grade digital camera and two image processing algorithms, is analysed relative to that of a system based on accelerometry. A full set of modal parameters is considered in this process, including modal frequency, damping, mass and mode shapes. It is shown that the camera-based vibration monitoring systems can provide high accuracy results, although their effective application requires consideration of a number of issues related to the sensitivity, nature of the excitation force, and signal and image processing. Based on these findings, suggestions for best practice are provided to aid in the implementation of camera-based vibration monitoring systems in experimental modal analysis.

Phenocave: An Automated, Standalone, and Affordable Phenotyping System for Controlled Growth Conditions.

Controlled plant growth facilities provide the possibility to alter climate conditions affecting plant growth, such as humidity, temperature, and light, allowing a better understanding of plant responses to abiotic and biotic stresses. A bottleneck, however, is measuring various aspects of plant growth regularly and non-destructively. Although several high-throughput phenotyping facilities have been built worldwide, further development is required for smaller custom-made affordable systems for specific needs. Hence, the main objective of this study was to develop an affordable, standalone and automated phenotyping system called “Phenocave” for controlled growth facilities. The system can be equipped with consumer-grade digital cameras and multispectral cameras for imaging from the top view. The cameras are mounted on a gantry with two linear actuators enabling XY motion, thereby enabling imaging of the entire area of Phenocave. A blueprint for constructing such a system is presented and is evaluated with two case studies using wheat and sugar beet as model plants. The wheat plants were treated with different irrigation regimes or high nitrogen application at different developmental stages affecting their biomass accumulation and growth rate. A significant correlation was observed between conventional measurements and digital biomass at different time points. Post-harvest analysis of grain protein content and composition corresponded well with those of previous studies. The results from the sugar beet study revealed that seed treatment(s) before germination influences germination rates. Phenocave enables automated phenotyping of plants under controlled conditions, and the protocols and results from this study will allow others to build similar systems with dimensions suitable for their custom needs.

Digital media archive for gross pathology images based on open-source tools and Fast Healthcare Interoperability Resources (FHIR).

Macroscopic examination of surgical pathology and autopsy cases is a fundamental component of anatomic pathology. The photographic documentation of such clinical specimens is essential, and it may be required in certain instances. Our department began using consumer-grade digital cameras in 2005 to improve the practice of gross photography. However, the lack of an application to correctly catalog the photographs resulted in thousands of digital image files scattered across shared network drives, with limited case and patient metadata, making image retrieval a cumbersome and sometimes impossible task. Thirteen years later, we examined the legacy method of acquiring and accessing gross photographs in our department and determined the need for a web-based digital media archive to capture images with structured metadata. Using several open-source tools, including MediaWiki, we developed a flexible platform for building our digital media archive with a data schema based on the Fast Healthcare Interoperability Resources standard. Following a short pilot, we replaced the legacy method of handling gross pathology images with a new acquisition workflow and digital media archive. Through March 2021, 233 distinct users have accessed the system, 58 of which have uploaded 21,024 images. Of those images, 13,684 (65.1%) correspond to surgical pathology images, 4045 (19.2%) belong to neuropathology cases, and 3295 (15.7%) originate from autopsies. We demonstrate the design and implementation of a customizable anatomic pathology digital media archive solution in an academic pathology department setting using a modern standard for exchanging healthcare information electronically. The system’s efficiency and scalability for our current operation will enable us to integrate with other applications and pathology informatics initiatives in the future.

Comparison of 3D models of an object placed in two different media (air and water) created on the basis of photos obtained with a mobile phone camera

Photogrammetry is one of the contactless geodetic techniques that provide reliable information about objects and their surface properties. Traditional photogrammetric methods require specialized equipment, such as metric cameras, which are very expensive and difficult to access, and the processing of the data obtained from them requires a significant investment of time. An alternative to this solution are generally available non-metric cameras, which can replace professional photogrammetric devices. Currently, consumer-grade digital cameras are enjoying the increase of popularity among photogrammeters thanks to the interchangeable lenses, good image sensor quality and the ability to manually set all camera parameters. However, due to the high costs of purchasing and maintaining such equipment, mobile phone digital cameras are increasingly used, the parameters of which are beginning to match many professional digital SLRs. The newest smartphones are also waterproof, so they can have potential applications in underwater photogrammetry.This article presents the possibilities of using mobile phone cameras in close-range photogrammetry and shows the beginning of research on the use of smartphones in underwater photogrammetry. Three-dimensional models of the gear wheel were made on the basis of photos obtained in two media: in the air and underwater. The two point clouds were then compared, obtaining an average distance difference of ±0,3 mm.

Development of an automated plant phenotyping system for evaluation of salt tolerance in soybean

Plant high-throughput phenotyping technology is taking more and more important roles in soybean breeding and genetic research thanks to the advance in sensing and data analytic technologies. However, commercial high-throughput phenotyping systems of general purpose are expensive and complicated for many research groups, and their data analytic methods are designed for specific research projects. The goal of this study was to develop and validate a customized image-based phenotyping system that was used to automatedly collect, process and analyze imagery data of soybean cultivars to evaluate their response to salt stress in controlled environments. The imaging system consisted of a consumer-grade digital camera and an automated platform was used to take sequential images of soybean plants of five cultivars under salt stress during the experimental period. An image processing and analytic pipeline was developed to automatically extract image features and evaluate their tolerance to salt stress. Results indicated that two image features, i.e. canopy area and ExV (the difference of excess green and excess red) were highly correlated with salinity tolerance trait of soybean. The image saturation and blue channel values were able to extract salt stress characteristics and identify different types of salt stress characteristics. In addition, the ratio of damaged leaf area to canopy area was extracted as a novel image feature to quantify the salinity tolerance grade. The overall results indicated that the automatic plant phenotyping system based on low-cost image sensors and automation platform was able to quantify plant stress due to salt stress and would be useful in soybean breeding programs.

Performance Evaluation of High Sensitivity Consumer Grade Digital Cameras as On-Board Camera at Night

Various kinds of cameras such as 360°camera, action cameras, smartphones, and drive recorders have been utilizing as an on-board cameras for the construction of the Intelligent Transport Systems. However, there are still problems with efficient imaging using on-board cameras. These problems are the influence of sunshine, and shadow or occlusion by the person or cars. On the other hand, the high sensitive functions of consumer grade digital cameras are amazingly increasing, and high sensitive imaging such as ISO409600 was achieved.

Quantitative Trimming for Ortho Imaging at Night by Onboard High Sensitivity Consumer Grade Digital Cameras

Various kinds of cameras have been utilizing as onboard cameras in the construction of Intelligent Transport Systems. In recently, utilization of the high sensitivity consumer grade digital cameras at night is attracting attention from the viewpoint of avoiding the effects of sunlight and congestion of people and cars. However, due to the image taken by the onboard cameras is a perspective projection image, the image is projected small at the far from car and the greater the influence of the lens distortion the farther from the image center. In order to avoid the issues, lower part of projection image or a bird's-eye view image is used, but the imaging of the bonnet part due to the vehicle model and the tilt of the camera becomes a new issue. Furthermore, a bird's-eye view image at night has to be trimmed to coincide with irradiation range since the irradiation distance and range for headlight are limited. On the other hand, feature quantities such as vanishing points and feature points on the lane have been used for projective transformation from a perspective projection image to a bird's-eye view image, but the projective transformation based on the feature quantity is an ill-posed problem.

Affordable Phenotyping of Winter Wheat under Field and Controlled Conditions for Drought Tolerance

Drought stress is one of the key plant stresses reducing grain yield in cereal crops worldwide. Although it is not a breeding target in Northern Europe, the changing climate and the drought of 2018 have increased its significance in the region. A key challenge, therefore, is to identify novel germplasm with higher drought tolerance, a task that will require continuous characterization of a large number of genotypes. The aim of this work was to assess if phenotyping systems with low-cost consumer-grade digital cameras can be used to characterize germplasm for drought tolerance. To achieve this goal, we built a proximal phenotyping cart mounted with digital cameras and evaluated it by characterizing 142 winter wheat genotypes for drought tolerance under field conditions. The same genotypes were additionally characterized for seedling stage traits by imaging under controlled growth conditions. The analysis revealed that under field conditions, plant biomass, relative growth rates, and Normalized Difference Vegetation Index (NDVI) from different growth stages estimated by imaging were significantly correlated to drought tolerance. Under controlled growth conditions, root count at the seedling stage evaluated by imaging was significantly correlated to adult plant drought tolerance observed in the field. Random forest models were trained by integrating measurements from field and controlled conditions and revealed that plant biomass and relative growth rates at key plant growth stages are important predictors of drought tolerance. Thus, based on the results, it can be concluded that the consumer-grade cameras can be key components of affordable automated phenotyping systems to accelerate pre-breeding for drought tolerance.

MAPPING QUALITY EVALUATION OF MONOCULAR SLAM SOLUTIONS FOR MICRO AERIAL VEHICLES

Abstract. Monocular simultaneous localization and mapping (SLAM) attracted much attention in the mobile-robotics domain over the past decades along with the advancements of small-format, consumer-grade digital cameras. This is especially the case for micro air vehicles (MAV) due to their payload and power limitations. The quality of global 3D reconstruction by SLAM solutions is a critical factor in occupancy-grid mapping, obstacle avoidance, and map representation. Although several benchmarks have been created in the past to evaluate the quality of vision-based localization and trajectory-estimation, the quality of mapping products has been rarely studied. This paper evaluates the quality of three state-of-the-art open-source monocular SLAM solutions including LSD-SLAM, ORB-SLAM, and LDSO in terms of the geometric accuracy of the global mapping. Since there is no ground-truth information of the testing environment in existing visual SLAM benchmark datasets (e.g., EuRoC, TUM, and KITTI), an evaluation dataset using a quadcopter and a terrestrial laser scanner is created in this work. The dataset is composed of the image data extracted from the recorded videos by flying a drone in the test environment and the high-fidelity point clouds of the test area acquired by a terrestrial laser scanner as the ground truth reference. The mapping quality evaluation of the three SLAM algorithms was mainly conducted on geometric accuracy comparisons by calculating the deviation distance between each SLAM-derived point clouds and the laser-scanned reference. The mapping quality was also discussed with respect to their noise levels as well as further applications.

Hyperspectral Image Recovery Using a Color Camera for Detecting Colonies of Foodborne Pathogens on Agar Plate

Hyperspectral imaging often requires a special camera system to obtain spectral images. The cost for acquisition and process of hyperspectral images is usually much higher than color images. On the contrary, typical consumer-grade digital color cameras are much cheaper to obtain and process spatially high-resolution images than hyperspectral cameras. This paper is concerned with the development of a hyperspectral image recovery technique that can reconstruct hyperspectral images only from color images obtained by a digital color camera. A sparse representation and least squares regression-based classification of foodborne pathogens on agar plates are presented. The target pathogen bacteria were the six representative non-O157 Shiga toxin-producing Escherichia coli (STEC) serogroups (O26, O45, O103, O111, O121, and O145). The wavelength range for the spectral recovery was from 400 to 1000 nm. Unlike many other studies using color charts with known and noise-free spectra for training their spectral recovery models, we directly used the hyperspectral and color images of real scenes for training the spectral recovery models. Both hyperspectral and color images were calibrated to percent reflectance values and then spatially registered. Two spectral recovery models including polynomial multivariate linear regression (MLR) and partial least squares regression (PLSR) were evaluated and compared by cross-validation and independent test. The spectral recovery results showed that the PLSR was more effective than the MLR. The classification accuracy measured with recovered spectra in an independent test set was about 5–10% less than the case of using the true hyperspectral images although the difference in maximum classification accuracy was only about 2%. The results suggested the potential of a cost-effective color imaging system using hyperspectral image classification algorithms for differentiating pathogens in agar plates.

CAMERA CALIBRATION WITH IRRATIONAL RADIAL DISTORTION MODEL WITH ANALYTICAL SOLUTIONS

Abstract. Consumer grade digital cameras are widely used in many applications including Photogrammetric mapping and 3D modelling. One common limitation found in such cameras is radial lens distortions. To produce wide angle lenses camera manufacturers reduce the amount of barrel distortion by minimizing both the central and edge distortion profiles, resulting in a mixture of pincushion and barrel distortions for a single lens. These lenses also lack symmetry, making some of the existing distortion models almost ineffective. The mostly used model for radial distortion corrections is the polynomial model which is difficult to solve analytically especially when the model possesses many quadratic terms. Suggestions were made for division models but such models are not suitable when the lens field of view exceeds 180 degrees and exhibit some instabilities when dealing with large magnitude of distortion coefficients. Moreover mathematical formulations of some models cannot handle negative distortion coefficients. Attempts to improve the division models were made with proposals for rational models which present the advantage of handling larger distortion magnitude with fewer terms. However some of these models do not account for all the distortion coefficients in their solutions, limiting the potential of the techniques. This study presents an irrational distortion model with analytical solutions. The proposed model was tested with imagery captured by wide angle lenses and the experimental results reveal that the technique produced the best estimates of radial distortion coefficients. The proposed model was also able to capture image distortions originating from projection errors by the wide angles lenses used in this study.

A SYSTEM FOR MONITORING OF UAV CAMERA ORIENTATION: DESIGN AND INITIAL ANALYSIS

Abstract. Many unmanned aerial vehicles (UAV) that are used for aerial mapping are equipped with consumer-grade digital cameras, which use CMOS (Complementary metal–oxide–semiconductor) image sensors. Majority of these sensors capture images using an electronic rolling shutter, which can cause distortions on the image if either the camera or the captured objects are moving. This phenomenon is usually ignored in aerial mapping with UAVs in practice. However, there is a lack of published research papers that would prove the effect can be neglected. In this paper, we present the design of the system for monitoring UAV camera orientation. Furthermore, the calibration process to get correct and reliable readings is described. The initial analysis of the data is focused on assessing the accuracy that can be achieved using the proposed system. The main component of our system is a MEMS (Microelectromechanical system) gyroscope. It was selected for its low weight and size, low price and high sampling rates which are all very beneficial characteristics for a system, mounted on a UAV. In a paper, a working prototype is presented that uses the selected MEMS gyroscope connected to a single-board computer. The presented initial analysis of collected data shows, that the system would be capable to indirectly detect the image distortions, caused by camera orientation changes during exposure, in the range of typical ground sample distance (GSD).