Mapping Camera Research Articles

Using a depth camera for crop row detection and mapping for under-canopy navigation of agricultural robotic vehicle

Computer vision provides local environmental information for robotic navigation in crop fields. It is particularly useful for robots operating under canopies of tall plants such as corns (Zea Mays) and sorghums (Sorghum bicolor), where GPS signal is not always receivable. However, the development of under-canopy navigation systems is still an open research area. The key contribution of our work is the development of a vision-based system for under-canopy navigation using a Time-of-Flight (ToF) camera. In the system, a novel algorithm was used to detect parallel crop rows from depth images taken under crop canopies. Two critical tasks in navigation were accomplished based on the detection results: 1) generating crop field maps as occupancy grids when reliable robot localization is available (from other sources such as GPS and IMU), and 2) providing inter-row vehicle positioning data when the field map is available and the localization is not reliable. The proposed system was evaluated in field tests. The test results showed that the proposed system was able to map the crop rows with mean absolute errors (MAE) of 3.4 cm and 3.6 cm in corn and sorghum fields, respectively. It provides lateral positioning data with MAE of 5.0 cm and 4.2 cm for positioning in corn and sorghum crop rows, respectively. The potential and limitations of using ToF cameras for under-canopy navigation were discussed.

SETUP OF A CORONA CAMERA AND IMAGE CO-REGISTRATION / CALIBRATION

Abstract. Power lines are an important infrastructure and need special attention. Their functionality is of high importance for our life. Their monitoring is important to guarantee their sustainable operation. It is done mostly by aerial survey using different technologies. In addition to standard setups, e.g. LiDAR and high resolution RGB cameras, often UV cameras, so called Corona Cameras, are used to detect unwanted discharges. In many cases this UV cameras are combined along with RGB cameras with a similar field of view in order to superimpose the detected UV signals onto a visual image of the inspected objects. Discharges at power lines indicate findings that should be monitored with such additional sensors like thermal or high resolution mapping cameras then.The use of Corona-Cameras for such inspection work needs several steps of calibration, co registering and direct referencing methods in order to get them in a good georeference to the other sensors.

Combined Geometric Positioning and Performance Analysis of Multi-Resolution Optical Imageries from Satellite and Aerial Platforms Based on Weighted RFM Bundle Adjustment

Combined geometric positioning using images with different resolutions and imaging sensors is being increasingly widely utilized in practical engineering applications. In this work, we attempt to perform the combined geometric positioning and performance analysis of multi-resolution optical images from satellite and aerial platforms based on weighted rational function model (RFM) bundle adjustment without using ground control points (GCPs). Firstly, we introduced an integrated image matching method combining least squares and phase correlation. Next, for bundle adjustment, a combined model of the geometric positioning based on weighted RFM bundle adjustment was derived, and a method for weight determination was given to make the weights of all image points variable. Finally, we conducted experiments using a case study in Shanghai with ZiYuan-3 (ZY-3) satellite imagery, GeoEye-1 satellite imagery, and Digital Mapping Camera (DMC) aerial imagery to validate the effectiveness of the proposed weighted method, and to investigate the positioning accuracy by using different combination scenarios of multi-resolution heterogeneous images. The experimental results indicate that the proposed weighted method is effective, and the positioning accuracy of different combination scenarios can give a good reference for the combined geometric positioning of multi-stereo heterogeneous images in future practical engineering applications.

Utilization of Smartphone Depth Mapping Cameras for App-Based Grading of Facial Movement Disorders: Development and Feasibility Study.

BackgroundFor the classification of facial paresis, various systems of description and evaluation in the form of clinician-graded or software-based scoring systems are available. They serve the purpose of scientific and clinical assessment of the spontaneous course of the disease or monitoring therapeutic interventions. Nevertheless, none have been able to achieve universal acceptance in everyday clinical practice. Hence, a quick and precise tool for assessing the functional status of the facial nerve would be desirable. In this context, the possibilities that the TrueDepth camera of recent iPhone models offer have sparked our interest.ObjectiveThis paper describes the utilization of the iPhone’s TrueDepth camera via a specially developed app prototype for quick, objective, and reproducible quantification of facial asymmetries.MethodsAfter conceptual and user interface design, a native app prototype for iOS was programmed that accesses and processes the data of the TrueDepth camera. Using a special algorithm, a new index for the grading of unilateral facial paresis ranging from 0% to 100% was developed. The algorithm was adapted to the well-established Stennert index by weighting the individual facial regions based on functional and cosmetic aspects. Test measurements with healthy subjects using the app were performed in order to prove the reliability of the system.ResultsAfter the development process, the app prototype had no runtime or buildtime errors and also worked under suboptimal conditions such as different measurement angles, so it met our criteria for a safe and reliable app. The newly defined index expresses the result of the measurements as a generally understandable percentage value for each half of the face. The measurements that correctly rated the facial expressions of healthy individuals as symmetrical in all cases were reproducible and showed no statistically significant intertest variability.ConclusionsBased on the experience with the app prototype assessing healthy subjects, the use of the TrueDepth camera should have considerable potential for app-based grading of facial movement disorders. The app and its algorithm, which is based on theoretical considerations, should be evaluated in a prospective clinical study and correlated with common facial scores.

Multi-Sensor Fusion: A Simulation Approach to Pansharpening Aerial and Satellite Images.

The growing demand for high-quality imaging data and the current technological limitations of imaging sensors require the development of techniques that combine data from different platforms in order to obtain comprehensive products for detailed studies of the environment. To meet the needs of modern remote sensing, the authors present an innovative methodology of combining multispectral aerial and satellite imagery. The methodology is based on the simulation of a new spectral band with a high spatial resolution which, when used in the pansharpening process, yields an enhanced image with a higher spectral quality compared to the original panchromatic band. This is important because spectral quality determines the further processing of the image, including segmentation and classification. The article presents a methodology of simulating new high-spatial-resolution images taking into account the spectral characteristics of the photographed types of land cover. The article focuses on natural objects such as forests, meadows, or bare soils. Aerial panchromatic and multispectral images acquired with a digital mapping camera (DMC) II 230 and satellite multispectral images acquired with the S2A sensor of the Sentinel-2 satellite were used in the study. Cloudless data with a minimal time shift were obtained. Spectral quality analysis of the generated enhanced images was performed using a method known as “consistency” or “Wald’s protocol first property”. The resulting spectral quality values clearly indicate less spectral distortion of the images enhanced by the new methodology compared to using a traditional approach to the pansharpening process.

Automated Processing of Declassified KH-9 Hexagon Satellite Images for Global Elevation Change Analysis Since the 1970s

Observing changes in Earth surface topography is crucial for many Earth science disciplines. Documenting these changes over several decades at regional to global scale remains a challenge due to the limited availability of suitable satellite data before the year 2000. Declassified analog satellite images from the American reconnaissance program Hexagon (KH-9), which surveyed nearly all land surfaces from 1972 to 1986 at metric resolution, provide a unique opportunity to fill the gap in observations. However, large-scale processing of analog imagery remains challenging. We developed an automated workflow to generate Digital Elevation Models (DEMs) and orthophotos from scanned KH-9 mapping camera stereo images. The workflow includes a preprocessing step to correct for film and scanning distortions and crop the scanned images, and a stereo reconstruction step using the open-source NASA Ames Stereo Pipeline. The processing of several hundreds of image pairs enabled us to estimate reliable camera parameters for each KH-9 mission, thereby correcting elevation biases of several tens of meters. The resulting DEMs were validated against various reference elevation data, including snow-covered glaciers with limited image texture. Pixel-scale elevation uncertainty was estimated as 5 m at the 68% confidence level, and less than 15 m at the 95% level. We evaluated the uncertainty of spatially averaged elevation change and volume change, both from an empirical and analytical approach, and we raise particular attention to large-scale correlated biases that may impact volume change estimates from such DEMs. Finally, we present a case study of long-term glacier elevation change in the European Alps. Our results show the suitability of these historical images to quantitatively study global surface change over the past 40-50 years.

Calibration of Industrial Cameras for Aerial Photogrammetric Mapping

This paper details the development of a camera calibration method purpose-built for use in photogrammetric survey production. The calibration test field was established in a hangar, where marker coordinates were measured using a high-precision survey methodology guaranteeing very high accuracy. An analytical model for bundle adjustment was developed that does not directly use the coordinates of field calibration markers but integrates bundle adjustment and survey observations into a single process. This solution, as well as a classical calibration method, were implemented in a custom software, for which the C++ source code repository is provided. The method was tested using three industrial cameras. The comparison was drawn towards a baseline method, OpenCV implementation. The results point to the advantages of using the proposed approach utilizing extended bundle adjustment.

Application of Spectral Mixture Analysis to Vessel Monitoring Using Airborne Hyperspectral Data

As marine transportation has increased in coastal regions, maritime accidents associated with vessels have steadily increased. Remotely sensed satellite or airborne images can aid rapid vessel monitoring over wide areas at high resolutions. In this study, airborne hyperspectral experiments were performed to detect marine vessels mainly including fishing boat and yacht by applying pixel-based mixture techniques and to estimate the size of the vessels through an objective ellipse fitting method. Various spectral libraries of marine objects and seawaters were constructed through in-situ experiments for spectral analysis of the internal structures of vessels. The hyperspectral images were dimensionally reduced through principal component analysis. Several hyperspectral mixture algorithms, such as N-FINDR, pixel purity index (PPI), independent component analysis (ICA), and vertex component analysis (VCA), were used for the detection of vessels. The N-FINDR and VCA techniques presented a total of 14 vessels, the ICA technique detected seven vessels, and the PPI technique detected two vessels. The pixel-based probability of detection (POD) and false alarm ratio (FAR) for all 14 vessels were 96.40% and 4.30%, respectively. The sizes of the vessels were estimated by extracting the boundaries of the vessels through a two-dimensional gradient and applying the ellipse fitting method. Compared with the digital mapping camera (DMC) images with resolutions of 0.10 m, the root-mean-square errors of the length and width of the vessels were approximately 1.19 m and 0.81 m, respectively. The application of spectral mixing methods provided a high probability of detecting the objects, as well as the overall structures of the decks of the vessels.

Angular effect in proximal sensing of leaf-level chlorophyll content using low-cost DIY visible/near-infrared camera

Proximal sensing is increasingly highlighted, owing to its potential for various applications in precision agriculture. However, the uses of its related special devices like visible/near-infrared (VNIR) cameras for crop mapping are yet limited in practice, due to their high costs in market. To handle this issue, this case study was dedicated to modifying a low-cost RGB camera as a VNIR one in a do-it-yourself (DIY) way – replacing its infrared-blocking optical filter with a red-blocking one – for estimating leaf-level chlorophyll content. To enhance its performance, the angular effect that proved to impact the quality of imaging in conventional remote sensing was examined here. Specifically, in the cases of 120 individual leaves for four plant species, 24 vegetation indices (VIs) were proposed and derived from the three bands (blue, green, and near-infrared) of the collected images; then, the same operation was repeated for a set of leaf inclinations (θ, the angle between the optical axis of the camera and the normal vector of the leaf-fixed plate, set from 0°, 5°, …, to 50°). The results showed that for each of the four plant species, different optimal VIs for leaf chlorophyll content retrieval were detected for different geometries of leaf reflectance. This suggested that more modular retrieval models capable of integrally reflecting the angular effect best should be developed for accurately sensing of leaf biochemical content. Overall, the contribution of this study is of fundamental implications for advancing quantitative proximal sensing in plant biochemistry and precision agriculture.

COMPARATIVE EVALUATION OF DERIVED IMAGE AND LIDAR POINT CLOUDS FROM UAV-BASED MOBILE MAPPING SYSTEMS

Abstract. Unmanned aerial vehicles (UAVs) have been widely used for 3D reconstruction/modelling in various applications such as precision agriculture, coastal monitoring, and emergency management. For such mapping applications, camera and LiDAR are the two most commonly used sensors. Mapping with imagery-based approaches is considered to be an economical and effective option and is often conducted using Structure from Motion (SfM) techniques where point clouds and orthophotos are generated. In addition to UAV photogrammetry, point clouds of the area of interest can also be directly derived from LiDAR sensors onboard UAVs equipped with global navigation satellite systems/inertial navigation systems (GNSS/INS). In this study, a custom-built UAV-based mobile mapping system is used to simultaneously collect imagery and LiDAR data. Derived LiDAR and image-based point clouds are investigated and compared in terms of their absolute and relative accuracy. Furthermore, stability of the system calibration parameters for the camera and LiDAR sensors are studied using temporal datasets. The results show that while LiDAR point clouds demonstrate a high absolute accuracy over time, image-based point clouds are not as accurate as LiDAR due to instability of the camera interior orientation parameters.

HIGH PRECISION FULLY INTEGRATED AIRBORNE DIGITAL MAPPING SYSTEMS – STATE OF THE ART AND PERFORMANCE ANALYSIS

Abstract. This paper introduces the Phase One Aerial System 150, a next generation fully integrated fully digital aerial camera system with one single digital camera head and lens which almost matches the perfect geometry of a film camera for all airborne mapping applications. It is the first true replacement for the simplicity, geometry and efficiency established by film cameras for traditional airborne mapping. Several test flights were planned to be flown with the Phase One Aerial System 150 over the Greater Denver Area, Colorado, U.S.A. during the Winter of 2020. Two lenses are planned to be used, namely: 1) 50 mm lens for wide coverage and a geometry closest to that of a film camera which is suitable for most mapping applications, and 2) 90 mm lens which provides a higher resolution (smaller GSD) and a narrower field of view which is suitable for applications where less building lean might be required. Multiple flight altitudes are flown in order to end up with a GSD of 10 cm, and 20 cm, respectively. One dual-altitude flight was planned to characterize and calibrate the integrated system including camera in-flight calibration and camera/IMU boresight calibration. The remaining flights are planned to be used to validate system accuracy and productivity as well as the mapping product accuracy. Due to the Covid-19 pandemic and poor weather, only one full dual-altitude light has been flown. Therefore, system calibration, assessment, and validation are done using this single test flight. The remainder of the test flights intended for map production evaluation and accuracy assessment will be flown during the Spring of 2020, which results will be shared with the ISPRS audience during the congress presentations.

WAVELETS FOR SELF-CALIBRATED BUNDLE BLOCK ADJUSTMENT

Abstract. This paper entails a methodological novelty and builds upon prior research on a wavelets-based model for digital camera self-calibration. We introduce a new kernel function based on the compactly supported orthogonal third-order asymmetric Daubechies wavelet to correct systematic image distortion errors. Tests are done by using aerial images taken with a high-resolution metric digital aerial mapping camera. The quality of experimental results is evaluated by using reliable and high precision ground check points in the calibration field. For example, a four-fold block with this wavelet self-calibration model has the external accuracy of about 0.28 GSD (=ground sampling distance) in the horizontal direction, and about 0.43 GSD in the vertical direction, respectively, where 1GSD ≈ 4.6cm. The posterior standard deviations σ̂0 of unit weight are reduced from 0.37 pixel to 0.27 pixel. The residual vector lengths are also significantly reduced after our wavelet additional parameters are used. Experimental results support the proposal and demonstrate the applicability of this new model.

AMS-3000 LARGE FIELD VIEW AERIAL MAPPING SYSTEM: BASIC PRINCIPLES AND THE WORKFLOW

Abstract. Three-line array stereo aerial survey camera is a typical mapping equipment of aerial photogrammetry. As one of the airborne equipment, it can quickly obtain a large range of basic geographic information with high precision. At present, typical three-line array stereoscopic aerial survey cameras, such as Leica ADS40 and 80, have the disadvantages of small field of view and low resolution, which makes it difficult to meet the demand of large-scale topographic mapping for economic construction. For the urgent need of domestic three linear array aerial mapping camera in our project, we developed the AMS-3000 camera system. Camera features include a large field of view, high resolution, low distortion and high environmental adaptability. The AMS-3000 system has reached the international advanced level on both software and hardware aspects.

PARTICIPATORY IMAGE-BASED MODELS’ ALIGNMENT FOR RECONSTRUCTING A LARGE-SCALE INDOOR MAPPING

Abstract. In this paper, we introduced a recently developed image-based model alignment technique for 3D reconstruction of large-scale indoor corridors. The proposed participatory model alignment technique enables crowd source single image-based modeling since it allows various participants to incorporate their images taken from different cameras for large-scale indoor mapping. This technique is robust against changes of camera orientation and prevents miss-association of a newly generated 3D model to the previously integrated models. To investigate the possibility of aligning two individual 3D models, their respective corridor topological graphs must match, and they need to geometrically transform into the same object space. Here 3D affine transformation is applied, and the transformation parameters are estimated through corresponding vertices of both 3D models. Having integrated two models in the same 3D space, they will be back projected into the image space for evaluation using Direct Linear Transformation. Note that the proposed method performs layout model matching in image space and considers information including layout topology and geometry as well as image information to address model alignment. The advantages of using layout information in the proposed alignment technique are twofold. First, a metric constraint is imposed to insure topological model consistency and balance 3D models scale issues. Second, it will reduce alignment ambiguity related to indoor corridor scenes, where the scene is enriched with multiple structural elements including various corridors junctions. To evaluate the performance of the proposed method, we have performed the experiments on a data set collected from Ross building corridors at York University. This dataset includes single images captured by a handheld wide-angle camera. The obtained results present the ability of the proposed method in alignment of single image-based 3D models while producing limited geometric errors.

Application of Machine learning Algorithms in Autonomous Vehicles Navigation System

Autonomous vehicles are the future of transportation and it also expected to become a fully-fledged reality within a decade. The major giants in the automotive industry like Toyota with Microsoft and Amazon, Mercedes with Bosch, Audi with Huawei, etc. are hard pressing their transition from conventional vehicle to autonomous vehicles. The state of Karnataka, for instance, had approximately 205,200 registered taxis higher than Madhya Pradesh 174,900 registered cabs from 2014 to 2015. This results in higher traffic congestion, pollution and high fuel consumption due to unorganized driving practice. As a consequence, this presents a great deal of opportunities for autonomous cars as they can significantly reduce the accident rate and improve effective and stress free parking, optimal running time, fuel economy etc. It also reduces traffic congestion as all the autonomous cars can be synchronized together through cloud. But the crucial part is their navigation system. Although a number of sophisticated forms of technology like such as radars, lasers, and high definition cameras for mapping, localization, obstacle detection, etc, are used, autonomous vehicles are still trying to reach perfection to navigate precisely under uninformed terrains and dynamics obstacles on the way like pedestrians, erratic drivers, traffic and climatic conditions. Events like heavy snow and rain, improper lane marking makes it hard to detect an obstacle on the road for evasive maneuver within limited response time period. Machine learning is one of the fast growing technologies that provide optimal solution to overcome the challenges in the autonomous navigation system. This paper focuses on the application of different machine learning algorithms and compares the use, challenge and scope of the same. The paper also focuses on the future direction of machine learning with reference to autonomous navigation system.

Optimal Spectral Combination of a Hyperspectral Camera for Intraoperative Hemodynamic and Metabolic Brain Mapping

Intraoperative optical imaging is a localization technique for the functional areas of the human brain cortex during neurosurgical procedures. These areas are assessed by monitoring the oxygenated (HbO2) and deoxygenated hemoglobin (Hb) concentration changes occurring in the brain. Sometimes, the functional status of the brain is assessed using metabolic biomarkers: the oxidative state of cytochrome-c-oxidase (oxCCO). A setup composed of a white light source and a hyperspectral or a standard RGB camera could be used to identify the functional areas. The choice of the best spectral configuration is still based on an empirical approach. We propose in this study a method to define the optimal spectral combinations of a commercial hyperspectral camera for the computation of hemodynamic and metabolic brain maps. The method is based on a Monte Carlo framework that simulates the acquisition of the intrinsic optical signal following a neuronal activation. The results indicate that the optimal spectral combination of a hyperspectral camera aims to accurately quantify the HbO2 (0.5% error), Hb (4.4% error), and oxCCO (15% error) responses in the brain following neuronal activation. We also show that RGB imaging is a low cost and accurate solution to compute Hb maps (4% error), but not accurate to compute HbO2 (48% error) or oxCCO (1036% error) maps.

Automatic Age Estimation: A Survey

Since, aging is non-reversible process. Human face and gait change with time which reflects major variations in appearance, the vast majority of people are able to easily recognize human traits like emotional states, where they can tell if the person is happy, sad or angry from the face, likewise, it is easy to determine the gender of the person. However, knowing person’s age is a very challenge task. Hence, significant interest in the computer vision and pattern recognition research community is given to automatic age estimation. This paper, presents a thorough analysis of recent research in aging and age estimation. Discuss popular algorithms used in age estimation and existing models. Underline age estimation challenges especially using RGB images. Finally insights for future research based on depth map and Kinect camera.

A geometric calibration model for the new ultra‐large frame aerial mapping camera DMZ II

AbstractThe DMZ II (DaMianZhen II) camera uses a multi‐frame optical plane assembly to form a virtual ultra‐large array sensor (23 000 × 23 000 pixels). The geometric model of this aerial mapping camera, with one lens and 12 component CCD panchromatic arrays, is proposed. The new geometric model combines the well‐known global parameters with sensor‐specific local parameters for the individually installed CCD arrays. Test flight evaluations show that the model can improve the geometric accuracy from ±6 μm (1 pixel) to ±1·5 μm (1/4 pixel), equivalent to that of traditional single‐array imaging sensors, providing a multiple‐array large‐frame aerial mapping camera with high accuracy and efficiency.

Inferring Surface Flow Velocities in Sediment-Laden Alaskan Rivers from Optical Image Sequences Acquired from a Helicopter

The remote, inaccessible location of many rivers in Alaska creates a compelling need for remote sensing approaches to streamflow monitoring. Motivated by this objective, we evaluated the potential to infer flow velocities from optical image sequences acquired from a helicopter deployed above two large, sediment-laden rivers. Rather than artificial seeding, we used an ensemble correlation particle image velocimetry (PIV) algorithm to track the movement of boil vortices that upwell suspended sediment and produce a visible contrast at the water surface. This study introduced a general, modular workflow for image preparation (stabilization and geo-referencing), preprocessing (filtering and contrast enhancement), analysis (PIV), and postprocessing (scaling PIV output and assessing accuracy via comparison to field measurements). Applying this method to images acquired with a digital mapping camera and an inexpensive video camera highlighted the importance of image enhancement and the need to resample the data to an appropriate, coarser pixel size and a lower frame rate. We also developed a Parameter Optimization for PIV (POP) framework to guide selection of the interrogation area (IA) and frame rate for a particular application. POP results indicated that the performance of the PIV algorithm was highly robust and that relatively large IAs (64–320 pixels) and modest frame rates (0.5–2 Hz) yielded strong agreement ( R 2 > 0.9 ) between remotely sensed velocities and field measurements. Similarly, analysis of the sensitivity of PIV accuracy to image sequence duration showed that dwell times as short as 16 s would be sufficient at a frame rate of 1 Hz and could be cut in half if the frame rate were doubled. The results of this investigation indicate that helicopter-based remote sensing of velocities in sediment-laden rivers could contribute to noncontact streamgaging programs and enable reach-scale mapping of flow fields.

Vision Guiding System Based on Strong Positioning and Hand-eye Calibration

In order to solve the current localization algorithm big background interference affects calibration methods exist damage, physical contact with the workpiece material on the surface of the problem, hand-eye calibration was proposed based on strong orientation and three visual guidance system. First, deepen the Hessian matrix, using Gaussian filter, using the maximum inhibition precisely determine the feature points and direction, in order to construct surf feature points description, finish strong positioning operator based on surf, to remove the background and the ambient light interference, the purpose of accurate positioning center of the target with Angle. And then put forward three points calibration method, calculation scale rotation matrix and translation matrix parameters, complete the Robot world coordinates and binding mapping camera image coordinates. Finally, program implementation algorithm and system, the experiment results showed that compared with the current workpiece material transfer system, this system has higher goal orientation and calibration transfer success rate.