Non-overlapping Fields Of View Research Articles

A novel global calibration method for vision measurement system based on mirror-image stereo target

Using three-dimensional (3D) targets to solve the transformation relationship between the local and global coordinate systems is a common global calibration method. However, 3D targets have complex structures and high costs. And multiple coordinate transformations during global calibration can cause significant cumulative errors. Therefore, based on reflection imaging, this paper designs a new mirror-image stereo target (MST) and analyzes its view area and three structural parameter constraints. A new global calibration method is proposed, as well as three optimization algorithms for pose solution between MST, optical 3D sensor and global camera to improve the global calibration accuracy. The experimental results show that the registration errors of two workpieces are 0.086 mm and 0.114 mm, respectively, confirming the feasibility and precision of the proposed method. Our method is flexible, requiring only a planar target and a mirror, rather than complex 3D targets, to achieve global calibration with non-overlapping field of view.

Adaptive Point-Line Fusion: A Targetless LiDAR-Camera Calibration Method with Scheme Selection for Autonomous Driving.

Accurate calibration between LiDAR and camera sensors is crucial for autonomous driving systems to perceive and understand the environment effectively. Typically, LiDAR-camera extrinsic calibration requires feature alignment and overlapping fields of view. Aligning features from different modalities can be challenging due to noise influence. Therefore, this paper proposes a targetless extrinsic calibration method for monocular cameras and LiDAR sensors that have a non-overlapping field of view. The proposed solution uses pose transformation to establish data association across different modalities. This conversion turns the calibration problem into an optimization problem within a visual SLAM system without requiring overlapping views. To improve performance, line features serve as constraints in visual SLAM. Accurate positions of line segments are obtained by utilizing an extended photometric error optimization method. Moreover, a strategy is proposed for selecting appropriate calibration methods from among several alternative optimization schemes. This adaptive calibration method selection strategy ensures robust calibration performance in urban autonomous driving scenarios with varying lighting and environmental textures while avoiding failures and excessive bias that may result from relying on a single approach.

TMIC-84. A DEEP LEARNING-BASED METHOD FOR RAPID, PATIENT-SPECIFIC ASSAY OF MACROPHAGE INFILTRATION IN HIGH-GRADE GLIOMA USING LABEL-FREE STIMULATED RAMAN HISTOLOGY

Abstract BACKGROUND 5-Aminolevulinic acid (5-ALA), the most widely used fluorophore in image-guided glioma surgery, identifies tumor-associated macrophages (TAM) in high-grade glioma (HGG) tissue. Infiltrating macrophages, the predominant immune cells in glioma, are highly implicated in tumor progression, recurrence, and treatment response – particularly in immunotherapy. Using a unique, paired dataset of label-free stimulated Raman histology (SRH) and two-photon microscopy (TPEF) images sharing one-to-one spatial resolution, we developed a deep-learning approach to automatically quantify TAM infiltration from intraoperatively acquired SRH images. METHODS We compiled a dataset of 906 paired whole-slide SRH/TPEF images from 79 patients with HGG. We first trained a pix2pix generative adversarial network to convert raw, label-free SRH into synthetic TPEF with identifiable macrophages. Next, we trained a MaskR-CNN model to locate and segment TAMs. The pix2pix network was trained using 5,531 hand-picked 300-by-300-pixel fields-of-view (FOV) best exemplifying TAMs. We used a human-in-the-loop approach to train the segmentation network on 1,000 hand-labeled FOVs. RESULTS Macrophage segmentation from purely histologic data was near-identical to the fluorescence-based ground truth, with a mean dice score of 90.3%. Predicted TAM density was highly correlated with r=0.735. Subgroup analyses of TAM density in 346,836 non-overlapping FOVs revealed significantly lower TAM density in IDH-mutant (p< 0.01) and MGMT-hypermethylated (p=0.03) tumors. Our end-to-end algorithm uses fresh, unlabeled tissue specimens in the operating room and takes just three minutes to analyze whole-slide SRH images. CONCLUSION We demonstrate how deep neural networks can be used to rapidly and quantitatively evaluate macrophage infiltration in HGG in the operating room. Our software enables analysis of glioma patients’ tumor immune environment without immunohistochemistry or fluorescent labels, and may be leveraged to study the effects of immune-modulating therapies in the tumor microenvironment. Future work will focus on evaluating macrophage density and distribution as biomarkers for response to immunotherapies showing efficacy in glioma patients.

외부 시스템을 이용한 Monocular camera-3D LiDAR 융합 시스템의 Non-overlapping 외부 교정 방법에 대한 연구

This study presents a novel framework for the extrinsic calibration of monocular camera and LiDAR with a non-overlapping field of view. The conventional non-overlapping extrinsic calibration utilizes a motion-based calibration method. However, motion-based calibration has a disadvantage in that the vehicle must be moving. In order to solve extrinsic calibration problems under such constraint configurations, the proposed solution uses external sensors that are located away from vehicle. First, the external camera and vehicle camera utilize an aruco marker. Second, the external LiDAR and vehicle LiDAR utilize a scan matching method. In this study, a Gazebo simulator was utilized to show the accuracy of the proposed method. The external LiDAR used Velodyne HDL 64ch, the vehicle LiDAR used Puck 16ch, and the camera used the Blackfly color sensor from FLIR.

Displacement measurement of large structures using nonoverlapping field of view multi‐camera systems under six degrees of freedom ego‐motion

AbstractIn this work, a non‐overlapping field of view (FOV) multi‐camera system (MCS) has been proposed to realize the motion self‐compensation, which can be flexibly arranged according to the field situation. The challenges of ensuring a complete fixed measurement platform owing to the self‐weight, unexpected collisions, and adjacent engineering constructions have been addressed in this study. MCS can be fixed on an unstable platform and employ two or more correcting cameras, without an overlapping FOV, to measure and compensate the six degrees of freedom (6‐DOF) ego‐motion of the platform, with high accuracy. The numerical simulation results show that, under the same configuration conditions, the 6‐DOF motion estimation accuracy of the proposed method is better than that of the Perspective‐n‐Points methods, especially for the estimation accuracy of the out‐of‐plane motion and the roll motion of the camera. The field demonstration has been performed on a suspension bridge, and the results demonstrate that the deflection measurements provided by the proposed MCS are consistent with those provided by the Robotic Total Station. The proposed MCS, which is not restrained by a strictly stable platform, proves to be an effective method to compensate for the errors induced by the platform ego‐motion, and it can meet the requirements of the displacement measurement of large structures needing high precision.

Global calibration method for non-overlapping cameras based on rigidly connected stereo targets.

In complex field of view (FOV) environments, a single camera's FOV measurement range is limited and cannot cover the entire object under test for global calibration. Multiple cameras are used mostly for large FOV environment measurements, but the traditional one- and two-dimensional targets used for global calibration in large FOV environments are prone to overlapping FOV. Furthermore, other large-sized targets are difficult to produce and process, and the laser projection method and plane mirror calibration methods are easily affected by the outdoor environment. To solve this problem, a non-common FOV binocular calibration method based on rigidly connected stereo targets is proposed. The calibration process is as follows: First, the rigidly connected target, which is composed of two plane targets with a checkerboard, is placed in front of the two cameras, and the vision sensor captures the corresponding sub-target image; then, the target is moved multiple times, and the transformation relationship between multiple vision sensors is obtained according to the spatial constraint characteristics of the rigidly connected target. Hence, the method overcomes the limitation of the non-overlapping FOV calibration method that relies on large measuring instruments. The experimental results show that the RMS error of the 13 mm distance is 0.16 mm. The proposed method is effective, simpler to operate than other methods, and does not rely on the constraint of complex targets. More importantly, this measurement method solves the difficult problem of measurement in non-public FOV, meeting the requirements of large FOV measurement ranges.

Multivision Sensor Extrinsic Calibration Method With Non-Overlapping Fields of View Using Encoded 1D Target

Multiple vision sensors (multi-VSs) are difficult to calibrate in large fields of view vision measurement owing to the lack of overlapping areas. Thus, this study proposes a novel method for the external parameter calibration of multi-VSs by using an encoded 1D target. The quick response code is used in this study to uniquely identify the feature circle. The Hough transform method and ellipse center solving method are jointly applied to optimize the feature points and further improve the collinearity of the feature points. As the cross-ratio of the collinearity feature points involves a projection invariance, the rotation and translation vectors between the multi-VSs are solved by the inverse process of projection and rigid transformation. The accuracy and advantages of the proposed method are verified by simulation and real experiments. The proposed target has a simple structure, the operation is convenient and flexible, and the method can be applied to the global calibration of multi-VSs with non-overlapping fields of view.

MC-Calib: A generic and robust calibration toolbox for multi-camera systems

In this paper, we present MC-Calib, a novel and robust toolbox dedicated to the calibration of complex synchronized multi-camera systems using an arbitrary number of fiducial marker-based patterns. Calibration results are obtained via successive stages of refinement to reliably estimate both the poses of the calibration boards and cameras in the system. Our method is not constrained by the number of cameras, their overlapping field-of-view (FoV), or the number of calibration patterns used. Moreover, neither prior information about the camera system nor the positions of the checkerboards are required. As a result, minimal user interaction is needed to achieve an accurate and robust calibration which makes this toolbox accessible even with limited computer vision expertise. In this work, we put a strong emphasis on the versatility and the robustness of our technique. Specifically, the hierarchical nature of our strategy allows to reliably calibrate complex vision systems even under the presence of noisy measurements. Additionally, we propose a new strategy for best-suited image selection and initial parameters estimation dedicated to non-overlapping FoV cameras. Finally, our calibration toolbox is compatible with both, perspective and fisheye cameras. Our solution has been validated on a large number of real and synthetic sequences including monocular, stereo, multiple overlapping cameras, non-overlapping cameras, and converging camera systems. Project page: https://github.com/rameau-fr/MC-Calib

Trends in Vehicle Re-Identification Past, Present, and Future: A Comprehensive Review

Vehicle Re-identification (re-id) over surveillance camera network with non-overlapping field of view is an exciting and challenging task in intelligent transportation systems (ITS). Due to its versatile applicability in metropolitan cities, it gained significant attention. Vehicle re-id matches targeted vehicle over non-overlapping views in multiple camera network. However, it becomes more difficult due to inter-class similarity, intra-class variability, viewpoint changes, and spatio-temporal uncertainty. In order to draw a detailed picture of vehicle re-id research, this paper gives a comprehensive description of the various vehicle re-id technologies, applicability, datasets, and a brief comparison of different methodologies. Our paper specifically focuses on vision-based vehicle re-id approaches, including vehicle appearance, license plate, and spatio-temporal characteristics. In addition, we explore the main challenges as well as a variety of applications in different domains. Lastly, a detailed comparison of current state-of-the-art methods performances over VeRi-776 and VehicleID datasets is summarized with future directions. We aim to facilitate future research by reviewing the work being done on vehicle re-id till to date.

Calibration of binocular cameras with non-overlapping fields of view based on planar mirrors

Multicamera systems are commonly applied in the large field-of-view (FOV) measurements. However, two cameras may have a non-overlapping FOV in some scenes, and traditional binocular camera calibration methods cannot be used to directly calibrate non-overlapping cameras. To solve this problem, our study proposes a calibration method for binocular cameras with non-overlapping FOVs on the basis of planar mirrors. According to the reflection characteristics of an optical planar mirror, the proposed method can use the same target to calibrate non-overlapping cameras. Hence, the method overcomes the limitation of non-overlapping cameras being unable to observe common targets. Experiments results show that the maximum RMS error does not exceed 0.53 mm. Hence, the proposed method is effective, and its measurement technique is simpler and more universal than those of other methods. In addition, it is applicable to a wide range of measurements.

Flexible Extrinsic Parameter Calibration for Multicameras With Nonoverlapping Field of View

In many measurement applications using multicameras, there is often a nonoverlapping field of view (nFOV) between multicameras. It benefits greatly from having all of the cameras unified in a single coordinate frame by means of common spatial constraints. However, it is very difficult to establish common spatial constraints for accurate and quick calibration of the extrinsic parameters between multicameras in the case of long working distance, asymmetric working angle, or limited working space. To overcome these issues, we propose a flexible calibration method using a camera rig, which can be easily modeled as a hand–eye calibration problem and solved by only several sets of calibration images. To further improve the calibration accuracy, the camera intrinsic parameters, lens distortion coefficients, and extrinsic parameters are optimized simultaneously, similar to binocular calibration. However, this causes calculation instability of multiparameters. We extend the epipolar constraint to two images with an nFOV and add it into the optimization objective function together with the reprojection error, which ensures the basic consistency of intrinsic parameters before and after optimization, and reduce the fluctuation caused by the number of images involved in calibration. Experiments and application have proved that the proposed new methods are feasible and effective.

Stereo calibration with absolute phase target.

Stereo cameras have been widely used for three-dimensional (3D) photogrammetry, and stereo calibration is a crucial process to estimate the intrinsic and extrinsic parameters. This paper proposes a stereo calibration method with absolute phase target by using horizontal and vertical phase-shifting fringes. The one-to-one mapping from the world points to the image points that can be recovered by referring to the absolute phase and then used to calibrate the stereo cameras. Compared with traditional methods that only use feature points within the overlapping field-of-view (FOV), the proposed method can use all feature points within the overlapping and non-overlapping FOVs. Besides, since phase is more robust against camera defocusing than intensity, the target images can be captured regardless of the depth-of-field (DOF). With the advantages of whole-field capability and defocusing tolerability, the target placement becomes very flexible. Both simulations and experiment results demonstrate the robustness and accuracy of the proposed method.

The Road is Enough! Extrinsic Calibration of Non-overlapping Stereo Camera and LiDAR using Road Information

This paper presents a framework for the targetless extrinsic calibration of stereo cameras and Light Detection and Ranging (LiDAR) sensors with a non-overlapping Field of View (FOV). In order to solve the extrinsic calibrations problem under such challenging configuration, the proposed solution exploits road markings as static and robust features among the various dynamic objects that are present in urban environment. First, this study utilizes road markings that are commonly captured by the two sensor modalities to select informative images for estimating the extrinsic parameters. In order to accomplish stable optimization, multiple cost functions are defined, including Normalized Information Distance (NID), edge alignment and, plane fitting cost. Therefore a smooth cost curve is formed for global optimization to prevent convergence to the local optimal point. We further evaluate each cost function by examining parameter sensitivity near the optimal point. Another key characteristic of extrinsic calibration, repeatability, is analyzed by conducting the proposed method multiple times with varying randomly perturbed initial points.

Subtleties of extrinsic calibration of cameras with non-overlapping fields of view

Abstract The calibration of the relative pose between rigidly connected cameras with non-overlapping fields of view (FOV) is a prerequisite for many applications. In this paper, the subtleties of the experimental realization of such calibration optimization methods like in (Z. Liu, et al., Measurement Science and Technology, 2011, Z. Li, V. Willert, Intelligent Transportation Systems (ITSC), 2018) are presented. Two strategies that could be adapted to certain optimization processes to find better local minima are evaluated. The first strategy is a careful measurement acquisition of pose pairs for solving the calibration problem, which improves the accuracy of the initial value for the following non-linear refinement. The second strategy is the introduction of a quality measure for the image data used for the calibration, which is based on the projection size of the known planar calibration patterns on the image. We show that introducing an additional weighting to the optimization objective chosen as a function of that quality measure improves calibration accuracy and increases robustness against noise. The above strategies are integrated into different setups and their improvement is demonstrated both in simulation and real-world experiment.

Extrinsic parameters calibration of multi-camera with non-overlapping fields of view using laser scanning.

An extrinsic parameters calibration method of multi-cameras with non-overlapping fields of view (FOV) using laser scanning is presented. Firstly, two lasers are mounted on a multi-degree-of-freedom manipulator and can scan objects freely by the projected line-structured light. Then, controlling the movement of the manipulator, the line-structured light is projected into the field of view of one of the multi-cameras, and the light plane equation in the camera coordinate frame is calibrated by the target. The manipulator is moved several times in small amplitude to change the position of structured light in the field of vision of the camera and to continue to calibrate the light plane. The light plane equation of line-structured light in the manipulator coordinate frame are solved by the hand-eye calibration method. Secondly, with the help of the light planes, projected into the field of vision of other cameras to be calibrated, the light plane equation in the camera coordinate frame is calibrated, and the external parameters between the camera coordinate frame and the manipulator coordinate frame are calculated, so that the calibration of the external parameters of multiple cameras can be realized. The proposed method connects the non-overlapping multi-cameras by the laser scanning. It can effectively solve the problem of multi-camera extrinsic parameter calibration under the conditions of long working distance and complex environment light.

Extrinsic Parameters Calibration Method of Cameras with Non-Overlapping Fields of View in Airborne Remote Sensing

Multi-camera systems are widely used in the fields of airborne remote sensing and unmanned aerial vehicle imaging. The measurement precision of these systems depends on the accuracy of the extrinsic parameters. Therefore, it is important to accurately calibrate the extrinsic parameters between the onboard cameras. Unlike conventional multi-camera calibration methods with a common field of view (FOV), multi-camera calibration without overlapping FOVs has certain difficulties. In this paper, we propose a calibration method for a multi-camera system without common FOVs, which is used on aero photogrammetry. First, the extrinsic parameters of any two cameras in a multi-camera system is calibrated, and the extrinsic matrix is optimized by the re-projection error. Then, the extrinsic parameters of each camera are unified to the system reference coordinate system by using the global optimization method. A simulation experiment and a physical verification experiment are designed for the theoretical arithmetic. The experimental results show that this method is operable. The rotation error angle of the camera’s extrinsic parameters is less than 0.001rad and the translation error is less than 0.08 mm.

Global calibration of non-overlapping cameras: State of the art

Multi-camera vision systems (MVS) have a larger field of view than a single camera and are often applied to acquire geometric information of large-scale objects or scenes. Data integration from multiple cameras is a very important part for vision systems which is usually solved by using a global calibration method. After installing the MVS, in some cases, cameras do not have a common field of view (CFV) or only a narrow CFV. Accurate global calibration of cameras with non-overlapping field of view (FOV) is a very challenging task. A variety of global calibration of non-overlapping multi-camera methods (GCNM) have been proposed to estimate the relative positions and orientations of cameras based on different types of media or techniques such as large-range measuring devices, large-scale calibration targets, optical mirrors, motion model, laser projection, visual measuring instruments, etc. However, the GCNM is not yet a completely solved problem. Choosing which type of GCNM method to use is highly dependent on the specific vision system. Thus, in this paper, we present a comparative review of different GCNM methods and analyze accuracy, range, defects, and applications. Researchers and developers can take it as background information for their future works.

Moving human tracking across multi-camera based on artificial immune random forest and improved colour-texture feature fusion

ABSTRACTIn multi-camera video tracking, the tracking scene and tracking-target appearance can become complex. To finish multi-camera tracking in these challenging, we first utilize an improved brightness transfer function to establish matching tasks of different cameras and to reduce the influence of brightness changes between multiple cameras. Then, we proposes an improved colour-texture feature fusion (ICTFF) that is composed of the colour features and texture features for multi-camera human tracking in non-overlapping field of view. It’s the first time to use artificial immune random forest with fully exploit the linear combination information of the feature and colour feature that can achieving the optimization of the number of decision trees and the effective classification of the target feature. Compared with state-of-the-art algorithms, our ICTFF algorithm can significantly improve the tracking accuracy in some complex scenes, such as changing the speed, changing the direction and appearance of the target.

Multi-camera visual SLAM for autonomous navigation of micro aerial vehicles

In this paper, we present a visual simultaneous localization and mapping (SLAM) system which integrates measurements from multiple cameras to achieve robust pose tracking for autonomous navigation of micro aerial vehicles (MAVs) in unknown complex environments. We analyze the iterative optimizations for pose tracking and map refinement of visual SLAM in multi-camera cases. The analysis ensures the soundness and accuracy of each optimization update. A well-known monocular visual SLAM system is extended to utilize two cameras with non-overlapping fields of view (FOVs) in the final implementation. The resulting visual SLAM system enables autonomous navigation of an MAV in complex scenarios. The theory behind this system can easily be extended to multi-camera configurations, when the onboard computational capability allows this. For operations in large-scale environments, we modify the resulting visual SLAM system to be a constant-time robust visual odometry. To form a full visual SLAM system, we further implement an efficient back-end for loop closing. The back-end maintains a keyframe-based global map, which is also used for loop-closure detection. An adaptive-window pose-graph optimization method is proposed to refine keyframe poses of the global map and thus correct pose drift that is inherent in the visual odometry. We demonstrate the efficiency of the proposed visual SLAM system for applications onboard of MAVs in experiments with both autonomous and manual flights. The pose tracking results are compared with ground truth data provided by an external tracking system.

Flexible global calibration of multiple cameras with nonoverlapping fields of view using circular targets.

Global calibration of multicamera systems is a difficult problem. The key is to find an appropriate calibration target. This paper proposes a flexible method to construct a global calibration target with circular targets. Any object that is to be measured by the system can be used as a calibration target with the help of a hand-held scanner. The calibration method does not need a special calibration target or multiple shots, which makes it flexible in applications. Circular targets pasted on the calibration target are used as features that are captured by cameras. The particle swarm optimization method is employed to correct the eccentricity error in the projection of circular targets. The eccentricity correction method does not need any prior knowledge except the cameras' intrinsic parameters. A synthetic data experiment was performed to validate the eccentricity correction method, and a physical experiment in a train wheelset inner-side distance measurement system was performed to validate the global calibration method. The measurement accuracy of the system was better than 0.1 mm, and the eccentricity correction method improved the three-dimensional reconstruction accuracy by about 0.1 mm.