Metaheuristic Camera Calibration for Optical Tomographic Imaging in Industrial Environments

Fish-eye lenses are common in several computer vision applications, such as four-camera surround view driver assistance, where a very wide angle (e.g., 180 degrees) of view is available. Nevertheless, their applicability is usually limited by the lack of an accurate and easy-to-use calibration procedure. In this paper, we present a camera calibration method for fish-eye lenses and a panoramic image stitching framework for calibrated surround images. To achieve the calibration of fish-eye captured images, it only requires to observe a reference planar pattern (e.g., chessboard), followed by offline estimating extrinsic and intrinsic parameters and save the related parameters. Each fish-eye distorted image can then be efficiently online corrected. Then, each calibrated image is transformed to its top-down view (or bird's-eye view) via the perspective transformation based on the estimated homography matrix. As a result, these surround bird'seye view images can be stitched to generate the final panoramic image. It is expected that the proposed framework would be applicable to AVM (around view monitoring) system or ADAS (advanced driver assistance system) of vehicles in the future.

This study uses machine vision, feature extraction, and support vector machine (SVM) to compose a vibration monitoring system (VMS) for an in situ evaluation of the performance of industrial motors. The vision-based system respectively offers a spatial and temporal resolution of 1.4 µm and 16.6 ms after the image calibration and the benchmark of a laser displacement sensor (LDS). The embedded program of machine vision has used zero-mean normalized correlation (ZNCC) and peak finding (PF) for tracking the registered characteristics on the object surface. The calibrated VMS provides time–displacement curves related to both horizontal and vertical directions, promising remote inspections of selected points without attaching additional markers or sensors. The experimental setup of the VMS is cost-effective and uncomplicated, supporting universal combinations between the imaging system and computational devices. The procedures of the proposed scheme are (1) setting up a digital camera, (2) calibrating the imaging system, (3) retrieving the data of image streaming, (4) executing the ZNCC criteria, and providing the time–displacement results of selected points. The experiment setup of the proposed VMS is straightforward and can cooperate with surveillances in industrial environments. The embedded program upgrades the functionality of the camera system from the events monitoring to remote measurement without the additional cost of attaching sensors on motors or targets. Edge nodes equipped with the image-tracking program serve as the physical layer and upload the extracted features to a cloud server via the wireless sensor network (WSN). The VMS can provide customized services under the architecture of the cyber–physical system (CPS), and this research offers an early warning alarm of the mechanical system before unexpected downtime. Based on the smart sensing technology, the in situ diagnosis of industrial motors given from the VMS enables preventative maintenance and contributes to the precision measurement of intelligent automation.

Disease model systems, such as the zebrafish, play an important role in understanding the onset of diseases like cancer and monitor the efficacy of new drugs. In the past, non-invasive methods for screening, diagnostics and treatment monitoring were intrinsically from the outside. In the past decades, there has been a strong drive to look inside these model systems, which resulted in the development of many small animal tomographic imaging techniques. Due to the absence of ionizing radiation, high-resolution, and cost efficiency, optical tomography is a popular imaging technique to study disease model systems such as zebrafish. The main obstacles in obtaining high-resolution imaging suitable for tissue characterization are the scattering of light in tissue and diffraction of optical waves. Scattering of light in tissue degrades the resolution of optical tomography systems, especially for thick samples. In this thesis, transmission optical coherence tomography (OCT) is used to select ballistic, non-scattered, from non-ballistic, scattered, light. We demonstrate that transmission optical coherence tomography is a versatile tool to measure optical properties of liquids, solids, and particle suspensions. The developed technique is used to perform quantitative optical tomography of the refractive index and attenuation coefficient. A good agreement is observed between our measurements and literature values for group refractive index, group velocity dispersion, and attenuation coefficient. Based on the tomographic reconstruction of transmission OCT measurements, the median attenuation coefficient, group refractive index and volumes of various organs of an adult zebrafish are segmented and quantified in optical coherence projection tomography reconstructions. In optical tomography light is imaged by a lens onto the camera. Due to the focusing of light onto the camera, this light is collected non-uniformly along the propagation direction from the sample. Consequently, the straight-ray assumption as in standard (pre-) clinical X-ray CT reconstruction is violated. Reconstruction of optical tomography images with standard filtered back projection (FBP) causes radial blurring and tangential blurring that becomes stronger with increasing distance to the rotation axis. We present 2D and 3D tomographic reconstruction algorithms that include the point spread function (PSF) of the imaging system. For emission optical projection tomography, these methods show greatly reduced radial and tangential blurring over the entire field of view 113 114 Summary and a significantly improved signal-to-noise ratio compared to FBP. The 3D PSF-based algorithm is evaluated using different initializations. When initialized with the 2D PSF-based reconstruction result, the 3D PSF-based reconstruction gives an improved signal-to-background and image quality in a useful timeframes. Besides including the physical point spread function (PSF) in the 2D tomographic reconstruction, the effect of the PSF also can be reduced by deconvolution of the FBP reconstructed image or filtering the sinogram before FBP reconstruction. We compared the performance of these techniques with each other based on simulations and the signal-to-noise ratio and the sharpness in reconstructed fluorescent beads and zebrafish OPT images. We demonstrate that the sinogram filtering performs poorly on data acquired with high numerical aperture optical imaging systems. We show that the deconvolution technique performs best for highly sparse, low signal-to-noise ratio objects. The PSF-based reconstruction method is superior for non-sparse objects and data of high signal-to-noise ratio. In this thesis, we developed novel algorithms for transmission OCT signal processing and PSF-based tomographic reconstruction. Our algorithms allow for high-resolution quantitative imaging in turbid media. These techniques can be used for quantitative optical imaging of disease model systems. Potentially this may lead to more insight in tissue development and disease onset, progression, and treatment.

Automated quality control in industrial environments requires sensors to obtain information about the manufactured products. Visual sensors are crucial for this task. In industrial environments, line-scan cameras are commonly used for their ability to acquire images from fast moving objects at high resolution. However, the information in the images must be translated from image coordinates to scene coordinates. This work proposes a calibration procedure for line-scan cameras particularly designed for industrial environments. The proposed approach is based on a novel calibration target that can be used in multi-camera architectures, a common scenario found in many product inspection pipelines. Industrial environments are very challenging for visual sensors, as they are affected by harsh conditions. The calibration procedure integrates an outlier rejection method to deal with these issues. In addition, the calibration target includes a large number of feature points. Consequently, the removal of some of these features does not affect the performance of the calibration. Additional signal processing methods are proposed to extract features from the images. The calibration procedure uses data acquired from multiple observations by optimizing the reprojection error, resulting in an accurate and robust method, suitable for in industrial environments. The design of the calibration target, the increased number of features, the outlier rejection method and other contributions of this work result in an accurate estimation of the camera parameters. The proposed method is validated using synthetic and real data acquired with a commercial line-scan camera.

A new method of camera calibration is proposed for high-precise videometrics in large field. The camera to be calibrated and the control points used in the new method are both close to the ground. In the condition that the control points and the optical center of the camera are approximately coplanar, the principle point position, focal length, lens distortion coefficients and the camera's position and attitude parameters are calibrated precisely by the method. Two calibration images are taken by the camera to be calibrated in measurement state and vertical rotation state respectively. If the vertical tangent lens distortion can be neglected or the movement field of the targets to be measured are close to the ground, only the measurement state calibration image is needed to calibrate the camera's parameters except the vertical tangent lens distortion coefficients. By the new method, to cali- brate the camera's intrinsic parameters in laboratory in advance is not needed. The new method breaks the localization for the traditional camera calibration methods in large field videometrics which require the control points must be distributed in space rationally.

Monocular cameras are commonly used in communication devices, entertainment, and industrial environments since they allow people to recognize rapidly scenes and objects. On the other hand, thermal cameras are mostly known in industrial environments to visualize the objects’ thermic radiance but they lose their visual details. This work presents the design and implementation of a thermography inspection tool – INVIfusion – to fuse infrared and visual spectrum images. This tool includes three modules: image acquisition and calibration module, multimodal image fusion module, and report generation module. The main contribution of this work compared with other software inspection tools is INVIfusion supports cameras from different manufacturers (infrared and visible spectrum), having different field of view and spatial resolutions. To validate the suggested system’s functionality and accuracy, quantitative and qualitative tests were performed considering different camera configurations. In all cases the thermal camera was a FLIR E320. Quantitative tests were performed measuring the mean symmetric re-projection error obtaining a maximum error of 2.08 ± 1.8 pixels.

A new kind of structured-lighting surface sensor based on computer vision is presented. The sensor can be used to complete dimensional measurements in an industrial environment with the help of smart calibration processes and image algorithms. The mathematical model and calibration techniques for the sensor are described in detail. In the local calibration, termed the differential method, structured parameters and the local coordinate system of the sensor are determined. Subsequently, a global calibration process is introduced briefly. As an application, a sample vision inspection system is built for checking dimensions on the left sideframe of a body-in-white (BIW). An automotive BIW is an assembled body before insertion of door, windshield, hood, decklid, and fender.

This paper presents and evaluates a passive device-free tracking system for localization of objects or persons. The system works with a continuous evaluation of the received signal strengths of connection links between fixed installed anchor nodes. Out of the evaluated measurements, an image of the scene can be generated based on Radio Tomographic Imaging (RTI). The major contribution of this paper is an application of segmentation algorithms for sharpening the image by reducing noise and interference. The applied combination of image segmentation with a connected component labeling algorithm enables an evaluation of certain areas in the image to estimate the position and the number of objects in the room. Finally, the application of particle filtering smoothes the estimated position and leads to a reliable and accurate position estimation, which is comparable with active solutions. As interference and noise in the link measurements caused by multipath propagation effects can harm the achievable accuracy, the foreseen application area - an industrial environment- poses a huge challenge, as the radio channel is highly affected by the metallic infrastructures.

Image distortion is a problem due to wide field-of-view cameras, and camera calibration is a fundamental step in various applications such as image undistortion, 3D reconstruction, and camera motion estimation to overcome this problem. In image calibration, intrinsic camera parameters such as focal length and distortion are estimated. The quality of the undistorted/enhanced image depends on the correctness of focal length and distortion. However, existing methods consist of two approaches: checkerboard, which requires manual interaction, and others are deep learning approaches. Most Deep Learning approaches are based on the Convolution Neural Network (CNN) framework, and it fails to capture the long-term dependency in a distorted image. This paper proposes a fully automated EnsembleNet method to infer the focal length and distortion parameters to overcome this problem. The proposed model extracts various contexts (local patches) by exploiting ViT(Vision Transformer) and spatial features from various CNN-based models using a single input image. The proposed model uses the differential evolution (DE) approach to learn the ensemble weights. The experiments show that the proposed EnsembleNet outperforms the state-of-the-art deep learning-based models in terms of mean squared error.

Trends in utilization of reflectance confocal microscopy in the United States, 2017-2019

The full-color LISS-4MN images with 5.8m spatial resolution obtained by Indian IRS-P6 satellite were used in this study.The technologies of band composition under MN mode,image calibration and data fusion of LISS-4MN and ETM+ were researched in the preliminary plan on soil and water conservation.Land use type,vegetation cover type,soil erosion and soil and water conservation in Xianzigou watershed of Liujiaxia reservoir were surveyed using IRS-P6 images based on artificial patches drawing,the features of watershed soil erosion,was learned comprehensively.A reasonable and effective soil erosion prevention system was established to improve the efficiency of soil and water conservation in ecological engineering.

Diffuse optical tomography (DOT) is emerging as a viable new biomedical imaging modality. Using visible and near‐infrared light, in the range of 500 to 900 nm, this technique can probe absorption as well as scattering properties of biological tissues. The main applications are currently in brain, breast, limb, and join imaging; however, the area of optical tomographic imaging of small animals is attracting increasing attention. This interest is mainly fueled by recent advances in transgenic manipulation of small animals that has led to many models of human diseases. Using these models it is possible to link specific genes, proteins, and enzymes to molecular, and cellular processes that underlie various disorders. In addition, the advent of novel biochemical markers that are sensitive to molecular processes, defect genes, and cell receptors, makes it for the first time possible to detect diseases on a molecular level long before actual phenotypical symptoms appear. Small animal optical tomography has several advantages over other, more traditional, imaging modalities. For example, optical markers emit low‐energy near‐infrared photons that are less harmful than more energetic gamma rays emitted from radioactive markers (used in SPECT and PET, for instance). This simplifies synthesis procedures and experimental designs and will be of particular importance for future applications in humans. Furthermore, optical methods typically offer higher sensitivity (as compared to MRI and SPECT) and are relatively inexpensive (as compared to PET, SPECT, and MRI). In this paper we will review underlying principles in optical tomographic imaging as they apply to studies involving small animals. We will describe the basic contrast mechanism involved in imaging of endogenous as well exogenous contrast agents. In addition we will discuss specific features and advantages of different types of optical instrumentation currently available, such as steady‐state, frequency‐domain, and time‐resolved imaging systems. Furthermore we will describe in detail the structure of commonly used image reconstruction schemes and algorithms and point to still existing challenges. Finally, we will provide an overview of the most recent literature in optical small animal imaging, specifically in the areas of blood oximetry, fluorescence and bioluminescence imaging.Educational Objectives:1. To understand significance, potential, and limits of optical tomographic small animal imaging.2. To understand the contrast mechanisms that underlie optical tomographic imaging.3. To understand the basic measurement instrumentation used in optical tomographic imaging.4. To understand the fundamental concepts and problems involving optical tomographic image reconstruction algorithms.5. To learn about the major applications of this novel technology.

In computer vision, camera calibration is essential for photogrammetric measurement. We propose a new stratified camera calibration method based on geometric constraints. This paper proposes several new theorems in 2D projective transformation: (1) There exists a family of lines whose parallelity remains invariable in a 2D projective transformation. These lines are parallel with the image of the infinity line. (2) There is only one line whose verticality is invariable with the family of parallel lines in a 2D projective transformation, and the principal point lies on this line. With the image of the infinite line and the dual conic of the circular points, the closed-form solution of the line passing through principal point is deduced. The angle among the target board and image plane, which influences camera calibration, is computed. We propose a new geometric interpretation of the target board's pose and solution method. To obtain appropriate poses of the target board for camera calibration, we propose a visual pose guide (VPG) of the target board system that can guide a user to move the target board to obtain appropriate images for calibration. The expected homography is defined, and its solution method is deduced. Experimental results with synthetic and real data verify correctness and validity of the proposed method.

Target-specified reference-based deep learning network for joint image deblurring and resolution enhancement in surgical zoom lens camera calibration

Background: Dynamic Diffuse optical imaging using near-infrared light was shown to be promising method for neoadjuvant therapy monitoring as an alternative functional imaging that is low-cost, non-invasive, portable, safe and simple to operate. While optical breast imaging methods rely on "static" assessments of tissue oxy- and deoxy- hemoglobin concentration without contrast agents, they are insufficient for clinical applications. Dynamic tomographic optical imaging induces tumor-sensitive hemodynamic variations, as a contrast mechanism, driven by fractional mammographic compression. These tumor contrast measurements are governed by interlay of tissue biomechanics and oxygen metabolism. In this study we seek to evaluate the predictive value of these biomarkers with respect to treatment outcome. Methods: A group of 22 patients with locally advanced breast cancer were scanned using our dynamic TOBI system before and during neoadjuvant chemotherapy. In this analysis we focused on pre-treatment, day 7 and day 30 post-treatment dynamic TOBI scans. Both breasts are compressed in turn to 4-8 lbs of force (depending on size) and optical images are acquired every 2 seconds over 2 minutes. We calculate the time course of oxy (HbO), deoxy (HbR)and total (HbT) hemoglobin concentration as well as the hemoglobin oxygen saturation (SO2). Regions of interest are defined in the optical images to correspond to the radiology identified tumor location, and the healthy tissue in the same breast, respectively. We compare the time courses in the two regions at baseline, day 7 and 30 days after initiation of treatment. Results: In this analysis we present results from 10 patients including 6 responders (defined as greater than 50% reduction in the largest tumor axis from baseline imaging and final pathology) and 4 non-responders. As the compression plates are held in place the tissue collagen matrix begins to stretch, effectively reducing the compression force. At baseline, all patients exhibit a decrease followed by delayed recovery in HbT, and SO2 in the tumor area, in contrast to immediate recovery in surrounding tissue. At day 7 and 30, this contrast is maintained in non-responders (<50% reduction in tumor maximum diameter); however, in responders, the contrast starts decreasing at day 7 and substantially disappears at day 30. Average changes in HbT and SO2, show that the contrast between normal and tumor increases somewhat at day 7 and more noticeably at day 30 in non-responders to NACT. Comparing hemodynamic changes in responders and non-responders, it is clear that at three selected time points (30, 60 and 90 s) during the scan, the contrast between tumor and normal tissue in both ΔHbT and ΔSO2 is reduced for the responders at day 7 and day 30. Conclusions: These initial results suggest that dynamic optical breast imaging can detect changes due to treatment and have predictive value for the treatment outcome. DTOBI can show the difference in hemodynamic response to compression between tumor and normal tissue and demonstrates the feasibility of using dynamic optical breast tomography for neoadjuvant chemotherapy monitoring. ongoing. Citation Format: Sajjadi AY, Singh B, Deng B, Boas DA, Moy B, Schapira L, Bardia A, Specht MC, Carp SA, Isakoff SJ. Dynamic tomographic optical breast imaging (TOBI) for neoadjuvant chemotherapy monitoring. [abstract]. In: Proceedings of the Thirty-Eighth Annual CTRC-AACR San Antonio Breast Cancer Symposium: 2015 Dec 8-12; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2016;76(4 Suppl):Abstract nr P4-03-04.