Actual Camera Research Articles

Feasibility Study on the Use of Infrared Cameras for Skin Cancer Detection under a Proposed Data Degradation Model.

Infrared thermography is considered a useful technique for diagnosing several skin pathologies but it has not been widely adopted mainly due to its high cost. Here, we investigate the feasibility of using low-cost infrared cameras with microbolometer technology for detecting skin cancer. For this purpose, we collected infrared data from volunteer subjects using a high-cost/high-quality infrared camera. We propose a degradation model to assess the use of lower-cost imagers in such a task. The degradation model was validated by mimicking video acquisition with the low-cost cameras, using data originally captured with a medium-cost camera. The outcome of the proposed model was then compared with the infrared video obtained with actual cameras, achieving an average Pearson correlation coefficient of more than 0.9271. Therefore, the model successfully transfers the behavior of cameras with poorer characteristics to videos acquired with higher-quality cameras. Using the proposed model, we simulated the acquisition of patient data with three different lower-cost cameras, namely, Xenics Gobi-640, Opgal Therm-App, and Seek Thermal CompactPRO. The degraded data were used to evaluate the performance of a skin cancer detection algorithm. The Xenics and Opgal cameras achieved accuracies of 84.33% and 84.20%, respectively, and sensitivities of 83.03% and 83.23%, respectively. These values closely matched those from the non-degraded data, indicating that employing these lower-cost cameras is appropriate for skin cancer detection. The Seek camera achieved an accuracy of 82.13% and a sensitivity of 79.77%. Based on these results, we conclude that this camera is appropriate for less critical applications.

A Distortion Correction Method Based on Actual Camera Imaging Principles.

In the human-robot collaboration system, the high-precision distortion correction of the camera as an important sensor is a crucial prerequisite for accomplishing the task. The traditional correction process is to calculate the lens distortion with the camera model parameters or separately from the camera model. However, in the optimization process calculate with the camera model parameters, the mutual compensation between the parameters may lead to numerical instability, and the existing distortion correction methods separated from the camera model are difficult to ensure the accuracy of the correction. To address this problem, this study proposes a model-independent lens distortion correction method based on the image center area from the perspective of the actual camera lens distortion principle. The proposed method is based on the idea that the structured image preserves its ratios through perspective transformation, and uses the local image information in the central area of the image to correct the overall image. The experiments are verified from two cases of low distortion and high distortion under simulation and actual experiments. The experimental results show that the accuracy and stability of this method are better than other methods in training and testing results.

BusEye: A multi-stream approach for driver behavior analysis on public bus driver cameras

In the contemporary era, modern public buses are equipped with multiple built-in cameras, offering real-time or recorded monitoring capabilities through centralized systems. This research endeavors to introduce a novel approach for classifying driver behaviors at the signalized road intersections with traffic lights by leveraging a public bus vehicular camera that encompasses the road and thus traffic lights. The proposed method involves a 2-step multi-head self-attention network that employs the extracted traffic light state, and optic flow information besides the raw vehicular camera video. Subsequently, a bi-directional long short-term memory based spatio-temporal aggregation strategy is applied to guide the learning final discriminative representation. To evaluate the proposed approach, we introduce a benchmark dataset called BusEye, which covers the challenging real-life scenarios that are captured on actual vehicular cameras mounted on public busses. We provide extensive analysis and ablation studies on the BusEye dataset that shows our proposed solution outperforms the state-of-the-art video classification methods both on a per-class basis and overall. Based on the outcomes derived from the validation and test data, the proposed method demonstrated remarkable efficacy in classifying driver behaviors at signalized intersections.

Application of a stand-alone RTI measuring system with an integrated camera in cultural heritage digitisation

The Multi-Light Image Collection (MLIC) data, in most cases processed with the Reflectance Transformation Imaging (RTI) technique, is one of the popular methods of documentation and digital representation of movable finds from an archaeological excavation. The manual (hand-held) approach to collecting this data is technically simple and still popular in the case of archaeology. However, an extensive amount of work during image collection is a strong motivation for building RTI acquisition systems, which are also popular, in most cases, in the shape of domes. The research presented in the article focuses on two aspects of building such a device. First, the study examines the effect of the placement and number of lights on the resulting RTI model for a system based on LED stripe-shaped circuits. Secondly, the camera component was studied.Recent development in the area of low-cost accessories for microcomputers resulted in a few new industrial-grade USB cameras with the possibility to take high-resolution images. Cameras designed for use with microcomputers offer broad control of the focus or exposition parameters and provide the possibility to build RTI domes with the fully integrated camera, therefore, almost complete automatization of the capturing and processing of MLIC data.This paper presents the main assumptions and elements of the dome-shaped measurement system, which enables automatic RTI data acquisition and processing together with the evaluation of light configuration used to build relightable models. Amphoras handles with stamps and ancient pottery were used as an objects to evaluate the quality of the relightable models generated from the images acquired by a programmable miniature Arducam camera. Results of comparison with professional high-quality mirrorless cameras show that the quality of images from miniature cameras can be satisfactory for some archaeological objects.

Evaluation of Camera Recognition Performance under Blockage Using Virtual Test Drive Toolchain.

This study is the first to develop technology to evaluate the object recognition performance of camera sensors, which are increasingly important in autonomous vehicles owing to their relatively low price, and to verify the efficiency of camera recognition algorithms in obstruction situations. To this end, the concentration and color of the blockage and the type and color of the object were set as major factors, with their effects on camera recognition performance analyzed using a camera simulator based on a virtual test drive toolkit. The results show that the blockage concentration has the largest impact on object recognition, followed in order by the object type, blockage color, and object color. As for the blockage color, black exhibited better recognition performance than gray and yellow. In addition, changes in the blockage color affected the recognition of object types, resulting in different responses to each object. Through this study, we propose a blockage-based camera recognition performance evaluation method using simulation, and we establish an algorithm evaluation environment for various manufacturers through an interface with an actual camera. By suggesting the necessity and timing of future camera lens cleaning, we provide manufacturers with technical measures to improve the cleaning timing and camera safety.

Attitude Analysis and Evaluation of Camera Based on Planar Double Joint Robot

Planar double-jointed robot is a kind of widely used robot. In the past decades, with the development of science and technology, planar double-jointed robot has a great breakthrough in many aspects, especially in the actual camera and attitude analysis. In recent years, pun-section manipulator has made great progress in technology, but its kinematic characteristics and the human body’s ability to move have a great gap. So, in the moving plane, we need to study how to improve its camera ability to find a more stable and effective camera mode, and to a certain extent, to meet its requirements in complex working conditions. Simulating the pose of two joints, we seek a more stable and effective camera mode to improve its moving speed, expand its moving space, and lay a solid foundation for its application. Planar double-jointed robot is the most representative one at present. It includes machinery, material, electronics, computer, sensor, drive, intelligent control, artificial intelligence, and so on. Because it is closer to the human body, more suitable for people’s work and work, so it does not need to change its environment, so it is regarded as having a high affinity for people’s machinery. Its characteristic is that it has the good adaptability and the flexibility, but the double joint camera’s support discontinuity enables it to be able to adapt each kind of complex situation well. It can not only climb the stairs but also in the messy ground smooth through and also in the inconvenient passage through. The aim of this paper is to analyze the unique advantages and irreplaceable characteristics of the planar double-jointed robot based on the bionic and mechanical research results. This paper analyzes the mechanical technology by means of literature review and actual data analysis. The experimental results show that the biggest difference between the two articulated robots and other robots is that they have two human-like joint motions.

Lightweight and Effective Convolutional Neural Networks for Vehicle Viewpoint Estimation From Monocular Images

Vehicle viewpoint estimation from monocular images is a crucial component for autonomous driving vehicles and for fleet management applications. In this paper, we make several contributions to advance the state-of-the-art on this problem. We show the effectiveness of applying a smoothing filter to the output neurons of a Convolutional Neural Network (CNN) when estimating vehicle viewpoint. We point out the overlooked fact that, under the same viewpoint, the appearance of a vehicle is strongly influenced by its position in the image plane, which renders viewpoint estimation from appearance an ill-posed problem. We show how, by inserting in the model a CoordConv layer to provide the coordinates of the vehicle, we are able to solve such ambiguity and greatly increase performance. Finally, we introduce a new data augmentation technique that improves viewpoint estimation on vehicles that are closer to the camera or partially occluded. All these improvements let a lightweight CNN reach optimal results while keeping inference time low. An extensive evaluation on a viewpoint estimation benchmark (Pascal3D+) and on actual vehicle camera data (nuScenes) shows that our method significantly outperforms the state-of-the-art in vehicle viewpoint estimation, both in terms of accuracy and memory footprint.

One-Dimensional Plasma Model of Electrical Discharge Machining in Deionized Water for Prediction of Plasma Characteristics Along the Interelectrode Gap

Abstract Electrical discharge machining (EDM) is a nonconventional machining process that involves the formation of a plasma in an interelectrode gap filled with a dielectric that melts and vaporizes the electrodes (tool and workpiece) when a voltage is applied across them. This work models the plasma discharge in the EDM process to describe plasma characteristics such as electron density, heavy species densities, plasma potential, and plasma temperature using chemical kinetics, fluid flow, and heat transfer mechanisms in a 1D domain in the direction of the gap. The 1D domain allows the model to utilize surface reactions on the electrode walls necessary for sustaining the plasma reactions. The domain also provides a perspective of the plasma characteristics near the workpiece. Temperature results are compared with the experimental data obtained from spectrometer measurements. Additionally, an estimate of the plasma diameter is made and compared with actual high-speed camera images. The effect of EDM parameters such as supply voltage and interelectrode gap size on plasma characteristics is studied. The model predicts the incumbent heat flux on the workpiece electrode for small gaps which could have applications in the development of melt-pool models of EDM. Finally, the model provides a physics-based understanding of the mechanisms of plasma generation in the EDM process.

Applicability of Convolutional Neural Network for Estimation of Turbulent Diffusion Distance from Source Point

For locating the source of leaking gas in various engineering fields, several issues remain in the immediate estimation of the location of diffusion sources from limited observation data, because of the nonlinearity of turbulence. This study investigated the practical applicability of diffusion source-location prediction using a convolutional neural network (CNN) from leaking gas instantaneous distribution images captured by infrared cameras. We performed direct numerical simulation of a turbulent flow past a cylinder to provide training and test images, which are scalar concentration distribution fields integrated along the view direction, mimicking actual camera images. We discussed the effects of the direction in which the leaking gas flows into the camera’s view and the distance between the camera and the leaking gas on the accuracy of inference. A single learner created by all images provided an inference accuracy exceeding 85%, regardless of the inflow direction or the distance between the camera and the leaking gas within the trained range. This indicated that, with sufficient training images, a high-inference accuracy can be achieved, regardless of the direction of gas leakage or the distance between the camera and the leaking gas.

An Improved Frequency Domain Guided Thermal Imager Strips Removal Algorithm Based on LRSID.

The thermal imaging image of the Sustainable Development Science Satellite (SDGSAT-1) is mainly used for high-resolution observations of the ground width, due to the influence of blind elements and non-uniformity of the detector, and the system is a pendulum sweep imaging mode, resulting in fringed noise in the image. In this paper, a Fringing algorithm based on LRSID (low-rank-based single-image decomposition) algorithm is proposed, which can effectively remove the lateral and vertical fringe noise of the thermal imager and maintain the detail and clarity of the image. First, pretreatment of the obvious light and dark stripes then, based on LLSID algorithm, the vertical direction pinstripes and horizontal stripes are processed; finally, the fringed frequency band of the original image is replaced in the frequency domain with the image frequency domain processed by the LRSID algorithm, and then the Fourier inverse transformation is performed to obtain the final image. Using the method proposed in this paper, the simulated and actual SDGSAT-1 thermal imaging camera remote sensing stripes images are removed, and the visual and quantitative indicators are compared with the processing results of other algorithms, and the results show that the proposed algorithm has the best performance to remove the stripes, which can effectively remove horizontal and vertical fringes at the same time, and retain the detail and clarity of the image.

Technology of Image Motion Compensation for Different Velocity of Aerial Camera Based on Integration Level and Frequency Division Control

When an aircraft is flying sideways, the aerial camera will be in a squint state, and there will be multiple image motions of different sizes on the imaging target at the same time. In this paper, the principle of different velocity image motion in squint imaging of an aircraft is analysed, and the compensation strategy of different velocity image motion is proposed. Then, three methods are used to analyse the adjustment strategy of different velocity image motion compensation, namely, adjusting the integration level of (Time Delayed and Integration) TDICCD, grouping and frequency division control of TDICCD on the image surface, and optimizing the control based on the combination of integration level and frequency division control. Through the simulation analysis, it is concluded that the adjustment of the integration level of TDICCD and the slice grouping control is the optimal method. This method not only ensures the required image SNR of the optical system but also reduces the complex problems of electronic circuit and control system caused by the presence of too many segments and groups. In the actual aerial camera system imaging, this method can be used as an effective method for the compensation for the image motion of different velocity when the aerial camera tilts. Then, the design of the driving circuit and timing generator are described. The validity of the method is verified by indoor simulations and a flight test.

Key Clips and Key Frames Extraction of Videos Based on Deep Learning

The surveillance camera network covering the city, while protecting the safety of people’s lives and property, generate a large amount of surveillance video data every day, but few videos contain useful information. Surveillance cameras in sparsely crowded areas may capture most of the video in the background. A large number of surveillance videos bring greater storage pressure, and also increase the difficulty of the staff’s work. In this paper, we use the auto-encoder network to extract features of video frames. By comparing the feature differences between video frames, we can automatically select key video clips and key frame images that contain useful information to slim down the surveillance video. Through experiments on actual cameras, we found that this method can achieve the purpose of reducing the pressure of storage and traversal.

Predictive display for teleoperation with virtual reality fusion technology

AbstractA virtual reality (VR) fusion technology of matching a virtual environment of simulated graphics models with actual camera views of the slave station task environment has been developed. This fusion technology enables high‐fidelity predictive displays with calibrated graphics overlay on the live video. Reliable and accurate parameters are achieved by camera calibration and feature point detection. Since the object pose becomes known through the virtual model, the operator can use the teleoperation system effectively. Predictive displays are very useful for both noncontact and contact tasks, providing an effective VR interface with immediate visual prediction to the operator. Besides, the feature points detection avoids the exacerbation of maneuverability caused by outside interference. Lastly, the experiment of endoscopic penetration is conducted to validate the system.

Signal-Dependent Noise Estimation for a Real-Camera Model via Weight and Shape Constraints

Most computer vision algorithms require parameter adjustment according to the image noise level. Conventionally, the additive white Gaussian noise (AWGN) model is widely used in most noise estimation algorithms; however, this assumption does not hold in the real world where the noise from cameras is more complex, and it is more appropriate to assume the signal-dependent noise (SDN) model. In this paper, we focus on the SDN model while considering the nonlinear radiometric calibration inside an actual camera, and propose an algorithm to efficiently estimate the noise level function (NLF), which is defined as the noise standard deviation with respect to image intensity. First, the input image is divided into overlapping patches, and noise samples are estimated in the linear transform domain. The confidence levels of the noise samples and the prior of the camera response function are then employed as constraints for the recovery of the NLF. Finally, the noise samples and constraints are represented in a convex optimization problem. The experimental results using both real and synthetic noisy images demonstrate the superiority of the proposed method. In addition, the estimated NLFs are incorporated into two well-known denoising schemes, non-local means and BM3D, and shows significant improvements in denoising SDN-polluted images.

Modal identification of building structures using vision-based measurements from multiple interior surveillance cameras

Surveillance cameras are ubiquitous within modern building systems, providing a wealth of data that, to date, has not been utilized for structural health monitoring and condition assessment. In this paper, a vision-based vibration-monitoring technique is introduced, which leverages existing cameras within building structures to extract modal information and assess for damage. Cameras already installed within buildings, such as interior surveillance cameras, are used to measure structural responses (inter-story drifts and rotations) by tracking the cameras’ motion relative to the floor below. By synchronizing vision-based inter-story measurements captured at multiple stories, a complete three-dimensional representation of the building’s motion is recovered. Output-only system identification (frequency domain decomposition) is applied to these vision-based measurements to extract modal information—frequencies and mode shapes. An experimental parametric study on a lab-scale three-story structure is conducted to assess the efficacy of the vision-based monitoring technique for structural modal identification. The primary set of structure/camera geometries considered is representative of actual surveillance camera installations. Broadband white noise is used to excite the structure uni- and bidirectionally, and the resulting ambient and transient responses are measured. The accuracy of the modal information extracted by means of the proposed vision-based technique is assessed through comparison with modal data from accelerometer-based measurements, quantified by the modal assurance criteria. Moreover, the precision, robustness, and repeatability of the method as well as its shortcomings are established and are described in this paper.

Neural Network-Based Pose Estimation for Noncooperative Spacecraft Rendezvous

This article presents the Spacecraft Pose Network (SPN), the first neural network-based method for on-board estimation of the pose, i.e., the relative position and attitude, of a known noncooperative spacecraft using monocular vision. In contrast to other state-of-the-art pose estimation approaches for spaceborne applications, the SPN method does not require the formulation of hand-engineered features and only requires a single grayscale image to determine the pose of the spacecraft relative to the camera. The SPN method uses a convolutional neural network (CNN) with three branches to solve the problem of relative attitude estimation. The first branch of the CNN bootstraps a state-of-the-art object detection algorithm to detect a 2-D bounding box around the target spacecraft in the input image. The region inside the 2-D bounding box is then used by the other two branches of the CNN to determine the relative attitude by initially classifying the input region into discrete coarse attitude labels before regressing to a finer estimate. The SPN method then estimates the relative position by using the constraints imposed by the detected 2-D bounding box and the estimated relative attitude. Further, with the detection of 2-D bounding boxes of subcomponents of the target spacecraft, the SPN method is easily generalizable to estimate the pose of multiple target geometries. Finally, to facilitate integration with navigation filters and perform continuous pose tracking, the SPN method estimates the uncertainty associated with the estimated pose. The secondary contribution of this article is the generation of the Spacecraft PosE Estimation Dataset (SPEED), which is used to train and evaluate the performance of the SPN method. SPEED consists of synthetic as well as actual camera images of a mock-up of the Tango spacecraft from the PRISMA mission. The synthetic images are created by fusing OpenGL-based renderings of the spacecraft's 3-D model with actual images of the Earth captured by the Himawari-8 meteorological satellite. The actual camera images are created using a seven degrees-of-freedom robotic arm, which positions and orients a vision-based sensor with respect to a full-scale mock-up of the Tango spacecraft with submillimeter and submillidegree accuracy. The SPN method, trained only on synthetic images, produces degree-level relative attitude error and cm-level relative position errors when evaluated on the actual camera images with a different distribution not used during training.

CMWMF: Constant Memory Architecture of Weighted Mode/Median Filter for Extremely Large Label Depth Refinement

In this study, we propose a constant memory hardware architecture that can support weighted mode, median, and joint bilateral filters, which is referred to as CMWMF. This work aims to meet the high memory and computation requirements of processing depth maps with a large number of depth candidates. In the proposed architecture, we leverage the geometry smoothing characteristic of natural images to reduce the static random access memory (SRAM) size for hardware implementation. The architecture preserves a constant number of disparity values instead of depending on the label count and size of the local supporting window. A novel weighted median search procedure is proposed, which assigns a computation to each input cycle, thereby rendering the process hardware friendly. An index-checking technique is proposed to process out-of-order joint histograms. We adopted the above-mentioned techniques in our architecture as they consume a constant SRAM size and supports multiple types of filters. As a result, this architecture reduces the SRAM size by 92.4% with a negligible decrease in performance. According to our analysis on the KITTI, and Middlebury datasets, and with actual depth cameras, the preserved information is sufficient. The proposed architecture is one of the most suitable depth refinement architectures for scenarios having a large number of depth candidates.

Salient View Selection for Visual Recognition of Industrial Components

We introduce a new method to find a salient viewpoint with a deep representation, according to ease of semantic segmentation. The main idea in our segmentation network is to utilize the multipath network with informative two views. In order to collect training samples, we assume all the information of designed components and even error tolerances are available. Before installing the actual camera layout, we simulate different model descriptions in a physically correct way and determine the best viewing parameters to retrieve a correct instance model from an established database. By selecting the salient viewpoint, we better understand fine-grained shape variations with specular materials. From the fixed top-view, our system initially predicts a 3-DoF pose of a testing object in a data-driven way, and precisely align the model with a refined semantic mask. Under various conditions of our system setup, the presented method is experimentally validated. A robotic assembly task with our vision solution is also successfully demonstrated.

Using the dead leaves pattern for more than spatial frequency response measurements

Simulation is an established tool to develop and validate camera systems. The goal of autonomous driving is pushing simulation into a more important and fundamental role for safety, validation and coverage of billions of miles. Realistic camera models are moving more and more into focus, as simulations need to be more then photo-realistic, they need to be physical-realistic, representing the actual camera system onboard the self-driving vehicle in all relevant physical aspects – and this is not only true for cameras, but also for radar and lidar. But when the camera simulations are becoming more and more realistic, how is this realism tested? Actual, physical camera samples are tested in laboratories following norms like ISO12233, EMVA1288 or the developing P2020, with test charts like dead leaves, slanted edge or OECF-charts. In this article we propose to validate the realism of camera simulations by simulating the physical test bench setup, and then comparing the synthetical simulation result with physical results from the real-world test bench using the established normative metrics and KPIs. While this procedure is used sporadically in industrial settings we are not aware of a rigorous presentation of these ideas in the context of realistic camera models for autonomous driving. After the description of the process we give concrete examples for several different measurement setups using MTF and SFR, and show how these can be used to characterize the quality of different camera models.

Mirror binocular calibration method based on sole principal point

In three-dimensional vision measurements, single-camera mirror binocular stereo vision systems composed of a camera and mirrors have become popular, owing to their high flexibility and compactness. However, the calibration of these systems is different from traditional two-camera stereo vision systems, because there is no rigid-body transformation between the actual camera and the virtual camera. A calibration method for single-camera mirror binocular vision system is proposed, which can establish the rigid-body transformation by rotating the image based on the sole principal point. The comparative experiments showed that the proposed method yielded a higher accuracy than the traditional method.