Actual Camera Research Articles

Improved Image-Based Localization Using SFM and Modified Coordinate System Transfer

Accurate localization of mobile devices based on camera-acquired visual media information usually requires a search over a very large GPS-referenced image database collected from social sharing websites like Flickr or services such as Google Street View. This paper proposes a new method for reliable estimation of the actual query camera location by optimally utilizing structure from motion (SFM) for three-dimensional (3-D) camera position reconstruction, and introducing a new approach for applying a linear transformation between two different 3-D Cartesian coordinate systems. Since the success of SFM hinges on effectively selecting among the multiple retrieved images, we propose an optimization framework to do this using the criterion of the highest intraclass similarity among images returned from retrieval pipeline to increase SFM convergence rate. The selected images along with the query are then used to reconstruct a 3-D scene and find the relative camera positions by employing SFM. In the last processing step, an effective camera coordinate transformation algorithm is introduced to estimate the query's geo-tag. The influence of the number of images involved in SFM on the ultimate position error is investigated by examining the use of three and four dataset images with different solution for calculating the query world coordinates. We have evaluated our proposed method on query images with known accurate ground truth. Experimental results are presented to demonstrate that our method outperforms other reported methods in terms of average error.

An Indoor Location-Based Positioning System Using Stereo Vision with the Drone Camera

Stereo vision is used to reconstruct 3D information of the space by estimating the depth value from the simulation of human eyes. Spatial restoration can be used as a means of location estimation in an indoor area, which is impossible to accomplish using the relative location estimation technology, GPS. By mapping the real world in virtual space, it is feasible to clear the boundary between real space and virtual space. This paper presents a method to control the drone indoors through a positioning system using Structure from Motion algorithm (SfM). SfM calculates the relative relationship between cameras based on images to be acquired from various locations and obtains disparity to enable restoration of 3D space. First, the 3D virtual space is reconstructed using several photographs taken from an indoor environment. Second, the real-time drone position is determined by comparing the 3D virtual space camera with the image displayed on the drone camera. In this case, if the direction of the virtual camera used for 3D virtual space construction is the same as the amount of yaw rotation of the drone, it is possible to quickly find the same position as the image seen in the real drone camera in the virtual space. As a result, if the scale of the actual camera image and the virtual camera image is 1 : 1 matched, then it is possible to know that the drone is in the position of the virtual camera. The proposed indoor location-based drone controlling method can be applied to various drone applications such as group flight in an indoor environment because of its ability to fly the drone without the use of the traditional remote-control and flight trajectory programming.

Aerial Wide-Area Motion Imagery Registration Using Automated Multiscale Feature Selection

Automatic registration of aerial wide-area motion imagery is required to correct the camera parameters in order to eliminate the geocoding errors arising from frequent reinstallation of the camera array on the carrier aircraft. Approaches developed to date solely rely on the information present in the imagery not using a priori knowledge about the environment and the features present in it in the sequence analysis. To this end, we propose a novel method based on dynamic feature extraction and automatic multiscale feature matching to produce per-frame camera pose corrections. The features are extracted from the imagery using one of the three dedicated classifiers and are then robustly matched to the features projected from the datum using a coarse-to-fine multiscale approach. Finally, the bias between the estimated and the actual camera pose is estimated using ordinary least squares optimization performed on the distances between the approved match candidate pairs. The application of the proposed method to 50 frames of very high-resolution aerial imagery captured over mixed terrain at an altitude of 5.3 km demonstrates significant reduction in position error of the features (from 47.76 to 12.31 m) and proves the attractiveness of the approach as an alternative to manual labeling methods using ground control points.

Computational Vision for Automatic Tracking and Objective Estimation of Mobile Robot Trajectory

Automatic tracking and evaluation of moving-object trajectories is critical in many applications such as performance estimation of mobile robot navigation. Mobile robot is an effective platform for stimulating student motivation at K-12 institutions as well as a good tool for rigorous engineering practices in colleges, universities, and graduate schools. Developing new mobile robot platforms and algorithms requires objective estimation of navigation performance in a quantitative manner. Conventional methods to estimate mobile robot navigation typically rely on manual usage of chronometer to measure the time spent for the completion of a given task or counting the success rate on the task. This paper proposes an alternative; a multi-camera vision system that can automatically track the movement of mobile robot and estimate it in terms of physics-based profiles: position, velocity, and acceleration of the robot in the trajectory with respect to a user-defined world-coordinate system. The proposed vision system runs two synchronized cameras to simultaneously capture and track the movement of the robot at 30 frames per second. The system runs a homography-based projection algorithm that converts the view-dependent appearance of the robot in the camera images to a view-independent orthographic projection mapped on the registered world coordinate system. This enables the human evaluator to view and estimate the robot navigation from a virtual top-down view embedded with the physics-based profiles regardless of the actual cameras’ viewing positions. The proposed system can also be used for other domains including highway traffic monitoring and intelligent video surveillance.

Video image target monitoring based on RNN-LSTM

Traditional image object classification and detection algorithms and strategies cannot meet the problem of video image acquisition and processing. Deep learning deliberately simulates the hierarchical structure of human brain, and establishes the mapping from low-level signals to high-level semantics, so as to achieve hierarchical feature representation of data. Deep learning technology has powerful visual information processing ability, which has become the forefront technology and domestic and international research hotspots to deal with this challenge. In order to solve the problem of target space location in video surveillance system, time-consuming and other problems, in this paper, we propose the algorithm based on RNN-LSTM deep learning. At the same time, according to the principle of OpenGL perspective imaging and photogrammetry consistency, we use 3D scene simulation imaging technology, relying on the corresponding relationship between video images and simulation images we locate the target object. In the 3D virtual scene, we set up the virtual camera to simulate the imaging processing of the actual camera, and the pixel coordinates in the video image of the surveillance target are substituted into the simulation image, next, the spatial coordinates of the target are inverted by the inverse process of the virtual imaging. The experimental results show that the detection of target objects has high accuracy, which has an important reference value for outdoor target localization through video surveillance images.

Assessing the Crop-Water Status in Almond (Prunus dulcis Mill.) Trees via Thermal Imaging Camera Connected to Smartphone.

Different tools are being implemented in order to improve the water management in agricultural irrigated areas of semiarid environments. Thermography has been progressively introduced as a promising technique for irrigation scheduling and the assessing of crop-water status, especially when deficit irrigation is being implemented. However, an important limitation is related to the cost of the actual cameras, this being a severe limitation to its practical usage by farmers and technicians. This work evaluates the potential and the robustness of a thermal imaging camera that is connected to smartphone (Flir One) recently developed by Flir Systems Inc. as a first step to assess the crop water status. The trial was developed in mature almond (Prunus dulcis Mill.) trees that are subjected to different irrigation treatments. Thermal information obtained by the Flir One camera was deal with the thermal information obtained with a conventional Thermal Camera (Flir SC660) with a high resolution, and subsequently, confronted with other related plant physiological parameters (leaf water potential, Ψleaf, and stomatal conductance, gs). Thermal imaging camera connected to smartphone provided useful information in estimating the crop-water status in almond trees, being a potential promising tool to accelerate the monitoring process and thereby enhance water-stress management of almond orchards.

Influence of Camera Rotation on Stereo-DIC and Compensation Methods

Camera rotation during stereo digital image correlation (DIC) measurements is always present in some amount and is caused by environmental vibrations during testing (e.g. blast testing, testing in industrial environments, vibration testing Helffrick et al., Mech Syst Signal Process 25(3):917–927, 2011, etc.). The impact of changing extrinsic parameters on stereo-DIC measurements was investigated in this paper by simulating several DIC experiments with a shaking camera system. To obtain an appropriate order of magnitude of the camera motion, the actual camera rotation during a test was observed and reported. The impact hereof was investigated by analyzing static images and by simulating camera rotation with a stereo-DIC simulator presented in Balcaen et al. (Exp Mech 57(5):703–718, 2017). Insight on this underestimated error source is explained by investigating which rotations are more critical to the resulting displacements and strains. Two possible compensation methods were investigated and their capability of compensating camera motion was evaluated.

Development and Tests of a 3D Fish-Tracking Videometry System for an Experimental Flume

To design effective and efficient fish passage facilities at hydropower plants, the knowledge of swim behaviour of fish is essential. Therefore, living wild fish were investigated at different fish guidance structures in an experimental flume in a test section of 11 m length and 2.5 m width at water depths of about 0.6 m. Besides analysis of time data and manual recordings of the fish behaviour, video recordings of the fish movements can allow more detailed analysis of fish behaviour in different hydraulic situations. Thus, a videometry system was installed consisting of eleven synchronous cameras with overlapping views lined-up under dry conditions outside the flume. A 3D tracking algorithm was developed and implemented to analyse the video data. Core of the code is a motion-based multiple object tracking method, in which several objects can be tracked in 2D pixel-frame coordinates at the same time. After undistorting and stereo-calibrating the cameras, the 2D tracks are transferred to a 3D metric-space according to their epipolar geometry. Within this paper video data from a single experimental run of 15 min with three fishes with lengths of 100–150 mm are analysed exemplarily. The path-time diagram gives a distinct ‘big picture’ of the fish movement, which helps to identify preferred and disliked regions. However, due to imperfect actual camera setup, a 3D view in the near field of the cameras and an automated separation of individual tracks in a group of fish remains challenging.

Blood on Boylston: Digital Memory and the Dramatisation of Recent History in <em>Patriots Day</em>

Blood on Boylston: Digital Memory and the Dramatisation of Recent History in <em>Patriots Day</em>

Characterization of Plenoptic Imaging Systems and Efficient Volumetric Estimation From Plenoptic Data

Light-field cameras in conjunction with computational refocusing can be used to produce volumetric estimates of an imaged scene. However, these estimates are often dominated by image blur in the depth direction from objects not in each synthesized focal plane. Tomographic algorithms have been shown to be effective in creating volumetric estimates from plenoptic data but are often prohibitively slow. Deconvolution would be an attractive solution due to processing speed, but existing image synthesis equations are shift-variant. This research proposes an alternate refocusing transformation that makes the core problem described in continuous coordinates shift-invariant so that deconvolution is a viable solution. Shift-invariance of the new refocusing transform is demonstrated mathematically. Furthermore, the discretization involved in the imaging system and refocusing algorithm are characterized with respect to shift-variance in order to identify potential sources of artifacts and to propose potential mitigating steps where possible. While the sampled light field is not directly invertible, experimental data are used to demonstrate that regularized deconvolution using the derived synthesis equations produces improved results compared to the base focal stack in both synthetic examples and actual camera data.

Investigation of 3D Reconstruction from Time-series Images by Towing Camera

This study shows the investigation of three dimensional (3D) reconstruction using a towing camera system in order to collect the information inside the nuclear plant for reactor decommissioning. The towing camera obtains time-series inside the nuclear plant by moving through a cable. The camera can move by pulling the cable. However, in this case, passive rotation affects the 3D reconstruction. In this study, in order to investigate effect of the passive rotation, the experiments are carried out for 3D reconstruction with actual camera at a mockup. In order to investigate influence of passive rotation, we prepare two types of camera. One is towing passive joint camera which is influenced by passive rotations and the other is towing fix joint camera which is not influenced by them. We compare 3D reconstruction from time-series images by these two cameras. A square log is adopted as a target for 3D reconstruction in mockup. Experimental results show that time-series images by a passive joint camera reconstruct the target more accurate and the floor more flatly in comparison to a fixed joint camera. Additionally, we verify that it is difficult to reconstruct the points with respect to the vertical height when towing camera is set to downward.

Modeling and Simulation of a Vision-Based Autonomous Vehicle

This paper presents a comprehensive mathematical simulation for the trajectory of a vision-based autonomous vehicle during moving between lane lines of the structured road. The simulation accomplished by using MATLAB/Simulink software. This simulation mimics the existence of an actual digital camera by using a novel 3D-vision block to simulate theactual images that assumed to be provided by a digital camera connected to an embedded computer. The 3D-vision block uses mathematical equations, execution sequence and logical conditions to create a virtual captured image. So, this virtual image is then used to detect the lane in the front of the vehicle depending on the virtual camera position and its parameters. Inside simulation environment that based on the kinematic model of the vehicle and vision model, the controller has been designed that will be coded in the embedded computer. Several evaluations are shown and discussed about the lane detection and 3D vision simulation.

Fast J-linkage algorithm for camera orientation applications

A computationally efficient approach to obtain vanishing points in images for camera orientation applications is presented in this paper. Modifications are made to the conventional J-linkage algorithm, named Fast J-linkage, so that the detection of vanishing points is achieved in a computationally efficient manner. This fast algorithm is then utilized to recover camera orientation or rotation matrices from video sequences. Both the computational efficiency and performance aspect of the Fast J-linkage algorithm are compared with the conventional J-linkage algorithm by examining the York Urban image database. In addition, the results of an actual camera rotation implementation is provided and validated by using inertial sensors data. The results obtained indicate the higher computational efficiency and accuracy aspects of the developed vanishing point detection.

Fast Orientation of Video Images of Buildings Acquired from a UAV without Stabilization.

The aim of this research was to assess the possibility of conducting an absolute orientation procedure for video imagery, in which the external orientation for the first image was typical for aerial photogrammetry whereas the external orientation of the second was typical for terrestrial photogrammetry. Starting from the collinearity equations, assuming that the camera tilt angle is equal to 90°, a simplified mathematical model is proposed. The proposed method can be used to determine the X, Y, Z coordinates of points based on a set of collinearity equations of a pair of images. The use of simplified collinearity equations can considerably shorten the processing tine of image data from Unmanned Aerial Vehicles (UAVs), especially in low cost systems. The conducted experiments have shown that it is possible to carry out a complete photogrammetric project of an architectural structure using a camera tilted 85°–90° ( or ω) and simplified collinearity equations. It is also concluded that there is a correlation between the speed of the UAV and the discrepancy between the established and actual camera tilt angles.

카메라 위치를 반영한 후진 주차 가이드라인 생성 연구

자동차 운행에 있어서 최종적인 단계는 주차로 마무리 되는데, 특히 운전을 처음 배우는 사람들이 가장 어려워하며, 협소한 주차공간에서의 주차는 초보자뿐만 아니라 일반 운전자도 편하지는 않다. 이러한 문제점을 해결하기 위해 주차 조향 보조 시스템, 자동 항속 제어 시스템 등이 개발되어 운전자의 편의성을 향상시키고 있고, 초음파, 카메라, 열화상 카메라 또는 레이더 등을 이용하여 운전자에게 주변 환경을 보다 정확하게 인지할 수 있도록 주차 보조 시스템이 개발되고 있다. 본 연구에서는 카메라 위치가 차량 후면 어느 곳에 위치하든 중앙부에 위치했을 때와 동일하게 영상을 처리할 수 있도록 후진 회전 반경 식을 제안하고, 실차 검증을 통해 주차 가이드라인을 생성하고자 한다. The final step of driving a car is parking, which is the most difficult part for people learning to drive. Parking in narrow parking spaces is difficult for both ordinary drivers and beginners. To solve this problem, the development of SPAS (Smart Parking Assist System), ACC (Automatic Control System) improves the convenience of drivers. In addition, parking assistance systems have been developed to recognize more accurately the surrounding environment to the driver using the ultrasound, camera, thermal camera, and radar. This paper proposes the reverse turning radius to process images as if the camera is located in the center of the vehicle regardless of the actual camera position. In addition, it generates the parking guidelines through verification using the vehicle.

Yawning Detection Using Embedded Smart Cameras

Yawning detection has a variety of important applications in a driver fatigue detection, well-being assessment of humans, driving behavior monitoring, operator attentiveness detection, and understanding the intentions of a person with a tongue disability. In all of the above applications, an automatic detection of yawning is one important system component. In this paper, we design and implement such automatic system, using computer vision, which runs on a computationally limited embedded smart camera platform to detect yawning. We use a significantly modified implementation of the Viola-Jones algorithm for face and mouth detections and, then, use a backprojection theory for measuring both the rate and the amount of the changes in the mouth, in order to detect yawning. As proof-of-concept, we have also implemented and tested our system on top of an actual smart camera embedded platform, called APEX from CogniVue Corporation. In our design and implementations, we took into consideration the practical aspects that many existing works ignore, such as real-time requirements of the system, as well as the limited processing power, memory, and computing capabilities of the embedded platform. Comparisons with existing methods show significant improvements in the correct yawning detection rate obtained by our proposed method.

Demosaicing Method for Digital Cameras with White-RGB Color Filter Array

Demosaicing, or color filter array (CFA) interpolation, estimates missing color channels of raw mosaiced images from a CFA to reproduce full-color images. It is an essential process for single-sensor digital cameras with CFAs. In this paper, a new demosaicing method for digital cameras with Bayer-like W-RGB CFAs is proposed. To preserve the edge structure when reproducing full-color images, we propose an edge direction–adaptive method using color difference estimation between different channels, which can be applied to practical digital camera use. To evaluate the performance of the proposed method in terms of CPSNR, FSIM, and S-CIELAB color distance measures, we perform simulations on sets of mosaiced images captured by an actual prototype digital camera with a Bayer-like W-RGB CFA. The simulation results show that the proposed method demosaics better than a conventional one by approximately +22.4% CPSNR, +0.9% FSIM, and +36.7% S-CIELAB distance.

Reference data set for camera spectral sensitivity estimation.

In this article, we describe a spectral sensitivity measurement procedure at the National Physical Laboratory, London, with the aim of obtaining ground truth spectral sensitivity functions for Nikon D5100 and Sigma SD1 Merill cameras. The novelty of our data is that the potential measurement errors are estimated at each wavelength. We determine how well the measured spectral sensitivity functions represent the actual camera sensitivity functions (as a function of wavelength). The second contribution of this paper is to test the performance of various leading sensor estimation techniques implemented from the literature using measured and synthetic data and also evaluate them based on ground truth data for the two cameras. We conclude that the estimation techniques tested are not sufficiently accurate when compared with our measured ground truth data and that there remains significant scope to improve estimation algorithms for spectral estimation. To help in this endeavor, we will make all our data available online for the community.

Multi-Camera View Synthesis Based on Depth Image Layer Separation (DILS) Algorithm

A multi-camera view synthesis known as Multi-Level View Synthesis (MLVS) approach based on Depth Image Layer Separation (DILS) algorithm for 3D vision and free-viewpoint video application is presented. The main aim of this technique is to create multi-camera view system through reduced number of actual image acquisition cameras, whilst maintaining the quality of the virtual view synthesis images. DILS separate the disparity depth map into several layers based on location and depth. Based on experimental results, the multi-camera of interview image synthesis provides good results compared with the conventional inter-view interpolation method. The proposed technique is shown to provide additional views with fewer cameras and lower complexity compared to conventional high volume camera configurations for free-viewpoint acquisition and light field imaging applications. Therefore, the multi-camera view synthesis can reduce the cost required.

Practical signal-dependent noise parameter estimation from a single noisy image.

The additive white Gaussian noise is widely assumed in many image processing algorithms. However, in the real world, the noise from actual cameras is better modeled as signal-dependent noise (SDN). In this paper, we focus on the SDN model and propose an algorithm to automatically estimate its parameters from a single noisy image. The proposed algorithm identifies the noise level function of signal-dependent noise assuming the generalized signal-dependent noise model and is also applicable to the Poisson-Gaussian noise model. The accuracy is achieved by improved estimation of local mean and local noise variance from the selected low-rank patches. We evaluate the proposed algorithm with both synthetic and real noisy images. Experiments demonstrate that the proposed estimation algorithm outperforms the state-of-the-art methods.