Camera Array System Research Articles

Infrared Camera Array System and Self-Calibration Method for Enhanced Dim Target Perception

Infrared Camera Array System and Self-Calibration Method for Enhanced Dim Target Perception

Polarizing Camera Array System Equipment and Calibration Method

Polarizing Camera Array System Equipment and Calibration Method

A novel audio-visual based beamformer using a 360° camera and planar microphone array system

Beamforming systems that rely only on microphone arrays may suffer from poor performance in complex acoustic environments with multiple sources and room reverberation. To overcome this limitation, we propose a novel method that combines a planar microphone array and a 360° camera to steer the beamformer. The camera provides visual information about the location and orientation of sound sources, which can be used to adjust the beamformer parameters. We will present results that compare the signal-to-noise ratio (SNR) of the proposed method with a conventional beamforming microphone-based method in a simulated room with different source positions, orientations, and noise levels to investigate the benefit of visual information for beamforming in different environments. The proposed method has the potential to improve the performance of beamforming systems in applications such as speech enhancement, speech recognition, and speaker localization.

Computational imaging and occluded objects perception method based on polarization camera array.

Traditional optical imaging relies on light intensity information from light reflected or transmitted by an object, while polarization imaging utilizes polarization information of light. Camera array imaging is a potent computational imaging technique that enables computational imaging at any depth. However, conventional imaging methods mainly focus on removing occlusions in the foreground and targeting, with limited attention to imaging and analyzing polarization characteristics at specific depths. Conventional camera arrays cannot be used for polarization layered computational imaging. Thus, to study polarization layered imaging at various depths, we devised a flexible polarization camera array system and proposed a depth-parallax relationship model to achieve computational imaging of polarization arrays and polarization information reconstruction under varying conditions and depths. A series of experiments were conducted under diverse occlusion environments. We analyzed the distinctive characteristics of the imaging results obtained from the polarization array, employing a range of array distribution methods, materials, occlusion density, and depths. Our research successfully achieved computational imaging that incorporates a layered perception of objects. Finally, we evaluated the object region's polarization information using the gray level co-occurrence matrix feature method.

Computational Super-Resolution Imaging With a Sparse Rotational Camera Array

A sparse camera array utilizes several identical cameras for multi-frame super-resolution. However, the influence of camera layout on super-resolution performance remains unclear. In conventional camera arrays, all cameras share a similar observation model, resulting in redundancy during information collection. This study presents a novel super-resolution framework that actively controls cameras in a sparse camera array to rotate to certain degrees. Through a detailed analysis of the forward imaging model, it is shown that the rotation of cameras uses the rotational asymmetry property of pixel layouts and shape. The proposed camera layout method provides more diverse and informative spatial sampling than a conventional camera array with only translations between cameras. A physics-enabled deep neural network was introduced for multi-frame super-resolution. With the explicitly defined gradients and affine models as guidance, it can effectively decode additional information collected by the rotational camera array and overcome the large-scale spatial feature alignment problem. The experimental results on synthetic and real prototype datasets validated the proposed theory and solution. We believe our study can inspire innovations for future task-specific camera array system design.

Quality Assessment of 3D Point Clouds on the Different Surface Materials Generated from iPhone LiDAR Sensor

Terrestrial Laser Scanning (TLS) is a powerful device that can capture 3D reality faster than ever before. For more industry nowadays, this equipment assists a lot in analyzing issues from artifacts, structures, buildings, and landscapes at required accuracy and precision depending on the applications. Nonetheless, the costs and complexity of TLS in many fields remain high. With the invention of the iPhone 12 Pro’s new Light Detection and Ranging (LiDAR) sensor, and the increased capability of the iPhone’s camera array system, the generation of 3D point clouds using this sensor led to several studies on their capability and resulting accuracy for some applications in Architecture, Engineering and Construction (AEC) industries, especially in planning and decision-making purposes towards 3D reconstructed model. Generally, this 3D model is incorporated from different surface materials which have different results when scanning via laser scanning platform. Therefore, several tests were conducted on the iPhone 12 Pro at different surface materials (e.g., plywood, aluminium, canvas, plastic board, and ceramic tile), where the density of 3D point clouds and accuracy of distance measurements generated by iPhone 12 Pro were used as the parameters of quality assessment while comparison to TLS served as reference data. The 3D scanning sessions on different object materials were carried out in a small classroom, approximately 100m2 where each object has its specific dimension and being placed side by side with each other. According to the test outcomes, the LiDAR sensor of iPhone 12 Pro was able to generate a good density of 3D point clouds which produced proximate value of actual object dimensions using meshing process in CloudCompare Software. Therefore, it can be seen that the output of 3D model generated from iPhone LiDAR Sensor is sufficed to replicate the 3D indoor building environment at small coverage area.

An efficient coral survey method based on a large-scale 3-D structure model obtained by Speedy Sea Scanner and U-Net segmentation

Over the last 3 decades, a large portion of coral cover has been lost around the globe. This significant decline necessitates a rapid assessment of coral reef health to enable more effective management. In this paper, we propose an efficient method for coral cover estimation and demonstrate its viability. A large-scale 3-D structure model, with resolutions in the x, y and z planes of 0.01 m, was successfully generated by means of a towed optical camera array system (Speedy Sea Scanner). The survey efficiency attained was 12,146 m2/h. In addition, we propose a segmentation method utilizing U-Net architecture and estimate coral coverage using a large-scale 2-D image. The U-Net-based segmentation method has shown higher accuracy than pixelwise CNN modeling. Moreover, the computational cost of a U-Net-based method is much lower than that of a pixelwise CNN-based one. We believe that an array of these survey tools can contribute to the rapid assessment of coral reefs.

A camera array system based on DSLR cameras for autostereoscopic prints

In this paper, we present a multi-view camera-array system by using commercial DSLR cameras. In order to produce quality autostereoscopic prints, we initially calibrated the color based on X-Rite color-checker, then automatically adjust the alignment of all images by one black-white checker. In this system, we also utilized an external electronic trigger based on Arduino for synchronizing all cameras to generate bullet-time effect photos. Finally, we converted all photos into multiplexed images then printed them on a lenticular lens panel to be an autostereoscopic photo frame.

High-precision and flexible calibration method of non-overlapping array cameras

In this paper, a high-precision and flexible intrinsic and extrinsic parameters calibration method for non-overlapping array cameras is proposed. In the camera array system, each camera employs an individual planar calibration board to achieve intrinsic parameters calibration. Without assistance of any other coded or un-coded target, the extrinsic parameters of array cameras can be flexibly calibrated based on close-range photogrammetry. To overcome the disadvantages of low precision under a large field of view in close-range photogrammetry, a refinement coordinates mapping (RCM) method to achieve high-precision reconstruction of targets is developed. The principles of single-camera calibration, close-range photogrammetry, and the extrinsic calibration method are introduced, respectively. Experiments are performed to analyze the stability of single-camera calibration and the world coordinates reconstruction of targets. The results of the re-projection error in extrinsic parameters calibration show that the developed RCM method has high precision.

Development of an Efficient Coral-Coverage Estimation Method Using a Towed Optical Camera Array System [Speedy Sea Scanner (SSS)] and Deep-Learning-Based Segmentation: A Sea Trial at the Kujuku-Shima Islands

Various methods have been developed and used for monitoring marine benthic habitats, such as coral reefs and seagrass meadows. However, the efficiency of general survey methods [e.g., line intercept transects and autonomous underwater vehicles (AUVs)] still is not high. In this article, we propose a practical coral-coverage estimation method combining an effective survey system [Speedy Sea Scanner (SSS)] and a deep-learning-based estimation method. The SSS is a towed-type system with six cameras arrayed on the platform. The depth rating of the system in our trial was 50 m. The length of the array baseline was 4.4 m, and six cameras were placed on the platform with equal spacing. The sea trial was conducted at Kujuku-Shima, Japan, on September 30, 2017. We successfully generated 3-D models and high-quality orthophotos of the seafloor with high resolution of about 1.5 mm/pixel. The survey efficiency of the SSS was about 7000 m <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> /h. In addition, the experimental results of coral-coverage estimation showed that the corals can be distinguished with accuracy of about 80% in places with relatively high transparency, and the error of coverage estimation was 10% or less. The proposed coral-coverage estimation method is more efficient than other survey techniques and costs less than AUV surveying; therefore, it is expected to become a promising tool for marine environmental surveying.

A Comparison of Plenoptic Imaging System and Camera Array System

A Comparison of Plenoptic Imaging System and Camera Array System

A Comparison of Plenoptic Imaging System and Camera Array System

A Comparison of Plenoptic Imaging System and Camera Array System

Research on Depth Estimation Method of Light Field Imaging Based on Big Data in Internet of Things From Camera Array

In recent years, optical field imaging technology has received extensive attention in the academic circle for its novel imaging characteristics of shooting first and focusing later, variable depth of field, variable viewpoint, and so on. However, the existing optical field acquisition equipment can only acquire a limited number of discrete angle signals, so image aliasing caused by under sampling of optical field angle signals reduces the quality of optical field images. Based on the camera array system as a platform, this paper studies the optical field imaging and depth estimation method based on the Big Data in Internet of Things obtained from camera array around the angle sampling characteristics of the optical field data set, and has achieved some innovative research results in the following aspects. On the basis of analyzing the characteristics of different depth clues in the optical field data set, a depth estimation method combining parallax method and focusing method is proposed. First, this paper analyzes the disparity clues and focus clues contained in the multi-view data set and the light field refocusing image set of the camera array, respectively, and points out the differences and relationships between the two depth clues extraction methods in the light field sampling frequency domain space, that is, the disparity method focuses on the energy concentration characteristics near the frequency domain spatial angle axis, while the focus method focuses on the high frequency proportion of energy distribution on the angle axis. Then, the weighted linear fusion method based on image gradient is used to fuse the two calculation results, which improves the accuracy and robustness of depth estimation. Finally, the results of depth estimation experiments on different sets of scenes show that compared with the method based on a single depth cue, the method in this paper has higher accuracy in depth calculation in discontinuous areas of scene depth and similar texture areas.

A Ground-Based Near Infrared Camera Array System for UAV Auto-Landing in GPS-Denied Environment.

This paper proposes a novel infrared camera array guidance system with capability to track and provide real time position and speed of a fixed-wing Unmanned air vehicle (UAV) during a landing process. The system mainly include three novel parts: (1) Infrared camera array and near infrared laser lamp based cooperative long range optical imaging module; (2) Large scale outdoor camera array calibration module; and (3) Laser marker detection and 3D tracking module. Extensive automatic landing experiments with fixed-wing flight demonstrate that our infrared camera array system has the unique ability to guide the UAV landing safely and accurately in real time. Moreover, the measurement and control distance of our system is more than 1000 m. The experimental results also demonstrate that our system can be used for UAV automatic accurate landing in Global Position System (GPS)-denied environments.

Synthetic aperture photography using a moving camera-IMU system

Occlusion poses as a critical challenge in computer vision for a long time. Recently, the technique of synthetic aperture photography using a camera array has been regarded as a promising way to address the problem of occluded object imaging. Unfortunately, the expensive cost of a standard camera array system with the required calibration procedure still limits the widespread popularity of this technique. To make synthetic aperture photography practical, in this paper, we propose a novel approach based on a mobile camera-IMU system moving on a planar surface. The contributions of our work include: (1) with geometric property assumptions, we mathematically prove that the transformation matrices for synthetic aperture image generation can be factorized into an infinite homography and a shift; (2) the IMU data are utilized to estimate the homography by selecting the plane at infinity as the reference plane, which is scene independent and does not require any calibration; (3) compared to the conventional camera array based approaches, the proposed method effectively improves the solution for synthetic aperture images capturing and generation by using a more flexible, convenient and inexpensive system. Extensive experimental results with qualitative and quantitative evaluations have demonstrated the see-through-occlusion performance with satisfactory accuracy of the proposed approach in different scenarios.

Using Marker-less Motion Capture Systems for Walk Path Analysis in Paced Assembly Flow Lines

In recent years, automotive industry is facing a turbulent environment with an increasing demand for mass-customization and shortened product life-cycles. For manual assembly, this trend has led to a rising planning complexity, since growing numbers of product variants are hitting mixed-model assembly lines. In this context, it is crucial for production planning to be aware of the actual state of an assembly line in order to identify inconsistencies between the situation in company-owned learning factories and the shop floor, especially when considering non-value-adding tasks (e.g. walk paths). However, a feedback loop for walk paths linking the assembly line with the planning department is not established in practice. Consequently, discrepancies between planned and real processes remain largely unknown since they only become apparent through production disruptions. In order to provide production planning with an objective tool for walk path assessment, this work proposes a novel tracking approach, being able to reconstruct operators’ motion within an assembly line. Based on a distributed depth camera array, a scalable and marker-less tracking system is presented that can be applied in productive environments. An in-depth evaluation underlines the performance of this novel approach and assesses the overall path accuracy. Finally, the proposed system is set up in an automotive final assembly line during operation. The gathered data is investigated regarding planning inconsistencies during operation.

A Surround Video Capture and Presentation System for Preservation of Eye-Gaze in Teleconferencing Applications

We propose a new video conferencing system that uses an array of cameras to capture a remote user and then show the video of that person on a spherical display. This telepresence system has two key advantages: (i) it can capture a near-correct image for any potential observer viewing direction because the cameras surround the user horizontally; and (ii) with view-dependent graphical representation on the spherical display, it is possible to tell where the remote user is looking from any viewpoint, whereas flat displays are visible only from the front. As a result, the display can more faithfully represent the gaze of the remote user. We evaluate this system by measuring the ability of observers to accurately judge which targets the actor is gazing at in two experiments. Results from the first experiment demonstrate the effectiveness of the camera array and spherical display system, in that it allows observers at multiple observing positions to accurately tell at which targets the remote user is looking. The second experiment further compared a spherical display with a planar display and provided detailed reasons for the improvement of our system in conveying gaze. We found two linear models for predicting the distortion introduced by misalignment of capturing cameras and the observer's viewing angles in video conferencing systems. Those models might be able to enable a correction for this distortion in future display configurations.

Simultaneous active camera array focus plane estimation and occluded moving object imaging

Automatically focusing and seeing occluded moving object in cluttered and complex scene is a significant challenging task for many computer vision applications. In this paper, we present a novel synthetic aperture imaging approach to solve this problem. The unique characteristics of this work include the following: (1) To the best of our knowledge, this work is the first to simultaneously solve camera array auto focusing and occluded moving object imaging problem. (2) A unified framework is designed to achieve seamless interaction between the focusing and imaging modules. (3) In the focusing module, a local and global constraint-based optimization algorithm is presented to dynamically estimate the focus plane of the moving object. (4) In the imaging module, a novel visibility analysis based active synthetic aperture imaging approach is proposed to remove the occluder and significantly improve the quality of occluded object imaging. An active camera array system has been set up and evaluated in challenging indoor and outdoor scenes. Extensive experimental results with qualitative and quantitative analyses demonstrate the superiority of the proposed approach compared with state-of-the-art approaches.

Simplified homography matrix for a large-scale camera array system

We want to use a large-scale camera array system in which each camera is placed at the desired position to photograph a subject and later render images of the subject viewed from various directions or render images for a three-dimensional display. The homography matrix for each camera should be calculated in advance to correct the captured images. In the case that each camera is physically facing toward the subject as precisely as possible but the captured image still includes geometrical distortion, if the expected error in the deviations from the ideal directions is assumed to be the zero vector, the homography matrix of each camera can be easily obtained.

Circular camera array system for 3‐D displays based on the reconstruction of parallax rays

Abstract— A circular camera system employing an image‐based rendering technique that captures light‐ray data needed for reconstructing three‐dimensional (3‐D) images by using reconstruction of parallax rays from multiple images captured from multiple viewpoints around a real object in order to display a 3‐D image of a real object that can be observed from multiple surrounding viewing points on a 3‐D display is proposed. An interpolation algorithm that is effective in reducing the number of component cameras in the system is also proposed. The interpolation and experimental results which were performed on our previously proposed 3‐D display system based on the reconstruction of parallax rays will be described. When the radius of the proposed circular camera array was 1100 mm, the central angle of the camera array was 40°, and the radius of a real 3‐D object was between 60 and 100 mm, the proposed camera system, consisting of 14 cameras, could obtain sufficient 3‐D light‐ray data to reconstruct 3‐D images on the 3‐D display.