Multiscale Camera Research Articles

A Dynamic Occupancy Model for Interacting Species with Two Spatial Scales

AbstractOccupancy models have been extended to account for either multiple spatial scales or species interactions in a dynamic setting. However, as interacting species (e.g., predators and prey) often operate at different spatial scales, including nested spatial structure might be especially relevant to models of interacting species. Here we bridge these two model frameworks by developing a multi-scale, two-species occupancy model. The model is dynamic, i.e. it estimates initial occupancy, colonization and extinction probabilities—including probabilities conditional to the other species’ presence. With a simulation study, we demonstrate that the model is able to estimate most parameters without marked bias under low, medium and high average occupancy probabilities, as well as low, medium and high detection probabilities, with only a small bias for some parameters in low-detection scenarios. We further evaluate the model’s ability to deal with sparse field data by applying it to a multi-scale camera trapping dataset on a mustelid-rodent predator–prey system. Most parameters are estimated with low uncertainty (i.e. narrow posterior distributions). More broadly, our model framework creates opportunities to explicitly account for the spatial structure found in many spatially nested study designs, and to study interacting species that have contrasting movement ranges with camera traps.Supplementary materials accompanying this paper appear online.

Scan-free end-to-end new approach for snapshot camera spectral sensitivity estimation.

Spectral sensitivity is largely related to sensor imaging, which has drawn widespread attention in computer vision. Accurate estimation becomes increasingly urgent because manufacturers rarely disclose it. In this Letter, we present a novel, compact, inexpensive, and real-time computational system for snapshot spectral sensitivity estimation. A multi-scale camera based on the multi-scale convolutional neural network is first proposed, to the best of our knowledge, to automatically extract multiplexing features of an input image by multiscale deep learning, which is vital to solving the inverse problem in sensitivity estimation. Our network is flexible and can be designed with different convolutional kernel sizes for a given application. We build a dataset with 10,500 raw images and generate an excellent pre-trained model. Commercial cameras are adopted to test model validity; the results show that our system can achieve estimation accuracy as high as 91.35%. We provide a method for system design, propose a deep learning network, build a dataset, demonstrate training process, and present experimental results with high precision. This simple and effective method provides an accurate approach for precise estimation of spectral sensitivity and is suitable for computational applications such as pathological digital stain, virtual/augmented reality display, and high-quality image acquisition.

A modular hierarchical array camera

Array cameras removed the optical limitations of a single camera and paved the way for high-performance imaging via the combination of micro-cameras and computation to fuse multiple aperture images. However, existing solutions use dense arrays of cameras that require laborious calibration and lack flexibility and practicality. Inspired by the cognition function principle of the human brain, we develop an unstructured array camera system that adopts a hierarchical modular design with multiscale hybrid cameras composing different modules. Intelligent computations are designed to collaboratively operate along both intra- and intermodule pathways. This system can adaptively allocate imagery resources to dramatically reduce the hardware cost and possesses unprecedented flexibility, robustness, and versatility. Large scenes of real-world data were acquired to perform human-centric studies for the assessment of human behaviours at the individual level and crowd behaviours at the population level requiring high-resolution long-term monitoring of dynamic wide-area scenes.

Field of view in monocentric multiscale cameras.

Conventionally, the field of view of a camera is understood as the angular extent of a convex circular or rectangular region. Parallel camera architectures with computational image stitching, however, allow implementation of a field of view with an arbitrary shape. Monocentric multiscale lenses further allow the implementation of an arbitrary field of view in camera volumes comparable to conventional single-lens systems. In contrast with conventional wide-field-of-view systems, multiscale design can also achieve nearly uniform resolution across the entire field of view. This paper presents several design studies obtaining unconventional fields of view using this approach.

Model-Based Multiscale Gigapixel Image Formation Pipeline on GPU

We present an efficient and flexible GPU implementation of a highly-parallelizable and scalable image formation pipeline for gigapixel images based on the MapReduce framework. The presented implementation was developed to operate with the AWARE multiscale gigapixel cameras, but it is also able to efficiently form gigapixel images from any source. The AWARE cameras are compact camera arrays that simultaneously collect images that span a wide field-of-view to generate high-resolution and high dynamic range panoramic images and video. The proposed GPU implementation exploits the mutiscale nature of the AWARE image acquisition, not only enabling the fast composition of gigapixel-scale panoramas, but also the rapid formation of images of arbitrary portions of the field-of-view at current display-scale resolutions at video rates.

Illumination compensation of microcamera image in monocentric multiscale foveated imaging system

A novel imaging system based on monocentric multiscale design and foveated imaging is proposed to realize wide field of view (FOV) and high resolution dynamic gaze of the moving target. Different from the existing monocentric multiscale cameras, the individual microcamera of the proposed imaging system relays a relatively large portion of the overall intermediate image. The proposed imaging system has many advantages, but the illumination of the microcamera image is nonuniform. The edge illumination of the microcamera image is low that makes it difficult to the subsequent image data processing. We analyze the illumination of the image theoretically and propose to use the relative illumination factor to compensate the low illumination region in corresponding fields. Finally, a specific microcamera is designed by using ZEMAX software and the illumination of the microcamera image is optimized by the proposed method, realizing the uniform illumination of the image.

Evaluations on multi-scale camera networks for precise and geo-accurate reconstructions from aerial and terrestrial images with user guidance

During the last decades photogrammetric computer vision systems have been well established in scientific and commercial applications. Recent developments in image-based 3D reconstruction systems have resulted in an easy way of creating realistic, visually appealing and accurate 3D models. We present a fully automated processing pipeline for metric and geo-accurate 3D reconstructions of complex geometries supported by an online feedback method for user guidance during image acquisition. Our approach is suited for seamlessly matching and integrating images with different scales, from different view points (aerial and terrestrial), and with different cameras into one single reconstruction. We evaluate our approach based on different datasets for applications in mining, archaeology and urban environments and thus demonstrate the flexibility and high accuracy of our approach. Our evaluation includes accuracy related analyses investigating camera self-calibration, georegistration and camera network configuration.

Characterization of the AWARE 40 wide-field-of-view visible imager

High-resolution, wide-field-of-view photography has undergone substantial improvements since the advent of multiscale lens design. Here, we present and evaluate a multiscale camera capable of a 10 μrad instantaneous field of view over a 36° field of view. We experimentally verify this camera’s optical performance at the individual microcamera level and report a 2.4 gigapixel panorama stitched from five snapshots. The results are consistent with the use of parallel camera arrays to produce scalable imagery at a substantial resolution range.

Feedback stitching for gigapixel video

Methods of stitching static panoramas are unsuitable for video-rate stitching from camera arrays, because these methods are too computationally intensive for real-time operation and do not take advantage of prior knowledge of camera positions or the coherence between successive frames of a video sequence. We propose feedback stitching, which embeds the stitching process in a feedback loop, so that as new frames are captured, any new stitching errors occurring in the video sequence are analyzed and corrected as the sequence progresses. These algorithms are suitable for multiscale cameras, a camera array technology proven to be capable of gigapixel snapshot and video imaging, to allow for real-time compensation of any registration or parallax errors.

상분할 방식의 기가픽셀 카메라를 위한 가우스 광학적인 구조설계

작년 6월 20일, 기존의 일반 카메라에 비해 화질이 1,000배 이상인 초고분해능의 기가픽셀카메라가 미국 듀크대의 과학자들에 의해 네이처에 보고되었고 이를 월스트리트 저널이 보도했다. 보도에 따르면, 이 카메라는 미군부의 필요에 의해 공중 및 지상배치감시를 목적으로 개발되었다는 것이다. 이 기술은 향후 국방과 산업분야의 전반에 걸쳐서 기술.경제적인 파급효과가 매우 클 것으로 예상되기 때문에 국내에서도 시급히 개발에 착수할 필요가 있다고 판단된다. 이에 본 연구에서는 슈퍼 분해능을 갖는 상분할 방식의 기가픽셀 카메라를 구현하기 위한 전초작업으로써 카메라 렌즈시스템의 광학적인 구조를 고찰하였고 더불어 구조와 관련된 렌즈사양 값들의 범위를 계산해 내었다. It was reported in Nature and the Wall Street Journal on June 20th, 2012 that scientists at Duke university have developed a gigapixel camera, capable of over 1,000 times the resolution of a normal camera. According to the reports, this super-resolution camera was motivated by the need of US military authorities to surveil ground and sky. We notice the ripple effect of this technology has spread into the area of national defense and industry, so that this research has started to realize the super-resolution camera as a frontier research topic. As a result, we can understand the optical structure of a super-resolution camera's lens system to be composed of a front, monocentric objective of a single lens plus 98 rear, multiscale camera lenses. We can also obtain the numerical ranges of specification factors related to the optical structure, such as the diameter of the aperture, and the focal length.

Autofocus for a multiscale gigapixel camera

In recent studies, the advanced wide field of view architectures for image reconstruction and exploitation (AWARE) multiscale camera, which is composed of a monocentric objective lens and an array of microcameras, was developed for the realization of snapshot wide field of view and high resolution imaging. This paper describes accelerated autofocus (AF) methods for the AWARE system based on a hierarchical spatial algorithm and an iterative temporal algorithm. In the algorithms, sensor positions of each microcamera are hierarchically scanned with contrast detection to effectively search for a focusing distance. The positions are then updated iteratively for dynamic scenes using temporal information. The algorithms are theoretically analyzed and experimentally demonstrated. The developed AF methods can be used for the realization of the temporal gigapixel imaging by the AWARE system.

Gigapixel Imaging with the AWARE Multiscale Camera

Gigapixel cameras have been confined to specialized applications such as aerial photography and astronomical observatories. A simplified architecture would better suit terrestrial imaging and reduce instrument cost and complexity. Our gigapixel AWARE camera is based on monocentric multiscale optical design principles that produce high-resolution images with a field of view (FOV) limited only by vignetting. This design allows resolution to approach the theoretical diffraction limits for a given entrance pupil size and FOV.

Development of a scalable image formation pipeline for multiscale gigapixel photography

We report on the image formation pipeline developed to efficiently form gigapixel-scale imagery generated by the AWARE-2 multiscale camera. The AWARE-2 camera consists of 98 "microcameras" imaging through a shared spherical objective, covering a 120° x 50° field of view with approximately 40 microradian instantaneous field of view (the angular extent of a pixel). The pipeline is scalable, capable of producing imagery ranging in scope from "live" one megapixel views to full resolution gigapixel images. Architectural choices that enable trivially parallelizable algorithms for rapid image formation and on-the-fly microcamera alignment compensation are discussed.

Engineering a gigapixel monocentric multiscale camera

Multiscale designs greatly simplify the large-scale optics needed to realize the resolution of a large aperture. Unlike most monolithic lens systems, multiscale cameras have interdependencies between the optics at various scales. To realize a successful multiscale design, the relationships between microcamera scale and overall camera scale parameters must be understood. Starting with a specification of the multiscale camera, we present a simplified model that allows paraxial optical quantities to be estimated. Based on these quantities, a monocentric objective and a microcamera can be designed. An example of a practical design using spherical glass optics is presented.

A rendering framework for multiscale views of 3D models

Images that seamlessly combine views at different levels of detail are appealing. However, creating such multiscale images is not a trivial task, and most such illustrations are handcrafted by skilled artists. This paper presents a framework for direct multiscale rendering of geometric and volumetric models. The basis of our approach is a set of non-linearly bent camera rays that smoothly cast through multiple scales. We show that by properly setting up a sequence of conventional pinhole cameras to capture features of interest at different scales, along with image masks specifying the regions of interest for each scale on the projection plane, our rendering framework can generate non-linear sampling rays that smoothly project objects in a scene at multiple levels of detail onto a single image. We address two important issues with non-linear camera projection. First, our streamline-based ray generation algorithm avoids undesired camera ray intersections, which often result in unexpected images. Second, in order to maintain camera ray coherence and preserve aesthetic quality, we create an interpolated 3D field that defines the contribution of each pinhole camera for determining ray orientations. The resulting multiscale camera has three main applications: (1) presenting hierarchical structure in a compact and continuous manner, (2) achieving focus+context visualization, and (3) creating fascinating and artistic images.

Microcamera aperture scale in monocentric gigapixel cameras

Multiscale cameras achieve wide-angle, high-resolution imaging by combining coarse image formation by a simplified wide-field objective with localized aberration correction in an array of narrow field microcameras. Microcamera aperture size is a critical parameter in multiscale design; a larger aperture has greater capacity to correct aberration but requires a more complex microcamera optic. A smaller aperture requires integration of more microcameras to cover the field. This paper analyzes multiscale system performance as a function of microcamera aperture for 2 and 40 gigapixel monocentric objective lenses. We find that microcamera aperture diameters of 3 to 12 mm paired with complementary metal oxide semiconductor sensors in the 1 to 15 megapixel range are most attractive for gigapixel-scale cameras.