Global Shutter Camera Research Articles

Integrated particle image velocimetry and fluid–structure interaction analysis for patient-specific abdominal aortic aneurysm studies

BackgroundUnderstanding the hemodynamics of an abdominal aortic aneurysm (AAA) is crucial for risk assessment and treatment planning. This study introduces a low-cost, patient-specific in vitro AAA model to investigate hemodynamics using particle image velocimetry (PIV) and flow-simulating circuit, validated through fluid–structure interaction (FSI) simulations.MethodsIn this study, 3D printing was employed to manufacture a flexible patient-specific AAA phantom using a lost-core casting technique. A pulsatile flow circuit was constructed using off-the-shelf components. A particle image velocimetry (PIV) setup was built using an affordable laser source and global shutter camera, and finally, the flow field inside the AAA was analyzed using open-source software. Fluid–structure interaction (FSI) simulations were performed to enhance our understanding of the flow field, and the results were validated by PIV analysis. Both steady-state and transient flow conditions were investigated.ResultsOur experimental setup replicated physiological conditions, analyzing arterial wall deformations and flow characteristics within the aneurysm. Under constant flow, peak wall deformations and flow velocities showed deviations within − 12% to + 27% and − 7% to + 5%, respectively, compared to FSI simulations. Pulsatile flow conditions further demonstrated a strong correlation (Pearson coefficient 0.85) in flow velocities and vectors throughout the cardiac cycle. Transient phenomena, particularly the formation and progression of vortex structures during systole, were consistently depicted between experimental and numerical models.ConclusionsBy bridging high-fidelity experimental observations with comprehensive computational analyses, this study underscores the potential of integrated methodologies in enhancing our understanding of AAA pathophysiology. The convergence of realistic AAA phantoms, precise PIV measurements at affordable cost point, and validated FSI models heralds a new paradigm in vascular research, with significant implications for personalized medicine and bioengineering innovations.

Dual-Shutter Optical Vibration Sensing.

Visual vibrometry is a highly useful tool for remote capture of audio, as well as the physical properties of materials, human heart rate, and more. While visually-observable vibrations can be captured directly with a high-speed camera, minute imperceptible object vibrations can be optically amplified by imaging the displacement of a speckle pattern created by shining a laser beam on the vibrating surface. In this paper, we propose a novel method for sensing vibrations at high speeds (up to 63kHz), for multiple scene sources at once, using sensors rated for only 130Hz operation. Our method relies on simultaneously capturing the scene with two cameras equipped with rolling and global shutter sensors, respectively. The rolling shutter camera captures distorted speckle images that encode the high-speed object vibrations. The global shutter camera captures undistorted reference images of the speckle pattern, helping to decode the source vibrations. We demonstrate our method by capturing vibration caused by audio sources (e.g., speakers, human voice, and musical instruments) and analyzing the vibration modes of a tuning fork.

Conception of a High-Level Perception and Localization System for Autonomous Driving

This paper describes the conception of a high level, compact, scalable, and long autonomy perception and localization system for autonomous driving applications. Our benchmark is composed of a high resolution lidar (128 channels), a stereo global shutter camera, an inertial navigation system, a time server, and an embedded computer. In addition, in order to acquire data and build multi-modal datasets, this system embeds two perception algorithms (RBNN detection, DCNN detection) and one localization algorithm (lidar-based localization) to provide real-time advanced information such as object detection and localization in challenging environments (lack of GPS). In order to train and evaluate the perception algorithms, a dataset is built from 10,000 annotated lidar frames from various drives carried out under different weather conditions and different traffic and population densities. The performances of the three algorithms are competitive with the state-of-the-art. Moreover, the processing time of these algorithms are compatible with real-time autonomous driving applications. By providing directly accurate advanced outputs, this system might significantly facilitate the work of researchers and engineers with respect to planning and control modules. Thus, this study intends to contribute to democratizing access to autonomous vehicle research platforms.

EVALUATION OF A COMPACT HELMET-BASED LASER SCANNING SYSTEM FOR ABOVEGROUND AND UNDERGROUND 3D MAPPING

Abstract. As a strategic resource, urban underground space can be used for rail transportation, commercial streets, which has high economic and social benefits, and is of great significance to sustainable city development. Due to denied Global Navigation Satellite System (GNSS) signal, traditional mobile mapping systems have difficulty collecting accurate 3D point clouds in urban underground space. Thus, a helmet-based laser scanning system, named "WHU-Helmet", is integrated in this paper to make up for the shortcomings of the existing traditional mobile mapping systems. "WHU-Helmet" is mainly equipped with four types of sensors: a GNSS receiver (optional), an IMU, a laser scanner, and a global shutter camera. "WHU-Helmet" is not relying on GNSS signal and has the advantages of low cost, small volume and easy operation. Using "WHU-Helmet", a multi-scale Normal Distributions Transform (NDT) based LiDAR-IMU SLAM is implemented to collect underground 3D point cloud in real-time. To validate the performance of "WHU-Helmet" in aboveground and underground 3D mapping, experiments were conducted in a typical urban metro station. The experiments show that the average and RMSE of HLS point errors of "WHU-Helmet" are 0.44 meters and 0.23 meters, respectively, showing great potential of "WHU-Helmet" in the application of aboveground and underground 3D mapping.

NTU VIRAL: A visual-inertial-ranging-lidar dataset, from an aerial vehicle viewpoint

In recent years, autonomous robots have become ubiquitous in research and daily life. Among many factors, public datasets play an important role in the progress of this field, as they waive the tall order of initial investment in hardware and manpower. However, for research on autonomous aerial systems, there appears to be a relative lack of public datasets on par with those used for autonomous driving and ground robots. Thus, to fill in this gap, we conduct a data collection exercise on an aerial platform equipped with an extensive and unique set of sensors: two 3D lidars, two hardware-synchronized global-shutter cameras, multiple Inertial Measurement Units (IMUs), and especially, multiple Ultra-wideband (UWB) ranging units. The comprehensive sensor suite resembles that of an autonomous driving car, but features distinct and challenging characteristics of aerial operations. We record multiple datasets in several challenging indoor and outdoor conditions. Calibration results and ground truth from a high-accuracy laser tracker are also included in each package. All resources can be accessed via our webpage https://ntu-aris.github.io/ntu_viral_dataset/.

Design and Implementation of 2D MIMO-Based Optical Camera Communication Using a Light-Emitting Diode Array for Long-Range Monitoring System.

Wireless technologies that use radio frequency (RF) waveforms are common in wireless communication systems, such as the mobile communication, satellite system, and Internet of Things (IoT) systems. It is more advantageous than wired communication because of the ease of installation. However, it can negatively impact human health if high frequencies are used to transmit data. Therefore, researchers are exploring the potential of optical wireless communication as an alternative, which uses the visible light bandwidth instead of RF waveforms. Three possibilities are being investigated: visible light communication, light fidelity, and optical camera communication. In this paper, we propose a multiple-input multiple-output modulation scheme using a light-emitting diode (LED) array, which is applicable to the IoT system, based on on–off keying modulation in the time domain. This scheme is compatible with the two popular types of camera in the market, rolling shutter cameras and global shutter cameras, as well as the closed-circuit television camera, which is used in factories, buildings, etc. Despite the small size of the LED array, implementing this scheme with 10 links in different positions at a communication distance of 20 m is possible for efficient performance (low error rate) by controlling the exposure time, shutter speed, focal length, channel coding and applying the matched filter.

Low-Cost, Continuous Motion Imaging, Computationally Augmented Whole Slide Imager for Digital Pathology

As digital pathology advances, systems for acquiring Whole Slide Images (WSI) need to become affordable and effective. Current research is majorly reported around the low-throughput stop-and-focus imaging systems that are too slow for clinical use. This work reports a system that acquires well focused images at a high throughput. It reduces the imaging time by an order of magnitude via the carefully engineered implementation of a global shutter camera which allows continuous motion imaging. The second challenge of maintaining good focus comes from three sources: the imperfections induced by low-cost optomechanics, smear surface undulations, and irregularity of the slide surface itself. These are minimized via a tiled-scan method supported by focal mapping, which predicts and achieves best focus on-the-fly. Finally, any remnant defocus is compensated for by a post-acquisition image enhancement method which sharpens the image without disturbing the essential features and color tones. Thus, this system delivers a 15 × 15 mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> scan in about 3 minutes at a resolution of 0.78 μm with a 40x microscopy objective. The bill of material cost is about US $1300 and hence it would be beneficial for telepathology in resource-constrained scenarios. Also, this device can serve as an enabler of Artificial Intelligence/Machine Learning algorithms to provide fully automated diagnosis. The developed codes for post-acquisition image enhancement are available at https://github.com/navchetan-awasthi/Microscopy.

Experimental Comparison between Event and Global Shutter Cameras.

We compare event-cameras with fast (global shutter) frame-cameras experimentally, asking: “What is the application domain, in which an event-camera surpasses a fast frame-camera?” Surprisingly, finding the answer has been difficult. Our methodology was to test event- and frame-cameras on generic computer vision tasks where event-camera advantages should manifest. We used two methods: (1) a controlled, cheap, and easily reproducible experiment (observing a marker on a rotating disk at varying speeds); (2) selecting one challenging practical ballistic experiment (observing a flying bullet having a ground truth provided by an ultra-high-speed expensive frame-camera). The experimental results include sampling/detection rates and position estimation errors as functions of illuminance and motion speed; and the minimum pixel latency of two commercial state-of-the-art event-cameras (ATIS, DVS240). Event-cameras respond more slowly to positive than to negative large and sudden contrast changes. They outperformed a frame-camera in bandwidth efficiency in all our experiments. Both camera types provide comparable position estimation accuracy. The better event-camera was limited by pixel latency when tracking small objects, resulting in motion blur effects. Sensor bandwidth limited the event-camera in object recognition. However, future generations of event-cameras might alleviate bandwidth limitations.

Design and Implementation of a Monitoring System using Optical Camera Communication for a Smart Factory

Wireless technologies that are based on radio frequencies are currently widely used, with numerous applications around the world. However, they pose some disadvantages to human health. High frequencies can have potentially harmful effects on children, hospital patients, and even healthy people if the signal power exceeds the permitted standard. Conversely, the use of visible light for data transmission is a trend that presents new options, including optical wireless communication, optical camera communication, and visible light communication. This paper proposes a modulation scheme based on on-off keying in the time domain, which is applied to a monitoring system using optical camera communication. This scheme has various compatible supports for the global-shutter camera and rolling-shutter camera, which are popular commercially available cameras. Furthermore, this scheme facilitates a low-cost monitoring system. By using small light-emitting diodes (LEDs) and controlling the exposure time in a single camera, the camera, as a receiver, can simultaneously detect signals from up to 10 sensor devices in different positions at a maximum distance of up to 50 m, with a low error rate.

INTEGRATION OF A LOW-COST MULTISENSORY UAV SYSTEM FOR FOREST APPLICATION

Abstract. To integrate the multi-spectral imagery and laser scanning data for forest management, a low-cost multisensory UAV system, named Kylin Cloud, is introduced in this paper. The Kylin Cloud is composed several low-cost sensors (i.e., GNSS receiver, IMU, global shutter camera, multispectral camera, and laser scanner), providing the fusion of the imagery and laser scanning data for reliable forest inventory. Experiments were undertaken in a forest park in Wuhan. Results showed that the registration error of the multispectral Digital Orthophoto Map (DOM) and laser scanning data is about one pixel, demonstrating a high potential of the proposed low-cost system.

3D MOBILE MAPPING WITH A LOW COST UAV SYSTEM

Abstract. In order to accomplish the automatic mobile mapping task in a small area of interest, a low cost UAV system is proposed in this paper. Multiple sensors including a global shutter camera and an inertial measurement unit are calibrated and synchronized to collect data from the area of interest. First the images are matched by the chronological order and the SfM is utilized. Then the origin SfM result is integrated with the IMU data by adding the IMU constraints into the bundle adjustment. At last the photogrammetry point clouds are generated using PMVS according to the extrinsic parameters. Experiments are undertaken in a typical scene with photogrammetry point clouds generated. The trajectory estimated by the proposed integration method are compared with the method that relies on image only, showing that the proposed method has better performance.

WIRED AND WIRELESS CAMERA TRIGGERING WITH ARDUINO

Abstract. Synchronous triggering is an important task that allows simultaneous data capture from multiple cameras. Accurate synchronization enables 3D measurements of moving objects or from a moving platform. In this paper, we describe one wired and four wireless variations of Arduino-based low-cost remote trigger systems designed to provide a synchronous trigger signal for industrial cameras. Our wireless systems utilize 315 MHz or 434 MHz frequencies with noise filtering capacitors. In order to validate the synchronization accuracy, we developed a prototype of a rotating trigger detection system (named RoTriDeS). This system is suitable to detect the triggering accuracy of global shutter cameras. As a result, the wired system indicated an 8.91 μs mean triggering time difference between two cameras. Corresponding mean values for the four wireless triggering systems varied between 7.92 and 9.42 μs. Presented values include both camera-based and trigger-based desynchronization. Arduino-based triggering systems appeared to be feasible, and they have the potential to be extended to more complicated triggering systems.

The event-camera dataset and simulator: Event-based data for pose estimation, visual odometry, and SLAM

New vision sensors, such as the dynamic and active-pixel vision sensor (DAVIS), incorporate a conventional global-shutter camera and an event-based sensor in the same pixel array. These sensors have great potential for high-speed robotics and computer vision because they allow us to combine the benefits of conventional cameras with those of event-based sensors: low latency, high temporal resolution, and very high dynamic range. However, new algorithms are required to exploit the sensor characteristics and cope with its unconventional output, which consists of a stream of asynchronous brightness changes (called “events”) and synchronous grayscale frames. For this purpose, we present and release a collection of datasets captured with a DAVIS in a variety of synthetic and real environments, which we hope will motivate research on new algorithms for high-speed and high-dynamic-range robotics and computer-vision applications. In addition to global-shutter intensity images and asynchronous events, we provide inertial measurements and ground-truth camera poses from a motion-capture system. The latter allows comparing the pose accuracy of ego-motion estimation algorithms quantitatively. All the data are released both as standard text files and binary files (i.e. rosbag). This paper provides an overview of the available data and describes a simulator that we release open-source to create synthetic event-camera data.

신경망 기반 차량 이미지센서 시스템을 위한 플리커 프리 공간-PSK 변조 기법

This paper introduces a novel modulation scheme for vehicular communication in taking advantage of existing LED lights available on a car. Our proposed 2-Phase Shift Keying (2-PSK) is a spatial modulation approach in which a pair of LED light sources in a car (either rear LEDs or front LEDs) is used as a transmitter. A typical camera (i.e. low frame rate at no greater than 30fps) that either a global shutter camera or a rolling shutter camera can be used as a receiver. The modulation scheme is a part of our Image Sensor Communication proposal submitted to IEEE 802.15.7r1 (TG7r1) recently. Also, a neural network approach is applied to improve the performance of LEDs detection and decoding under the noisy situation. Later, some analysis and experiment results are presented to indicate the performance of our system

PHOTOGRAMMETRIC ACCURACY AND MODELING OF ROLLING SHUTTER CAMERAS

Abstract. Unmanned aerial vehicles (UAVs) are becoming increasingly popular in professional mapping for stockpile analysis, construction site monitoring, and many other applications. Due to their robustness and competitive pricing, consumer UAVs are used more and more for these applications, but they are usually equipped with rolling shutter cameras. This is a significant obstacle when it comes to extracting high accuracy measurements using available photogrammetry software packages. In this paper, we evaluate the impact of the rolling shutter cameras of typical consumer UAVs on the accuracy of a 3D reconstruction. Hereto, we use a beta-version of the Pix4Dmapper 2.1 software to compare traditional (non rolling shutter) camera models against a newly implemented rolling shutter model with respect to both the accuracy of geo-referenced validation points and to the quality of the motion estimation. Multiple datasets have been acquired using popular quadrocopters (DJI Phantom 2 Vision+, DJI Inspire 1 and 3DR Solo) following a grid flight plan. For comparison, we acquired a dataset using a professional mapping drone (senseFly eBee) equipped with a global shutter camera. The bundle block adjustment of each dataset shows a significant accuracy improvement on validation ground control points when applying the new rolling shutter camera model for flights at higher speed (8m=s). Competitive accuracies can be obtained by using the rolling shutter model, although global shutter cameras are still superior. Furthermore, we are able to show that the speed of the drone (and its direction) can be solely estimated from the rolling shutter effect of the camera.

PHOTOGRAMMETRIC ACCURACY AND MODELING OF ROLLING SHUTTER CAMERAS

Unmanned aerial vehicles (UAVs) are becoming increasingly popular in professional mapping for stockpile analysis, construction site monitoring, and many other applications. Due to their robustness and competitive pricing, consumer UAVs are used more and more for these applications, but they are usually equipped with rolling shutter cameras. This is a significant obstacle when it comes to extracting high accuracy measurements using available photogrammetry software packages. In this paper, we evaluate the impact of the rolling shutter cameras of typical consumer UAVs on the accuracy of a 3D reconstruction. Hereto, we use a beta-version of the Pix4Dmapper 2.1 software to compare traditional (non rolling shutter) camera models against a newly implemented rolling shutter model with respect to both the accuracy of geo-referenced validation points and to the quality of the motion estimation. Multiple datasets have been acquired using popular quadrocopters (DJI Phantom 2 Vision+, DJI Inspire 1 and 3DR Solo) following a grid flight plan. For comparison, we acquired a dataset using a professional mapping drone (senseFly eBee) equipped with a global shutter camera. The bundle block adjustment of each dataset shows a significant accuracy improvement on validation ground control points when applying the new rolling shutter camera model for flights at higher speed (8m=s). Competitive accuracies can be obtained by using the rolling shutter model, although global shutter cameras are still superior. Furthermore, we are able to show that the speed of the drone (and its direction) can be solely estimated from the rolling shutter effect of the camera.