Traditional Camera Research Articles

How Many Events are Needed for One Reconstructed Image Using an Event Camera?

Abstract. Event cameras offer significant advantages over traditional cameras, including high temporal resolution, high dynamic range, and low power consumption. However, reconstructing images from the asynchronous events generated by these cameras presents unique challenges. This paper investigates the optimal number of events needed for high-quality image reconstruction using event cameras. We evaluate two primary reconstruction strategies—fixed time window and fixed number of events—across various dynamic and static scenes. Our study includes scenarios with different lighting conditions and camera movements. Using the state-of-the-art E2VID algorithm, we perform both qualitative and quantitative analyses of the reconstructed images, comparing them with reference frames from a traditional RGB camera. Our results demonstrate the trade-offs between temporal resolution and image quality for each reconstruction strategy, providing insights into the optimal settings for different applications. This research offers practical guidelines for selecting appropriate reconstruction parameters to achieve the better image quality from event cameras.

Asynchronous Visual-Inertial Odometry for Event Cameras

Abstract. Event cameras are bio-inspired visual sensors that output changes in pixel-level brightness asynchronously instead of standard intensity frames at a fixed rate. These cameras offer reliable visual information in high-speed motion and high dynamic range (HDR) scenes, addressing the limitations of traditional cameras in such scenarios. Therefore, research of integrating event cameras into established visual algorithms holds significant value. In this study, based on traditional Visual-Inertial Odometry (VIO) frameworks, we proposed an innovative asynchronous monocular event-based inertial odometry method to fully exploit the benefits of event cameras. First, the corner features are extracted separately from the raw event stream and the time surface map, followed by uniform feature selection to accurately describe three-dimensional spatial geometry. Then, feature tracking is achieved by integrating these two event representation methods. In addition, our method obtains stable and high frequency state estimation by fusing event and IMU measurements through graph optimization. We validate the effectiveness of our proposed approach, comparing with several state-of-the-art EVIO systems and VIO systems.

Hierarchical Event-RGB Interaction Network for single-eye expression recognition

The Single-eye Expression Recognition task stands as a crucial vision task, aimed at decoding human emotional states through careful examination of the eye region. Nevertheless, traditional cameras face challenges in detecting and capturing relevant biological information, especially under demanding lighting conditions such as dim environments, high exposure scenarios, or when other radiation sources are present. In this regard, we use a new type of sensor data that can resist extreme lighting conditions, namely event camera data, to improve the performance of single-eye expression recognition. To this end, we propose a novel Hierarchical Event-RGB Interaction Network (HI-Net), to fully integrate RGB and event data to overcome the extreme lighting challenges faced by the single-eye expression recognition task. The HI-Net contains two novel designs: Event-RGB Semantic Interaction Mechanism (ER-SIM) and Hierarchical Semantics Modeling (HSM) Scheme. The former aims to achieve interaction between Event and RGB modality semantics, while the latter aims to obtain high-quality modality semantic representations. In the ER-SIM, we employ an effective cross-attention mechanism to facilitate information fusion, to adaptively integrate and complement multi-scale Event and RGB semantics to cope with extreme lighting conditions. In HSM Scheme, we first explore multi-scale contextual semantics for the event modality and the RGB modality respectively. Then, we perform a semantics interaction strategy for these multi-scale contextual semantics, to enhance each modality's semantic representation. Extensive experiments demonstrate that our HI-Net significantly outperforms many state-of-the-art methods on the single-eye expression recognition task, especially under degraded lighting conditions.

Exploring event-based human pose estimation with 3D event representations

Human pose estimation is a fundamental and appealing task in computer vision. Although traditional cameras are commonly applied, their reliability decreases in scenarios under high dynamic range or heavy motion blur, where event cameras offer a robust solution. Predominant event-based methods accumulate events into frames, ignoring the asynchronous and high temporal resolution that is crucial for distinguishing distinct actions. To address this issue and to unlock the 3D potential of event information, we introduce two 3D event representations: the Rasterized Event Point Cloud (RasEPC) and the Decoupled Event Voxel (DEV). The RasEPC aggregates events within concise temporal slices at identical positions, preserving their 3D attributes along with statistical information, thereby significantly reducing memory and computational demands. Meanwhile, the DEV representation discretizes events into voxels and projects them across three orthogonal planes, utilizing decoupled event attention to retrieve 3D cues from the 2D planes. Furthermore, we develop and release EV-3DPW, a synthetic event-based dataset crafted to facilitate training and quantitative analysis in outdoor scenes. Our methods are tested on the DHP19 public dataset, MMHPSD dataset, and our EV-3DPW dataset, with further qualitative validation via a derived driving scene dataset EV-JAAD and an outdoor collection vehicle. Our code and dataset have been made publicly available at https://github.com/MasterHow/EventPointPose.

Focus-Switchable Piezoelectric Actuator: A Bionic Thin-Plate Design Inspired by Conch Structure

Traditional camera zoom mechanisms, relying on stepper motors and intricate transmission components, suffer from bulky designs that restrict their applicability in compact systems. This study presents a novel piezoelectric focus-switchable mechanism (PFSM) directly driven by a biomimetic radial-mode piezoelectric actuator (RMPA) with inclined driving structure mimicking the conch shell. The PFSM utilizes rotational motion for optical zoom, providing a more compact and efficient alternative to conventional linear motion-based systems. By utilizing finite element method (FEM) optimization, we developed a compact prototype (34×34×3mm³, 14.83g) and experimentally verified its performance, achieving a peak rotational speed of 1573.14 RPM, torque output of 3.6 mN·m, and step displacement resolution of 45.8 μrad. These attributes enable smooth lens module switching for optical zoom, thus demonstrating the mechanism’s feasibility. Importantly, the PFSM offers precise positioning without relying on additional transmission parts and features a self-locking capability when de-energized, rendering it an ideal choice for camera zoom applications. In summary, this study underscores the PFSM’s potential as a compact, lightweight, and efficient camera zoom mechanism, contributing meaningfully to the field of imaging technology and optical systems.

A novel affordable user interface for robotic surgery training: design, development and usability study.

The use of robotic systems in the surgical domain has become groundbreaking for patients and surgeons in the last decades. While the annual number of robotic surgical procedures continues to increase rapidly, it is essential to provide the surgeon with innovative training courses along with the standard specialization path. To this end, simulators play a fundamental role. Currently, the high cost of the leading VR simulators limits their accessibility to educational institutions. The challenge lies in balancing high-fidelity simulation with cost-effectiveness; however, few cost-effective options exist for robotic surgery training. This paper proposes the design, development and user-centered usability study of an affordable user interface to control a surgical robot simulator. It consists of a cart equipped with two haptic interfaces, a VR visor and two pedals. The simulations were created using Unity, which offers versatility for expanding the simulator to more complex scenes. An intuitive teleoperation control of the simulated robotic instruments is achieved through a high-level control strategy. Its affordability and resemblance to real surgeon consoles make it ideal for implementing robotic surgery training programs in medical schools, enhancing accessibility to a broader audience. This is demonstrated by the results of an usability study involving expert surgeons who use surgical robots regularly, expert surgeons without robotic surgery experience, and a control group. The results of the study, which was based on a traditional Peg-board exercise and Camera Control task, demonstrate the simulator's high usability and intuitive control across diverse user groups, including those with limited experience. This offers evidence that this affordable system is a promising solution for expanding robotic surgery training.

Improved multi-input parameter optimization method for camera colorimetric characterization

In order to improve the accuracy of camera colorimetric characterization, a multi-input parameter optimization method was proposed in this paper. The input parameters of the traditional camera characterization method were generally RGB values; in the proposed method, the luminance parameter L was introduced in addition to RGB values, and the four-input parameters of RGBL were used as input parameters for the conversion model. In the experiment, 549 colors were uniformly selected from the Munsell Book of Color (Matte Edition), and the RGBL values and corresponding CIEXYZ values of the selected colors were measured by a spectroradiometer and three cameras, including an imaging luminance meter, respectively. Then, a polynomial model and a backpropagation (BP) neural network model were employed to establish the improved color conversion model with RGBL four-input parameters, which was compared with three-input parameter models to verify the effectiveness of the proposed method. Experimental results show that the proposed method can significantly improve the conversion accuracy and reduce the color difference with a maximum reduction of 57.7% in CIELAB.

Event Vision-based Corner Detection with Count-normalized Multi-Layer Perceptron and Throughput Indicator

The advancement of event cameras has sparked a revolution in imaging technology, presenting exciting opportunities for vision-based measurement tasks. Event cameras operate on an innovative asynchronous imaging principle, which offers several advantages over traditional cameras, including ultra-high dynamic range and exceptional temporal resolution. As a result, event cameras excel in challenging environments characterized by motion blur, overexposure, or underexposure, outperforming conventional frame-based cameras. However, the asynchronous signal streams generated by event cameras pose compatibility challenges with existing vision-based measurement algorithms. This paper focuses specifically on corner detection, a critical vision-based measurement task, tailored for event cameras. To address this challenge, we propose novel corner detectors that leverage advanced optimization techniques, including enhanced time surface representations, multi-layer perceptron classifiers, and an innovative throughput mechanism. Through rigorous experimentation, our method consistently shows lower projection errors compared to state-of-the-art methods across all datasets while also maintaining longer tracking times in low-textured scenarios. Specifically, our CMLP and CMLP-T methods achieve an average valid tracking rate of 83.38% and 84.65%, respectively, on the DAVIS240C dataset collection, surpassing all existing methods. We validate the effectiveness of our proposed corner detectors by demonstrating their enhanced performance compared to state-of-the-art methods. Furthermore, our work contributes to the application of machine learning in event signal processing for vision-based measurement tasks, providing insights into optimizing models for the unique characteristics of event cameras.

Seeing Through Muddy Water: Laser-Induced Graphene for Portable Tomography Imaging.

Due to its outstanding physical and chemical properties, graphene synthesized by laser scribing on polyimide (PI) offers excellent opportunities for photothermal applications, antiviral and antibacterial surfaces, and electrochemical storage and sensing. However, the utilization of such graphene for imaging is yet to be explored. Herein, using chemically durable and electrically conductive laser-induced graphene (LIG) for tomography imaging in aqueous suspensions is proposed. These graphene electrodes are designed as impedance imaging units for four-terminal electrical measurements. Using the real-time portable imaging prototypes, the conductive and dielectric objects can be seen in clear and muddy water with equivalent impedance modeling. This low-cost graphene tomography measurement system offers significant advantages over traditional visual cameras, in which the suspended muddy particles hinder the imaging resolution. This research shows the potential of applying graphene nanomaterials in emerging marine technologies, such as underwater robotics and automatic fisheries.

A Smart Visual Sensor for Smoke Detection Based on Deep Neural Networks.

The automatic detection of smoke by analyzing the video stream acquired by traditional surveillance cameras is becoming a more and more interesting problem for the scientific community thanks to the necessity to prevent fires at the very early stages. The adoption of a smart visual sensor, namely a computer vision algorithm running in real time, allows one to overcome the limitations of standard physical sensors. Nevertheless, this is a very challenging problem, due to the strong similarity of the smoke with other environmental elements like clouds, fog and dust. In addition to this challenge, data available for training deep neural networks is limited and not fully representative of real environments. Within this context, in this paper we propose a new method for smoke detection based on the combination of motion and appearance analysis with a modern convolutional neural network (CNN). Moreover, we propose a new dataset, called the MIVIA Smoke Detection Dataset (MIVIA-SDD), publicly available for research purposes; it consists of 129 videos covering about 28 h of recordings. The proposed hybrid method, trained and evaluated on the proposed dataset, demonstrated to be very effective by achieving a 94% smoke recognition rate and, at the same time, a substantially lower false positive rate if compared with fully deep learning-based approaches (14% vs. 100%). Therefore, the proposed combination of motion and appearance analysis with deep learning CNNs can be further investigated to improve the precision of fire detection approaches.

Recent Advances in Bio-Inspired Vision Sensor: A Review

Event-based cameras, also known as biologically inspired visual sensors, are capable of capturing real-time scene changes efficiently. Unlike traditional frame-based cameras, event cameras solely report triggered pixel-level brightness changes which are referred to as events. Event-based cameras show many advantages such as high temporal resolution, low latency, and high dynamic range, making them very attractive in robotics and computer vision, especially in challenging scenarios that are too demanding for traditional cameras. In this paper, we provide a comprehensive overview of the emerging field of event-based vision, focusing on the operation principle, sampling mechanisms, and algorithms that take advantage of their superior features. We also delve into the various tasks for which event cameras are utilized, such as object tracking, optical flow estimation, 3D reconstruction, SLAM, image reconstruction, and recognition. Additionally, we highlight the challenges and future opportunities for event cameras, seeking a more efficient way for machines to perceive and interact with the world.

Camera traps equipped with macro lenses as a tool for monitoring arboreal small mammals: a case study in an agroecosystem (NE Italy)

Despite their increasing use, camera traps as a monitoring tool for arboreal small mammals leave room for further improvements to increase their effectiveness. In the summer of 2023, we conducted a small mammal survey in a wooded area of a lowland agroecosystem in the Veneto region, using standard camera traps equipped with macro lenses for close-up shooting. This camera trap technique made it possible to contact three species of small mammals in the tree-shrub layer: Eurasian red squirrel Sciurus vulgaris, wood mouse Apodemus sylvaticus, and black rat Rattus rattus. The use of macro lenses combined with the standard camera trapping technique made it possible to obtain better quality images and more information even on smaller species compared to more traditional camera traps.

Enhancing robustness in asynchronous feature tracking for event cameras through fusing frame steams

Event cameras produce asynchronous discrete outputs due to the independent response of camera pixels to changes in brightness. The asynchronous and discrete nature of event data facilitate the tracking of prolonged feature trajectories. Nonetheless, this necessitates the adaptation of feature tracking techniques to efficiently process this type of data. In addressing this challenge, we proposed a hybrid data-driven feature tracking method that utilizes data from both event cameras and frame-based cameras to track features asynchronously. It mainly includes patch initialization, patch optimization, and patch association modules. In the patch initialization module, FAST corners are detected in frame images, providing points responsive to local brightness changes. The patch association module introduces a nearest-neighbor (NN) algorithm to filter new feature points effectively. The patch optimization module assesses optimization quality for tracking quality monitoring. We evaluate the tracking accuracy and robustness of our method using public and self-collected datasets, focusing on average tracking error and feature age. In contrast to the event-based Kanade–Lucas–Tomasi tracker method, our method decreases the average tracking error ranging from 1.3 to 29.2% and boosts the feature age ranging from 9.6 to 32.1%, while ensuring the computational efficiency improvement of 1.2–7.6%. Thus, our proposed feature tracking method utilizes the unique characteristics of event cameras and traditional cameras to deliver a robust and efficient tracking system.

Leveraging BIM for Enhanced Camera Allocation Planning at Construction Job Sites: A Voxel-Based Site Coverage and Overlapping Analysis

In the construction industry, the imperative for visual surveillance mechanisms is underscored by the need for safety monitoring, resources, and progress tracking, especially with the adoption of vision intelligence technology. Traditional camera installation plans often move toward coverage and cost objectives without considering substantial coverage overlap, inflating processing and storage requirements, and complicating subsequent analyses. To address these issues, this research proposes a voxel-based site coverage and overlapping analysis for camera allocation planning in parametric BIM environments, called the PBA approach. The first step is to collect information from the BIM model, which is the input for the parametric modeling step. After that, the PBA approach simulates the virtual devices and the construction layout by employing visual language programming and then generates a coverage area. Lastly, the performance simulation and evaluation of various placement scenarios against predefined criteria are conducted, including visual coverage and overlapping optimization for eliminating data redundancy purposes. The proposed approach is evaluated through its application to construction projects. The results from these various implementations indicate a marked decrease in data overlap and an overall enhancement in surveillance efficacy. This research contributes a novel, BIM-centric solution to visual information adoption in the construction industry, offering a scalable approach to optimize camera placement while mitigating overlapping areas.

CFZA camera: a high-resolution lensless imaging technique based on compound Fresnel zone aperture.

In lensless imaging using a Fresnel zone aperture (FZA), it is generally believed that the resolution is limited by the outermost ring breadth of the FZA. The limitation has the potential to be broken according to the multi-order property of binary FZAs. In this Letter, we propose to use a high-order component of the FZA as the point spread function (PSF) to develop a high-order transfer function backpropagation (HBP) algorithm to enhance the resolution. The proportion of high-order diffraction energy is low, leading to severe defocus noise in the reconstructed image. To address this issue, we propose a Compound FZA (CFZA), which merges two partial FZAs operating at different orders as the mask to strike a balance between the noise and resolution. Experimental results verify that the CFZA-based camera has a resolution that is double that of a traditional FZA-based camera with an identical outer ring breadth and can be reconstructed with high quality by a single HBP without calibration. Our method offers a cost-effective solution for achieving high-resolution imaging, expanding the potential applications of FZA-based lensless imaging in a variety of areas.

Structural vibration frequency monitoring based on event camera

Compared with traditional cameras, event cameras (ECs) have the significant advantages of high temporal resolution, low data redundancy, and microsecond delay, which are beneficial in structural monitoring to extract the dense response of structures in both spatial and temporal dimensions. In this paper, the vibration frequency detection method based on ECs is studied. This study investigates vibration frequency detection methods based on ECs, and proposes two algorithms for vibration frequency detection based on event streams: marker tracking and event count. Experimental verification is conducted through forced vibration experiments. The results indicate that the event count method achieves high-precision measurement of vibration frequencies in the range of 10–190 Hz for different vibration scales, with a maximum relative error of 1% and an average relative error of 0.673%. The marker tracking method demonstrates a maximum relative error of 1.43% and an average relative error of 0.575% in frequency measurement for large-amplitude vibrations. However, as the amplitude decreases, the frequency measurement error increases. When the amplitude is less than 3 pixels, the frequency measurement error exceeds 30%, rendering the measurement results unreliable. This research provides technical support for high-precision structural vibration frequency monitoring and further expands the application of ECs in structural monitoring.

Efficient event-based robotic grasping perception using hyperdimensional computing

Grasping is fundamental in various robotic applications, particularly within industrial contexts. Accurate inference of object properties is a crucial step toward enhancing grasping quality. Dynamic and Active Vision Sensors (DAVIS), increasingly utilized for robotic grasping, offer superior energy efficiency, lower latency, and higher temporal resolution than traditional cameras. However, the data they generate can be complex and noisy, necessitating substantial preprocessing. In response to these challenges, we introduce GraspHD, an innovative end-to-end algorithm that leverages brain-inspired hyperdimensional computing (HDC) to learn about the size and hardness of objects and estimate the grasping force. This novel approach circumvents the need for resource-intensive pre-processing steps, capitalizing on the simplicity and inherent parallelism of HDC operations. Our comprehensive analysis reveals that GraspHD surpasses state-of-the-art approaches in terms of overall classification accuracy. We have also implemented GraspHD on an FPGA to evaluate system efficiency. The results demonstrate that GraspHD operates at a speed 10x faster and offers an energy efficiency 26x higher than existing learning algorithms while maintaining robust performance in noisy environments. These findings underscore the significant potential of GraspHD as a more efficient and effective solution for real-time robotic grasping applications.

MVT: Multi-Vision Transformer for Event-Based Small Target Detection

MVT: Multi-Vision Transformer for Event-Based Small Target Detection

CamGNN: Cascade Graph Neural Network for Camera Re-Localization

In response to the inaccurate positioning of traditional camera relocation methods in scenes with large-scale or severe viewpoint changes, this study proposes a camera relocation method based on a cascaded graph neural network to achieve accurate scene relocation. Firstly, the NetVLAD retrieval method, which has advantages in image feature representation and similarity calculation, is used to retrieve the most similar images to a given query image. Then, the feature pyramid is employed to extract features at different scales of these images, and the features at the same scale are treated as nodes of the graph neural network to construct a single-layer graph neural network structure. Secondly, a top–down connection is used to cascade the single-layer graph structures, where the information of nodes in the previous graph is fused into a message node to improve the accuracy of camera pose estimation. To better capture the topological relationships and spatial geometric constraints between images, an attention mechanism is introduced in the single-layer graph structure, which helps to effectively propagate information to the next graph during the cascading process, thereby enhancing the robustness of camera relocation. Experimental results on the public dataset 7-Scenes demonstrate that the proposed method can effectively improve the accuracy of camera absolute pose localization, with average translation and rotation errors of 0.19 m and 6.9°, respectively. Compared to other deep learning-based methods, the proposed method achieves more than 10% improvement in both average translation and rotation accuracy, demonstrating highly competitive localization precision.

Service Quality and Residents' Preferences for Facilitated Self-Service Fundus Disease Screening: Cross-Sectional Study.

Fundus photography is the most important examination in eye disease screening. A facilitated self-service eye screening pattern based on the fully automatic fundus camera was developed in 2022 in Shanghai, China; it may help solve the problem of insufficient human resources in primary health care institutions. However, the service quality and residents' preference for this new pattern are unclear. This study aimed to compare the service quality and residents' preferences between facilitated self-service eye screening and traditional manual screening and to explore the relationships between the screening service's quality and residents' preferences. We conducted a cross-sectional study in Shanghai, China. Residents who underwent facilitated self-service fundus disease screening at one of the screening sites were assigned to the exposure group; those who were screened with a traditional fundus camera operated by an optometrist at an adjacent site comprised the control group. The primary outcome was the screening service quality, including effectiveness (image quality and screening efficiency), physiological discomfort, safety, convenience, and trustworthiness. The secondary outcome was the participants' preferences. Differences in service quality and the participants' preferences between the 2 groups were compared using chi-square tests separately. Subgroup analyses for exploring the relationships between the screening service's quality and residents' preference were conducted using generalized logit models. A total of 358 residents enrolled; among them, 176 (49.16%) were included in the exposure group and the remaining 182 (50.84%) in the control group. Residents' basic characteristics were balanced between the 2 groups. There was no significant difference in service quality between the 2 groups (image quality pass rate: P=.79; average screening time: P=.57; no physiological discomfort rate: P=.92; safety rate: P=.78; convenience rate: P=.95; trustworthiness rate: P=.20). However, the proportion of participants who were willing to use the same technology for their next screening was significantly lower in the exposure group than in the control group (P<.001). Subgroup analyses suggest that distrust in the facilitated self-service eye screening might increase the probability of refusal to undergo screening (P=.02). This study confirms that the facilitated self-service fundus disease screening pattern could achieve good service quality. However, it was difficult to reverse residents' preferences for manual screening in a short period, especially when the original manual service was already excellent. Therefore, the digital transformation of health care must be cautious. We suggest that attention be paid to the residents' individual needs. More efficient man-machine collaboration and personalized health management solutions based on large language models are both needed.