Zoom Camera Research Articles

Developing a self-calibrating system for volume measurement of spheroidal particles using two acoustically levitated droplets.

Acoustically levitated droplets in the nanoliter to microliter range are studied in various fields. The volume measurements of these are conventionally done using image analysis. A precision-produced calibration sphere is often used to calibrate the recording equipment, which is time-consuming and expensive. This paper describes a self-calibrating method to measure the volumes of acoustically levitated droplets as a versatile and low-cost alternative. The distance between two levitated droplets in a horizontally oriented acoustic trap is processed via real-time or recorded frame data using image analysis. To assist in setting the cavity length for the acoustic trap, a simulation of the acoustic field is utilized based on the temperature in the trap, thereby also predicting the distance between the central nodes used to determine the scale factor. The volumes of the spheroidal-shaped levitated droplets can then be calculated from the pixel data. We use a modified version of the well-known TinyLev, and our method has been tested with two types of transducer packing. Its accuracy for volume measurements has been verified in comparison with the standard calibration sphere technique. Self-calibration of the system is demonstrated by changing the camera zoom during data collection, with negligible effects on measured volume. This is something that could not be achieved with conventional static methods.

Online Optical Axis Parallelism Measurement Method for Continuous Zoom Camera Based on High-Precision Spot Center Positioning Algorithm

Ensuring precise alignment of the optical axis is critical for achieving high-quality imaging in continuous zoom cameras. However, existing methods for measuring optical axis parallelism often lack accuracy and fail to assess parallelism across the entire focal range. This study introduces an online measurement method designed to address these limitations by incorporating two enhancements. First, image processing methodologies enable sub-pixel-level extraction of the spot center, achieved through improved morphological processing and the incorporation of an edge tracing algorithm. Second, measurement software developed using Qt Creator can output real-time data on optical axis parallelism across the full focal range post-measurement. This software features a multi-threaded architecture that facilitates the concurrent execution of image acquisition, data processing, and serial communication. Experimental results derived from simulations and real data indicate that the maximum average error in extracting the center of the spot is 0.13 pixels. The proposed system provides critical data for optical axis calibration during camera adjustment and inspection.

Pupil-centric imaging model for zoom camera calibration.

Zoom camera calibration has always been challenging, as arbitrary zoom/focus settings change the camera parameters. Current calibration methods are based on the pinhole imaging model, which results in coupled calibration parameters due to the lack of geometric/physical constraints. In this Letter, we present a novel, to the best of our knowledge, pupil-centric imaging model that accounts for the camera's radial, decentering, and mustache distortions at various zoom settings using exit pupil offsets and a non-frontal lens-sensor model. Therefore, we provide a reasonable physical explanation for the different distortion effects. Global optimization is performed based on the proposed initial camera calibration and bundle adjustment under several zoom and autofocus setting combinations. Experiments using three representative zoom cameras demonstrate the effectiveness of the proposed method. Its relative measurement accuracy is better than that of current state-of-the-art methods.

Focus-switchable piezoelectric actuator: A bionic thin-plate design inspired by conch structure

Traditional camera zoom mechanisms, relying on stepper motors and intricate transmission components, suffer from bulky designs that restrict their applicability in compact systems. This study presents a novel piezoelectric focus-switchable mechanism (PFSM) directly driven by a biomimetic radial-mode piezoelectric actuator (RMPA) with inclined driving structure mimicking the conch shell. The PFSM utilizes rotational motion for optical zoom, providing a more compact and efficient alternative to conventional linear motion-based systems. By utilizing finite element method (FEM) optimization, we developed a compact prototype (34 × 34 × 3 mm³, 14.83 g) and experimentally verified its performance, achieving a peak rotational speed of 1573.14 RPM, torque output of 3.6 mN·m, and step displacement resolution of 45.8 μrad. These attributes enable smooth lens module switching for optical zoom, thus demonstrating the mechanism’s feasibility. Importantly, the PFSM offers precise positioning without relying on additional transmission parts and features a self-locking capability when de-energized, rendering it an ideal choice for camera zoom applications. In summary, this study underscores the PFSM’s potential as a compact, lightweight, and efficient camera zoom mechanism, contributing meaningfully to the field of imaging technology and optical systems.

Observations of Igneous Subsurface Stratigraphy during the Jezero Crater Floor Rapid Traverse from the RIMFAX Ground-penetrating Radar

Perseverance traversed the eastern, northern, and western margins of the Séítah formation inlier on the rover’s western fan front approach. Mapping the stratigraphy and extent of the Máaz and Séítah formations is key to understanding the depositional history and timing of crater floor resurfacing events. Perseverance's rapid progress across the Jezero crater floor between the Octavia E. Butler landing site and the western fan front resulted in limited contextual images of the deposits from the Navigation Camera and Mast Camera Zoom. By combining the limited surface images with continuous subsurface sounding by the Radar Imager for Mars’ Subsurface Experiment (RIMFAX) ground-penetrating radar, Jezero crater floor stratigraphy was inferred along this rapid traverse. We produced the first subsurface map of the Máaz formation thickness and elevation of the buried Séítah formation for 2.3 km of the rapid traverse. Three distinct reflector packets were observed in RIMFAX profiles interspersed with regions of low-radar reflectivity. We interpret these reflector packets with increasing depth to be the Roubion member of the Máaz formation (covered in places with regolith), the Rochette member, and the Séítah formation. We found a median permittivity of 9.0 and bulk density of 3.2 g cm−3 from hyperbola fits to RIMFAX profiles, which suggests a mafic composition for Máaz and Séítah. The low-radar reflectivity regions within each reflector packet could indicate potential depositional hiatuses where low-density material like sediment or regolith could have accumulated between successive Máaz formation lava flows and the Séítah formation at depth.

Data-driven hierarchical learning approach for multi-point servo control of Pan–Tilt–Zoom cameras

Pan–Tilt–Zoom (PTZ) cameras, with their significant features of free rotation and zoom, are widely used in areas such as border security, ecological conservation, emergency management, and the military. PTZ cameras can achieve automatic monitoring of a selected area through multiple servo operations, known as multi-point servo control. However, due to the deficiencies in the servo control Software Development Kit (SDK), hardware wear, and interference from complex external environments, the multi-point servo control process generates significant errors. This paper proposes a precise multi-point servo control framework based on Deep Reinforcement Learning (DRL) to address this issue. The complexity of real-world environments necessitates a reward function design that fully considers various factors, for which we propose the directional gravity reward function. Due to the instability during the training process, prolonged trial-and-error interactions between the agent and the equipment can cause irreversible damage to the devices. This framework employs a phased training approach, where the agent learns sequentially from offline, off-policy, and on-policy data, reducing direct interaction with the equipment while enhancing the agent’s overall performance. Additionally, real-time and accurate device status can be obtained by performing feature matching on images from adjacent time frames, which is crucial for the system’s operation. Evaluation results indicate that our proposed precise multi-point servo control framework significantly outperforms other methods in 4-point servo control tasks in both virtual and real-world scenarios. Additionally, the operational process fully considers safety issues.

Negotiating Visibility: Mediating Presence through Zoom Camera Choices in Post-Secondary Students during COVID-19

Students at a large and socially diverse urban university completed an anonymous survey examining online learning experiences during the COVID-19 pandemic with an emphasis on decisions to keep their cameras on or off during synchronous class. The 505 student respondents used 7-point scales to assess their school performance and everyday life experiences during the pandemic, general classroom values, pre-pandemic and current pandemic experiences, technological proficiencies related to Zoom, and camera on/off attitudes, as well as the online behavior of professors, and the role of social media in their everyday lives. The findings underscored two motivations underlying school engagement. Students could be motivated by a need for belonging involving authentic self-presentation while experiencing the emotional presence of others, and/or be instrumentally motivated by a need to perform well and advance their careers. The importance of professors creating a safe online space to foster a sense of belonging was highlighted. Finally, the findings show that feelings about having one’s camera on or off during online classes are related to everyday social media experiences. The social-emotional and pragmatic aspects of university education are complementary facets of a university experience.

Towards Fully Autonomous Drone Tracking by a Reinforcement Learning Agent Controlling a Pan–Tilt–Zoom Camera

Towards Fully Autonomous Drone Tracking by a Reinforcement Learning Agent Controlling a Pan–Tilt–Zoom Camera

Active Visual Perception Enhancement Method Based on Deep Reinforcement Learning

Traditional object detection methods using static cameras are constrained by their limited perspectives, hampering the effective detection of low-confidence targets. To address this challenge, this study introduces a deep reinforcement learning-based visual perception enhancement technique. This approach leverages pan–tilt–zoom (PTZ) cameras to achieve active vision, enabling them to autonomously make decisions and actions tailored to the current scene and object detection outcomes. This optimization enhances both the object detection process and information acquisition, significantly boosting the intelligent perception capabilities of PTZ cameras. Experimental findings demonstrate the robust generalization capabilities of this method across various object detection algorithms, resulting in an average confidence level improvement of 23.80%.

Bidirectional Temporal Pose Matching for Tracking

Multi-person pose tracking is a challenging task. It requires identifying the human poses in each frame and matching them across time. This task still faces two main challenges. Firstly, sudden camera zooming and drastic pose changes between adjacent frames may result in mismatched poses between them. Secondly, the time relationships modeled by most existing methods provide insufficient information in scenarios with long-term occlusion. In this paper, to address the first challenge, we propagate the bounding boxes of the current frame to the previous frame for pose estimation, and match the estimated results with the previous ones, which we call the Backward Temporal Pose-Matching (BTPM) module. To solve the second challenge, we design an Association Across Multiple Frames (AAMF) module that utilizes long-term temporal relationships to supplement tracking information lost in the previous frames as a Re-identification (Re-id) technique. Specifically, we select keyframes with a fixed step size in the videos and label other frames as general frames. In the keyframes, we use the BTPM module and the AAMF module to perform tracking. In the general frames, we propagate poses in the previous frame to the current frame for pose estimation and association, which we call the Forward Temporal Pose-Matching (FTPM) module. If the pose association fails, the current frame will be set as a keyframe, and tracking will be re-performed. In the PoseTrack 2018 benchmark tests, our method shows significant improvements over the baseline methods, with improvements of 2.1 and 1.1 in mean Average Precision (mAP) and Multi-Object Tracking Accuracy (MOTA), respectively.

IoT-Based Strawberry Disease Detection With Wall-Mounted Monitoring Cameras

This paper proposes StrawberryTalk, an Internet of Things (IoT) platform for image-based strawberry disease detection. StrawberryTalk reuses the wall-mounted monitoring cameras without extra hardware cost. The contributions of StrawberryTalk are the utilization of IoT for automatic photo shoot and the wind detection mechanism to eliminate the obscure photos due to the wind effects. Also, the data preprocessing and the infection detection models are manipulated as IoT devices to simplify the implementation of the multi-cascade Artificial Intelligence (AI) models. We derive the relationship between the camera zoom factor and the distance between the camera and the strawberries for optimal disease detection. Accuracy of detection may be affected by obscure photos. In terms of eliminating obscure photos due to wind effect, we analytically derive the relationship between the wind alert delay and the number of obscure photos that must be retaken. For the greenhouses in the Bao Mountain, we only need to retake one photo. Based on the experiments, the mean average precision (mAP) of StrawberryTalk (to detect exact spots in a leaf) can be up to 92.37%, which is better than the previous approaches. To detect if a pot has infected leaves, the accuracy of StrawberryTalk can be up to 97.92% at the zoom factor 30X. In commercial operation, it is important to detect all infected strawberry pots. StrawberryTalk is able to detect all infected pots (i.e., recall is 100%) with a camera of zoom factor of 12X. The accuracy is 96.88%.

Self-Supervised Learning for Real-World Super-Resolution from Dual and Multiple Zoomed Observations.

In this paper, we consider two challenging issues in reference-based super-resolution (RefSR) for smartphone, (i) how to choose a proper reference image, and (ii) how to learn RefSR in a self-supervised manner. Particularly, we propose a novel self-supervised learning approach for real-world RefSR from observations at dual and multiple camera zooms. Firstly, considering the popularity of multiple cameras in modern smartphones, the more zoomed (telephoto) image can be naturally leveraged as the reference to guide the super-resolution (SR) of the lesser zoomed (ultra-wide) image, which gives us a chance to learn a deep network that performs SR from the dual zoomed observations (DZSR). Secondly, for self-supervised learning of DZSR, we take the telephoto image instead of an additional high-resolution image as the supervision information, and select a center patch from it as the reference to super-resolve the corresponding ultra-wide image patch. To mitigate the effect of the misalignment between ultra-wide low-resolution (LR) patch and telephoto ground-truth (GT) image during training, we first adopt patch-based optical flow alignment to obtain the warped LR, then further design an auxiliary-LR to guide the deforming of the warped LR features. To generate visually pleasing results, we present local overlapped sliced Wasserstein loss to better represent the perceptual difference between GT and output in the feature space. During testing, DZSR can be directly deployed to super-solve the whole ultra-wide image with the reference of the telephoto image. In addition, we further take multiple zoomed observations to explore self-supervised RefSR, and present a progressive fusion scheme for the effective utilization of reference images. Experiments show that our methods achieve better quantitative and qualitative performance against state-of-the-arts. The code and pre-trained models will be publicly available.

Effectiveness of optical, digital, and hybrid zoom equipped drones for use in reading livestock ear tags for individual animal identification

Predicting how advertised zoom capabilities of commercially available drones being deployed for animal management will perform can be difficult, as promotional and marketing materials supplied by the manufacturer do not necessarily reflect real-world performance. We compared our ability to read livestock ear tags used for individual animal identification using various drone models with differing zoom capabilities. Drone models were assessed at various distances using a veterinary bovine head model to determine their ability to read livestock ear tags of various colours and sizes, and to establish observational distance limits. Results indicate that while drones that primarily utilize optical zoom are preferable, newer model drones equipped with hybrid zoom cameras that utilize computational photography are superior to 5-year-old drone models equipped with only digital zoom cameras. Recently released drone models are now capable of reading livestock ear tags at distances exceeding 60 m and perform equivalent to binoculars in terms of discerning numbers printed on various coloured livestock ear tags.

AEFormer: Zoom Camera Enables Remote Sensing Super-Resolution via Aligned and Enhanced Attention

Reference-based super-resolution (RefSR) has achieved remarkable progress and shows promising potential applications in the field of remote sensing. However, previous studies heavily rely on existing and high-resolution reference image (Ref), which is hard to obtain in remote sensing practice. To address this issue, a novel structure based on a zoom camera structure (ZCS) together with a novel RefSR network, namely AEFormer, is proposed. The proposed ZCS provides a more accessible way to obtain valid Ref than traditional fixed-length camera imaging or external datasets. The physics-enabled network, AEFormer, is proposed to super-resolve low-resolution images (LR). With reasonably aligned and enhanced attention, AEFormer alleviates the misalignment problem, which is challenging yet common in RefSR tasks. Herein, it contributes to maximizing the utilization of spatial information across the whole image and better fusion between Ref and LR. Extensive experimental results on benchmark dataset RRSSRD and real-world prototype data both verify the effectiveness of the proposed method. Hopefully, ZCS and AEFormer can enlighten a new model for future remote sensing imagery super-resolution.

An adaptive crack inspection method for building surface based on BIM, UAV and edge computing

Unmanned aerial vehicle (UAV) and building information modeling (BIM) have been applied to inspect building surfaces to improve efficiency and reduce labor costs. However, most of the current research focus on the inspection mode of “first collect and then identify” with limited intelligence. Therefore, this study proposes a “global-local” adaptive inspection method for building surfaces by integrating BIM, UAV (with multifunction camera), and edge computing, enabling real-time synchronization of crack data collection and analysis. First, the building surface area to be inspected is extracted from BIM to generate global inspection points using the selected camera's field of view (FOV) and considering coordinates transformation between BIM and reality. Then, A star algorithm integrating with genetic algorithm (GA) is used to optimize the global flight path based on generated inspection points. Finally, the coordinates of the inspection points based on BIM are converted into real-world coordinates, and the flight parameters such as the flight yaw angle (FYA) and the gimbal yaw angle (GYA) are calculated to generate the inspection mission. During global inspection, a local adaptive inspection is proposed for each FOV through real-time local crack identification and flight replanning using edge computing and zoom camera. In addition, multiple battery supply points are automatically added considering the endurance of the UAV. A real building has been used to validate the proposed “global-local” adaptive inspection method. The experiment results show that the inspection time has been significantly reduced compared with the traditional method, and the crack identification of building surfaces can be implemented with high efficiency, precision, quality, and low cost.

Marker-Based Localization System Using an Active PTZ Camera and CNN-Based Ellipse Detection

Localization in GPS-denied environments is challenging, and many existing solutions have infrastructural and on-site calibration requirements. This article tackles these challenges by proposing a localization system that is infrastructure free and does not require on-site calibration, using a single active pan–tilt–zoom camera to detect, track, and localize a circular LED marker. We propose to use a convolutional neural network (CNN) trained using only synthetic images to detect the LED marker as an ellipse and show that our approach is more robust than using traditional ellipse detection without requiring tuning of parameters for feature extraction. We also propose to leverage the predicted elliptical angle as a measure of uncertainty of the CNN's predictions and show how it can be used in a filter to improve marker range estimation and 3-D localization. We evaluate our system's performance through localization of a unmanned aerial vehicle in real-world flight experiments and show that it can outperform alternative methods for localization in GPS-denied environments. We also demonstrate our system's performance in indoor and outdoor environments.

Diverse Lava Flow Morphologies in the Stratigraphy of the Jezero Crater Floor

AbstractWe present a combined geomorphologic, multispectral, and geochemical analysis of crater floor rocks in Jezero crater based on data obtained by the Mast Camera Zoom and SuperCam instruments onboard the NASA Mars 2020 Perseverance rover. The combined data from this analysis together with the results of a comparative study with geologic sites on Earth allows us to interpret the origins of rocks exposed along the Artuby ridge, a ∼900 m long scarp of lower Máaz formation rocks. The ridge exposes rocks belonging to two morphologically distinct members, Artuby and Rochette, both of which have basaltic composition and are spectrally indistinguishable in our analysis. Artuby rocks consist of morphologically distinct units that alternate over the ridge, bulbous, hummocky, layers with varying thicknesses that in places appear to have flowed over underlying strata, and sub‐planar thinner laterally continuous layers with variable friability. The Rochette member has a massive appearance with pronounced pitting and sub‐horizontal partings. Our findings are most consistent with a primary igneous emplacement as lava flows, through multiple eruptions, and we propose that the thin layers result either from preferential weathering, interbedded ash/tephra layers, ʻaʻā clinker layers, or aeolian deposition. Our analyses provide essential geologic context for the Máaz formation samples that will be returned to Earth and highlight the diversity and complexity of geologic processes on Mars not visible from orbit.

Deep learning enables parallel camera with enhanced- resolution and computational zoom imaging

High performance imaging in parallel cameras is a worldwide challenge in computational optics studies. However, the existing solutions are suffering from a fundamental contradiction between the field of view (FOV), resolution and bandwidth, in which system speed and FOV decrease as system scale increases. Inspired by the compound eyes of mantis shrimp and zoom cameras, here we break these bottlenecks by proposing a deep learning-based parallel (DLBP) camera, with an 8-μrad instantaneous FOV and 4 × computational zoom at 30 frames per second. Using the DLBP camera, the snapshot of 30-MPs images is captured at 30 fps, leading to orders-of-magnitude reductions in system complexity and costs. Instead of directly capturing photography with large scale, our interactive-zoom platform operates to enhance resolution using deep learning. The proposed end-to-end model mainly consists of multiple convolution layers, attention layers and deconvolution layer, which preserves more detailed information that the image reconstructs in real time compared with the famous super-resolution methods, and it can be applied to any similar system without any modification. Benefiting from computational zoom without any additional drive and optical component, the DLBP camera provides unprecedented-competitive advantages in improving zoom response time (~ 100 ×) over the comparison systems. Herein, with the experimental system described in this work, the DLBP camera provides a novel strategy to solve the inherent contradiction among FOV, resolution and bandwidth.

Human Detection from Drone using You Only Look Once (YOLOv5) for Search and Rescue Operation

Drones are unmanned aerial vehicles that can be remotely operated to perform a variety of tasks. They have been used in search and rescue operations since the early 2000s and have proven to be invaluable tools for quickly locating missing persons in difficult terrain and environment. In certain cases, automated human detection on drone camera feed can help the responder to locate the victims more effectively. In this work, we propose the use of a deep learning method called You Only Look Once version 5, or YOLOv5. The YOLOv5 model is trained using data collected during a simulation of search and rescue operations, where mannequins were used to represent human victims. Video was acquired using DJI Matrice 300 drone with Zenmuse H20T camera which flew around an area with various terrains such as farms, ravines, and river of more than 15,000 m2, at a height of 40 meters. The drone used grid, circular and zigzag flying patterns, with three different levels of camera zooms, and the data was captured on different days and times. The total duration of the video collected at 1080p@30fps is 148 minutes 26 seconds. Five pretrained models of YOLOv5 with different complexities were trained and tested using this dataset. Results showed that pretrained yolov5l6 model delivered the best precision, recall and mAP50 rate at 0.668, 0.303 and 0.346 respectively. Besides, the experiment also showed that we can improve the overall performance by using images acquired at 6x zoom magnification level where precision, recall, and mAP50 rate are increased to 0.846, 0.543, and 0.591 respectively. yolov5l6 model also delivered an acceptable inference time of 43ms per 1920x1080 resolution image, thus it can run at a respectable 23fps.

Patient Experience with Video Telemedicine Neurology Follow-up Visits at Community Clinics (P1-4.001)

<h3>Objective:</h3> To assess patient satisfaction and perception of physician empathy following implementation of video telemedicine neurology follow-up visits at community clinics <h3>Background:</h3> There is a growing shortage of neurologists in the US who provide care outside of metropolitan areas[1]. Within the region of our academic medical center, neurologists drive approximately 2 hours round-trip each day to provide care at community clinics. Telemedicine minimizes travel time for both patients and physicians while maintaining/improving access. We aimed to develop a telemedicine service and assess patient experience. <h3>Design/Methods:</h3> We implemented video conferencing with tablet for patient to interact with neurologist and a pan and tilt zoom camera attached to a moveable cart to enable neurologist to perform neurological exam. Telemedicine visits were offered to patients who had previously been seen face-to-face by the neurologist performing follow-up. Implementation was at a clinic 45 miles from tertiary center(site 1) between February 2020–January 2021 and at a clinic 43 miles from tertiary center(site 2) between July 2021–January 2022. After the telemedicine visit, a survey was completed by patient and/or caregiver, which included the consultation and relational empathy(CARE)[1] and telemedicine patient satisfaction measures(TPSM)[2]. <h3>Results:</h3> 31 patients from site 1 and 38 patients from site 2 participated in telemedicine follow-up visit and completed the survey. Mean age was 66 years. Follow-up reasons were Parkinson disease/parkinsonism(36.2%), migraine(20.3%), dementia(17.4%), epilepsy(14.5%), and other(11.6%). CARE measure mean responses indicated patients’ perception of relational empathy was excellent(65%), and very good(25%). TPSM mean responses were strongly agreed(71%) and agreed(28%) with regard to patient satisfaction. The neurologists involved anecdotally reported improved work-life balance. <h3>Conclusions:</h3> Patient satisfaction and perceived physician empathy was high for video telemedicine neurology follow-up at our community clinics, supporting expansion of this model. This model also improved work-life balance for the neurologist. Study of care outcomes and use for new patient consultations is needed. <b>Disclosure:</b> Dr. Truitt has stock in NVTA. Dr. Truitt has stock in QTRX. Dr. Truitt has stock in EXAS. Dr. Truitt has stock in ALC. Dr. Truitt has stock in ACAD. Dr. Truitt has stock in ABBV. Dr. Truitt has stock in NVS. Dr. Truitt has stock in VRTX. Dr. Truitt has stock in MRK. Dr. Truitt has stock in ATRI. Dr. Truitt has stock in HZNP. Dr. Truitt has stock in ZM. Dr. Arumaithurai has nothing to disclose. Dr. Young has nothing to disclose.