Camera Frame Research Articles

On methods of setting the coordinates of the mouse pointer in the case of alternative options for constructing human-computer interaction

Objective. The aim of the study is to compare and generalize different methods of contactless setting of computer mouse pointer coordinates. Method. To create a contactless human-machine interface, computer vision technologies are used to select a control point on a video camera frame, with the help of which the computer cursor is moved on the user's monitor. General logical methods of scientific research, namely, generalization, are used. An analysis of the considered cursor coordinate control systems is carried out with the allocation of common parts for further generalization and description of formulas for setting the coordinates of the computer mouse cursor. Scientific works with absolute positioning of the computer cursor and works with control of the cursor position shift relative to its coordinates on the previous frame were taken for analysis. Result. As a result of the analysis, strengths and weaknesses of each of the approaches under consideration were indicated. Absolute positioning requires less time to set the cursor to the desired position. It is determined that cursor shift control is not so demanding on the accuracy of calculating the coordinates of the control point. Formulas are given that generalize the methods of moving the computer cursor. Conclusion. The proposed approach to calculating new coordinates of the mouse pointer allows implementing both presented approaches to positioning the computer cursor in the software system, since each of them has advantages depending on the specific situation.

Improvements to monoscopic analysis for imaging atmospheric Cherenkov telescopes: Application to H.E.S.S.

Imaging atmospheric Cherenkov telescopes (IACTs) detect rays by measuring the Cherenkov light emitted by secondary particles in the air shower when the rays hit the atmosphere of the Earth. Given usual distances between telescopes in IACT arrays, at low energies (lesssim 100 GeV), the limited amount of Cherenkov light produced typically implies that the event is registered by one IACT only. Such events are called monoscopic events, and their analysis is particularly difficult. Challenges include the reconstruction of the event's arrival direction, energy, and the rejection of background events due to charged cosmic rays. Here, we present a set of improvements, including a machine-learning algorithm to determine the correct orientation of the image in the camera frame, an intensity-dependent selection cut that ensures optimal performance across all energies, and a collection of new image parameters. To quantify these improvements, we make use of simulations and data from the 28-m diameter central telescope of the IACT array. Knowing the correct image orientation, which corresponds to the arrival direction of the photon in the camera frame, is especially important for the angular reconstruction, which could be improved in resolution by 57% at 100,GeV. The event selection cut, which now depends on the total measured intensity of the events, leads to a reduction of the low-energy threshold for source analyses by ∼50%. The new image parameters characterize the intensity and time distribution within the recorded images and complement the traditionally used Hillas parameters in the machine learning algorithms. We evaluate their importance to the algorithms in a systematic approach and carefully evaluate associated systematic uncertainties. We find that including subsets of the new variables in machine-learning algorithms improves the reconstruction and background rejection, resulting in a sensitivity improved by 41% at the low-energy threshold. Finally, we apply the new analysis to data from the Crab Nebula and estimate systematic uncertainties introduced by the new method.

A comparison of machine learning methods for queue length detection

Queues are formed by people waiting for a service in public institutions and they can be defined as orderly groups of people. Automatically counting the number of people waiting in a queue through video camera footage would provide these institutions with valuable information with regards to customer service quality. In this paper, our goal is to compare several machine learning methods for finding the total number of people waiting in a queue given video camera frames. We approached this problem as a regression task. We used a subset of the Collective Activity Dataset and compared three different methods. The first two methods used bounding box coordinates and orientations provided by the dataset, while the last method utilized the bounding box coordinates to extract feature maps from the frames using RoiAlign. The first method used XGBoost, while the latter methods used Convolutional Neural Networks (CNNs). Results show that the method using RoiAlign presents the best prediction performance in terms of mean squared error and mean absolute error, compared to other methods.

Interaction of the Shock Train Leading Edge and Filamentary Plasma in a Supersonic Duct

Quasi-direct current (Q-DC) filamentary electrical discharges are used to control the shock train in a back-pressured Mach 2 duct flow. The coupled interaction between the plasma filaments and the shock train leading edge (STLE) is studied for a variety of boundary conditions. Electrical parameters associated with the discharge are recorded during actuation, demonstrating a close correlation between the STLE position and dynamics. High-speed self-aligned focusing schlieren (SAFS) and high frame-rate color camera imaging are the primary optical diagnostics used to study the flowfield and plasma morphology. Shock tracking and plasma characterization algorithms are employed to extract time-resolved quantitative data during shock–plasma interactions. Four distinct shock–plasma interaction types are identified and outlined, revealing a strong dependence on the spacing between the uncontrolled STLE and discharge electrodes and a moderate dependence on flow parameters.

Pix2HDR - A Pixel-Wise Acquisition and Deep Learning-Based Synthesis Approach for High-Speed HDR Videos.

Accurately capturing dynamic scenes with wide-ranging motion and light intensity is crucial for many vision applications. However, acquiring high-speed high dynamic range (HDR) video is challenging because the camera's frame rate restricts its dynamic range. Existing methods sacrifice speed to acquire multi-exposure frames. Yet, misaligned motion in these frames can still pose complications for HDR fusion algorithms, resulting in artifacts. Instead of frame-based exposures, we sample the videos using individual pixels at varying exposures and phase offsets. Implemented on a monochrome pixel-wise programmable image sensor, our sampling pattern captures fast motion at a high dynamic range. We then transform pixel-wise outputs into an HDR video using end-to-end learned weights from deep neural networks, achieving high spatiotemporal resolution with minimized motion blurring. We demonstrate aliasing-free HDR video acquisition at 1000 FPS, resolving fast motion under low-light conditions and against bright backgrounds - both challenging conditions for conventional cameras. By combining the versatility of pixel-wise sampling patterns with the strength of deep neural networks at decoding complex scenes, our method greatly enhances the vision system's adaptability and performance in dynamic conditions.

Vibration monitoring of rotating shafts using DIC and compressed sensing

Monitoring vibrations in rotating shafts is essential for diagnosing and detecting mechanical faults. Digital image correlation (DIC), an optical full-field measurement technique, is increasingly employed in experimental mechanics. This study introduces a novel measurement approach that combines DIC with compressed sensing to monitor high-speed rotating shafts accurately. Traditionally, analyzing rotating shafts requires high-speed sampling devices, which increases experimental costs and reduces spatial resolution. To overcome these limitations, a random exposure sampling method was developed to capture speckle images on the shaft surface with a low frame-rate camera. By leveraging compressed sensing and DIC, the method reconstructs high-speed vibration signals from captured images. Step motion experiments demonstrated that the DIC system achieves a measurement accuracy of 10 µm. Experimental validation was conducted using various setups, including low-speed motors, high-speed rotating shafts, and milling machines, demonstrating the effectiveness of the approach. The measurement results of rotating shafts at 1200 rpm and 6000 rpm, compared with those obtained from laser Doppler vibrometry, demonstrated the effectiveness of the method in vibration measurement. Additionally, experiments on a milling machine showed that vibrations reconstructed using compressed sensing closely matched those measured with a high-speed camera. This measurement system shows significant potential for accurately assessing vibrations in high-speed rotating shafts, offering valuable insights for machinery monitoring and fault diagnosis.

MOVRO2: Loosely coupled monocular visual radar odometry using factor graph optimization

Ego-motion estimation is an indispensable part of any autonomous system, especially in scenarios where wheel odometry or global pose measurement is unreliable or unavailable. In an environment where a global navigation satellite system is not available, conventional solutions for ego-motion estimation rely on the fusion of a LiDAR, a monocular camera and an inertial measurement unit (IMU), which is often plagued by drift. Therefore, complementary sensor solutions are being explored instead of relying on expensive and powerful IMUs. In this paper, we propose a method for estimating ego-motion, which we call MOVRO2, that utilizes the complementarity of radar and camera data. It is based on a loosely coupled monocular visual radar odometry approach within a factor graph optimization framework. The adoption of a loosely coupled approach is motivated by its scalability and the possibility to develop sensor models independently. To estimate the motion within the proposed framework, we fuse ego-velocity of the radar and scan-to-scan matches with the rotation obtained from consecutive camera frames and the unscaled velocity of the monocular odometry. We evaluate the performance of the proposed method on two open-source datasets and compare it to various mono-, dual- and three-sensor solutions, where our cost-effective method demonstrates performance comparable to state-of-the-art visual-inertial radar and LiDAR odometry solutions using high-performance 64-line LiDARs.

Automated cell tracking using 3D nnUnet and Light Sheet Microscopy to quantify regional deformation in zebrafish.

Light Sheet Microscopy (LSM) in conjunction with embryonic zebrafish, is rapidly advancing three-dimensional, in vivo characterization of myocardial contractility. Preclinical cardiac deformation imaging is predominantly restricted to a low-order dimensionality image space (2D) or suffers from poor reproducibility. In this regard, LSM has enabled high throughput, non-invasive 4D (3d+time) characterization of dynamic organogenesis within the transparent zebrafish model. More importantly, LSM offers cellular resolution across large imaging Field-of-Views at millisecond camera frame rates, enabling single cell localization for global cardiac deformation analysis. However, manual labeling of cells within multilayered tissue is a time-consuming task and requires substantial expertise. In this study, we applied the 3D nnU-Net with Linear Assignment Problem (LAP) framework for automated segmentation and tracking of myocardial cells. Using binarized labels from the neural network, we quantified myocardial deformation of the zebrafish ventricle across 4-6 days post fertilization (dpf). Our study offers tremendous promise for developing highly scalable and disease-specific biomechanical quantification of myocardial microstructures.

Dynamic digital image correlation method for rolling convective contact

Digital image correlation (DIC) is an increasingly popular and effective non-contact method for measuring full-field displacements and strains of deformable bodies under load. Current DIC methods applied to bodies undergoing large displacements and rotations require a large measurement area for both the reference (i.e., undeformed) image and the deformed images. This can limit the resulting resolution of the displacement and strain fields. To address this issue, we propose a two-stage dynamic DIC method capable of measuring displacements and strains under material convection with high resolution. During the first stage, the reference image is assembled from smaller, high-resolution images of the undeformed object obtained using a spatially-fixed or moving frame. Following capture, each sub-image is rigidly translated and rotated into its appropriate place, thereby producing a full, high-resolution image of the reference body. In stage two, images of the loaded and deformed body, again obtained using a small camera frame with high resolution, are aligned with matching regions of the undeformed composite image using BRISK feature detection before performing DIC. We demonstrate the method on a contact problem whereby an elastomeric roller travels along a rigid surface. In doing so, we obtain fine-resolution measurements of the state of strain of the region of the roller sidewall in contact with the substrate, even as new material convects through the region of interest. We present these measurements as a series of images and videos capturing strain evolution as the roller transitions from static loads to a fully dynamic steady-state, documenting the effectiveness of the method.

End-to-end training of acoustic scene classification using distributed sound-to-light conversion devices: verification through simulation experiments

We propose a framework for classifying acoustic scenes utilizing distributed sound sensor devices capable of sound-to-light conversion, which we term as Blinkies. These Blinkies can convert acoustic signals into varying intensities of light via an inbuilt light-emitting diode. By using Blinkies, we can aggregate the spatial acoustic information across a wide region by recording the fluctuating light intensities of numerous Blinkies distributed throughout the region. Nonetheless, the signal communicated is subject to the bandwidth limitation imposed by the frame rate of the video camera, typically capped at 30 frames per second. Our objective is to refine the process of transforming sound into light for the purpose of acoustic scene classification within these bandwidth confines. While traversing through the air, a light signal is affected by inherent physical limitations such as the attenuation of light and interference from noise. To account for these factors, we have integrated these physical constraints into differentiable physical layers. This approach enables us to jointly train a pair of deep neural networks for the conversion of sound to light and for the classification of acoustic scenes. Our simulation studies, which employed the SINS database for acoustic scene classification, demonstrated that our proposed framework outperforms the previous one that utilized Blinkies. These findings emphasize the effectiveness of Blinkies in the field of acoustic scene classification.

Potential of Non-Contact Dynamic Response Measurements for Predicting Small Size or Hidden Damages in Highly Damped Structures.

Vibration-based structural health monitoring (SHM) is essential for evaluating structural integrity. Traditional methods using contact vibration sensors like accelerometers have limitations in accessibility, coverage, and impact on structural dynamics. Recent digital advancements offer new solutions through high-speed camera-based measurements. This study explores how camera settings (speed and resolution) influence the accuracy of dynamic response measurements for detecting small cracks in damped cantilever beams. Different beam thicknesses affect damping, altering dynamic response parameters such as frequency and amplitude, which are crucial for damage quantification. Experiments were conducted on 3D-printed Acrylonitrile Butadiene Styrene (ABS) cantilever beams with varying crack depth ratios from 0% to 60% of the beam thickness. The study utilised the Canny edge detection technique and Fast Fourier Transform to analyse vibration behaviour captured by cameras at different settings. The results show an optimal set of camera resolutions and frame rates for accurately capturing dynamic responses. Empirical models based on four image resolutions were validated against experimental data, achieving over 98% accuracy for predicting the natural frequency and around 90% for resonance amplitude. The optimal frame rate for measuring natural frequency and amplitude was found to be 2.4 times the beam's natural frequency. The findings provide a method for damage assessment by establishing a relationship between crack depth, beam thickness, and damping ratio.

High speed fast-ion D-alpha spectrometer for the NSTX-U tokamak.

Here, we present the design and first calibration results of a new single-channel Fast-Ion D-Alpha (FIDA) spectrometer to be employed at the National Spherical Torus Experiment Upgrade (NSTX-U). The Czerny-Turner-type spectrometer uses a custom-designed aspherical lens setup instead of mirrors and achieves excellent spectral resolution, with high photon throughput through a round-to-linear fiber bundle, and camera frame rates around 8.4kHz. The spectrometer uses a blocking bar to avoid saturation effects of the cold D-alpha emission line and will allow for detailed studies of the fast-ion confinement in NSTX-U. Expected synthetic spectra predicted with the TRANSP and FIDASIM codes show that the spectral range from 648.5 to 658nm will sufficiently cover halo, the red-shifted beam emission, and the blue-shifted portion of FIDA emission in NSTX-U, which is sufficient for fast-ion transport studies of co-rotating fast ions.

ChromaFlash: Snapshot Hyperspectral Imaging Using Rolling Shutter Cameras

Hyperspectral imaging captures scene information across narrow, contiguous bands of the electromagnetic spectrum. Despite its proven utility in industrial and biomedical applications, its ubiquity has been limited by bulky form factors, slow capture times, and prohibitive costs. In this work, we propose a generalized approach to snapshot hyperspectral imaging that only requires a standard rolling shutter camera and wavelength-adjustable lighting. The crux of this approach entails using the rolling shutter as a spatiotemporal mask, varying incoming light quicker than the camera's frame rate in order for the captured image to contain rows of pixels illuminated at different wavelengths. An image reconstruction pipeline then converts this coded image into a complete hyperspectral image using sparse optimization. We demonstrate the feasibility of this approach by deploying a low-cost system called ChromaFlash, which uses a smartphone's camera for image acquisition and a series of LEDs to change the scene's illumination. We evaluated ChromaFlash through simulations on two public hyperspectral datasets and assessed its spatial and spectral accuracy across various system parameters. We also tested the real-world performance of our prototype by capturing diverse scenes under varied ambient lighting conditions. In both experiments, ChromaFlash outperformed state-of-the-art techniques that use deep learning to convert RGB images into hyperspectral ones, achieving snapshot performance not demonstrated by prior attempts at accessible hyperspectral imaging.

A novel detection method for warhead fragment targets in optical images under dynamic strong interference environments

A measurement system for the scattering characteristics of warhead fragments based on high-speed imaging systems offers advantages such as simple deployment, flexible maneuverability, and high spatiotemporal resolution, enabling the acquisition of full-process data of the fragment scattering process. However, mismatches between camera frame rates and target velocities can lead to long motion blur tails of high-speed fragment targets, resulting in low signal-to-noise ratios and rendering conventional detection algorithms ineffective in dynamic strong interference testing environments. In this study, we propose a detection framework centered on dynamic strong interference disturbance signal separation and suppression. We introduce a mixture Gaussian model constrained under a joint spatial-temporal-transform domain Dirichlet process, combined with total variation regularization to achieve disturbance signal suppression. Experimental results demonstrate that the proposed disturbance suppression method can be integrated with certain conventional motion target detection tasks, enabling adaptation to real-world data to a certain extent. Moreover, we provide a specific implementation of this process, which achieves a detection rate close to 100% with an approximate 0% false alarm rate in multiple sets of real target field test data. This research effectively advances the development of the field of damage parameter testing.

L-PBF High-Throughput Data Pipeline Approach for Multi-modal Integration

Metal-based additive manufacturing requires active monitoring solutions for assessing part quality. Multiple sensors and data streams, however, generate large heterogeneous data sets that are impractical for manual assessment and characterization. In this work, an automated pipeline is developed that enables feature extraction from high-speed camera video and multi-modal data analysis. The framework removes the need for manual assessment through the utilization of deep learning techniques and training models in a weakly supervised paradigm. We demonstrate this pipeline’s capability over 700,000 high-speed camera frames. The pipeline successfully extracts melt pool and spatter geometries and links them to corresponding pyrometry, radiography, and processparameter information. 715 individual prints are examined to reveal melt pool areas that exceeds 0.07 mm2 and pyrometry signal over a threshold (375 pyrometry units) were more likely to have defects. These automated processes enable massive throughput of characterization techniques.

High repetition-rate dual-channel X-ray spectrometer for high-intensity laser-plasma experiments.

We describe the development and demonstration of a high-repetition-rate-capable dual-channel (DC) x-ray spectrometer designed for high-intensity laser-plasma experiments (≥1×1021 W/cm2). The spectrometer, which operates at high repetition rates, is limited only by the refresh rate of targets and the camera's frame rate. It features two channels, each equipped with a flat highly oriented pyrolytic graphite (HOPG) crystal and a unique detector plane, allowing it to resolve two distinct x-ray bands: approximately 7-10 and 10-13keV. Each detector plate carrier holds two slots for active (scintillators) or passive (imaging plates) x-ray detectors. We present the design and testing of the HR-DC-HOPG using both the COMET laser (10J, 0.5ps shot/4min) at LLNL's Jupiter Laser Facility and the SCARLET laser (10J, 30fs shot/min) at Ohio State University. The results demonstrate the spectrometer's performance across various laser energies, target materials, pulse shapes, and detector types.

Humidity Effects on the Positive Leader Steps in Laboratory Long Spark Discharges

AbstractThe stepwise development of positive lightning leaders is still not well understood. A recent laboratory study indicated, at high absolute humidity, positive leaders can do steps due to the merging of a separate luminous structure and the primary leader channel, similar to the steps of negative leaders. The humidity may play a key role in the formation of positive leader steps, however, the humidity effect on the positive leader steps has never been explored. In this paper, we examine numerous positive long spark discharges at different humidity levels with the synchronized discharge current and high‐speed camera frames recording the evolution of leader channel. The positive leader propagation manners at different humidity levels are compared both morphologically and electrically. The effect of humidity on steps is further analyzed statistically. We found that the positive leader steps characterized by steep‐rise current pulse and abrupt channel elongation, which may be led by separate luminous structures, only appear under the condition that high absolute humidity is above a certain threshold. As the ambient humidity increases, these positive leader steps occur more frequently.

Integrated photonic encoder for low power and high-speed image processing.

Modern lens designs are capable of resolving greater than 10 gigapixels, while advances in camera frame-rate and hyperspectral imaging have made data acquisition rates of Terapixel/second a real possibility. The main bottlenecks preventing such high data-rate systems are power consumption and data storage. In this work, we show that analog photonic encoders could address this challenge, enabling high-speed image compression using orders-of-magnitude lower power than digital electronics. Our approach relies on a silicon-photonics front-end to compress raw image data, foregoing energy-intensive image conditioning and reducing data storage requirements. The compression scheme uses a passive disordered photonic structure to perform kernel-type random projections of the raw image data with minimal power consumption and low latency. A back-end neural network can then reconstruct the original images with structural similarity exceeding 90%. This scheme has the potential to process data streams exceeding Terapixel/second using less than 100 fJ/pixel, providing a path to ultra-high-resolution data and image acquisition systems.

High-speed eye tracking based on a synchronized imaging mechanism by a dual-ring infrared lighting source

It is a challenge for conventional monocular-camera single-light source eye-tracking methods to achieve high-speed eye tracking. In this work, a dual-ring infrared lighting source was designed to achieve bright and dark pupils in high speed. The eye-tracking method used a dual-ring infrared lighting source and synchronized triggers for the even and odd camera frames to capture bright and dark pupils. A pupillary corneal reflex was calculated by the center coordinates of the Purkinje spot and the pupil. A map function was established to map the relationship between the pupillary corneal reflex and gaze spots. The gaze coordinate was calculated based on the mapping function. The average detection time of each gaze spot was 3.76 ms.

Iterative reconstruction methods and the resolution principle for fast-ion loss detector measurements

Fast-ion loss detectors (FILDs) are crucial for analyzing fast-ion dynamics in magnetically confined fusion plasmas. A core challenge is to derive an accurate ion velocity distribution, requiring treatment of thousands of remapped camera frames for a full discharge. The ill-posed nature of this task necessitates regularization with a well-chosen regularization parameter and computationally efficient methods. In this work, we introduce the ‘resolution principle,’ a novel criterion for selecting the optimal regularization parameter, providing a distinction between genuine features and artefacts smaller than the diagnostic resolution in the reconstruction, thereby preventing misinterpretations. This principle, coupled with three iterative reconstruction techniques—Kaczmarz’s method, coordinate descent, and Cimmino’s method—demonstrates enhanced reconstruction capabilities compared to conventional methods like Tikhonov regularization. Utilizing these techniques allows rapid processing of measurements from full discharges, removing the computational bottleneck and facilitating between-discharge reconstructions. By reconstructing 6000 camera frames from an ELMy H-mode discharge at ASDEX Upgrade, we capture the temporal evolution of gyroradii and pitch angles, unveiling a direct correlation between pitch-angle behavior and changes in the toroidal magnetic field for a specific subset of lost ions accelerated by edge-localized modes (ELMs) to energies approximately twice that of the injection energy.